Installing Curb with Ruby 1.9.1 in Windows

Disclaimer: This is about how I installed curb in Windows XP. This is probably not the only way and certainly not the best way. I'm not a Windows developer (I just play one on TV) and haven't done any work with C in an embarrassingly long time. I'll be happy to help with any problems, but please don't expect me to be able to fix this at 5pm on Friday when your production machine decides to blow up.

Now that we got that out of the way...

How I finally got curb to install in Windows.

1. Install the latest Ruby installer and DevKit.
   The finer points of doing that are outside the scope of this post, so if you have trouble with those things I'd reccommend this tutorial, the DevKit Howto and the excellent RubyInstaller mailing list.
   
   Just for the record, I'm using the latest as of the time I'm writing this: ruby-1.9.1-p429 and devkit-3.4.5r3-20091110.7z.
   


2. Get libcurl.
   I, again, am using the latest as of the time I'm writing this: 7.21.0. If you need a different version for whatever reason, the libcurl downloads page has a few different versions available, so make sure to get the one for mingw32 and that you get libcurl and not plain-old curl.
   
   With that downloaded, I extracted the .zip file to the root of my C:\ drive. The location you put this is important for the next steps.


3. Put the curl binaries in your path.
   My preferred way is to add the libcurl \bin folder to the Windows path, so that typing 'path' at a Windows command prompt gives a result like this:
   
   C:\>path
PATH=C:\Ruby191-p429\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\curl-7.21.0-devel-mingw32\bin;
   
   Other things you can do here are copy the libcurl .dll files into your ruby's \bin folder (or anywhere else in the PATH) but I think that's kinda lame.


4. Install curb.
   The magic incantation I found for this was the following:
   
   gem install curb -- --with-curl-lib=C:\curl-7.21.0-devel-mingw32\bin --with-curl-include=C:\curl-7.21.0-devel-mingw32\include
   
   Every other combination of flags and paths resulted in compilation errors, so you've been warned. Also, you'll need to change the curl paths if you decided to install to a different place or with a different version.
   


5. Test it.
   Here's a very simple script that will load curb and GET a url:
   require 'curb'

c = Curl::Easy.new
c.url = "http://example.com"
c.http_get
puts c.body_str.size

# Output:
# 574

Did this work for you? Let me know in the comments!

Getting S3 and SWFUpload to Cooperate in Rails

Updated on 4/27: Added the workaround for the bug in the Mac's flash plugin.

(First things first, there is a rails plugin that automates a lot of this process. I didn't like the way it worked for a few reasons (like making swfobject.js into one big javascript variable), so I decided not to use it. PJ at GitHub mentioned that they use it, so it must not be all bad.)

These are some stumbling blocks I ran into and I thought I'd try to save others some frustration. I'll try to link or acknowledge the particular sources where I read these nuggets, but I've been googling like mad the last few days and might have forgotten.

Signature and Policy generation
elcgit decided to reinvent some wheels with the signing and policy generation bit, and wrote his own base64 encoding routines. This is unnecessary, because ruby includes openssl and base 64 encoding capabilities in the stdlib. In fact, amazon provides sample code for this on this helpful documentation page:


require 'base64'
require 'openssl'
require 'digest/sha1'

policy = Base64.encode64(policy_document).gsub("\n","")

signature = Base64.encode64(
OpenSSL::HMAC.digest(
OpenSSL::Digest::Digest.new('sha1'),
aws_secret_key, policy)
).gsub("\n","")


S3 Post Params

• How to pass the required POST parameters to S3 with SWFUpload might seem somewhat obvious, but I wasn't sure how to do this in the beginning. You merely need a line like when setting up your SWFUpload object:
  
  
post_params : <%= @s3.to_json %>,

  
  I made a class to encapsulate all the S3 stuff, created an instance in the controller, and then call the .to_json method (delegated to hash) so that I didn't have to specify all that stuff in the view.



• S3 requires a field called "file" to be present in the POST parameters, but by default Flash (and therefore SWFUpload) name this field something else. You can change this by including this line in your SWFUpload configuration:
  
  
file_post_name : "file",

  



• S3 requires all POST parameters to be present in the policy that's signed and passed in the parameters, so make sure to add this line to tell S3 to expect a parameter that SWFUpload provides by default:
  
  
["starts-with", "$Filename", ""]

  


• To work around an apparent bug in the Mac's flash plugin, I had to add the "success_action_status" key to the S3 policy and POST parameters with a value of 201. According to the S3 docs, the default return code of 204 is sent with no response body, which apparently triggers the bug.
  

SWFUpload and Return Codes
Depending on a few options you specify, S3 can return a 200, 201 or 204 for a successful upload. By default, SWFUpload treats any return code other than 200 as an error. You can fix this on the S3 side by supplying the success_action_status parameter, but I chose to make it so that SWFUpload would regard those three return codes as successes by adding this to my SWFUpload config:


http_success : [ 200, 201, 204 ],


Host the SWFUpload File on S3
SWFUpload does not have very good support for uploading to a different domain than the actual .swf file is hosted on due to crossdomain issues. I spent a day or two trying to make this work with the proper settings in a crossdomain.xml file, but gave up and decided to host the .swf on S3. This adds a bit of a deployment headache, but it's a much simpler solution (and possibly the only one).

Debug By Uploading to Rails
SWFUpload does not provide a lot of debugging help; sure, it has a debug mode which gives some info, but for a lot of server-side things it's not very helpful (I'd have been saved a lot of trouble if SWFUpload would display the response body and not just the return code on a failed upload, for example). You can get a look at the POST parameters being sent if you set SWFUpload's upload_url setting to a dummy action in your app and looking at the logs.

Functional and Acceptance Testing
I've recently started using Cucumber for testing and love it. Unfortunately, I wasn't able to get it to work with SWFUpload properly; this led to me essentially "flying blind" with no tests and having to do a lot of testing in the browser (yuck!). If I can find out how to do this (or someone else lets me know), I'll update this entry to have that info.

Testing for database insert with ActiveRecord

Often in my tests, I want to test to see whether a database insert occurred; more often I want to make sure that an insert did not happen. I've found a few approaches using mocking, neither of which I was very satisfied with.

Mocking AR's connection's insert

This approach won't win you any code cleanliness awards, but when you absolutely must tell whether or not your code is trying to insert records, this will get the job done.

def test_that_fails_if_any_inserts_happen
flexmock( ActiveRecord::Base.connection ).should_receive( :insert ).never

# code to test
end


This doesn't make an assertion, but flexmock is setting an expectation that the method insert will never be called on AR's connection object before flexmock's teardown gets called.

Mocking model's create

This is a little higher level, but even more deeply coupled with your code.

def test_that_some_model_never_creates
flexmock( MyModel ).should_receive( :create, :create! ).never

# code to test
end

Doing the test this way seems a little cleaner at first, but there's one major caveat: create and create! are only two avenues to doing an insert in the database; this test would pass if the code was refactored to make a raw sql insert, if it used ar-extension's import method, etc. You could remember to change the test if either of those things were to ever happen, but it's best to just avoid this unless you have few options.

How does ActiveRecord test this?

After being unsatisfied with the last two ways to test this, I got the bright idea to look at AR's tests and see how they test this. Surely they need to test whether objects get created, right? Right. This is taken from AR 2.1.1's finder_test.rb, lines 598-604:

def test_find_or_create_from_one_attribute
number_of_companies = Company.count
sig38 = Company.find_or_create_by_name("38signals")
assert_equal number_of_companies + 1, Company.count
assert_equal sig38, Company.find_or_create_by_name("38signals")
assert !sig38.new_record?
end

As you can see, to test whether a database record is created, they take the count of the table before and after the code they're testing, and make sure the final count matches their expectation. Simple.
I was a little underwhelmed by their solution. It's clean, but it's lacking something that I can't put my finger on. Overall. I like their way of testing this over my two solutions, so I've started refactoring my tests to work this way.

Has anyone out there run into this problem, and come up with a better solution?

Running a subset of shoulda tests from the command line

Someone finally revealed a solution for something bothering me for a long time: how to run a subset of shoulda tests. This should help a lot for those really ugly bugs where my last resort is to look at the logs and see exactly what's going on.

Apparently code order matters in Ruby

I was recently working on some code that had a structure like this:

module OuterModule
  class MyObject
    include InnerModule
  end

  module InnerModule
    def hello
      "hello"
    end
  end
end

include OuterModule

m = MyObject.new

puts m.hello

When I ran it, I would get this unfortunate output:

test_ruby_class_heirarchy.rb:3: uninitialized constant OuterModule::MyObject::InnerModule (NameError)

I'm too embarrased to list all the voodoo I tried to get this to work, but simply putting the definition of MyObject below that of InnerModule did the trick:

module OuterModule
  module InnerModule
    def hello
      "hello"
    end
  end

  class MyObject
    include InnerModule
  end
end

include OuterModule

m = MyObject.new

puts m.hello

This gives me the expected output:

hello

Going into this, I would have never guessed that the order in which I defined things would have this effect.

A stupid reason I like shoulda

I've been interested shoulda for a while but hadn't played around with it until today. I really like the idea of seperate contexts and not having to name methods stuff like "test_process_should_create_record_if_one_doesnt_exist". With some playing around though, I found out that shoulda helps me work around a bug I found in flexmock and rails where the teardown method wasn't getting called correctly in my tests.

As an addendum to that mailing list post, I get this output:

Loaded suite test/unit/test_shoulda_test
Started
inside setup
inside test
inside teardown
.
Finished in 0.069689 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

... when I run this code:

require File.join( File.dirname(__FILE__), '..', 'test_helper' )
require 'test/unit'
require 'flexmock/test_unit'

class TestTest < Test::Unit::TestCase
  context "This test" do
    setup do
      $stderr.puts "inside setup"
    end

    teardown do
      $stderr.puts "inside teardown"
    end

    should "pass" do
      $stderr.puts "inside test"
      assert true
    end
  end
end

Thanks shoulda!

Testing Tidbts #1: Be explicit

I was recently refactoring some tests (you do refactor your tests, right? hmm, sounds like an idea for another post..) and I came across something like this, trying to test whether a time field on a rails model was updated correctly:

assert_in_delta mq.ended_at,
                Time.now,
                1

What's that 1 for at the end? What's the third parameter for assert_in_delta, anyways?

If I had been more explicit, that might be a little more apparent:

assert_in_delta mq.ended_at,
                Time.now,
                1.second

It's a small change, and totally uneccessary in real, working code, but I feel like it improves the readability a great deal.

Awesome: Easy Join Table Conditions

Most of the new-in-edge-rails features elicit a yawn from me (at best), but this one caught my eye. Some of the projects at work that I assist on could be cleaned up a great deal with this functionality.

sphinx charset_table with unicode character folding

In a project at work I needed to have sphinx treat accented characters and their unaccented versions the same, eg: é is equivalent to e, etc. I took this list I found on the sphinx wiki and transformed it into a sphinx friendly charset_table.

Now when I do a search for the string "Héctor Lavoe" it matches the string "Hector Lavoe". Awesome!

Edit: fixed missing "é" in first sentence.

Splitting long lines in text files with vim

Recently I needed to split long lines in a text file, and it took me a few minutes to figure out how to do this neatly in vim. In case I need it again, here's the regexp I used:

%s/\(.\{250\}[^ ]* \)/\1 \\^M/gc

250 is an arbitrary number I picked, and the "^M" part is a newline entered in vim by pressing ctrl-v, ctrl-m. Also, I split the line where there was a space, because I knew the data had frequent spaces and I didn't want to split a word.

ActiveRecord #find syntax and you

I've recently been adding eager-loading of ActiveRecord associations into a project at work, and I've run into a few problems because of how the original #find queries were wrtten. I did a little research, and found out how you can help out AR a bit and save yourself (and me) some hassle.

The easiest way to query AR is to simply use an id, like this:

User.find(10)

This uses this sql, nicely referenced by the table name:

SELECT * FROM users WHERE (users.`id` = 10)

The downside to this is that a plain #find call will throw an exception if no record exists, and that’s not good (unless caught, the client will see a 500 at that point).

We’ve been getting around this by using find calls like this:

User.find( :all, :conditions => ['id = ?', 10] )

Which generates this sql:

SELECT * FROM users WHERE (id = 10)

This way is good because it will return nil if no record is found, instead of throwing an exception. The subtle downside is that the string in the conditions parameter is copied directly into the sql string, because AR doesn’t know how to do anything with it.

If instead you use a hash for the conditions parameter, like this:

User.find( :all, :conditions => { :id => 10 } )

AR knows that you’re referencing a column from that model’s table, and gives you the safe sql with no bad exception aftertaste:

SELECT * FROM users WHERE (users.`id` = 10)

As an added bonus, because AR is allowed to do it’s magic, if you decide to pass it an array instead of a single integer, it can do the right thing without you having to change your #find code:

User.find( :all, :conditions => { :id => [10,12] } )

Which automatically comes out as:

SELECT * FROM users WHERE (users.`id` IN (10,12))

Summary:

Don’t do this:

User.find(10)

or this: User.find( :all, :conditions => ['id = ?', 10] ).

Do this:

User.find( :all, :conditions => { :id => 10 } )

Rescue Me!

With all the talk recently about elegant nil handling in Ruby (like andand and if_not_nil and #try), I was surprised that no one mentioned using rescue.

Surely every ruby programmer knows rescue when used with its friend begin, but did you know rescue has a statement modifier form?

For example, the following throws an exception:
>> [ "Bacon Lettuce Tomato", nil ].collect {|s| s.upcase.reverse[0,6].reverse }
NoMethodError: You have a nil object when you didn't expect it!
The error occurred while evaluating nil.upcase
from (irb):15
from (irb):15:in `collect'
from (irb):15

Adding rescue nil after that ugly bit of method chaining gets me this:
>> [ "Bacon Lettuce Tomato", nil ].collect {|s| s.upcase.reverse[0,6].reverse rescue nil }
=> ["TOMATO", nil]

... and rescue "" gets me this, useful if I want to return all the same types:
>> [ "Bacon Lettuce Tomato", nil ].collect {|s| s.upcase.reverse[0,6].reverse rescue "" }
=> ["TOMATO", ""]

I haven't used this other than playing around in a console, so I don't really know if this has nasty side effects or not. Maybe there's a reason all these Ruby gurus haven't mentioned it...