Kerry Buckley

What’s the simplest thing that could possibly go wrong?

Archive for the ‘Ruby’ tag

Upgrading to Snow Leopard

one comment

A quick list of things I had to sort out after upgrading to Mac OS X 10.6 Snow Leopard:

Developer Tools

Dont’ forget to run the XCode installer on the Snow Leopard DVD, otherwise you’ll have trouble getting stuff to compile, even if you don’t use XCode. You’ll also have to download and install the iPhone SDK separately if you need it (and possibly even if you don’t – I installed it anyway, just in case).

Ruby and RubyGems

I had both of these installed from source, and although most things seemed to work OK, I couldn’t get Passenger to work at all until I reinstalled them. Instructions for installing are available on HiveLogic – this will overwrite any existing versions, assuming they’re in /usr/local (the system version of Ruby isn’t touched).

Before installing rubygems I removed all my installed gems (gem list|awk '{print $1}'|xargs sudo gem unin -a – there’s probably an easier way), then I reinstalled the ones I needed afterwards.

MySQL

Although I mostly use Postgres, I reinstalled MySQL following the instructions on the Norbauer blog.

MacPorts

Apparently you can rebuild your ports by just running sudo port upgrade --force installed, but by the time I came across that I’d already trashed and reinstalled as recommended on the link above.

For some reason the MacPorts installer hung while running the postinstall scripts, but after force-quitting the installer then running sudo port sync everything seemed fine.

I added +svn to the arguments for installing git-core (as if it didn’t have enough dependencies to build already!), and also installed postgresql84-server and imagemagick.

Apache and Passenger

I tried a whole bunch of stuff to get Passenger running, but it turned out in the end that rebuilding Ruby was the answer (see above). Once I’d done that, it was a simple case of installing the passenger gem and running sudo passenger-install-apache2-module to install the module.

Vim

The standard version of MacVim mostly works under 10.6, but there’s a custom-built binary that seems much more stable and a bit snappier.

Reader Notifier

The release version of Reader Notifier doesn’t work on 10.6, but for now there’s a patched version.

Safari Plugins

ClickToFlash needs to be upgraded to 1.5fc2.

DeliciousSafari hasn’t been updated for 64-bit Safari yet, but as a workaround you can force Safari to run in 32-bit mode. Do a ‘get info’ on the Safari app (in the Applications folder), and tick ‘Open in 32-bit mode’.

iStat Menus

Turns out I was using an old version (1.3) of iStat Menus, which doesn’t work in 10.6 (I noticed the missing menu when I went to check how high all the port install shenanigans were pushing the CPU temperatures). Upgrading to 2.0 sorted it out.

FlickrExport

Again, I was using an old version of this iPhoto exporter, but £6.90 and an upgrade to 3.0.2 later and everything was working again.

Remapping caps lock to escape

Not strictly 10.6-specific, but this was something I’d been meaning to get round to since switching back to vim. I was already to start installing input manager hacks until I stumbled across a blog post somewhere mentioning that it is already configurable (and has been for a while). Just open the keyboard preferences, hit ‘Modifier Keys…’ and change the action.

Written by Kerry

September 9th, 2009 at 2:33 pm

Posted in Apple

Tagged with , , , , , , , ,

Naresh Jain’s refactoring teaser

leave a comment

Naresh Jain recently posted a refactoring teaser. The original code was in Java, but I thought I'd have a go at refactoring it in Ruby instead. I deliberately didn't look at Naresh's solution beforehand, so mine goes in a rather different direction.

My code's here (with the specs in the same file for simplicity), and you can step through the history to see the state at each step of the refactoring. The links in the text below take you to the relevant version of the file, and the Δs next to them link to the diffs.

Firstly, I converted the code to Ruby, and made sure the tests (which I converted to RSpec) still passed. It's a fairly straight conversion, although I made a couple of changes while I was at it – mainly turning it into a module which I mixed into String. I changed the name of the method to phrases, to avoid conflicting with the built-in String#split. The regular expression to split on is much simpler too, because Ruby didn't understand the original, and I had no idea what it was supposed to do anyway.

Here's my initial Ruby version:

RUBY:
  1. #!/usr/bin/env ruby
  2. #
  3. #See http://blogs.agilefaqs.com/2009/07/08/refactoring-teaser-part-1/
  4.  
  5. require 'spec'
  6.  
  7. module StringExtensions
  8.   REGEX_TO_SPLIT_ALONG_WHITESPACES = /\s+/
  9.  
  10.   def phrases(number)
  11.     list_of_keywords = ""
  12.     count = 0
  13.     strings = split(REGEX_TO_SPLIT_ALONG_WHITESPACES)
  14.     all_strings = single_double_triple_words(strings)
  15.     size = all_strings.size
  16.     all_strings.each do |phrase|
  17.       break if count == number
  18.       list_of_keywords += "'" + phrase + "'"
  19.       count += 1
  20.       if (count <size && count <number)
  21.         list_of_keywords += ", "
  22.       end
  23.     end
  24.     return list_of_keywords
  25.   end
  26.  
  27.   private
  28.  
  29.   def single_double_triple_words(strings)
  30.     all_strings = []
  31.     num_words = strings.size
  32.  
  33.     return all_strings unless has_enough_words(num_words)
  34.  
  35.     # Extracting single words. Total size of words == num_words
  36.  
  37.     # Extracting single-word phrases.
  38.     (0...num_words).each do |i|
  39.       all_strings <<strings[i]
  40.     end
  41.  
  42.     # Extracting double-word phrases
  43.     (0...num_words - 1).each do |i|
  44.       all_strings <<"#{strings[i]} #{strings[i + 1]}"
  45.     end
  46.  
  47.     # Extracting triple-word phrases
  48.     (0...num_words - 2).each do |i|
  49.       all_strings <<"#{strings[i]} #{strings[i + 1]} #{strings[i + 2]}"
  50.     end
  51.     return all_strings
  52.   end
  53.  
  54.   def has_enough_words(num_words)
  55.     num_words>= 3
  56.   end
  57. end
  58.  
  59. String.send(:include, StringExtensions)
  60.  
  61. describe StringExtensions do
  62.   it 'finds all phrases' do
  63.     'Hello World Ruby'.phrases(6).should == "'Hello', 'World', 'Ruby', 'Hello World', 'World Ruby', 'Hello World Ruby'"
  64.   end
  65.  
  66.   it 'returns all phrases when asked for more than exist' do
  67.     'Hello World Ruby'.phrases(10).should == "'Hello', 'World', 'Ruby', 'Hello World', 'World Ruby', 'Hello World Ruby'"
  68.   end
  69.  
  70.   it 'returns the first n phrases when asked for fewer than exist' do
  71.     'Hello World Ruby'.phrases(4).should == "'Hello', 'World', 'Ruby', 'Hello World'"
  72.   end
  73.  
  74.   it 'returns the first word when asked for one phrase' do
  75.     'Hello World Ruby'.phrases(1).should == "'Hello'"
  76.   end
  77. end

I didn't change the specs at all during the refactoring (because I didn't change the API or add any new public methods or classes), and made sure they all passed at each step.

The first thing Δ I changed was to simplify that big iterator in phrases that loops through the list of phrases, formatting the output string. Basically all this does is to put each phrase in quotes, then stitch them all together separated by a comma and a space. The first of those tasks is a simple map, and the second is a join. The whole method collapses down to this (ruby methods automatically return the result of their last statement):

RUBY:
  1. def phrases(number)
  2.   strings = split(REGEX_TO_SPLIT_ALONG_WHITESPACES)
  3.   all_strings = single_double_triple_words(strings)
  4.   all_strings[0, number].map {|s| "'#{s}'"}.join(', ')
  5. end

Next Δ I remembered that by default String#split splits at whitespace anyway, so I did away with the regular expression. Then Δ I renamed the strings variable to words to make its purpose a little clearer, leaving the phrases method looking like this:

RUBY:
  1. def phrases(number)
  2.   words = split
  3.   all_strings = single_double_triple_words(words)
  4.   all_strings[0, number].map {|s| "'#{s}'"}.join(', ')
  5. end

The section of single_double_triple_words that extracted the single words seemed redundant, as we already had that list – the original words. I removed it Δ, and initialised all_strings to the word list instead (not forgetting to rename single_double_triple_words to match its new behaviour):

RUBY:
  1. module StringExtensions
  2.   def phrases(number)
  3.     words = split
  4.     all_strings = words
  5.     all_strings += double_triple_words(words)
  6.     all_strings[0, number].map {|s| "'#{s}'"}.join(', ')
  7.   end
  8.  
  9.   private
  10.  
  11.   def double_triple_words(strings)
  12.     all_strings = []
  13.     num_words = strings.size
  14.  
  15.     return all_strings unless has_enough_words(num_words)
  16.  
  17.     # Extracting double-word phrases
  18.     (0...num_words - 1).each do |i|
  19.       all_strings <<"#{strings[i]} #{strings[i + 1]}"
  20.     end
  21.  
  22.     # Extracting triple-word phrases
  23.     (0...num_words - 2).each do |i|
  24.       all_strings <<"#{strings[i]} #{strings[i + 1]} #{strings[i + 2]}"
  25.     end
  26.     return all_strings
  27.   end
  28.  
  29.   def has_enough_words(num_words)
  30.     num_words>= 3
  31.   end
  32. end

That has_enough_words method seemed a bit odd – particularly the way it was only called once, rather than after extracting each set of phrases. I decided it was probably a premature and incomplete attempt at optimisation, and removed it Δ for now.

My next target was the duplication in the blocks that calculate double- and triple-word phrases. First Δ I extracted them into separate methods:

RUBY:
  1. module StringExtensions
  2.   def phrases(number)
  3.     words = split
  4.     all_strings = words
  5.     all_strings += double_words(words)
  6.     all_strings += triple_words(words)
  7.     all_strings[0, number].map {|s| "'#{s}'"}.join(', ')
  8.   end
  9.  
  10.   private
  11.  
  12.   def double_words(strings)
  13.     all_strings = []
  14.     num_words = strings.size
  15.  
  16.     # Extracting double-word phrases
  17.     (0...num_words - 1).each do |i|
  18.       all_strings <<"#{strings[i]} #{strings[i + 1]}"
  19.     end
  20.     return all_strings
  21.   end
  22.  
  23.   def triple_words(strings)
  24.     all_strings = []
  25.     num_words = strings.size
  26.  
  27.     (0...num_words - 2).each do |i|
  28.       all_strings <<"#{strings[i]} #{strings[i + 1]} #{strings[i + 2]}"
  29.     end
  30.     return all_strings
  31.   end
  32. end

I decided that the num_words variable in the two new methods wasn't really necessary (it was only used once, and I think strings.size expresses intent perfectly clearly), so I inlined it Δ (and the same in triple_words):

RUBY:
  1. def double_words(strings)
  2.   all_strings = []
  3.  
  4.   # Extracting double-word phrases
  5.   (0...strings.size - 1).each do |i|
  6.     all_strings <<"#{strings[i]} #{strings[i + 1]}"
  7.   end
  8.   return all_strings
  9. end

Most of the code in double_words and triple_words was obviously very similar, so I created Δ a general extract_phrases method, and called it from both. The new method uses the start position and length version of Array#[] to extract the appropriate number of words, then Array#join to string them together separated by spaces:

RUBY:
  1. def double_words(strings)
  2.   extract_phrases(strings, 2)
  3. end
  4.  
  5. def triple_words(strings)
  6.   extract_phrases(strings, 3)
  7. end
  8.  
  9. def extract_phrases(strings, number_of_words)
  10.   result = []
  11.   (0...strings.size - number_of_words + 1).each do |i|
  12.     phrase = strings[i, number_of_words].join(' ')
  13.     result <<phrase
  14.   end
  15.   result
  16. end

At this point double_words and triple_words have become just dumb wrappers around extract_phrases, so I removed them Δ and just called extract_phrases directly:

RUBY:
  1. def phrases(number)
  2.   words = split
  3.   all_strings = words
  4.   all_strings += extract_phrases(words, 2)
  5.   all_strings += extract_phrases(words, 3)
  6.   all_strings[0, number].map {|s| "'#{s}'"}.join(', ')
  7. end

Rather than hardcoding the calls for two and three words, I changed it Δ to use a loop:

RUBY:
  1. def phrases(number)
  2.   words = split
  3.   all_strings = words
  4.   (2..words.size).each do |number_of_words|
  5.     all_strings += extract_phrases(words, number_of_words)
  6.   end
  7.   all_strings[0, number].map {|s| "'#{s}'"}.join(', ')
  8. end

I decided this was the point to put the optimisation back Δ and stop looking for phrases once we had enough:

RUBY:
  1. def phrases(number)
  2.   words = split
  3.   all_strings = words
  4.   (2..words.size).each do |number_of_words|
  5.     break if all_strings.length>= number
  6.     all_strings += extract_phrases(words, number_of_words)
  7.   end
  8.   all_strings[0, number].map {|s| "'#{s}'"}.join(', ')
  9. end

I decided that number_of_words wasn't particularly clear, so changed it Δ to phrase_length, then Δ made the iterator in extract_phrases more ruby-like by using map:

RUBY:
  1. def extract_phrases(strings, phrase_length)
  2.   (0...strings.size - phrase_length + 1).map do |i|
  3.     strings[i, phrase_length].join(' ')
  4.   end
  5. end

I then noticed that I hadn't been consistent in changing strings to words, so fixed that Δ.

Lastly Δ, I decided that even though it was only one line, the code that formats the output deserved pulling out into a method.

Here's my final version of the code:

RUBY:
  1. module StringExtensions
  2.   def phrases(number)
  3.     words = split
  4.     all_strings = words
  5.     (2..words.size).each do |phrase_length|
  6.       break if all_strings.length>= number
  7.       all_strings += extract_phrases(words, phrase_length)
  8.     end
  9.     format_output(all_strings[0, number])
  10.   end
  11.  
  12.   private
  13.  
  14.   def extract_phrases(words, phrase_length)
  15.     (0...words.size - phrase_length + 1).map do |i|
  16.       words[i, phrase_length].join(' ')
  17.     end
  18.   end
  19.  
  20.   def format_output(phrases)
  21.     phrases.map {|s| "'#{s}'"}.join(', ')
  22.   end
  23. end

Written by Kerry

July 17th, 2009 at 8:03 am

Posted in Ruby

Tagged with , ,

A couple of rspec mocking gotchas

3 comments

Just a couple of things that have caused a bit of head-scratching lately when writing RSpec specs using the built-in mocking framework.

Catching StandardError

Watch out if the code you're testing catches StandardError (of course you're not catching Exception, right?). Try this:

RUBY:
  1. require 'rubygems'
  2. require 'spec'
  3.  
  4. class Foo
  5.   def self.foo
  6.     Bar.bar
  7.   rescue StandardError
  8.     # do something here and don't re-raise
  9.   end
  10. end
  11.  
  12. class Bar
  13.   def self.bar
  14.   end
  15. end
  16.  
  17. describe 'Calling a method that catches StandardError' do
  18.   it 'calls Bar.bar' do
  19.     Bar.should_receive :bar
  20.     Foo.foo
  21.   end
  22. end

Nothing particularly exciting there. Let's run it and check that it passes:

$ spec foo.rb
.

Finished in 0.001862 seconds

1 example, 0 failures

However, what if we change the example to test the opposite behaviour?

RUBY:
  1. describe 'Calling a method that catches StandardError' do
  2.   it 'does NOT call Bar.bar' do
  3.     Bar.should_not_receive :bar
  4.     Foo.foo
  5.   end
  6. end

$ spec foo.rb
.

Finished in 0.001865 seconds

1 example, 0 failures

Wait, surely they can't both pass? Let's take out the rescue and see what's going on:

RUBY:
  1. class Foo
  2.   def self.foo
  3.     Bar.bar
  4.   end
  5. end

$ spec foo.rb
F

1)
Spec::Mocks::MockExpectationError in 'Calling a method that catches StandardError does NOT call Bar.bar'
 expected :bar with (no args) 0 times, but received it once
./foo.rb:6:in `foo'
./foo.rb:18:

Finished in 0.002276 seconds

1 example, 1 failure

That's more like it.

Of course, what's really happening here is that Spec::Mocks::MockExpectationError is a subclass of StandardError, so is being caught and silently discarded by our method under test.

If you're doing TDD properly, this won't result in a useless test (at least not immediately), but it might cause you to spend a while trying to figure out how to get a failing test before you add the call to Foo.foo (assuming the method with the rescue already existed). Generally you can solve the problem by making the code a bit more selective about which exception class(es) it catches, but I wonder whether RSpec exceptions are special cases which ought to directly extend Exception.

Checking receive counts on previously-stubbed methods

It's quite common to stub a method on a collaborator in a before block, then check the details of the call to the method in a specific example. This doesn't work quite as you would expect if for some reason you want to check that the method is only called a specific number of times:

RUBY:
  1. require 'rubygems'
  2. require 'spec'
  3.  
  4. class Foo
  5.   def self.foo
  6.     Bar.bar
  7.     Bar.bar
  8.   end
  9. end
  10.  
  11. class Bar
  12.   def self.bar
  13.   end
  14. end
  15.  
  16. describe 'Checking call counts for a stubbed method' do
  17.   before do
  18.     Bar.stub! :bar
  19.   end
  20.  
  21.   it 'only calls a method once' do
  22.     Bar.should_receive(:bar).once
  23.     Foo.foo
  24.   end
  25. end

$ spec foo.rb
.

Finished in 0.001867 seconds

1 example, 0 failures

I think what's happening here is that the mock object would normally receive an unexpected call, causing the expected :bar with (any args) once, but received it twice error that you'd expect. Unfortunately the second call to the method is handled by the stub, so never triggers the error.

You can fix it, but it's messy:

RUBY:
  1. it 'only calls a method once' do
  2.   Bar.send(:__mock_proxy).reset
  3.   Bar.should_receive(:bar).once
  4.   Foo.foo
  5. end

$ spec foo.rb
F

1)
Spec::Mocks::MockExpectationError in 'Checking call counts for a stubbed method only calls a method once'
 expected :bar with (any args) once, but received it twice
./foo.rb:23:

Finished in 0.002542 seconds

1 example, 1 failure

Does anyone know a better way?

The full example code is in this gist.

Written by Kerry

May 28th, 2009 at 12:05 pm

Posted in rspec

Tagged with ,

API vs RSI

one comment

World's-most-famous-twitterer Stephen Fry has a system for handling follow requests: you tweet using the #followmestephen hashtag, and he wades diligently through them, manually following people.

This seems an odd sort of thing to do – most people choose whom to follow based on whether they know them or like what they say, rather than on request – but I suppose when you have over a quarter of a million followers things work a little differently. It also creates lots of work , and looks like an ideal candidate for automation.

I thought I'd have a quick play with the Twitter API this morning (no doubt I'm not the only one), and cobbled together the script below, which you can also download as follow_me_stephen.rb (although if you're not Mr Fry I'm not sure why you would want to). Save the file, and run using ruby follow_me_stephen.rb.

I wanted to avoid having too many dependencies, so I didn't use the twitter gem, or the excellent httparty, but I was too lazy to figure out all the XPaths to handle the Atom version of the API. This means you need to have the JSON gem installed, which is as simple as sudo gem install json (omit the sudo on Windows).

The script's pretty dumb, in that it grabs the whole set of search results every time, and blindly requests to follow everyone, regardless of whether you're already following them.

RUBY:
  1. #!/usr/bin/env ruby
  2.  
  3. require 'net/http'
  4.  
  5. begin
  6.   require 'json'
  7. rescue LoadError
  8.   STDERR.puts <<EOF
  9.  
  10. No JSON parser found. Please run the following command to install:
  11.  
  12.   sudo gem install json
  13.  
  14. EOF
  15.   raise
  16. end
  17.  
  18. module FollowMeStephen
  19.   def run
  20.     auth_user, password = get_user_details
  21.     requestors = fetch_requestors
  22.     requestors.each do |user|
  23.       follow user, auth_user, password
  24.     end
  25.   end
  26.  
  27.   private
  28.  
  29.   def get_user_details
  30.     print 'Please enter your Twitter username: '
  31.     auth_user = gets.chomp
  32.     print 'Please enter your Twitter password: '
  33.     password = gets.chomp
  34.     return auth_user, password
  35.   end
  36.  
  37.   def fetch_requestors
  38.     requestors = []
  39.     puts 'Searching for hashtag "followmestephen"...'
  40.     query = '?q=%23followmestephen&rpp=150'
  41.     while query do
  42.       search = JSON.parse(get("/search.json#{query}"))
  43.       puts "Received page #{search['page']}"
  44.       requestors += search['results'].map {|r| r['from_user']}
  45.       query = search['next_page']
  46.     end
  47.     requestors.uniq
  48.   end
  49.  
  50.   def follow user, auth_user, password
  51.     print "Following #{user}... "
  52.     result = post "/friendships/create/#{user}.json", auth_user, password
  53.     puts result
  54.   end
  55.  
  56.   def get path
  57.     Net::HTTP.get 'search.twitter.com', path
  58.   end
  59.  
  60.   def post path, auth_user, password
  61.     request = Net::HTTP::Post.new path
  62.     request.basic_auth auth_user, password
  63.     response = Net::HTTP.new('twitter.com').start {|http| http.request(request)}
  64.     (response.kind_of? Net::HTTPSuccess) ? 'OK' : 'Failed'
  65.   end
  66. end
  67.  
  68. include FollowMeStephen
  69. run

Written by Kerry

March 15th, 2009 at 12:16 pm

Posted in Ruby

Tagged with , , ,