Kerry Buckley

[Update] The story runner is now included in the 1.1 release of RSpec, so a lot of the hackery mentioned below is no longer required. See http://rspec.info/ for details.

Background

A while ago I cobbled together some code to drive Selenium from Exactor (which was the acceptance testing framework we were using at the time). That project was offshored shortly afterwards, and I'm pretty sure Selenium never actually got integrated into the build (sneaking a peek at the continuous integration server reveals that even the pre-Selenium acceptance test build hasn't run successfully for months), but I was convinced of the value of Selenium for testing non-trivial web applications.

On my next project (the Web21C SDK portal) we used Selenium on Rails heavily for automated acceptance testing. This worked well, although the fact that the tests are deployed with the application limits what you can do – you can't interrogate the database after a test, for example, or dynamically set up stubs for systems you interface with (unless you also deploy the stubs as part of the application, and drive them from the browser).

With Mojo there's an additional complication: as well as being accessible through a browser, most functionality is also available as a RESTful API, using digest-based authentication. To keep our acceptance tests in one place, we wanted to be able to drive a browser using Selenium, alongside other tests which talked to the server directly. The obvious answer was to use Selenium Remote Control, and I also liked the look of RBehave, which has now been incorporated into RSpec (but not released yet).

The RSpec story runner

The story runner in the upcoming release of RSpec is an evolution of Dan North's RBehave, which in turn is based on JBehave (from the same author). In the past couple of weeks, David Chelimsky has done some excellent work on extracting the text of stories from the code, leading to the plain text story runner.

Read on for the gory details of my solution to running Selenium tests from the story runner.

Putting it all together

Here's what I did to get Selenium working from the story runner (you need to check out RSpec trunk – I'm currently using r2791 of rspec and r2814 or rspec_on_rails).

I'm sure when the next release of RSpec comes out a lot of this will be unnecessary, and there will probably be standard places for the stories and ways of running them, but for now I've just hacked something together that's simple and works. Apologies for any pieces of string or duct tape that are still showing!

The files described below are laid out as follows, relative to the root of the Rails project:

+-- lib
| +-- rspec/
| +-- rspec_on_rails/
| +-- selenium-ruby-client-driver/
| | +-- selenium.rb
| +-- selenium-server/
| | +-- selenium-server.jar
| +-- tasks/
|   +-- acceptance.rake
+-- stories/
| +-- all.rb
| +-- helper.rb
| +-- steps/
| | +-- login.rb
| | +-- selenium.rb
| +-- stories/
| | +-- login.rb
| | +-- login.txt

Here's an example story:

stories/stories/login.txt

Each 'Given', 'When' and 'Then' corresponds to a step (an 'And' is just syntactic sugar for a repeat of the same type of step). The steps are defined in separate files, which can be included as required (see below). This story uses two sets of steps:

stories/steps/login.rb

stories/steps/selenium.rb

Note the use of placeholder parameters, which are parsed out of the lines in the story. The Selenium steps use a global instance of Selenium::SeleniumDriver, which is created in the helper (again, see below). I originally created this in a "Given a browser" step, but it seemed simpler to just create it up front, before running the stories.

stories/stories/login.rb

This is just a wrapper for the textual story, which specifies which steps are required.

stories/all.rb

Another simple wrapper, this time to run all stories.

stories/helper.rb

Warning: This file is where all the ugly code's hiding!

Most of this code doesn't really belong 'loose' in a helper file, but I didn't want to expend too much effort on getting everything working tidily with a pre-release version of RSpec.Hopefully there are enough comments to give a vague idea what's going on, but here are some more notes:

• Lines 5–10: There still seem to be some bugs in trunk RSpec which cause existing specs to fail (or possibly bugs in our specs that don't show up in 1.0.8). I've installed the trunk versions of the plugins under lib, and here I pull the real plugins out of the load path and insert the trunk versions instead.
• Lines 17–21: Create a selenium driver, and open a browser session.
• Lines 23–28: By default, RSpec adds an ActiveRecordSafetyListener to the story runner. This rolls back database changes between scenarios, which is great if your calling your code directly, but obviously means that if you write to the database, the server that Selenium's talking to can't see them. There's probably a cleaner way of disabling it.
• Lines 30–37: A convenience method to remove some code from the story wrappers. Basically just replaces '.rb' with '.txt' and runs that file with the specified steps. However, I can't find a clean way of finding the file name of the code that called a method, so I'm pulling it out of the call stack. Ugh.
• Lines 39–56: The method names in the ruby interface to Selenium RC are rather java-like (eg get_title and is_text_present). I've used a bit of method_missing hackery to wrap these methods with more ruby-like names (title and has_text_present? for those specific cases), which allows us to use rspec matchers like $selenium.title.should == foo and $selenium.should have_title bar.

lib/tasks/acceptance.rake

Finally, I've created some rake tasks to wrap everything up, and also to start and stop the selenium server.

Still to do

There's plenty of scope for improvement here. For a start, I haven't figured out how to generate a report for the test run, apart from the plain text output from the console (which is only usable if you start the selenium server in a separate shell). Also, I can't see an obvious way of returning a success/fail status at the end of the run, which makes it difficult to use in a continuous integration process.

The story runner looks like a very promising tool though, and I can't wait for the stable release.

Technorati Tags: bdd, rspec, rbehave, selenium, acceptance testing