I agree with the intention, but I’m not sure I agree. I’m happy to lose the ‘test’ &ndash it’s losing the ‘development’ that bothers me. I’ve encountered far too many ‘designers’ who haven’t written a line of code in years, and I worry that they’d read ‘design by example’ as simply starting their design document with a few test cases. I’ve already seen TDD redefined as test-driven design and interpreted that way.

Of course for people who already grok TDD this isn’t an issue, but it’s not them that a name change is aimed at. Test-driven development might not be the perfect name, but at least there are already plenty of people who do understand what it really means to correct any misapprehension. The term BDD (which I use myself from time to time) has already started sowing confusion, without adding yet another.

On the other hand, anything that gets people discussing how to get the most from the practice has to be a good thing.

I guess I'm fortunate to work for a company (and in a country) where by and large there isn't a culture of firing people who don't meet performance targets – instead we have a system where an ever-increasing proportion of people's pay packet is made up of a performance-related bonus, rather than a fixed salary.

So, how do you go about getting your bonus each quarter? Simple: just meet your agreed targets.

Of course, nothing in a large corporation can ever be that simple, so in practice there's a complex tiered system of company, business unit, programme, project, team and individual targets, which are combined in a magic spreadsheet to generate how much bonus everyone gets. Each of these targets, and the performance against them, has to be agreed, monitored, quantified, audited and levelled. At each stage politics comes into play. Those that enjoy playing systems try to set targets they know they can achieve. People concentrate on meeting the letter of the objectives, possibly to the detriment of other activities, like helping colleagues or making process improvements.

Now step away from this corporate dystopia for a moment, into the world of Agile Software Development. A world where we value Individuals and interactions over processes and tools, and Responding to change over following a plan. A world where we strive to build projects around motivated individuals, give them the environment and support they need, and trust them to get the job done. Where working software is the primary measure of progress. Where at regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. Where the self-organising team, accountable directly to it customer or product owner, is responsible for its own delivery and processes.

Now, when agile teams in large organisations are forced to jump through the externally-imposed hoops of objectives, development action plans and post-implementation reviews (which add little visible benefit and take up time that could be spent delivering business value), is it any wonder that the carrot of performance-related pay feels like it's more of a punishment than an incentive?

Technorati Tags: agile, performance management, enterprise, corporate, hr

Public domain photo from Wikimedia Commons.

BT: A case study in agile programming

The cost to the taxpayer of abandoned Whitehall computer projects since 2000 has reached almost £2bn - not including the bill for an online crime reporting site that was cancelled this week, a survey by the Guardian reveals.

I have no doubt whatsoever that the government wastes a vast amount on IT contracts, but I think that, by concentrating on the cost of cancelled projects, the article misses the point slightly.

If a project is clearly never going to deliver, it's far better to cancel it than to fall into the trap of the sunk cost fallacy, and keep pouring money in in the hope that eventually everything will turn out OK in the end.

The real questions for me are 'are the government's IT needs being met at the most cost-effective manner?' and 'why does it take so long to realise that a project is doomed?'

I don't know anything about the projects discussed in the article, or about how government IT contracts are handled in general, but I'd be willing to bet that the following guesses aren't too wide of the mark:

• Someone decides that a new system is required, and produces a huge list of everything they think it needs to do. This goes out to tender, and the job goes to whoever manages to produce the lowest quote while still giving a reasonably credible impression that they can actually complete the work in the specified time.
• The contractor goes off and starts work. They talk to the civil servants who are responsible for specifying the system, but probably not to the people who will actually have to use it. They then go off and produce a design, get it signed off, and set up teams to work on all the identified subsystems.
• Every few months they deliver a progress report, assuring the client that everything's progressing according to plan. After a year or so the schedule probably slips a bit, but they assure everyone that it's jsut a blip, and the final delivery won't be significantly affected (we can always trim the testing phase a little, right?)
• Because the contract fixed time, cost and scope, there's only one thing that can be adjusted to keep the project profitable when the estimates turn out to have been optimistic: quality. Of course this ripples forward, with more and more time spent chasing problems caused by poor quality work in existing parts of the system.
• When (eventually) the first version of the system is delivered, there are integration problems, it doesn't quite do what the people specifying it actually wanted, and it turns out that large parts of the specification weren't that important, but some vital features have been missed out altogether. Depending on just how big a disaster it all was, one of two things happens:
• The project gets cancelled. The contractor moves on to the next lucrative contract, and an enquiry's set up to investigate the specific reasons for the project failure, completely missing the big picture.
• The problems are slowly ironed out, and the missing features are added to the requirements for the next release. The contractor rubs its collective hands at the thought of the massive fees they can charge for the change requests, and a huge amount of time is wasted arguing about whether each change is a new feature or a bug request.

I'm not (quite) naive enough to suggest that all these problems could be solved by wholesale adoption of XP, but I get the impression that the government (and the media reporting on these fiascos) isn't even aware that there is a better way. With major companies adopting an agile approach now (or at least pretending to), how long before the people responsible for spending our taxes wake up?

Technorati Tags: government, it, waste, agile

I've already mentioned Jeff Patton's XPDay keynote, where he talked about the importance of working in iterations (as opposed to increments). Jeff has now written a column, The Neglected Practice of Iteration, on StickyMinds, making the same point (but without the music!)

Excellent stuff, and well worth a read. I suspect most people who claim to follow an iterative development process are prone to lapsing into an incremental mentality – I know I am.

Technorati Tags: Jeff Patton, iteration

The twist was that the dojo was run using Inform 6, which is a domain-specific language for writing text-based adventure games ("You are in a maze of twisty little passages, all alike..."). this meant that none of the participants were familiar with the language, which forced us to take very small steps rather than quickly hacking together large chunks of code based on experience.

The baby steps thing seemed to work quite well, helped by the strict five minute pair rotations (driver back to audience, navigator to driver, audience member to navigator). Those rotations weren't quite like the way we used to do it (we never really settled on a fixed policy, and experimented with ping-pong pairing, swapping both people out of the pair at once and increasing timeslots to ten minutes). We also weren't as strict about banning suggestions and comments from the floor – stopping that seemed to help avoid circular conversations and let the pair get on with coding, although the unusual language may have helped here too. It was very frustrating though when you could see an mistake that was causing compilation errors, but the pair hadn't seen it yet.

As it turned out we only got about halfway through the tasks we'd been set. It was quite an enjoyable hour and a half, but I'm not sure how much I learned.

Technorati Tags: xpday, coding dojo, pair programming, inform6

Many companies are now adopting agile methodologies such as Scrum to jump on the agile bandwagon, and are looking for consultants to provide training, and to come in and tell them all the answers to their problems. Because they are looking for prescriptive processes that they can put in place for people to follow, we are starting to see ever-more complex systems for things that were previously considered simple tasks (such as prioritising user stories). This is what Fred Brooks referred to in No Silver Bullet as 'accidental complexity' – complexity in the activities around writing software, rather than in the domain of the business problem you're supposed to be solving.

Is this process rigor, or rigor mortis?

We have forgotten the meaning of iteration, and are working in increments instead. We are expecting an 'iteration' to contain a number of fully-completed stories, which we can then forget about and move on to new features. The problem with this is that when we deliver those stories, the customer is not entirely satisfied, and we have to introduce new stories to refine the design. This in turn affects our velocity, because we find ourselves spending more and more of our time enhancing features that we thought were done.

It's not iteration if you only do it once.

Alastair Cockburn has suggested a 'three story' approach, where you make the iteration explicit by putting two additional cards behind each user story – one to fix the original story, and another to fix the second attempt. Jeff's only concern with this was that people might take it too literally, doind a seond rework even if the customer was already happy, or stopping after the third story when the feature clearly wasn't really done.

Along with YAGNI, Jeff suggested adding YAGRI (you ain't gonna release it). This was to remind us that it's OK to present non-production features to the customer, in the knowledge that they won't be released until they've been iterated into a deliverable state. To me that seems OK for shrink-wrapped software and other things where iterations aren't released, but I wonder how it applies to web applications, where it is becoming the norm to release small changes frequently (perhaps multiple times per iteration). In that case, we're used to making the assumption that everything is always releasable. Maybe the answer is to separate new, 'beta-quality' features from the main body of functionality.

Three strategies for coping with the inevitable uncertainties in building a release were described:

Follow the Money

Prioritise the business goals. By focussing on the things with the highest value to the business you restrict the user constituencies which you should target first, and hence identify the highest priority user stories.

Don't Choose the Solution Too Early

Focus on the user goals rather than the perceived requirements on the application, and consider and discuss alternative ways of meeting those goals. It may be that a simpler or cheaper implementation than the immediately-obvious one may be sufficient.

Build up Quality in Iterations

Sometimes the customer really does need everything to be implemented before the product can be released. In this situation, you can start by implementing stories to a very basic, almost prototype, level. You then build up the quality iteratively, using feedback from the rough-cut implementations to guide you. One way of doing this is to grade each story (eg A+ down to F, plus I for incomplete), then have a 'passing grade' at which the feature is considered fit for release.

One consequence of actually iterating, rather than using 'iteration' as a label for an incrementl chunk of functionality, is that there's a lot more work for the customer (or product owner, or whatever you want to call them). They need to work frequently with the developers to evaluate each version of a story's implementation and describe the changes necessary to make it fit their needs.

The summary of the talk was to question common practice and to trust your instincts. As an aside, that leads into a rant I keep meaning to get round to, on the subject of people who say that a particular behaviour "isn't agile", because it doesn't fit the letter of their particular methodology of choice. Watch this space.

Technorati Tags: xpday, iteration, agile

Background

A while ago I cobbled together some code to drive Selenium from Exactor (which was the acceptance testing framework we were using at the time). That project was offshored shortly afterwards, and I'm pretty sure Selenium never actually got integrated into the build (sneaking a peek at the continuous integration server reveals that even the pre-Selenium acceptance test build hasn't run successfully for months), but I was convinced of the value of Selenium for testing non-trivial web applications.

On my next project (the Web21C SDK portal) we used Selenium on Rails heavily for automated acceptance testing. This worked well, although the fact that the tests are deployed with the application limits what you can do – you can't interrogate the database after a test, for example, or dynamically set up stubs for systems you interface with (unless you also deploy the stubs as part of the application, and drive them from the browser).

With Mojo there's an additional complication: as well as being accessible through a browser, most functionality is also available as a RESTful API, using digest-based authentication. To keep our acceptance tests in one place, we wanted to be able to drive a browser using Selenium, alongside other tests which talked to the server directly. The obvious answer was to use Selenium Remote Control, and I also liked the look of RBehave, which has now been incorporated into RSpec (but not released yet).

The RSpec story runner

The story runner in the upcoming release of RSpec is an evolution of Dan North's RBehave, which in turn is based on JBehave (from the same author). In the past couple of weeks, David Chelimsky has done some excellent work on extracting the text of stories from the code, leading to the plain text story runner.

Read on for the gory details of my solution to running Selenium tests from the story runner.

Putting it all together

Here's what I did to get Selenium working from the story runner (you need to check out RSpec trunk – I'm currently using r2791 of rspec and r2814 or rspec_on_rails).

I'm sure when the next release of RSpec comes out a lot of this will be unnecessary, and there will probably be standard places for the stories and ways of running them, but for now I've just hacked something together that's simple and works. Apologies for any pieces of string or duct tape that are still showing!

The files described below are laid out as follows, relative to the root of the Rails project:

+-- lib
| +-- rspec/
| +-- rspec_on_rails/
| +-- selenium-ruby-client-driver/
| | +-- selenium.rb
| +-- selenium-server/
| | +-- selenium-server.jar
| +-- tasks/
|   +-- acceptance.rake
+-- stories/
| +-- all.rb
| +-- helper.rb
| +-- steps/
| | +-- login.rb
| | +-- selenium.rb
| +-- stories/
| | +-- login.rb
| | +-- login.txt

Here's an example story:

stories/stories/login.txt

Each 'Given', 'When' and 'Then' corresponds to a step (an 'And' is just syntactic sugar for a repeat of the same type of step). The steps are defined in separate files, which can be included as required (see below). This story uses two sets of steps:

stories/steps/login.rb

stories/steps/selenium.rb

Note the use of placeholder parameters, which are parsed out of the lines in the story. The Selenium steps use a global instance of Selenium::SeleniumDriver, which is created in the helper (again, see below). I originally created this in a "Given a browser" step, but it seemed simpler to just create it up front, before running the stories.

stories/stories/login.rb

This is just a wrapper for the textual story, which specifies which steps are required.

stories/all.rb

Another simple wrapper, this time to run all stories.

stories/helper.rb

Warning: This file is where all the ugly code's hiding!

Most of this code doesn't really belong 'loose' in a helper file, but I didn't want to expend too much effort on getting everything working tidily with a pre-release version of RSpec.Hopefully there are enough comments to give a vague idea what's going on, but here are some more notes:

• Lines 5–10: There still seem to be some bugs in trunk RSpec which cause existing specs to fail (or possibly bugs in our specs that don't show up in 1.0.8). I've installed the trunk versions of the plugins under lib, and here I pull the real plugins out of the load path and insert the trunk versions instead.
• Lines 17–21: Create a selenium driver, and open a browser session.
• Lines 23–28: By default, RSpec adds an ActiveRecordSafetyListener to the story runner. This rolls back database changes between scenarios, which is great if your calling your code directly, but obviously means that if you write to the database, the server that Selenium's talking to can't see them. There's probably a cleaner way of disabling it.
• Lines 30–37: A convenience method to remove some code from the story wrappers. Basically just replaces '.rb' with '.txt' and runs that file with the specified steps. However, I can't find a clean way of finding the file name of the code that called a method, so I'm pulling it out of the call stack. Ugh.
• Lines 39–56: The method names in the ruby interface to Selenium RC are rather java-like (eg get_title and is_text_present). I've used a bit of method_missing hackery to wrap these methods with more ruby-like names (title and has_text_present? for those specific cases), which allows us to use rspec matchers like $selenium.title.should == foo and $selenium.should have_title bar.

lib/tasks/acceptance.rake

Finally, I've created some rake tasks to wrap everything up, and also to start and stop the selenium server.

Still to do

There's plenty of scope for improvement here. For a start, I haven't figured out how to generate a report for the test run, apart from the plain text output from the console (which is only usable if you start the selenium server in a separate shell). Also, I can't see an obvious way of returning a success/fail status at the end of the run, which makes it difficult to use in a continuous integration process.

The story runner looks like a very promising tool though, and I can't wait for the stable release.

Technorati Tags: bdd, rspec, rbehave, selenium, acceptance testing

In other words, agile developers do just as much designing as those still following waterfall methods, but we do it at the same time as writing the code.

Reflecting afterwards, I realised that I've been phrasing the message the wrong way. As anyone who really 'gets' test-driven (and particularly behaviour-driven) development knows, it's not really about the testing, but about the design process. A waterfall 'designer' starts from an understanding of the problem and builds up some kind of model for a solution, which they then pass on to the implementors An agile developer does exactly the same, but the language they use for the model happens to be executable source code rather than documents or UML. This choice of modeling language has the rather large benefit of being testable, and once your design is finished, you're done. No need for someone to try to interpret and implement it. No scope for misunderstandings between designer and developer. No danger that the design is incomplete, or that part of it turns out to be unworkable.

It's not that we start writing code and design as we go along.

We are always designing – we just write the code as we go along.

Technorati Tags: agile, tdd, bdd