Kerry Buckley

Over the years I've tried a number of approaches for managing gem dependencies in a Rails project. Here's a quick round-up of what I've tried, and the pros and cons of each.

Just use what's on the system

This is probably most people's default approach when first starting with Rails. Just sudo gem install whatever you need, require the appropriate gems (either in environment.rb or in the class that uses them), and you're away.

This mostly works OK for small projects where you're the only developer, but you still need to make sure the right gems are installed on the machine you're deploying the application to.

Worse, though, is what happens when you come back to the project after a while, various gems have been updated, and things mysteriously don't work any more. Not only do you have to mess around getting the code to work with the latest gem versions, but you probably don't even know exactly which versions it used to work with.

Freeze (unpack) gems

I think I first came across this technique in Err the Blog's Vendor Everything post. The idea is to install copies of all your gems into the project's vendor/gems directory, meaning that wherever the code is running, you can guarantee that it has the correct versions of all its dependencies.

This got much easier in Rails 2.1, which allowed you to specify all your gems using config.gem lines in environment.rb (you can also put gems only needed in specific environments in the appropriate file, eg you might only want to list things like rspec and cucumber in config/enviroments/test.rb). You can then run sudo rake gems:install to install any gems that aren't on your system, and rake gems:unpack to freeze them into vendor/rails, and be sure that wherever you check out or deploy the code, you'll be running the same versions of the gems. There's even a gems:build task to deal with gems that have native code (but more on that later).

Subsequent versions of Rails have improved on the original rake tasks – dependencies are now handled much better, for example – but there are still a few problems. The main one is the handling of gems that are required by rake tasks in your project, rather than just from your application code.

When you call a rake task in your Rails project, this is more-or-less what happens (I may have got some of the details slightly wrong):

1. The top-level Rakefile is loaded.
2. This in turn requires config/boot.rb, but not config/environment.rb.
3. It then requires some standard rake stuff, and finally tasks/rails (which is part of Rails – specifically railties). This finds and requires all the .rake files in your plugins and your project's lib/rake directory.

The problems start when you have a task depends on the rails environment task, and also requires a gem which is listed in environment.rb. Because the gem-loading magic only happens when the environment is loaded, the rake task will be blissfully unaware of your frozen gems, and will load them from the system instead.

If the system gem is newer than the frozen one, you get errors like this:

can't activate foo (= 1.2.3, runtime) for [], already activated foo-1.2.4 for []

If you work on two projects that use different versions of a gem like this, you end up having to uninstall and reinstall them as you switch from one to the other, which gets tedious fairly quickly.

Specify gems, but don't freeze

You can get round the wrong-version problem to some extent by specifying version numbers in environment.yml as '>=x.z.y' (or by not specifying them at all). If you're doing that, though, there's not really much benefit in unpacking the gems, and you may as well just use rake gems:install to make sure they're on the system. Of course the downside of this approach is that you can't be sure that everyone's running the exact same versions of the gems. Worse still, you can't be sure that what's on your production box matches your development and test environments.

GemInstaller

GemInstaller solves most of the problems with the built-in Rails gem management by running as a preinitializer, meaning it gets loaded before the other boot.rb gubbins.

GemInstaller uses the gems installed on the system rather than freezing them into the project, but because it gets to run first it ensures that the correct versions are used, even if there are newer versions installed. By default it checks your project's gem list and installs anything that's missing every time it runs (which is whenever you start a server, run the console, execute a rake task etc). You create a YAML file listing the gems you need (dependencies are handled automatically), and other options such as an HTTP proxy if necessary.

Of course on Unix-like systems, which is most of them (although I hear there are still people developing Rails projects on Windows), gems are generally installed as root. GemInstaller can get round this in two ways – either by setting the --sudo option and setting a rule in /etc/sudoers to allow the appropriate user(s) to run the gem commands as root without having to provide a password, or by using the built-in gem behaviour that falls back to installing in ~/.gem.

Personally I like to keep all my gems in one place, accessible to any user, so I went for the sudo approach. The only problem with this is that it uses sudo for all gem commands, rather than just install or update, which means it runs a sudo gem list every time your app starts up. Depending on the way you have Apache and Passenger set up this may mean granting sudo access to what should be a low-privileged user.

I ended up disabling the automatic updating of gems, and just warning when they're missing instead. In fact later versions of GemInstaller don't try to handle the update automatically anyway.

I created a separate script to do the update, which can be run manually, on a post-merge git hook, or as part of the Capistrano deployment task.

Because GemInstaller needs to go out to the network to fetch any new or updated gems, things get a bit more painful (as always) if you are unfortunate enough to be stuck behind a corporate HTTP proxy. Actually it's easy enough to configure if you're always behind a proxy, but it gets slightly trickier if your web access is sometimes proxied and sometimes direct. Nothing that can't be solved of course.

Unfortunately you can still end up with version conflicts if a gem is required by one you have specified, then you explicitly require an older version, but these can usually be resolved by shuffling the order of the gems in geminstaller.yml.

Bundler

Bundler is the newest kid on the gem management block, and looks to have solved pretty much all the problems faced by the other approaches. It's based on the gem management approach from Merb, and can be used in any Ruby project (not just Rails).

Bundler works by unpacking gems into the project (I recommend using a directory other than the default vendor/gems to avoid confusing Rails – this can be configured by setting bundle_path and bin_path in the Gemfile), but the intention is that you only commit the .gem files in the cache directory to source control. Gems are then installed locally within the project, including any platform-specific native code as well as the commands in bin.

Because Bundler resolves all dependencies up-front, you only need to specify the gems you're using explicitly, and let it handle the rest, which hopefully means an end to version conflicts at last.

Here's an example Gemfile:

Note the two additional sources (rubyforge.org is configured by default), the path overrides, and the last line, which removes the system gems from the paths, avoiding any potential confusion.

I've put this in config/preinitializer.rb to update from the cached gems on startup (this doesn't hit the network):

To avoid any startup delays after an upgrade, I also call system 'gem bundle --cached' from the after_update_code hook in the capfile.

Finally, to make sure only the .gem files are checked in, add these lines to .gitignore (you'll still need to explicitly git add the bundled_gems/cache directory):

vendor/bundled_gems
!vendor/bundled_gems/cache

[Update 3 November] Yehuda Katz just posted an article all about Bundler, including features coming in the imminent 0.7 release.

Technorati Tags: ruby, rails, gems, rubygems, geminstaller, bundler

Our team been using git at work for the past couple of years, but there's now a corporate push to keep everything in a centrally-managed subversion repository. We lost the battle to get corporate approval for git (apparently we're happy to employ people to write our code that we wouldn't trust to be able to use a DVCS), but at least we have agreement that we can continue using git as long as we mirror the code into subversion.

This isn't entirely trivial to do using git-svn (which is really intended for use with subversion acting as the master), but I found a few sets of instructions on the web. The simplest one was this Nano Rails blog post, which is what the steps below are based on.

The end result should be a subversion trunk which is a mirror of the git repository's master branch. This is only intended to be a one-way mirror – I wouldn't recommend also trying to commit into subversion and merge those commits back upstream into git.

You need to have git-svn installed (this comes with the default installation, or you can use the +svn option when installing via MacPorts).

Create subversion repository

Create the subversion repository in the usual way, using svnadmin.

Once you've got an empty repository to point to (we'll imagine it's at http://svn.example.com/foo), you also need to commit an initial version (I also created a trunk directory in this step, in case we later decide to mirror branches too):

svn co http://svn.example.com/foo
cd myproj
svn mkdir trunk
svn commit -m'Created trunk directory'

Once this is done, you can throw away the directory you checked out of subversion.

Set up the subversion remote

This step, and subsequent ones, need to be performed on whichever git repository you want to mirror from.

In our case, we have a central repository running on a local installation of Gitorious. This is a bare repository, which makes things a little tricker, as git-svn requires a working copy. To get round this, we create a clone, which we'll use as an intermediate step in the mirroring process. If you're not mirroring a bare repository, you can omit this step.

The repositories we want to mirror are in ~git/repositories, and we've created a directory ~git/repositories/svn-mirror where we'll put the clones. For this example, we'll use a repository called foo/mainline.git.

Create the clone:

git clone ~git/repositories/foo/mainline.git ~git/repositories/svn-mirror/foo
cd ~git/repositories/svn-mirror/foo

Now add the following to .git/config (with the correct svn URI, of course):

[svn-remote "svn"]
	url = http://svn.example.com/foo/trunk
	fetch = :refs/remotes/git-svn

Now do an initial fetch of the empty subversion remote, and check it out as a new git branch (called svn):

git svn fetch svn
git checkout -b svn git-svn

You can now merge in all your commits from master, and push them to subversion. You'll probably want to go and make a coffee or something while the dcommit runs – if you haven't used subversion for a while you've probably forgotten just how much slower it is than git.

git merge master
git svn dcommit

To allow pushing to svn from master, rebase master to the svn branch (which can then be deleted):

git checkout master
git rebase svn
git branch -d svn

At this point you should be able to manually update subversion at any time by running git svn dcommit from the master branch.

Automating subversion updates

In theory it should be possible to set up post-receive hooks to push from the gitorious repository to the clone and from there to subversion, but I decided to separate the two by just periodically polling for changes, as we don't really care about subversion being right up-to-the-minute. To poll hourly, add something like this to the git user's crontab:

0 * * * * (cd /usr/local/git/repositories/svn-mirror/foo;/usr/local/bin/git pull origin master;/usr/local/bin/git svn dcommit) >>/usr/local/git/gitorious/log/svn-mirror.log 2>&1

Technorati Tags: svn, subversion, git, mirror

I tend to agree that comments are, in most cases, evil (or at least mildly malevolent), but I did come across one of the exceptions to the rule today.

While doing a bit of drive-by refactoring while fixing a bug, I reflexively changed this line:

to this:

Then reading the comment above the line, expecting to delete it, it all came flooding back:

Now you could maybe argue that this unexpected behaviour is because httparty uses just a little too much of that old method missing proxy magic (which of course isn't really magic at all), but that's not the point of this post.

In the end I pulled it out into a private method to make it clearer what was going on, but decided to leave the comment in.

Naresh Jain recently posted a refactoring teaser. The original code was in Java, but I thought I'd have a go at refactoring it in Ruby instead. I deliberately didn't look at Naresh's solution beforehand, so mine goes in a rather different direction.

My code's here (with the specs in the same file for simplicity), and you can step through the history to see the state at each step of the refactoring. The links in the text below take you to the relevant version of the file, and the Δs next to them link to the diffs.

Firstly, I converted the code to Ruby, and made sure the tests (which I converted to RSpec) still passed. It's a fairly straight conversion, although I made a couple of changes while I was at it – mainly turning it into a module which I mixed into String. I changed the name of the method to phrases, to avoid conflicting with the built-in String#split. The regular expression to split on is much simpler too, because Ruby didn't understand the original, and I had no idea what it was supposed to do anyway.

Here's my initial Ruby version:

I didn't change the specs at all during the refactoring (because I didn't change the API or add any new public methods or classes), and made sure they all passed at each step.

The first thing Δ I changed was to simplify that big iterator in phrases that loops through the list of phrases, formatting the output string. Basically all this does is to put each phrase in quotes, then stitch them all together separated by a comma and a space. The first of those tasks is a simple map, and the second is a join. The whole method collapses down to this (ruby methods automatically return the result of their last statement):

Next Δ I remembered that by default String#split splits at whitespace anyway, so I did away with the regular expression. Then Δ I renamed the strings variable to words to make its purpose a little clearer, leaving the phrases method looking like this:

The section of single_double_triple_words that extracted the single words seemed redundant, as we already had that list – the original words. I removed it Δ, and initialised all_strings to the word list instead (not forgetting to rename single_double_triple_words to match its new behaviour):

That has_enough_words method seemed a bit odd – particularly the way it was only called once, rather than after extracting each set of phrases. I decided it was probably a premature and incomplete attempt at optimisation, and removed it Δ for now.

My next target was the duplication in the blocks that calculate double- and triple-word phrases. First Δ I extracted them into separate methods:

I decided that the num_words variable in the two new methods wasn't really necessary (it was only used once, and I think strings.size expresses intent perfectly clearly), so I inlined it Δ (and the same in triple_words):

Most of the code in double_words and triple_words was obviously very similar, so I created Δ a general extract_phrases method, and called it from both. The new method uses the start position and length version of Array#[] to extract the appropriate number of words, then Array#join to string them together separated by spaces:

At this point double_words and triple_words have become just dumb wrappers around extract_phrases, so I removed them Δ and just called extract_phrases directly:

Rather than hardcoding the calls for two and three words, I changed it Δ to use a loop:

I decided this was the point to put the optimisation back Δ and stop looking for phrases once we had enough:

I decided that number_of_words wasn't particularly clear, so changed it Δ to phrase_length, then Δ made the iterator in extract_phrases more ruby-like by using map:

I then noticed that I hadn't been consistent in changing strings to words, so fixed that Δ.

Lastly Δ, I decided that even though it was only one line, the code that formats the output deserved pulling out into a method.

Here's my final version of the code:

Just a couple of things that have caused a bit of head-scratching lately when writing RSpec specs using the built-in mocking framework.

Catching StandardError

Watch out if the code you're testing catches StandardError (of course you're not catching Exception, right?). Try this:

Nothing particularly exciting there. Let's run it and check that it passes:

$ spec foo.rb
.

Finished in 0.001862 seconds

1 example, 0 failures

However, what if we change the example to test the opposite behaviour?

$ spec foo.rb
.

Finished in 0.001865 seconds

1 example, 0 failures

Wait, surely they can't both pass? Let's take out the rescue and see what's going on:

$ spec foo.rb
F

1)
Spec::Mocks::MockExpectationError in 'Calling a method that catches StandardError does NOT call Bar.bar'
 expected :bar with (no args) 0 times, but received it once
./foo.rb:6:in `foo'
./foo.rb:18:

Finished in 0.002276 seconds

1 example, 1 failure

That's more like it.

Of course, what's really happening here is that Spec::Mocks::MockExpectationError is a subclass of StandardError, so is being caught and silently discarded by our method under test.

If you're doing TDD properly, this won't result in a useless test (at least not immediately), but it might cause you to spend a while trying to figure out how to get a failing test before you add the call to Foo.foo (assuming the method with the rescue already existed). Generally you can solve the problem by making the code a bit more selective about which exception class(es) it catches, but I wonder whether RSpec exceptions are special cases which ought to directly extend Exception.

Checking receive counts on previously-stubbed methods

It's quite common to stub a method on a collaborator in a before block, then check the details of the call to the method in a specific example. This doesn't work quite as you would expect if for some reason you want to check that the method is only called a specific number of times:

$ spec foo.rb
.

Finished in 0.001867 seconds

1 example, 0 failures

I think what's happening here is that the mock object would normally receive an unexpected call, causing the expected :bar with (any args) once, but received it twice error that you'd expect. Unfortunately the second call to the method is handled by the stub, so never triggers the error.

You can fix it, but it's messy:

$ spec foo.rb
F

1)
Spec::Mocks::MockExpectationError in 'Checking call counts for a stubbed method only calls a method once'
 expected :bar with (any args) once, but received it twice
./foo.rb:23:

Finished in 0.002542 seconds

1 example, 1 failure

Does anyone know a better way?

The full example code is in this gist.

Just came across an interesting error message from rspec. I had a spec that looked like this:

Obviously it failed, as I hadn't written the code yet, but there was more in the error message than I expected:

..........F

1)
RuntimeError in 'Blog should not mass-assign 'confirmed''
'should_not be  true' not only FAILED,
it is a bit confusing.
It might be more clearly expressed in the positive?
.../spec/models/blog_spec.rb:20:

Finished in 0.06192 seconds

11 examples, 1 failure

In fact, rewriting this as should be_false wouldn't work, as the expected value is nil. I took the hint though, and rewrote it as should be_nil.

Brad Wilson argues the case for renaming TDD to 'Design by Example' (or DbE), to emphasise the fact that TDD (and BDD, which is really just TDD done well) are about design rather than testing.

I agree with the intention, but I'm not sure I agree. I'm happy to lose the 'test' &ndash it's losing the 'development' that bothers me. I've encountered far too many 'designers' who haven't written a line of code in years, and I worry that they'd read 'design by example' as simply starting their design document with a few test cases. I've already seen TDD redefined as test-driven design and interpreted that way.

Of course for people who already grok TDD this isn't an issue, but it's not them that a name change is aimed at. Test-driven development might not be the perfect name, but at least there are already plenty of people who do understand what it really means to correct any misapprehension. The term BDD (which I use myself from time to time) has already started sowing confusion, without adding yet another.

On the other hand, anything that gets people discussing how to get the most from the practice has to be a good thing.

World's-most-famous-twitterer Stephen Fry has a system for handling follow requests: you tweet using the #followmestephen hashtag, and he wades diligently through them, manually following people.

This seems an odd sort of thing to do – most people choose whom to follow based on whether they know them or like what they say, rather than on request – but I suppose when you have over a quarter of a million followers things work a little differently. It also creates lots of work , and looks like an ideal candidate for automation.

I thought I'd have a quick play with the Twitter API this morning (no doubt I'm not the only one), and cobbled together the script below, which you can also download as follow_me_stephen.rb (although if you're not Mr Fry I'm not sure why you would want to). Save the file, and run using ruby follow_me_stephen.rb.

I wanted to avoid having too many dependencies, so I didn't use the twitter gem, or the excellent httparty, but I was too lazy to figure out all the XPaths to handle the Atom version of the API. This means you need to have the JSON gem installed, which is as simple as sudo gem install json (omit the sudo on Windows).

The script's pretty dumb, in that it grabs the whole set of search results every time, and blindly requests to follow everyone, regardless of whether you're already following them.

Last Thursday I attended the SC2009 conference at the BBC Media Centre in London. Here are a few notes (assuming I get round to finishing the post – I see I still have an unfinished draft about a session from XPDay2007).

MappingPersonal Practices (Ade Oshineye)

This was a simple exercise where everyone spent five minutes drawing a mind-map of the software development practices they find most important to them, then spent 90 seconds each listing and describing them. The intention was to then go round again and say which practices you'd heard others mention that you wanted to try, but time ran out.

Here's my selection, converted to Graphviz to save you the pain of trying to decipher my handwriting:

Of the practices mentioned by others, the ones I noted down that I ought to try are speaking at conferences, learning other languages (I haven't played with a new one for a while), getting feedback from others in the team and reading classic journal papers (conveniently Michael Feathers has just posted a list of recommended ones).

Ruby Kata & Sparring (Micah Martin)

Micah (who had come all the way from Chicago just for this conference!) described the coding kata, linking it to the martial arts practice from which the idea was taken. Trivia fact: the name of Micah's company, 8th Light, is the literal translation of Hakkoryu, the school of Jujitsu that Micah followed.

He mentioned two different views of katas: Dave Thomas's suggestion to practice solving the same problem many times and reflect on what you are learning, and Uncle Bob Martin's approach where you watch a master solve the problem and mimic their process (the latter seems closer to the practice of kata in martial arts). Micah suggested that both of these are missing an important aspect of martial art kata: performance.

Performing a code kata in front of other craftsman gives you a purpose for practicing it. You can receive feedback, learn and address your weaknesses and measure your skill against others, and it shows respect for the craft.

Micah performed a code kata himself, implementing Langston's Ant in Ruby and asking for a score out of ten at the end (most people gave seven or eight). Finally he mentioned sparring as another technique borrowed from martial arts, and talked about the Battleship tournament which he set up. Apparently this will be repeated soon, using Hangman as the problem to solve.

If you want to see more, there's a video of Micah making the same presentation at RubyConf 2008, and also a screencast of the Langston's Ant kata.

Three Paradigms: Taking An Extreme Position on Code Style in a Safe Environment (Keith Braithwaite)

The purpose of this session was to take a piece of contrived "enterprisey" code, make three separate refactorings on it and see what effect they had on the ease of making changes to the code. The refactorings were basically removing conditional logic, removing getters and setters, and making certain classes unmodifiable.

The example code was available in Java and C# (the so-called country-and-western languages – "We got both kinds…"). Unfortunately, like many people there my Java skills were a bit rusty (and my C# skills non-existent), so by the time I'd grabbed the code off one of the memory sticks that were floating round, set up a project in Eclipse, realised that I didn't have junit.jar and found someone to copy it from, I didn't really have enough time to complete any of the refactorings. I also spent a while puzzling over what the purpose of the UserDTO class was, as it seemed like it would have been easier to just pass User objects around. Turns out that the point was to show that there was no point (if you see what I mean). Fortunately I seem to have avoided being exposed to the kind of pattern-obsessed enterprise code where these useless nojos are considered good practice. It doesn't surprise me though – to quote Keith, "Curly bracket languages make you stupid".

I don't think anyone got as far as finishing all the refactorings, or making behavioural changes to the resulting code, but there was some interesting discussion about ways to achieve the various refactorings (including one person who admitted to removing getters and setters by making the fields public!) It's a shame there wasn't a bit more time available.

Responsibility-driven Design with Mock Objects (Willem van den Ende & Marc Evers)

This was a quick introduction to the use of mock objects as a design tool (refreshingly a show of hands indicated that everyone present practiced TDD, and most used mocks), followed by a randori-style session (in Ruby, by popular vote) designing some classes for a simple adventure game.

As with the previous session, time constraints meant that we didn't progress that far with the code, but there was a lot of discussion around various design decisions, which of course is the really valuable part of this kind of exercise. A lot of this was kicked of by Steve Freeman (co-inventor of mock objects, and co-author of the excellent Mock Roles, not Objects, where the proper use of mocks is explained), who piped up when someone suggested moving when there was some debate about the correct name for a method. From what I can remember, his comment was something along the lines of "No! Getting the right names for things is the whole point – if you aren't going to do that, you might as well not bother!" I was reminded of the quote from Phil Karlton: "There are only two hard problems in Computer Science: cache invalidation and naming things."

The main thing I took away from this session was some confirmation that the way I currently try to do TDD with mocks is fairly close to what seems to be considered good practice. Interestingly the number of people who agreed that the techniques shown were similar to how they currently used mocks was a fraction of the number that had earlier identified themselves as mock users, but unfortunately there wasn't time to find out how their approaches differed.

My Defining Moments (Steve Freeman)

This was similar in a way to Ade's, but instead of everyone talking for 90 seconds about their personal practices, a dozen or so volunteers talked for up to five minutes about a 'breakthrough experience' where they've learned something important, and how that breakthrough happened. Here are some of the ones I noted down (apologies if I've misrepresented anyone, but I only wrote down a few words for each and I may have forgotten the details):

Steve himself talked about when he was at DEC, where 'everything just worked'. He described the best sysadmins he'd ever seen, who would, unasked, perform 'random acts of helpfulness'. The main lesson he learnt from this was that despite the mediocrity found in most organisations, it was possible to be that good if you had the right people.

Micah described how some ways of working when he joined ObjectMentor hadn't felt quite right to him, and his realisation that in the end you can't impose one person's standards on another: we all have to develop our own which we're comfortable with.

Ade talked about his first project at ThoughtWorks. He described how his boss had shouted at him in a client meeting for describing himself as "just a developer". His main point though was about working with a team of developers from Dixons, whose sole Java experience was a two-week course from Dan North. Obviously the lack of experience wasn't ideal, but the fact that they'd learnt everything they knew from Dan meant that they only knew the right way to do things. For example if you suggested to them that something was too simple to need a test, they'd say "No, you have to start with a test!"

Willem van den Ende described an internship spent refactoring a hideous, bug-ridden, copy-and-pasted mudball of GUI code, which contained 40,000 lines of code and which no-one dared touch. In four months, by slowly removing duplication and cleaning up the code, he reduced it to a tenth of the size. He also found that he fixed most of the bugs without really trying, as a side-effect of removing duplication. The original code had taken two people three months to write.

Gojko Adzic talked about a project for the National Grid, where a graphical map view they'd added at the end of the project was by far the most popular feature of the software, despite the fact that the customer hadn't asked for it. He highlighted the discrepancy between this and the common agile view that says you should only work on features (user stories) that the customer has explicitly asked for. My take on this would be that it's part of the developers' job to propose new or alternative solutions to the customer, but generally you should still get their agreement.

Finally John Daniels talked about a 'lightbulb moment' when moving from ADA to Smalltalk, where he suddenly understood what OO was really about. He said it was 20 years before he saw another language with a development environment as good as the Smalltalk one (something I've heard from others too).

I didn't put my name down to talk, but for the record my defining moment would have actually have been a couple of events which have influenced my understanding and practice of TDD. From my initial one-test-per-method attempts at unit testing, reading about TestDox and RSpec introduced me to the idea of tests as specification. Then there were a couple of sessions from XPDay06, especially Are your tests really driving your development, which made me realise that my tests still weren't expressing intent as well as they could have done. Finally the aforementioned Mock Roles, not Objects showed me the right way to use mocks and stubs.

Test-driven Development of Asynchronous Systems (Nat Pryce)

I'd already read the notes from this presentation, which Nat originally gave at XPDay08, but seeing the talk in person made it much clearer. I don't think there's much I can add to the notes.

General observations

Firstly, it was great to spend a day with a hundred-odd people who obviously cared about their work (not just their career) as software developers. The other thing that struck me was that, despite this not being explicitly an Agile or XP conference, it seemed to be taken for granted that practices like TDD and refactoring were basic necessities, not optional extras.

More randomly, there were far more non-Apple laptops in evidence than you would see at a Ruby conference or a Barcamp, presumably because a lot of delegates work for large companies and brought their work laptop. That said, most of the ones I saw seemed to be running Linux rather than Windows, but I wasn't keeping count. More Java and .NET people too, although a fair proportion working in Ruby and other languages. On the phone front though, I don't think I've ever seen so many iPhones in the same room! It seemed like every other person had one (even Ade, who was also showing off his Android phone, a Christmas present from his employer).

finally, a huge thank you to Jason Gorman and everyone else involved in organising the day, and to the BBC for hosting it.

[Update 18/3/2009: added single-tunnel example]

This post is mainly for my own benefit, because every time I need to do this I find I've forgotten how and need to look it up again.

Say you want to reach dest, but have to tunnel through foo because you don't have direct access to port 22 on dest.

ssh -NL 65001:dest:22 foo &
ssh localhost -p 65001
Welcome to dest!
$

If you need to tunnel through multiple gateways to reach the machine you want to connect to, this is how to do it. Now let's say you have to jump through foo, bar and wibble to get to dest.

ssh -NL 65001:bar:22 foo &
ssh -NL 65002:wibble:22 localhost -p 65001 &
ssh -NL 65003:dest:22 localhost -p 65002 &
ssh localhost -p 65003
Welcome to dest!
$

Obviously you don't need to use ports starting with 65001, but can pick any convenient unused local ports.

You can use different usernames and SSH ports if necessary, eg if you have to connect to wibble as dave on port 222, that line becomes:

ssh -NL 65002:dave@wibble:222 localhost -p 65001

Technorati Tags: ssh, chain, tunnel