Quite often I'll make small changes to a project's code which I'm 99% sure won't break the build. Of course thanks to Murphy's Law that probability falls to about 10% if I decide to risk it push the changes upstream without running the full build locally.

Now even if the build only takes a few minutes to run, that's a few minutes which I can't spend doing anything else with the code, because if I save a file I can't be sure whether the original or changed version will be used in the tests. What I'd like to be able to do is to run the build in the background, with changes automatically pushed upstream if everything's green, or an alert shown if either the build fails or the push is rejected (because someone else has pushed a change while the build was running).

Fortunately with a distributed version control system like git, this is fairly easy.

Firstly, clone the local working repository and set up a remote pointing to the clone:

Now set up the cloned copy:

Do anything that needs doing to allow the cloned project to build (if you have a test database, create a separate instance to avoid tests interacting with your main workspace), and verify that the build runs.

Now add the following to .git/hooks/post-receive:

This will reset your checked-out working copy to the pushed master, then run .git/hooks/post-reset in the background. Create that file with the following content:

Make sure both these hooks are executable:

Now, back in your normal working repository you should just be able to type git push staging (add --force if you've rebased your normal master and get an error about the push not being a fast-forward), and the project will build in the staging repo. If everything works, it'll push upstream and you'll get a Growl notification (assuming you have Growl and growlnotify, or the equivalents for your OS, installed). If it fails, you'll also get a Growl notification. I've made the alerts sticky, so I don't miss them – if you don't want that, just remove the -s flag. The build errors will be in the build_output file.

This isn't entirely trivial to do using git-svn (which is really intended for use with subversion acting as the master), but I found a few sets of instructions on the web. The simplest one was this Nano Rails blog post, which is what the steps below are based on.

The end result should be a subversion trunk which is a mirror of the git repository's master branch. This is only intended to be a one-way mirror – I wouldn't recommend also trying to commit into subversion and merge those commits back upstream into git.

You need to have git-svn installed (this comes with the default installation, or you can use the +svn option when installing via MacPorts).

Create subversion repository

Create the subversion repository in the usual way, using svnadmin.

Once you've got an empty repository to point to (we'll imagine it's at http://svn.example.com/foo), you also need to commit an initial version (I also created a trunk directory in this step, in case we later decide to mirror branches too):

svn co http://svn.example.com/foo
cd myproj
svn mkdir trunk
svn commit -m'Created trunk directory'

Once this is done, you can throw away the directory you checked out of subversion.

Set up the subversion remote

This step, and subsequent ones, need to be performed on whichever git repository you want to mirror from.

In our case, we have a central repository running on a local installation of Gitorious. This is a bare repository, which makes things a little tricker, as git-svn requires a working copy. To get round this, we create a clone, which we'll use as an intermediate step in the mirroring process. If you're not mirroring a bare repository, you can omit this step.

The repositories we want to mirror are in ~git/repositories, and we've created a directory ~git/repositories/svn-mirror where we'll put the clones. For this example, we'll use a repository called foo/mainline.git.

Create the clone:

git clone ~git/repositories/foo/mainline.git ~git/repositories/svn-mirror/foo
cd ~git/repositories/svn-mirror/foo

Now add the following to .git/config (with the correct svn URI, of course):

[svn-remote "svn"]
	url = http://svn.example.com/foo/trunk
	fetch = :refs/remotes/git-svn

Now do an initial fetch of the empty subversion remote, and check it out as a new git branch (called svn):

git svn fetch svn
git checkout -b svn git-svn

You can now merge in all your commits from master, and push them to subversion. You'll probably want to go and make a coffee or something while the dcommit runs – if you haven't used subversion for a while you've probably forgotten just how much slower it is than git.

git merge master
git svn dcommit

To allow pushing to svn from master, rebase master to the svn branch (which can then be deleted):

git checkout master
git rebase svn
git branch -d svn

At this point you should be able to manually update subversion at any time by running git svn dcommit from the master branch.

Automating subversion updates

In theory it should be possible to set up post-receive hooks to push from the gitorious repository to the clone and from there to subversion, but I decided to separate the two by just periodically polling for changes, as we don't really care about subversion being right up-to-the-minute. To poll hourly, add something like this to the git user's crontab:

0 * * * * (cd /usr/local/git/repositories/svn-mirror/foo;/usr/local/bin/git pull origin master;/usr/local/bin/git svn dcommit) >>/usr/local/git/gitorious/log/svn-mirror.log 2>&1

Technorati Tags: svn, subversion, git, mirror

When you rebase a branch, you are changing its history in a way that will cause problems for anyone who already has a copy of the branch in their repository and tries to pull updates from you. You should understand the implications of using git rebase on a repository that you share.

Do you get annoyed by seeing things like this in your git history?

commit a0b46a7c57e37f5dc43373ba9167ad2da32c1ec5
Merge: c2d8046... 73e0e15...
Author: Fred Bloggs 
Date:   Tue Jun 17 17:30:49 2008 +0100

    Merge branch 'master' into new_feature

commit c2d8046c038d47940944e5b343d281b1d0c4d2b3
Author: Fred Bloggs 
Date:   Tue Jun 17 17:30:43 2008 +0100

    Added cool new feature

This happens when you use merge instead of rebase to keep a development branch up-to-date with master. Let's watch what happens in each case.

The wrong way (merge)

You're working on a cool new feature in your new_feature branch. You've committed two changes, and in the meantime there have been three other changes on master. Here's the branch visualisation from gitk --all:

You use merge to pull in those other three changes to your branch:

$ git merge master
Merge made by recursive.
 file_1 |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

Now what this has actually done is added the changes from master as a new commit in new_feature. You can see this in the history:

myproj(new_feature) $ git log -1
commit cbc97a909641d3c325c6023a2459e556e62182e6
Merge: 024dc64... 0a71c4c...
Author: Kerry Buckley 
Date:   Wed Jun 18 18:52:48 2008 +0100

    Merge branch 'master' into new_feature

And also in the gitk graph:

Now let's say your new feature is all finished, so you merge it into master:

myproj(new_feature) $ git checkout master
Switched to branch "master"
myproj(master) $ git merge new_feature
Updating 0a71c4c..cbc97a9
Fast forward
 file_3 |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

Let's see what that's done to the history:

myproj(master) $ git log
commit cbc97a909641d3c325c6023a2459e556e62182e6
Merge: 024dc64... 0a71c4c...
Author: Kerry Buckley 
Date:   Wed Jun 18 18:52:48 2008 +0100

    Merge branch 'master' into new_feature

commit 0a71c4c90aee5eeb60d15f199c4f8151756a8ae8
Author: Kerry Buckley 
Date:   Wed Jun 18 18:48:19 2008 +0100

    Third change in master

commit 9970c0b72e7741804fc07bba50450b3d512e5572
Author: Kerry Buckley 
Date:   Wed Jun 18 18:48:10 2008 +0100

    Second change in master

commit c44af4dee449082adf6741540d2f9e70968cf41e
Author: Kerry Buckley 
Date:   Wed Jun 18 18:46:33 2008 +0100

    First change in master

commit 024dc64022932a5a7b56c4fd7c7cf4a59d72e825
Author: Kerry Buckley 
Date:   Wed Jun 18 18:45:15 2008 +0100

    New feature finished

commit eb4f05fb8ef8b93cf639b1e06528ef075f19f323
Author: Kerry Buckley 
Date:   Wed Jun 18 18:45:01 2008 +0100

    New feature partly done

commit b4ffa1d35f808cc38e5f74fb2592224dd6f0e027
Author: Kerry Buckley 
Date:   Wed Jun 18 18:44:02 2008 +0100

    Last commit before creating branch

Or graphically:

The problem here is that the merge has applied all of the commits which were on new_feature but not on master, including the merge commit. That's just ugly.

The right way (rebase)

Now, from exactly the same point, we'll use rebase instead:

myproj(new_feature) $ git rebase master
First, rewinding head to replay your work on top of it...
HEAD is now at a644c41 Third change in master
Applying New feature partly done
Applying New feature finished

Basically, as it says, this has rewound all your changes since the new_feature branch diverged from master, moved the branch point up to the tip of master, then replayed your changes on top. The effect of this is as if you had created the branch from the current latest master, so no separate merge commit is required. Again, gitk --all can confirm this visually:

Now merge the changes into master exactly as before:

myproj(new_feature) $ git checkout master
Switched to branch "master"
myproj(master) $ git merge new_feature
Updating a644c41..d7d2233
Fast forward
 file_3 |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

No merge commit in the history this time, and a nice simple graph:

So there you have it. No excuse for polluting your history with merges any more!

Technorati Tags: git, merge, rebase, branch