Kerry Buckley

What’s the simplest thing that could possibly go wrong?

GenServer: Elixir’s Swiss Army Knife

leave a comment

If you ask someone what the coolest thing about Elixir is, chances are they’ll start with pipelines (the |> operator even made it to the front cover of the PragProg book). But the next thing they mention will probably have something to do with OTP in general, and/or GenServer specifically, and these are arguably more powerful.

I’ve been using Elixir more and more lately, and have started to get my head round how to use GenServer (and friends) for a bunch of different things. I thought I’d write up some of those examples, in case it’s helpful to other Elixir learners (it takes a while to get your head round this different way of structuring code, and I’m definitely not there yet).

You can find the code used in the examples on GitHub, and the commit history should follow the stages in text.

Important caveat

I have no idea what I'm doing

As I said above, I’m quite new to all this, so take everything with a pinch of salt. There are probably better ways to solve all these problems, and better ways to structure GenServer applications. Also, in the interests of preventing an already far too long post getting longer, I’ve completely ignored things like testing, code organisation and so on.

Hello world

First of all, we’ll create a standard mix application:

$ mix new hello_world
* creating README.md
* creating .gitignore
* creating mix.exs
* creating config
* creating config/config.exs
* creating lib
* creating lib/hello_world.ex
* creating test
* creating test/test_helper.exs
* creating test/hello_world_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd hello_world
    mix test

Run "mix help" for more commands.

Mix gives us a skeleton top level module for the application – in this case HelloWorld. Let’s tweak that slightly to make a function that simply prints a message to the terminal:

lib/hello_world.ex:

defmodule HelloWorld do
  def hello do
    IO.puts "Hello World"
  end
end

Nothing particularly interesting there, but we’ll just check that it works in iex:

$ iex -S mix
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Compiling 1 file (.ex)
Generated hello_world app
Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> HelloWorld.hello
Hello World
:ok

That all looks OK then. From here on I’ll tidy up the output from iex sessions a bit for readability, removing the version info, compiler output etc.

Testing, 1 2 3

OK, on to our first task. We want to implement a counter, so that successive calls to HelloWorld.hello will print “Hello World 1”, “Hello World 2” etc. We’ll split this task into a few steps, starting with simply implementing a counter.

Now, the thing with functional languages is that they don’t have an obvious place to store mutable state between function calls. In Elixir’s case, the answer to this (borrowed from the underlying world of Erlang and OTP) is to maintain state in the event loops of processes. OTP provides a handy abstraction for this pattern in the form of gen_server, and Elixir wraps this up in GenServer, along with a sprinkling of metaprogramming to remove a lot of the tedious admin.

In fact Elixir has a more specific version of GenServerAgent – whose only job is to hold state, but we’ll stick with the vanilla GenServer for the sake of this example.

Here’s our first stab at a counter server:

lib/hello_world/counter.ex

defmodule  HelloWorld.Counter do
  use GenServer

  def handle_call(:next, _from, count) do
    {:reply, count, count + 1}
  end
end

There’s not a huge amount of code there (although admittedly more than there would be in a simple class or struct in an OO language), but let’s run through what there is.

First we use the GenServer module, which triggers some macro/metaprogramming magic to pull default implementations of all the callbacks required by the GenServer behaviour into the HelloWorld.Counter module. This saves a lot of boilerplate compared to the underlying gen_server from Erlang.

Now all we need to do is implement whatever custom behaviour we need – in this case storing an integer, and when asked returning its current value and incrementing the stored value. Our counter is a tiny server running in its own Erlang process, and at its centre is an event loop, which sits waiting until it receives a message, then handles the message appropriately. The event loop is initialised with some initial state (which could be anything, but here it’ll just be a simple number), and every time it handles a message it calls itself again with the new value for the state.

Thanks to GenServer, we don’t need to implement the event loop ourselves; we just need to write callbacks to handle the messages we expect to receive. The three main callbacks are handle_call (when the caller is expecting a response), handle_cast (when it isn’t), and handle_info (for messages that aren’t sent using call or cast). We need to pass back the next counter value when asked, so we’ve implemented the former.

The contract says that handle_call has to take three arguments: the message received (we pattern-match this to :next, which is the only message we care about), information about the sender (which we ignore), and the current state (in our case the counter’s value). There are several valid response tuples, but we’ve used the most common. This consists of the atom :reply (to indicate that we want to send a reply to the sender of the message), the reply itself (for us that’s the counter value), and the new state (the incremented count).

OK, let’s try that out in iex. First we’ll use GenServer.start_link to start a new process for our counter. We pass in our counter module, along with the initial state (the number we want to start counting from). We’ll pattern-match the reply to the expected success tuple, capturing the process ID (pid):

$ iex -S mix
iex(1)> {:ok, pid} = GenServer.start_link HelloWorld.Counter, 1
{:ok, #PID<0.206.0>}

Now let’s check that the counter works as expected, by repeatedly sending the :next message to the pid of the counter we started:

iex(2)> GenServer.call pid, :next
1
iex(3)> GenServer.call pid, :next
2
iex(4)> GenServer.call pid, :next
3

Good, we seem to have a working counter. There’s a small snag though: we have to keep hold of its pid (another piece of state) for as long as we want to use it, so we seem to be back where we started. Fortunately Elixir has us covered. We can specify a name when starting a server, then use that name instead of the pid when sending messages, and have Elixir find the correct process for us:

iex(1)> GenServer.start_link HelloWorld.Counter, 1, name: :hello_counter
{:ok, #PID<0.151.0>}
iex(2)> GenServer.call :hello_counter, :next
1
iex(3)> GenServer.call :hello_counter, :next
2
iex(4)> GenServer.call :hello_counter, :next
3

So far, so good. It seems a bit clunky to have to call GenServer functions directly from any code that wants to use the counter though, so let’s add a couple of wrappers in our own module. We’ll add our own start_link function, which takes a tuple of the counter name and the initial value, and a next function that wraps the Genserver.call:

lib/hello_world/counter.ex

defmodule  HelloWorld.Counter do
  use GenServer

  def start_link({name, start_at}) do
    GenServer.start_link __MODULE__, start_at, name: name
  end

  def next(name) do
    GenServer.call name, :next
  end

  def handle_call(:next, _from, count) do
    {:reply, count, count + 1}
  end
end

Let’s give that a spin, and while we’re at it prove that counters with different names behave independently:

iex(1)> HelloWorld.Counter.start_link {:counter_1, 1}
{:ok, #PID<0.240.0>}
iex(2)> HelloWorld.Counter.start_link {:counter_2, 10}
{:ok, #PID<0.242.0>}

iex(3)> HelloWorld.Counter.next :counter_1
1
iex(4)> HelloWorld.Counter.next :counter_1
2
iex(5)> HelloWorld.Counter.next :counter_2
10
iex(6)> HelloWorld.Counter.next :counter_1
3
iex(7)> HelloWorld.Counter.next :counter_2
11

All looking good so far. All we need to do now is to start a counter with our application, so we can access it from HelloWorld without having to worry about whether it’s already running. For this, we’ll need a Supervisor. Normally you’d generate a supervised application by providing the --sup flag to mix new, which we didn’t do. No problem though: we can just step up a directory and re-run the command, making sure not to let it overwrite our modified HelloWorld module:

$ cd ..
$ mix new --sup hello_world
The directory "hello_world" already exists. Are you sure you want to continue? [Yn] y
* creating README.md
README.md already exists, overwrite? [Yn] y
* creating .gitignore
.gitignore already exists, overwrite? [Yn] y
* creating mix.exs
mix.exs already exists, overwrite? [Yn] y
* creating config
* creating config/config.exs
config/config.exs already exists, overwrite? [Yn] y
* creating lib
* creating lib/hello_world.ex
lib/hello_world.ex already exists, overwrite? [Yn] n
* creating lib/hello_world/application.ex
* creating test
* creating test/test_helper.exs
test/test_helper.exs already exists, overwrite? [Yn] y
* creating test/hello_world_test.exs
test/hello_world_test.exs already exists, overwrite? [Yn] y

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd hello_world
    mix test

Run "mix help" for more commands.
$ cd -

This has created the module HelloWorld.Application (in lib/hello_world/application.ex) to describe our application, and also added an extra line to mix.exs setting this as the main module to run at startup. Here’s the default module as generated:

lib/hello_world/application.ex

defmodule HelloWorld.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
      # Starts a worker by calling: HelloWorld.Worker.start_link(arg)
      # {HelloWorld.Worker, arg},
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: HelloWorld.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

All we need to do is add a worker for our counter. We supply the module name and the argument to pass to start_link:

lib/hello_world/application.ex

  ...
  def start(_type, _args) do
    children = [
      {HelloWorld.Counter, {:hello, 1}},
    ]
  ...

Now we can use the counter in our original hello function, safe in the knowledge that it’ll have been started before our application runs:

lib/hello_world.ex

defmodule HelloWorld do
  def hello do
    IO.puts "Hello World #{HelloWorld.Counter.next :hello}"
  end
end

Look! It works!

iex(1)> HelloWorld.hello
Hello World 1
:ok
iex(2)> HelloWorld.hello
Hello World 2
:ok
iex(3)> HelloWorld.hello
Hello World 3
:ok

Tick, tock

So far, so good. It’s tedious having to keep asking the computer to greet us though – what we really want (bear with me here) is for it to keep doing so, once every second. How can we build a timer to make this dream a reality? The answer, I’m sure you’ll be unsurprised to hear, is with another GenServer. Here’s one way of doing it:

lib/hello_world/timer.ex

defmodule  HelloWorld.Timer do
  use GenServer

  def start_link(args) do
    GenServer.start_link __MODULE__, args
  end

  def init(_args) do
    send self(), :tick
    {:ok, nil}
  end

  def handle_info(:tick, state) do
    IO.puts "TICK"
    HelloWorld.Counter.next(:hello) |> HelloWorld.hello
    Process.send_after self(), :tick, 1_000
    IO.puts "TOCK"
    {:noreply, state}
  end
end

So, what’s going on here? Firstly, we implement start_link again. This time we’ve decided to make the timer specific to this one very important job, so there’ll only ever be one instance of it running. This means we don’t need to keep track of separate pids or names, so we’ll simply name the process after the current module (ie HelloWorld.Timer), which we obtain using the __MODULE__ macro. Our server doesn’t require any arguments, but we’ll dutifully pass whatever we’re given to GenServer.start_link anyway.

In the init callback, which is called on startup, we’ll just send ourselves (the call to self() simply returns the current pid) an initial :tick message, to start things going. With that message dispatched in the background, we return a tuple indicating that all is well, with nil as our initial (and, as it turns out, permanent) state.

The main work happens in handle_info, which is the callback that handles receipt of arbitrary messages. When a :tick message arrives, we grab the next counter value and pipe it into HelloWorld.hello (which we’re about to change to accept a number rather than accessing the counter itself). Once that’s done, we trigger the next clock tick by sending a delayed message, using Process.send_after. This isn’t part of GenServer, which is why we’re using plain messages instead of cast.

We also top and tail things by printing “TICK” and “TOCK”, purely so we can see the order that things are happening (this will become useful shortly).

We can add the timer as another child process:

lib/hello_world/application.ex

  ...
  def start(_type, _args) do
    children = [
      {HelloWorld.Counter, {:hello, 1}},
      {HelloWorld.Timer, {}},
    ]
  ...

Finally here’s our pared-down HelloWorld module:

lib/hello_world.ex

defmodule HelloWorld do
  def hello(n) do
    IO.puts "Hello World #{n}"
  end
end

And this is what happens when we fire up iex again:

$ iex -S mix
TICK
Hello World 1
TOCK
TICK
Hello World 2
TOCK
TICK
Hello World 3
TOCK
[ ... and so on ]

Everything but the kitchen async

This all seems to be working, but what if saying hello was a really slow operation? As things stand, the call to HelloWorld.hello is synchronous, so we’re holding up the timer’s event loop while we wait for it to complete. Let’s push the work into the background, by turning HelloWorld itself into yet another GenServer. This time we’ll make it a single shot fire-and-forget job (normally you’d probably want to do this with a Task – another of Elixir’s GenServer specialisations). We’ll add a random sleep, to simulate something that took some time to complete.

Here’s the code:

lib/hello_world.ex

defmodule HelloWorld do
  use GenServer

  def hello(n) do
    GenServer.start HelloWorld, n
  end

  def init(n) do
    GenServer.cast self(), :hello
    {:ok, n}
  end

  def handle_cast(:hello, n) do
    :rand.uniform * 1900 |> round |> Process.sleep
    IO.puts "Hello World #{n}"
    {:stop, :normal, n}
  end
end

The hello function has been modified to start a server, with the number to append to the message supplied as an initial argument. In init, we send ourselves a :hello message, then return (with the number as the value to be used as the initial state), allowing the calling process to carry on with its business.

In handle_cast (with the message pattern-matched to :hello), we first sleep for up to two seconds, then print our traditional message, and finally instruct GenServer to cleanly terminate the process by returning :stop, with a reason of :normal and a value for the final state.

Let’s see what happens when we run the new asynchronous, sleepy hello program:

$ iex -S mix
TICK
TOCK
Hello World 1
TICK
TOCK
Hello World 2
TICK
TOCK
Hello World 3
TICK
TOCK
TICK
TOCK
Hello World 5
Hello World 4

It all starts out promisingly, with the message appearing after the timer tick finishes, which means the hello job is successfully running in the background. The fourth and fifth calls go a bit wonky though, appearing out of order (clearly the fourth one decided to sleep for over a second longer then the fifth). Depending on what we’re doing, this might not be a problem, but sometimes it is. The real-life application that inspired this blog post involved periodically sending configuration to routers. Most of the time one set of commands finished before the next was sent, but occasionally a particularly large job could overrun, in which case the next one needed to wait its turn.

Form an orderly queue

Let’s assume that we don’t want one hello to start until the current one has finished. To accomplish this, we need some kind of queue. Erlang/OTP does actually have a queue module, but that might be overkill for our needs. At this point we remember that the mechanism that passes the messages into our GenServer event loop is itself a queue – maybe that’s all we need?

Let’s tweak our module slightly, making it run continuously as a single server, rather than firing up a new one every second (this is probably far better practice anyway!):

lib/hello_world.ex

defmodule HelloWorld do
  use GenServer

  def start_link(args) do
    GenServer.start_link __MODULE__, args, name: __MODULE__
  end

  def hello(n) do
    GenServer.cast __MODULE__, {:hello, n}
  end

  def handle_cast({:hello, n}, state) do
    :rand.uniform * 1900 |> round |> Process.sleep
    IO.puts "Hello World #{n}"
    {:noreply, state}
  end
end

Note how hello is no longer responsible for creating a new process, but is now just a wrapper round cast, sending ourselves a message asking for the hello operation to happen. A server can only handle one message at a time, so any new request while it’s sleeping will be queued up in the process’s inbox waiting for the next go round the event loop. This will happen as soon as handle_cast completes (we’ve changed it to return a :noreply tuple instead of :stop, so the loop will continue).

We need to add the server to the child list:

lib/hello_world/application.ex

  ...
  def start(_type, _args) do
    children = [
      {HelloWorld.Counter, {:hello, 1}},
      {HelloWorld.Timer, {}},
      {HelloWorld, {}},
    ]
  ...

Let’s run it one last time:

$ iex -S mix
TICK
TOCK
TICK
TOCK
Hello World 1
TICK
TOCK
Hello World 2
TICK
TOCK
Hello World 3
Hello World 4
TICK
TOCK
TICK
TOCK
Hello World 5
TICK
TOCK
Hello World 6

Well that seems to have done the trick, and is probably more than enough GenServer for one post (sorry, it ended up much longer than I expected – thanks for sticking with it to the end!)

Written by Kerry

August 13th, 2017 at 1:22 pm

Posted in Elixir

The value of spiking: a cautionary tale

leave a comment

A while ago I built a little web app using the Strava API, that tells you how far you’ve run so far this year, and how many miles you’re projected to do my the end of the year if you keep up the same average.

As Strava lets you see your friends’ year-to-date mileage on their profile pages, I thought it would be a nice addition to generate a leaderboard to show where you stood against your running friends. The URI to query an athlete’s stats is https://www.strava.com/api/v3/athletes/:id/stats, which I’m already using to get the stats for the person using my app, so all I have to do is query that person’s friends list and map over all the IDs calling the same API with their IDs, right?

First things first, mock up the UI for the new page. After a bit of wrestling with Bootstrap I had some navigation elements I was happy with, so moved onto the core app. It turned out that the Strava library I’m using didn’t have a wrapper for the friends/followers API, but it was pretty easy to add one, so I made the required changes and submitted a pull request. Then a nice test-driven feature from the outside of the app, stubbing the boundary to the function that retrieved the stats for each friend, as that was already there for the current user and didn’t need to be changed. A data structure specifically to hold the friend’s name, profile URL and mileage to display in the table, and a public function to return a list of those structs, and the core was all done.

Finally I put it all together, wired the controller in the web app up to the core function to make the required query, and modified the view to display each friend’s details in a row in the leaderboard. Tests all green! Finally! Let’s try it out!

500 internal server error

Wait, that’s not supposed to happen. Lets look at the logs. Hmm, the call to the Strava API seems to be failing. Unauthorised. That’s odd.

Better take another look at the API documentation.

Totals and stats

Returns recent (last 4 weeks), year to date and all time stats for a given athlete. Only available for the authenticated athlete. This is the recommended endpoint when polling for athlete upload events.

Wait, what’s that bit in the middle again? “Only available for the authenticated athlete”. Oh. Realisation dawns. No leaderboard then, and a couple of days’ work wasted on a branch that’s never going to make it into master.

So what’s the moral of the story? If you’re building a feature that relies on something outside your control, don’t make assumptions about what works and what doesn’t. Build a spike first, to test your assumptions before ploughing your time into building the real thing.

Oh, and read the documentation.

Written by Kerry

August 2nd, 2017 at 7:31 pm

Phoenix ecosystem for Rails developers

leave a comment

I’ve been tinkering with Elixir and Phoenix a bit recently. I really like the language (it’s the first time since Ruby that a new (to me) language has seemed this much fun) and Phoenix seems like a rock solid framework, but I was interested in what the surrounding ecosystem was like compared to the much more mature Ruby/Rails landscape. This was just a tyre-kicking exercise – I only really took each tool as far as making sure I could install it and do the basics. Incidentally, a great place to find popular Elixir resources is Awesome Elixir.

The app I started creating (which does virtually nothing!) is on github, and here are some brief notes of the stuff I tried out:

BDD executable specifications

White Bread

Ruby equivalent: Cucumber

White Bread (because that’s what you make cucumber sandwiches with) is a Gherkin parser/runner. It works in much the same way as Cucumber.

It took me a bit of fiddling to get it working with Phoenix, but nothing too difficult. According to the docs it should generate a sample context (equivalent to a Cucumber step definitions file) when you first run it, but that failed. Once I’d copied the example from the docs though, it worked fine.

The most obvious difference in the steps is that they operate (as you would expect) in a functional style, passing the context from one step to the next rather than relying on shared state. This is actually quite nice, as it makes it very clear what data is being passed between steps.

Test data creation

Ex Machina

Ruby equivalent: Factory Girl

Completely painless to install and get started with, and seems to work very much like Factory Girl (although I only scraped the surface).

Blacksmith is another option, which I didn’t investigate.

Browser simulation/automation

Hound

Ruby equivalent: Capybara

First I tried TucoTuco (apparently it’s an animal similar to a capybara). The first problem I had was that Webdriver (a dependency) failed to compile under Elixir 2.2, so I had to drop back to Elixir 1.1.1 (I raise an issue against the project, and got a quick reply saying it’ll be fixed soon). I also had to explicitly add ibrowse as a dix dependency, as it seems to only be available from GitHub, not hex. I found TucoTuco’s documentation to be a bit sketchy, and never quite got it working with Phoenix.

Switching to Hound, I found the documentation to be much better. It still uses webdriver, so had the same compatibility issue as TucoTuco, but not ibrowse.

One difference compared with Capybara is that you need to manually start phantomjs (phantomjs --wd), rather than it being started for you. It should be easy enough to wrap it in an Elixir application, but I haven’t tried that yet.

Elixir version management

Exenv

Ruby equivalent: rbenv/RVM

Previously I’d just been using the latest version of Elixir from Homebrew, but having to downgrade because of the webdriver incompatibility set me looking for a version manager. Exenv follows the philosophy of rbenv rather than rvm, is dead simple to install and use, and just keeps out of your way and works flawlessly.

Unit tests/specs

ESpec/ESpec_Phoenix

Ruby equivalent: RSpec/RSpec-Rails

Pretty similar to RSpec. Includes the same kind of matchers, mocking, stubbing etc.

Style checking

Dogma

Ruby equivalent: Rubocop

The docs a bit out of date, but otherwise works more-or-less as expected, failing the build if the rules you specify are broken. I suspect it doesn’t have quite the range of rules or configurability as rubocop (yet).

If you just want gentle style suggestions rather than enforcement, consider Credo instead (which I didn’t try).

Slim templating

Phoenix_slime

Ruby equivalent: Slim

Works as expected. Seems to show up on Mix as phoenix_slim too, which is the one I specified. Might have been renamed to phoenix_slime? Who knows?

Test coverage

Ruby equivalent: Simplecov

Coverage is built into exunit and espec, using Erlang’s cover tool:

mix espec --cover

The output’s not great though, so enter Coverex, which produces output much more like simplecov’s.

Unfortunately both tools seem to mark things like use statements within defmodule blocks as uncovered, which leads to misleadingly low coverage scores. I can’t see any obvious way to have it fail the build on low coverage either. In ruby I normally fail if any line is uncovered (if you’re test-driving the code, that’s never going to happen, right?), so that’s two reasons I can’t do the same thing in Elixir.

Debugging

Ruby equivalent: ByeBug or Pry

This is built into IEx. In your code, do:

require IEx
IEx.pry

Make sure you start the server attached to an IEx session:

iex -S mix phoenix.server

Written by Kerry

February 6th, 2016 at 3:06 pm

Posted in Software

TDD example with Rails, Cucumber and RSpec

one comment

In the unlikely event that anyone wants to scroll through the whole history of the TDD demo Rails app I did the other day, I wrote a little script to munge it all into one page. And here it is.

Read the rest of this entry »

Written by Kerry

March 27th, 2015 at 11:22 pm

Posted in Uncategorized

Telling a story with git

leave a comment

This may be the ugliest bash one-liner ever:

git log --pretty=%h | tail -r | while read sha ; do if [[ -z $(git rev-parse --verify $sha^2 2> /dev/null) ]] ; then echo '----' ; git show --summary $sha | tail -n+5 | egrep -v '^ (?:create|delete) mode' ; git --no-pager diff --color -U999 $sha~ $sha | awk '/diff --git/ { sub("^a/", "", $3) ; print "===" $3 } /diff --git/, /@@/ { next } { print }' ; fi ; done | ansifilter -H | sed 's/^----$/<hr \>>/;s/===\(.*\)/<h2>\1<\/h2>/' > history.html

It runs through the git history of a project, printing the full message for each commit, followed by a complete listing of each changed file, with added, removed or changed lines highlighted. It then outputs the result into a big ugly HTML file. Like this.

Written by Kerry

March 27th, 2015 at 11:21 pm

Posted in Software

Setting up a new Mac

leave a comment

I decided to do a clean install on my new [work] Mac, rather than just copying everything across from my old one. I thought I’d note down some of the things I installed, in case it comes in handy for anyone else (particularly new Mac users).

I’ve mostly ignored common cross-platform stuff like DropBox, 1Password etc, and setting up things like git and ruby. If you just want to quickly set up a Mac for ruby development, Thoughtbot’s laptop project (or Adam’s fork) might be the way to go.

If you’ve got any other can’t-live-without Mac apps that I might have missed, let me know!

From the app store:

Other apps:

Packages from homebrew (see above)

brew install direnv dos2unix elixir git mysql node phantomjs postgresql qt redis sqlite ssh-copy-id the_silver_searcher tig youtube-dl

Hidden settings

Show ~/Library in Finder:

chflags nohidden ~/Library

Enable copying from quick look preview windows:

defaults write com.apple.finder QLEnableTextSelection -bool TRUE; killall Finder

Written by Kerry

November 8th, 2014 at 10:57 am

Posted in Apple

Quick-and-dirty “edit text in vim” Mac service

one comment

Today I wanted to edit a big chunk of text (the source of a wiki page) in a browser textarea, and remembered that I used to have an app that let me open any text in vim, then paste the result back where it came from. Eventually I found the app I was thinking of – QuickCursor – but it turns out it’s no longer available.

Coincidentally I was looking into how to do something else tonight, and the answer was to create a service that ran an AppleScript, and it occurred to me that I could probably make a basic “edit in vim” service fairly easily. Turns out I was right…

Open the “Automator” app, and create a new service. Add a shell script action, and paste in the following script (this assumes you’ve installed the mvim script that comes with MacVim):

filename=`mktemp ~/.edit-in-vim-XXXX`
cat > $filename
/usr/local/bin/mvim -f $filename
cat $filename
rm $filename

Creating a service in Automator

Select “Output replaces selected text” in Automator, save the service, and you’re done. Just select some text (it doesn’t select-all for you like QuickCursor did), right-click and run your new service. You can also assign a keyboard shortcut in System Preferences:

Setting a keyboard shortcut

Written by Kerry

September 5th, 2013 at 8:11 pm

Posted in Software

Tagged with , , ,

FESuffolk Jasmine lightning talk

leave a comment

At last night’s FESuffolk meeting I gave a lightning talk on getting started with unit-testing Javascript/coffeescript using jasmine-headless-webkit. I made a screen recording, so if you missed it you can still experience my frantic babbling (I had Keynote set to auto-advance, using the Ignite 20 × 15 second slide format, and as usual I tried to cram too much into each slide).

If you don’t want to watch the recording, you can look at the slides and guess what I might have been saying, read the example code (the commits try to build up in small steps), or just play with the utterly pointless app that the example implements.

Written by Kerry

March 26th, 2013 at 8:00 pm

Interesting little rSpec gotcha

one comment

This one had Adam and me stumped for a while. Trying to check that a method is only called on objects that respond to it:

describe "Foo#call_all_the_things" do
  let(:foo_1) { stub :foo_1, bar: "hello" }
  let(:foo_2) { stub :foo_2 }
  subject { Foo.new foo_1, foo_2 }

  it "only calls bar on objects that respond to it" do
    foo_1.should_receive :bar
    foo_2.should_not_receive :bar
    subject.call_all_the_things(:bar)
  end
end

class Foo
  def initialize *things
    @things = things
  end

  def call_all_the_things method
    @things.each do |thing|
      thing.send method if thing.respond_to? method
    end
  end
end


  1) Foo#call_all_the_things only calls bar on objects that respond to it
     Failure/Error: thing.send method if thing.respond_to? method
       (Stub :foo_2).bar(no args)
           expected: 0 times
           received: 1 time

Hmm. Why is it calling bar on the thing that doesn’t respond to it? Perhaps rSpec doubles don’t handle respond_to? properly?

[1] pry(main)> require "rspec/mocks/standalone"
=> true
[2] pry(main)> foo = stub foo: 123
=> #<RSpec::Mocks::Mock:0x3fd07d246f34 @name=nil>
[3] pry(main)> foo.respond_to? :foo
=> true
[4] pry(main)> foo.respond_to? :bar
=> false

Nope.

FX: lightbulb above head

Of course! To do the should_not_receive check, it needs to stub the method, which means it responds to it!

Two possible solutions: either let the fact that the missing method isn’t called be tested implicitly, or specify that when objects that don’t respond to the method exist, no NoMethodError is raised.

Written by Kerry

March 20th, 2013 at 3:04 pm

Posted in rspec,Ruby,Software

Adastral Park code retreat

leave a comment

Code retreat poster

Ever since attending code retreat at Bletchley Park (the UK leg of Corey Haines’s code retreat tour) last year, I’d been vaguely intending to try and run an internal one at work. I finally got round to it this month, when a few people expressed an interest in a code retreat as a one-off alternative to the usual developer conferences we run a couple of times a year. I thought I’d share my experiences facilitating a code retreat for the first time, in case it’s helpful to anyone thinking of doing the same.

Beforehand

The “official” code retreat site has some good information on things to consider when hosting and facilitating. I suspect organising an internal company event is easier than a public one, because hopefully you’ll have a venue available and you won’t need to worry about sponsorship. The only one of Corey’s rules I ignored was the one about giving people decent food at lunchtime, but we did at least manage to beg enough budget (thanks Mel!) to be able to provide cheap Tesco sandwiches at lunchtime and a few random snacks and coffee during the day. I don’t feel too bad because rather than asking people to “come out at eight in the morning to spend the day coding”, we were giving them an excuse to spend a work day away from their projects, getting paid to practice.

We also planned to start at nine o’clock rather than the traditional eight. I think for public events the early start, usually on a Saturday, is a good way of selecting only people who really want to take part, but running it at work I wanted to swing the balance a bit towards encouraging people to come (I work on a large site with probably a couple of hundred developers scattered amongst the 3000-odd BT people, and part of the motivation for running this kind of event is to try and put developers from different teams in touch with each other).

With a bit of publicity via mailing lists and posters on site, 27 people had signed up. That seemed like about the right number, although I had no idea how many of those would turn up, or whether anyone would turn up without registering. I turned up at the office in plenty of time, and was lucky enough to get some last-minute tips from the master:

tips

Getting started

As it turned out we had a few glitches to deal with before we could get going – firstly the venue was locked and we only managed to get in because Rupert knew how to find the secret back entrance, then Adam and Anna got kicked out of Tesco for a fire alarm while buying the food – so we ended up starting at closer to 9.30.

Coffee machine hack

Potentially the biggest disaster though was that the hopper was missing from the coffee machine (most likely locked away to stop people like us using it with our own beans instead of paying for catering). Fortunately it takes more than a little detail like that to defeat a room full of engineers, and we soon had a workaround in place using a plastic cup with the bottom cut out.

Once the caffeine supply was assured and we were finally ready to start, I gave a quick presentation covering the purpose of the day, the four rules of simple design and Conway’s game of life:

First session

With everyone hopefully having a reasonable idea of what was going on, people paired up and started the first session. There were about 25–30 people (I forgot to count), split something like 70:20:10 between Java, Ruby and Groovy. In the first session, as expected, everyone was mostly just getting a feel for the problem, and realising that it’s not quite as simple as it appears from the list of rules.

Getting started

First session

After 45 minutes, I put on my best exam invigilator voice and got everyone to stop typing, delete their code and gather round. Pairs were split roughly evenly between those that started with a grid and those that started with a cell, and we had a short discussion about how a grid approach using a two-dimensional array fitted in with the “infinite grid” constraint. Some of those who had started from a cell were thinking about approaches for allowing cells to know about their neighbours, but no-one had got that far yet.

Mixing it up

For the second session, I suggested that those that had started with a grid tried starting from a cell instead, and vice versa. There were also quite a few people who wanted a chance to try Ruby, so they tried to grab the rubyists before anyone else did. Little did they know that they would find themselves exposed to Vim (and in some cases Mac OS X) at the same time, but that’s just a bonus.

In general people who tried both seemed to prefer the cell approach, partly because it made it easier to test-drive the rules. With a grid, people found they were having to implement some kind of to_string or render method to allow the transformed grid to be compared with the expected result. The biggest puzzle with the cell approach was how to handle the relationship between multiple cells.

I introduced a list of possible constraints people might want to consider to make things more interesting:

  • No if statements (or other conditionals).
  • No loops.
  • No methods longer than five lines (or four, or three…)
  • No more than three methods per class.
  • No ‘naked’ language primitives.
  • No built-in data structures.

I also attempted to explain TDD as if you meant it, in case anyone fancied giving it a go. Unfortunately my description was rather garbled, not helped by the fact that I’m not sure I fully understand it myself (I’m still annoyed that I picked the wrong track at SC2009 and missed Keith’s original presentation of the exercise, and it turns out Adam made the same mistake at SPA2010).

Pairing

Flying fingers

Session three

By the third session several pairs were starting to experiment with storing cell objects either in a set (with each cell knowing its position) or a hash, keyed by position. Some people used a comma-separated “x,y” string as the key, but with a bit of prodding most ended up replacing it with a point class of some kind.

One pair decided to start with a single cell, then initialise other cells with their neighbours. This initially seemed promising, but soon led to complications with circular references, as well as the need to keep dead cells around to maintain a connection between cells with space between them.

Another pair started developing the rules with a cell class, and initially had an alive? method which performed the life calculation. After some discussion they split the class into two – one for a living cell and another for a dead one. This removed a level of nesting from the calculations (which ended up in a different method), but interestingly left alive? in both classes, hardcoded to true and false respectively. We decided that this would have looked less odd in Java, where the method would have been declared abstractly in a superclass or interface.

One pair had decided to try implementing a solution with no if statements, but resorted to cheating by using a case instead!

In general, a lot of people – particularly those new to TDD – were starting to get frustrated, feeling like they were just getting to the point where they were “about to start the interesting bit” as the 45 minutes were up. I told them that we’d just have two sessions after lunch rather than three, with the second session lengthened to give people a chance of implementing a working game.

The countdown begins

Fourth – and as it turned out final – session

After lunch we started the fourth session, but it soon became clear that several pairs had already decided to abandon TDD and clean code in favour of trying to solve the problem. I completely understand how they felt – I wasted most of the sessions at the code retreat I attended simply pushing for the finish line instead of letting it go and concentrating on perfecting the internal quality of the code – but it seemed like we’d lost sight of the key goal of the day a little. I suspect I’m mostly to blame for that, and I’m sure a more experienced facilitator could have done a better job of gently guiding people in the right direction. Anyway, it seemed sensible to bow to the inevitable, let everyone carry on for another 45 minutes and wrap up a bit earlier than planned.

Basking in the glow

With the extra time, a few pairs did manage to get a working implementation, but the prize for the most impressive (cancelled out by a booby prize for abandoning TDD and neglecting the quality of their code!) goes to Jon and Jia Yan, who produced an SWT app that allowed graphical editing of the grid, variable speed and saving and loading files. They used the “set of live cells” approach, with an effectively infinite grid. Resizing the window on screen allowed you to show or hide cells, but the ones off the edge of the visible area continued to multiply.

Matthew and Rupert decided to look at someone else’s solution in Clojure, to understand how a functional approach using list comprehensions can lead to a very concise solution. They also played around with a Ruby version using the same techniques. Looking at the Clojure code led to a slightly rearranged version of the rules for life at a particular location after a clock tick:

  1. Cells which had three neighbours are alive
  2. Cells which had two neighbours are unchanged
  3. All other cells are dead

Paul, stuck on his own after his pair got dragged off onto a conference call (the perils of being a senior manager!), had a play with TDD as if you meant it. I think by the end of the session he was starting to get somewhere. Further than me, at any rate.

Wrapping it all up

After letting Jon show off a glider gun running in their SWT universe, we gathered round for the closing circle, asking everyone what they’d learned, what had surprised them, and what they’d do differently back at work on Monday as a result. Probably the most common comments revolved around the fact that neither of the two most obvious solutions turned out to be the best, suggesting that it’s worth spiking a few alternative approaches before heading blindly off down what may be a blind alley. Quite a few people were new to pairing, TDD or both, and generally seemed to agree that both provided benefits, although a couple of people felt that TDD was making things take much longer. Reassuringly, of the pairs that had abandoned TDD to concentrate on trying to get a working solution, most had found themselves descending into confusion without the tests to guide them.

Final thoughts

Overall, the event seemed to be a success. For quite a few people this was their first experience of TDD, and the opportunity to get first-hand experience by pairing with someone more experienced seemed to be useful. I was particularly pleased to see someone who’s recently come back into software development after a long time away doing a great job of holding his pair back from rushing into implementation without a failing test.

On a less positive note, I didn’t feel we really concentrated enough on “perfect” code and the four rules of simple design. I know from experience that it’s hard to let go of the natural desire to try and finish solving the problem, and I felt that a lot of people were getting frustrated enough without me peering over their shoulders picking holes in their code (however nicely I tried to do it). Perhaps this is a case of horses for courses, with the concentration on design perfection more suitable for a different mix of participants, or perhaps it just reflects my lack of experience facilitating this kind of event.

Also, I don’t think we really had enough sessions for the code retreat magic to start kicking in properly. We’d planned to have the recommended five or six, but a combination of the late start and a general loss of direction after lunch reduced that to four (with the fourth being double-length).

The important thing, of course, is that people learned something by participating, and I hope that was the case. Maybe we’ll do another one one day, and use the experience of this one to make it even better.

Written by Kerry

November 1st, 2011 at 11:42 am