Categories
Weeknotes

Weeknotes 2022-06

On time this week! Although I type this having not written the actual post, so I may be tempting fate. Once again it turns out I haven’t done much, and am starting to doubt the wisdom of committing to writing these.

Work-wise, we did the planned coding dojo. Because I hadn’t got round to setting up or testing a shared editing environment to be able to rotate pairs, we did more of a mobbing approach (which I think probably worked better), with me as the dumb driver, trying desperately to write the code other people suggested rather than what I wanted. This proved surprisingly difficult, as it turns out that my preferences for writing unit tests have diverged from other people’s (annoyingly I suspect I’m now arguing against things I encouraged people to do in the first place). I was largely successful in doing what I was told, albeit with possibly a bit too much delay while I tried and failed to talk them round. I’m certainly not claiming that my way is the right way, but I feel like I need to put together a blog post or screencast at some point to attempt to justify my preferences.

There’s yet another professional community type initiative going on at the moment, and the successful applicants for “distinguished engineer” status (not a promotion or an actual role; more a vague acknowledgement of influence) were finally announced this week. Needless to say, I missed the cut.

I’ve been quite enjoying the zombie shenanigans All of Us are Dead on Netflix, who seem to be on a roll with their Korean dramas at the moment.

An uneventful week running-wise, although I was a bit quicker than I have been recently on both the “Thursday Tempo Ten” and parkrun. And I think I’ve got to the bottom of why Garmin has consistently shown my training load as above the recommended band recently – since I did a factory reset on my watch a while ago, apparently it’s assumed my maximum heart rate was about 20bpm too low (I’m pretty sure it must have measured rates well above that, so I don’t know why it didn’t update itself, or at least let me know).

Categories
Weeknotes

Weeknotes 2022-05

Oops, nearly a week late. Not a very eventful week anyway.

I’ve started using Obsidian for random note-taking. I’ve got it synced between home and work Macs and my phone via DropBox, and so far it seems fine, although I’ve only really scratched the surface of its features so far. The vim mode is one of the better ones I’ve come across (not on the phone, obviously).

I got dragged into a call about the fact that the Ipswich GoodGym group has fizzled out, but I think I managed to avoid getting talked into taking on any responsibility for getting it going again. We’ll see what happens.

At work we’re trying to get more team-wide learning stuff going again, and started this week with a discussion about various design topics and quandaries that had been suggested. We didn’t reach any conclusions (or expect to), but I think the conversations were valuable. We’re planning to alternate this kind of thing with a coding dojo.

It was the Great Bentley Half on Sunday (ie “today”, if I were writing this on time). The weather threatened to be appalling, but it cleared up to sunshine just in time for the start. There was a bit of wind, and a downpour in the middle, but the sun came out with enough time that my shorts and vest had mostly dried out by the finish. Got round in 1:36:10, which is a lot slower than 2019, but also a lot faster than the nightmare race I had in 2020.

Categories
Weeknotes

Weeknotes 2022-04

This week I had to go back to an old (2015) Rails app, after the (internal) customer asked for a second version for a slightly different set of users. It was the standard thing of “can you create and deploy a duplicate version, then just make this minor change …”, when the “create and deploy” turned out to be about 30% of the work, and the “minor change” the remaining 70%.

Fortunately it was a fairly simple app, we seem to have done a reasonable job of writing it cleanly in the first place, and I’ve made at least some effort to keep it up-to-date with Rails versions etc in the meantime, so it wasn’t too horrendous. These days I spend 99% of my time with Elixir and Phoenix, but apparently I still remember most of my Rails knowledge. As expected, there were a few things that felt a bit too magical compared to Phoenix’s intentional focus on clarity, but generally the “it just works” stuff still did. The only thing I think we might have over-complicated was the acceptance/integration tests, which in retrospect seemed to have a few too many levels of abstraction (Cucumber features, step definitions, SitePrism page objects and custom matchers).

I initially intended to deploy a second instance with its own database, and control the small behavioural differences with an environment variable, but abandoned that halfway through in favour of leaving it as a single app with two separate DNS entries, with users and root-level models partitioned by app, which was determined based on hostname. Even with the false start, that bit only took a day. The last half hour of that day was a bit of search-and-replace and repeating the DNS registration and SSL cert ordering, once I realised that I’d called the new version IPCC, when it should have been ITCC. Oops.

The “minor change” bit was one of those judgement calls where it was clear that the design wasn’t ideal – if it had been something under active development I’d definitely have spent some time refactoring, but as I suspect it’ll go back to humming along without me touching it for a few years I decided to put up with a bit of cruft.

Back in Elixir, I spent Friday fiddling with some potential improvements to a Phoenix app which might entail some proper BEAM stuff like custom supervision trees and ETS tables, but it’s all still a bit vague.

Garmin has continued to accuse me of overreaching, and after an easy run on Thursday it recommended 3.5 days of rest, but it seems to have settled down again now. It’s the Great Bentley half next weekend – I think that’ll be my first half marathon race since lockdown.

I’ve been doing a bit more unicycle practice this week, and can now just about wobble along for a few feet, using the garage door for balance but no longer holding on for dear life. It feels like actually riding it might not be too far off now.

I was surprised to find a couple of new chillis on last year’s plants that are still sitting on the kitchen windowsill, just when I was about to turf them out and think about planting new ones. In other culinary news, he third and final batch of marmalade is done, and turned out the best. It was a bit of a Goldilocks situation, with the first lot too runny and the second burnt and over-set.

Categories
Weeknotes

Weeknotes 2022-03

Let’s see if I can get this one written on time …

Having my finger on the pulse as ever, I finally got around to playing Untitled Goose Game, to see what all the fuss was about however long ago it was that that was the thing that was all over Twitter, rather than Wordle (which I’m playing in a less anachronistic fashion, on account of not having to wait for it to appear in the Steam sale). It’s moderately entertaining, but I have to admit to getting stuck/bored with a few of the tasks and googling for hints. The game also features a Mini, which is always good, even if it’s just static scenery.

A proper Mini!

I had a couple of days off this week, and attempted some long overdue DIY tasks. I successfully managed to get on the roof and remove the bits of dead creeper that were adorning the chimney (without falling off). but the other two jobs were less successful. My kitchen sink’s been emptying very slowly recently, but attacking it from outside with drain rods and from inside with a flexible brush thing both failed. The gap in the middle where neither tool reached can’t be that large, but apparently I need a plan C. Then I finally replaced the bulb in the floodlight that vaguely points at the patio, and which hasn’t worked for years. I bought an LED equivalent, expecting the kind of eye-melting power that you get from even basic bike lights these days, but it seems to throw out the equivalent of about half a candle (a tungsten bulb of the same wattage would be brighter). Ah well, back to halogen I guess – it’s not like it’s on that often, so it’s hardly an energy hog.

So now I own five ladders

At work I got volunteered to run a five minute “ice breaking” session for a team meeting, so I came up with something based on Gary Bernhardt’s WAT talk (which coincidentally was published ten years ago this week). I found a few more bits of Javascript unexpectedness (did you know that 'B' + 'a' + + 'B' + 'a' is 'BaNaNa'? Or that while 1 < 2 < 3 is true, 3 > 2 > 1 is false, but 2 > 1 > 0 is true?) and got people to shout out what they thought would be returned. Then afterwards I spent far too long writing up explanations for them all, which I doubt anyone read.

After Sunday’s cross country, Garmin says I’m “over-reaching”, so I ought to start next week with a rest day I suppose (because I’m the kind of person who takes advice from random pieces of consumer electronics).

Finally, I’ve made the first two of three batches of marmalade to last the year. The first lot, with huge oranges from a local greengrocer, came out a bit runny and seemed to have virtually no peel in it, whereas the second, with tiny oranges from the supermarket, was mostly peel and has probably set to a thick gloop (though I haven’t opened any of the second batch yet). Peel goes with r² but weight with r³, I guess.

Categories
Weeknotes

Weeknotes 2022-02

OK, so it’s week two and I’m posting this four days late (just edited that from the original “a day late”). At this rate, expect the final post of this year some time in 2028.

A couple of weeks ago I took my Strava heatmap and drew round it to show the area enclosed by places I’d run to starting and finishing at home:

Someone on Facebook saw it and mentioned CityStrides, which is what in the early 2000s we would have called a mashup of activity tracker data with Open Streetmap, showing you how many of the streets in your town you’ve run along. I’m always a sucker for this kind of nonsense, so I signed up and synced it with Garmin (Strava didn’t work for some reason – I suspect API rate limiting). It turned out I was in third place for Ipswich, with 461 of 1,412 streets completed (and I know both the people above me in the table). Of course when I mentioned this three other people signed up, pushing me down to 6th, but I did add some convoluted meanderings to a couple of runs this week to pick up another 24 streets.

With the end of the grace period for free corporate use of Docker Desktop fast approaching, I finally got round to switching to MiniKube. I tried the default option of Hyperkit, but it wouldn’t start (maybe something to do with security settings on my work Mac), but it was fine with VirtualBox, and actually seems much quicker than Docker Desktop. I don’t use Docker for anything fancy – just to create dummy instances of other systems that my application can ssh to for integration testing, and to allow me to compile the OTP application for Linux on my Mac – and the only change I had to make was to have the app connect to the containers using the VM’s IP address, whereas the previous setup had automatically mapped the ports to the host address. I feel slightly guilty about being part of the problem of big companies using open source without paying anything back, but apparently someone else is looking at getting a company licence for Docker Desktop, so maybe they’ll get some of our money eventually.

Friday saw the third “agile fest” at work, with a variety of talks (I wasn’t involved this time). All good stuff, although I did get into a minor disagreement in the chat when someone suggested we needed a single mandated structure for Jira, rather than teams doing what works best for them (it would be bad enough if someone told me I had to use Jira at all, let alone also telling me how to use it!)

Categories
Weeknotes

Weeknotes 2022-01

First proper week of the year, and I nearly forgot to do this, so that bodes well for the experiment continuing.

Back to work properly this week (well, apart from Monday), and other people were mostly back too, so meetings and stuff have started happening again. One of those annoying Fridays where something turns out to be more complicated than I thought, so ended up leaving it half-done to pick up on Monday. It was one of those rabbit holes where I started adding a new feature, realised that the related tests didn’t quite cover all the permutations of existing behaviour, then discovered that the untested bits didn’t quite behave as I expected, and had to be fixed too. I seem to remember there being another layer of yak shaving too, but I can’t remember what it was (which doesn’t bode well for the morning).

I got my unicycle out for the first time in ages. I bought it at the beginning of 2020, but stopped trying to learn to ride it when I decided that turning up at A&E having injured myself during lockdown might not be looked on too favourably. This time I’m practicing outside, for the sake of the floor and walls in the hall, where I started before. Still at the “pedal a couple of times while using a wall for stability, then fall off” stage. Hopefully when (if) I finally get the knack, it’ll be a useless skill that never goes away, like riding half a bicycle.

I finally watched Don’t Look Up, which seems to be dividing opinion, but lots of people had recommended it. Not terrible, but at least 45 minutes too long, and felt at times like it wasn’t sure whether it was serious satire or slapstick.

Just short of 40 miles of running this week, culminating in a very muddy, but almost enjoyable, county cross-country championships (which sounds more impressive than it is – you only have to be a member of a club to enter, and not actually be fast or anything).

Categories
Weeknotes

Weeknotes: 2022 week zero

Having gone almost two years without posting here (and I can’t even really blame the pandemic), I thought I’d have a go at writing weeknotes. I’m not sure yet whether I’ll really have anything interesting to include each week (I suspect not), but we’ll see. Maybe it’ll even encourage me to do more stuff so I can write about it.

Apparently week one of 2022 is the one that started today, so this is more of a preface and a trial run. I’d almost forgotten I had this blog too – I might migrate it to something more towards the static end of things at some point. Or, you know …

As usual, running featured quite heavily (47.3 miles), including drawing a reindeer, and the traditional NYE idiocy of cycling to Felixstowe, running on the prom for 45 minutes, getting in the sea (shoulders under and minimum one stroke – them’s the rules), then drying off and heading home via various pubs. Ended up in the Cricketers (’spoons), where they had Abbot Special Reserve for £1.99, and – as usual – Mr Harper was a bad influence.

I’m never usually one for staying up to see in the new year, but this time I excelled myself by going to bed at around 8pm. Got woken up by the fireworks, naturally.

Did a bit of geocaching for the first time in ages. A few years ago I reached the point where I’d either found, or failed to find, all the ones within cycling distance, but there are plenty more now, and I bagged a few while I was out running.

I compiled another cryptic crossword; something else I haven’t done for a few months. And I hit a 365-day streak learning Spanish on Duolingo, although I had cheated slightly by using a “streak freeze” a couple of times – once when I went to do my daily practice and realised it was gone midnight, and again on New Year’s Eve, having run out of hearts (and time) in the morning before quite finishing a lesson, and unsurprisingly forgotten to try again once I arrived home after the Abbot Special Reserve.

I’m still intermittently practicing drums – mostly using Melodics at the moment for a bit more structure, after spending a few months just repeatedly trying to learn and play along to the same few songs.

I even did a couple of days’ work between Christmas and New Year because I was bored (though that doesn’t help with using up the leave I need to take before April). For some reason it seems easier to spend time at this time of year on technical debt type stuff more than new features, although I did a bit of both. Nothing very exciting, although I will say that I still really like Phoenix LiveView (at least for the kind of work I mostly do, which is internal-use web apps with no API, and no mobile apps).

Well that’s more than enough for one week I reckon. I wonder how long I’ll manage to keep this up …

Categories
Food

Marmalade

A few people have asked for my marmalade recipe over the years, and up to now I’ve never bothered writing it down because it’s pretty much just the basic one you’d find in any simple cookbook. Nevertheless, here it is. Partly so I have somewhere to point people, and partly as a handy place to remind myself how much I normally make, so I know I’ll have enough to last the year.

TL;DR

Here’s the basics:

1:2:2 oranges/sugar/water, plus half a lemon for every pound of oranges. Pips & pith in muslin, then boil with water, juice and orange peel for 1½ to 2 hours, until half the water’s gone. Add sugar, heat gently and stir until dissolved, then boil for 15 minutes. Test for set, and boil more if necessary. Pour into heated jars.

Now for the long version …

Ingredients

This is how much I make in one batch, and will produce somewhere around 12–14 jars. Two batches lasts me nearly a year, and three gives me a few jars left over (I eat a lot of marmalade). I guess 2½ would be about right.

Seville oranges generally only appear in the shops for a few weeks around January, at least in the UK, so get them while you can. Quantities aren’t particularly critical, which is why the imperial and metric versions don’t quite line up.

  • 4lb/2kg Seville oranges
  • Two lemons
  • 8lb/4kg preserving sugar (not jam sugar, which has added pectin)
  • 8pt/4l water

You’ll also need a jam pan (or some other massive saucepan – it should be big enough for at least twice the amount of water you’re starting with), some muslin squares (Lakeland sell them), jars, and a nice sharp chef’s knife.

Preparation

Put the muslin in a sieve above a bowl (or the pan), and squeeze the lemon juice and pips into it. Discard the outside of the lemons.

Get rid of the little stalk things from the oranges, and cut them into quarters (the oranges, not the stalks).

Cut the oranges into quarters

Give each quarter a quick squeeze into the muslin, then scrape the pips and pith in as well. You want to boil these bits up to get the pectin that makes the marmalade set, but you don’t want them in the finished product, which is where the muslin comes in.

Squeeze out the juice, and scrape the pips & pith into the muslin

Once everything’s in the muslin square (you might need two), pull the corners together, give it a good squeeze, then tie it up with string. Put the tied-up bag(s) into the pan, along with the orange juice and the water.

Tie up the muslin square

Now for the tedious bit – cutting all those orange peel quarters into thin strips. If you like chunky marmalade you’re in luck, as it won’t take quite as long, but either way you’ll probably want to put the radio or a podcast or something on to relieve the boredom. Keep your wits about you though, as it’s easy to lose concentration and shave a bit off the end of one of your fingers.

I find the easiest technique is to cut each bit of peel into three lengthwise, then finely chop across the three bits.

Finely chop the peel

Add the chopped peel to the pan, and bring to the boil. Simmer uncovered for 1½ to 2 hours, until the volume of water has reduced by about half (and what’s left will mostly be peel). Remove the bag(s) of pips, then scrape any pieces of peel back into the pan, and squeeze as much juice as you can back into the pan too (this is harder than it sounds, because it’ll obviously be far too hot to touch).

Boil the peel and bag of pips

Now add the sugar. This is the point when you realise that marmalade is basically just sugar with some bits of orange peel in it.

At this point, put your jars (without their lids) in the oven at 100ºC (to avoid any risk of cracking when you pour the marmalade in), and put a small plate or saucer in the fridge (you’ll use this to test for the setting point later).

Add the sugar (so much sugar!)

Heat gently, and keep stirring until the sugar has all dissolved. Then turn up the heat and bring to the boil. Boil vigorously for 15 minutes, stirring occasionally to stop it sticking to the bottom of the pan. A word of warning: as you scrape the bottom of the pan with the spoon, it tends to encourage some mildly volcanic eruptions, and you probably don’t want to be splashing boiling syrup on yourself.

If you do end up burning some of the peel, don’t worry too much – it doesn’t seem to particularly affect the final flavour. You’ll end up with a darker finished product though, so maybe chuck in the word “caramelised” and pretend it was deliberate.

Double, double toil and trouble …

After 15 minutes, turn off the heat, take half a teaspoon of marmalade out of the pan and tip it onto the cold plate from earlier. Leave it for a minute, then drag your fingertip across it. If it forms a skin (or has already taken on a jelly-like consistency), then your marmalade is ready. If not, boil for another five minutes and try again. I’ve occasionally had nightmare batches that took ages; probably because I hadn’t boiled away enough water.

Leave to cool slightly (five minutes or so is fine), then pour, ladle or otherwise decant into jars. A jam funnel will greatly reduce the amount of mess generated by this stage, although it won’t be able to help with the inevitable sticky splashes you’ll already have on and around your cooker.

Transfer to jars

And that’s it! Ready to spread on your toast in the morning.

The finished product
Categories
Elixir Software

Playing with property-based tests

I’ve never really tried property-based testing in anger before (just a brief play with QuickCheck during an ill-fated attempt to learn me some Haskell), but I recently came across a practice kata that seemed perfectly suited, so thought I’d give it a go. Unfortunately I haven’t got round to reading the book that I got in a recent PragProg sale, so it was mostly fumbling around and a lot of Googling.

The problem, basically, is this:

Given a list of names, generate a set of randomised “secret Santa” gift tags, such that no-one ends up giving themself a present.

This kind of thing isn’t always easy to test using traditional example-based testing, because of its intrinsic unpredictability, but using a property-based approach instead allows us to define a set of rules that should always hold true, and have the test framework automatically throw a variety of input data at your code and check that the rule isn’t broken.

The rules I came up with were:

  • returns the same number of tags as supplied names
  • includes each name once as a sender
  • includes each name once as a recipient
  • does not tag anyone’s present as from themself

The framework I used was PropCheck, which is an Elixir wrapper around the Erlang proper library. It acts as an extension to ExUnit, so you can mix and match example- and property-based tests.

PropCheck comes with a number of built-in generators, which will produce instances of primitive types like integers, strings, lists etc. Strings behave a bit oddly, thanks to the way Elixir strings are based on Erlang binaries, whereas Erlang natively uses charlists (also supported, but not commonly used, in Elixir). When QuickCheck generates binaries, it tends towards using zeros as the individual codepoint values, rather than a printable character. This means you’ll see values like <<0, 1, 0>>, rather than "ABA" (which is equivalent to <<65, 66, 65>>).

Initially, I picked a simple list of binaries as the input, meaning I could define my first property like this:

property "returns the same number of tags as supplied names" do forall names <- list(binary()) do length(SecretSanta.tags(names)) == length(names) end end
Code language: PHP (php)

For the later properties though, it became clear that not all lists were valid input. Specifically neither empty nor single-element lists really made sense, and neither did zero-character names. This is one of the advantages of this kind of testing – the framework will come up with all sorts of edge cases, forcing you to stop and think whether they need to be handled, or explicitly excluded.

Restricting the input data meant writing a simple custom generator. I actually split it up into several smaller generators, which built on each other. First, let’s have some more realistic names. For simplicity, let’s say a firstname and a surname are each eight characters long (and are mysteriously made up of non-printable characters!):

defp name() do let([first <- binary(8), second <- binary(8)], do: first <> " " <> second) end
Code language: JavaScript (javascript)

The let macro allows us to take some existing generators, and transform their output – in this case joining two strings with a space to make a name.

Now we can create a list of names. My first approach to making a minimum length list didn’t work, but after getting some help via a GitHub issue I ended up with this, which basically says “append two names to a list of names”. That list may be empty, but by appending two values to it we are guaranteed a minimum of two elements:

defp at_least_two_names() do let([first <- name(), second <- name(), many <- list(name())], do: [first, second | many] ) end
Code language: PHP (php)

Again, although it seemed at first that this was enough, the tests revealed another hidden assumption: the names in the list need to be unique. I extended the previous generator to enforce this using such_that. This applies a guard to the generated values, forcing a new one to be produced if the condition is not met. If it can’t find a suitable value after a configurable number of tries, it will give up and fail the test.

defp at_least_two_unique_names do such_that(names <- at_least_two_names(), when: length(Enum.uniq(names)) == length(names)) end
Code language: HTML, XML (xml)

With all those in place, here are the implementations of the properties listed earlier:

property "returns the same number of tags as supplied names" do forall names <- at_least_two_unique_names() do length(SecretSanta.tags(names)) == length(names) end end property "includes each name once as a sender" do forall names <- at_least_two_unique_names() do tags = SecretSanta.tags(names) names -- Enum.map(tags, & &1.from) == [] end end property "includes each name once as a recipient" do forall names <- at_least_two_unique_names() do tags = SecretSanta.tags(names) names -- Enum.map(tags, & &1.to) == [] end end property "does not tag anyone's present as from themself" do forall names <- at_least_two_unique_names() do tags = SecretSanta.tags(names) tags |> Enum.all?(&(&1.from != &1.to)) end end
Code language: JavaScript (javascript)

And for completeness, here’s my implementation of the algorithm. It may not be pretty or efficient, but at least I know it works!

defmodule SecretSanta do def tags(names) do allocate(Enum.shuffle(names), Enum.shuffle(names)) end defp allocate([], []), do: [] # guard against leaving the same last person in both lists defp allocate([from, name], [to, name]) do allocate([from, name], [name, to]) end # if the two heads are the same, shuffle the second list defp allocate(from, to) when hd(from) == hd(to) do allocate(from, Enum.shuffle(to)) end defp allocate([from | from_tail], [to | to_tail]) do [%{from: from, to: to} | allocate(from_tail, to_tail)] end end
Code language: PHP (php)

Categories
Elixir

GenServer: Elixir’s Swiss Army Knife

If you ask someone what the coolest thing about Elixir is, chances are they’ll start with pipelines (the |> operator even made it to the front cover of the PragProg book). But the next thing they mention will probably have something to do with OTP in general, and/or GenServer specifically, and these are arguably more powerful.

I’ve been using Elixir more and more lately, and have started to get my head round how to use GenServer (and friends) for a bunch of different things. I thought I’d write up some of those examples, in case it’s helpful to other Elixir learners (it takes a while to get your head round this different way of structuring code, and I’m definitely not there yet).

You can find the code used in the examples on GitHub, and the commit history should follow the stages in text.

Important caveat

I have no idea what I'm doing

As I said above, I’m quite new to all this, so take everything with a pinch of salt. There are probably better ways to solve all these problems, and better ways to structure GenServer applications. Also, in the interests of preventing an already far too long post getting longer, I’ve completely ignored things like testing, code organisation and so on.

Hello world

First of all, we’ll create a standard mix application:

$ mix new hello_world
* creating README.md
* creating .gitignore
* creating mix.exs
* creating config
* creating config/config.exs
* creating lib
* creating lib/hello_world.ex
* creating test
* creating test/test_helper.exs
* creating test/hello_world_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd hello_world
    mix test

Run "mix help" for more commands.

Mix gives us a skeleton top level module for the application – in this case HelloWorld. Let’s tweak that slightly to make a function that simply prints a message to the terminal:

lib/hello_world.ex:

defmodule HelloWorld do
  def hello do
    IO.puts "Hello World"
  end
end

Nothing particularly interesting there, but we’ll just check that it works in iex:

$ iex -S mix
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Compiling 1 file (.ex)
Generated hello_world app
Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> HelloWorld.hello
Hello World
:ok

That all looks OK then. From here on I’ll tidy up the output from iex sessions a bit for readability, removing the version info, compiler output etc.

Testing, 1 2 3

OK, on to our first task. We want to implement a counter, so that successive calls to HelloWorld.hello will print “Hello World 1”, “Hello World 2” etc. We’ll split this task into a few steps, starting with simply implementing a counter.

Now, the thing with functional languages is that they don’t have an obvious place to store mutable state between function calls. In Elixir’s case, the answer to this (borrowed from the underlying world of Erlang and OTP) is to maintain state in the event loops of processes. OTP provides a handy abstraction for this pattern in the form of gen_server, and Elixir wraps this up in GenServer, along with a sprinkling of metaprogramming to remove a lot of the tedious admin.

In fact Elixir has a more specific version of GenServerAgent – whose only job is to hold state, but we’ll stick with the vanilla GenServer for the sake of this example.

Here’s our first stab at a counter server:

lib/hello_world/counter.ex

defmodule  HelloWorld.Counter do
  use GenServer

  def handle_call(:next, _from, count) do
    {:reply, count, count + 1}
  end
end

There’s not a huge amount of code there (although admittedly more than there would be in a simple class or struct in an OO language), but let’s run through what there is.

First we use the GenServer module, which triggers some macro/metaprogramming magic to pull default implementations of all the callbacks required by the GenServer behaviour into the HelloWorld.Counter module. This saves a lot of boilerplate compared to the underlying gen_server from Erlang.

Now all we need to do is implement whatever custom behaviour we need – in this case storing an integer, and when asked returning its current value and incrementing the stored value. Our counter is a tiny server running in its own Erlang process, and at its centre is an event loop, which sits waiting until it receives a message, then handles the message appropriately. The event loop is initialised with some initial state (which could be anything, but here it’ll just be a simple number), and every time it handles a message it calls itself again with the new value for the state.

Thanks to GenServer, we don’t need to implement the event loop ourselves; we just need to write callbacks to handle the messages we expect to receive. The three main callbacks are handle_call (when the caller is expecting a response), handle_cast (when it isn’t), and handle_info (for messages that aren’t sent using call or cast). We need to pass back the next counter value when asked, so we’ve implemented the former.

The contract says that handle_call has to take three arguments: the message received (we pattern-match this to :next, which is the only message we care about), information about the sender (which we ignore), and the current state (in our case the counter’s value). There are several valid response tuples, but we’ve used the most common. This consists of the atom :reply (to indicate that we want to send a reply to the sender of the message), the reply itself (for us that’s the counter value), and the new state (the incremented count).

OK, let’s try that out in iex. First we’ll use GenServer.start_link to start a new process for our counter. We pass in our counter module, along with the initial state (the number we want to start counting from). We’ll pattern-match the reply to the expected success tuple, capturing the process ID (pid):

$ iex -S mix
iex(1)> {:ok, pid} = GenServer.start_link HelloWorld.Counter, 1
{:ok, #PID<0.206.0>}

Now let’s check that the counter works as expected, by repeatedly sending the :next message to the pid of the counter we started:

iex(2)> GenServer.call pid, :next
1
iex(3)> GenServer.call pid, :next
2
iex(4)> GenServer.call pid, :next
3

Good, we seem to have a working counter. There’s a small snag though: we have to keep hold of its pid (another piece of state) for as long as we want to use it, so we seem to be back where we started. Fortunately Elixir has us covered. We can specify a name when starting a server, then use that name instead of the pid when sending messages, and have Elixir find the correct process for us:

iex(1)> GenServer.start_link HelloWorld.Counter, 1, name: :hello_counter
{:ok, #PID<0.151.0>}
iex(2)> GenServer.call :hello_counter, :next
1
iex(3)> GenServer.call :hello_counter, :next
2
iex(4)> GenServer.call :hello_counter, :next
3

So far, so good. It seems a bit clunky to have to call GenServer functions directly from any code that wants to use the counter though, so let’s add a couple of wrappers in our own module. We’ll add our own start_link function, which takes a tuple of the counter name and the initial value, and a next function that wraps the Genserver.call:

lib/hello_world/counter.ex

defmodule  HelloWorld.Counter do
  use GenServer

  def start_link({name, start_at}) do
    GenServer.start_link __MODULE__, start_at, name: name
  end

  def next(name) do
    GenServer.call name, :next
  end

  def handle_call(:next, _from, count) do
    {:reply, count, count + 1}
  end
end

Let’s give that a spin, and while we’re at it prove that counters with different names behave independently:

iex(1)> HelloWorld.Counter.start_link {:counter_1, 1}
{:ok, #PID<0.240.0>}
iex(2)> HelloWorld.Counter.start_link {:counter_2, 10}
{:ok, #PID<0.242.0>}

iex(3)> HelloWorld.Counter.next :counter_1
1
iex(4)> HelloWorld.Counter.next :counter_1
2
iex(5)> HelloWorld.Counter.next :counter_2
10
iex(6)> HelloWorld.Counter.next :counter_1
3
iex(7)> HelloWorld.Counter.next :counter_2
11

All looking good so far. All we need to do now is to start a counter with our application, so we can access it from HelloWorld without having to worry about whether it’s already running. For this, we’ll need a Supervisor. Normally you’d generate a supervised application by providing the --sup flag to mix new, which we didn’t do. No problem though: we can just step up a directory and re-run the command, making sure not to let it overwrite our modified HelloWorld module:

$ cd ..
$ mix new --sup hello_world
The directory "hello_world" already exists. Are you sure you want to continue? [Yn] y
* creating README.md
README.md already exists, overwrite? [Yn] y
* creating .gitignore
.gitignore already exists, overwrite? [Yn] y
* creating mix.exs
mix.exs already exists, overwrite? [Yn] y
* creating config
* creating config/config.exs
config/config.exs already exists, overwrite? [Yn] y
* creating lib
* creating lib/hello_world.ex
lib/hello_world.ex already exists, overwrite? [Yn] n
* creating lib/hello_world/application.ex
* creating test
* creating test/test_helper.exs
test/test_helper.exs already exists, overwrite? [Yn] y
* creating test/hello_world_test.exs
test/hello_world_test.exs already exists, overwrite? [Yn] y

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd hello_world
    mix test

Run "mix help" for more commands.
$ cd -

This has created the module HelloWorld.Application (in lib/hello_world/application.ex) to describe our application, and also added an extra line to mix.exs setting this as the main module to run at startup. Here’s the default module as generated:

lib/hello_world/application.ex

defmodule HelloWorld.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
      # Starts a worker by calling: HelloWorld.Worker.start_link(arg)
      # {HelloWorld.Worker, arg},
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: HelloWorld.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

All we need to do is add a worker for our counter. We supply the module name and the argument to pass to start_link:

lib/hello_world/application.ex

  ...
  def start(_type, _args) do
    children = [
      {HelloWorld.Counter, {:hello, 1}},
    ]
  ...

Now we can use the counter in our original hello function, safe in the knowledge that it’ll have been started before our application runs:

lib/hello_world.ex

defmodule HelloWorld do
  def hello do
    IO.puts "Hello World #{HelloWorld.Counter.next :hello}"
  end
end

Look! It works!

iex(1)> HelloWorld.hello
Hello World 1
:ok
iex(2)> HelloWorld.hello
Hello World 2
:ok
iex(3)> HelloWorld.hello
Hello World 3
:ok

Tick, tock

So far, so good. It’s tedious having to keep asking the computer to greet us though – what we really want (bear with me here) is for it to keep doing so, once every second. How can we build a timer to make this dream a reality? The answer, I’m sure you’ll be unsurprised to hear, is with another GenServer. Here’s one way of doing it:

lib/hello_world/timer.ex

defmodule  HelloWorld.Timer do
  use GenServer

  def start_link(args) do
    GenServer.start_link __MODULE__, args
  end

  def init(_args) do
    send self(), :tick
    {:ok, nil}
  end

  def handle_info(:tick, state) do
    IO.puts "TICK"
    HelloWorld.Counter.next(:hello) |> HelloWorld.hello
    Process.send_after self(), :tick, 1_000
    IO.puts "TOCK"
    {:noreply, state}
  end
end

So, what’s going on here? Firstly, we implement start_link again. This time we’ve decided to make the timer specific to this one very important job, so there’ll only ever be one instance of it running. This means we don’t need to keep track of separate pids or names, so we’ll simply name the process after the current module (ie HelloWorld.Timer), which we obtain using the __MODULE__ macro. Our server doesn’t require any arguments, but we’ll dutifully pass whatever we’re given to GenServer.start_link anyway.

In the init callback, which is called on startup, we’ll just send ourselves (the call to self() simply returns the current pid) an initial :tick message, to start things going. With that message dispatched in the background, we return a tuple indicating that all is well, with nil as our initial (and, as it turns out, permanent) state.

The main work happens in handle_info, which is the callback that handles receipt of arbitrary messages. When a :tick message arrives, we grab the next counter value and pipe it into HelloWorld.hello (which we’re about to change to accept a number rather than accessing the counter itself). Once that’s done, we trigger the next clock tick by sending a delayed message, using Process.send_after. This isn’t part of GenServer, which is why we’re using plain messages instead of cast.

We also top and tail things by printing “TICK” and “TOCK”, purely so we can see the order that things are happening (this will become useful shortly).

We can add the timer as another child process:

lib/hello_world/application.ex

  ...
  def start(_type, _args) do
    children = [
      {HelloWorld.Counter, {:hello, 1}},
      {HelloWorld.Timer, {}},
    ]
  ...

Finally here’s our pared-down HelloWorld module:

lib/hello_world.ex

defmodule HelloWorld do
  def hello(n) do
    IO.puts "Hello World #{n}"
  end
end

And this is what happens when we fire up iex again:

$ iex -S mix
TICK
Hello World 1
TOCK
TICK
Hello World 2
TOCK
TICK
Hello World 3
TOCK
[ ... and so on ]

Everything but the kitchen async

This all seems to be working, but what if saying hello was a really slow operation? As things stand, the call to HelloWorld.hello is synchronous, so we’re holding up the timer’s event loop while we wait for it to complete. Let’s push the work into the background, by turning HelloWorld itself into yet another GenServer. This time we’ll make it a single shot fire-and-forget job (normally you’d probably want to do this with a Task – another of Elixir’s GenServer specialisations). We’ll add a random sleep, to simulate something that took some time to complete.

Here’s the code:

lib/hello_world.ex

defmodule HelloWorld do
  use GenServer

  def hello(n) do
    GenServer.start HelloWorld, n
  end

  def init(n) do
    GenServer.cast self(), :hello
    {:ok, n}
  end

  def handle_cast(:hello, n) do
    :rand.uniform * 1900 |> round |> Process.sleep
    IO.puts "Hello World #{n}"
    {:stop, :normal, n}
  end
end

The hello function has been modified to start a server, with the number to append to the message supplied as an initial argument. In init, we send ourselves a :hello message, then return (with the number as the value to be used as the initial state), allowing the calling process to carry on with its business.

In handle_cast (with the message pattern-matched to :hello), we first sleep for up to two seconds, then print our traditional message, and finally instruct GenServer to cleanly terminate the process by returning :stop, with a reason of :normal and a value for the final state.

Let’s see what happens when we run the new asynchronous, sleepy hello program:

$ iex -S mix
TICK
TOCK
Hello World 1
TICK
TOCK
Hello World 2
TICK
TOCK
Hello World 3
TICK
TOCK
TICK
TOCK
Hello World 5
Hello World 4

It all starts out promisingly, with the message appearing after the timer tick finishes, which means the hello job is successfully running in the background. The fourth and fifth calls go a bit wonky though, appearing out of order (clearly the fourth one decided to sleep for over a second longer then the fifth). Depending on what we’re doing, this might not be a problem, but sometimes it is. The real-life application that inspired this blog post involved periodically sending configuration to routers. Most of the time one set of commands finished before the next was sent, but occasionally a particularly large job could overrun, in which case the next one needed to wait its turn.

Form an orderly queue

Let’s assume that we don’t want one hello to start until the current one has finished. To accomplish this, we need some kind of queue. Erlang/OTP does actually have a queue module, but that might be overkill for our needs. At this point we remember that the mechanism that passes the messages into our GenServer event loop is itself a queue – maybe that’s all we need?

Let’s tweak our module slightly, making it run continuously as a single server, rather than firing up a new one every second (this is probably far better practice anyway!):

lib/hello_world.ex

defmodule HelloWorld do
  use GenServer

  def start_link(args) do
    GenServer.start_link __MODULE__, args, name: __MODULE__
  end

  def hello(n) do
    GenServer.cast __MODULE__, {:hello, n}
  end

  def handle_cast({:hello, n}, state) do
    :rand.uniform * 1900 |> round |> Process.sleep
    IO.puts "Hello World #{n}"
    {:noreply, state}
  end
end

Note how hello is no longer responsible for creating a new process, but is now just a wrapper round cast, sending ourselves a message asking for the hello operation to happen. A server can only handle one message at a time, so any new request while it’s sleeping will be queued up in the process’s inbox waiting for the next go round the event loop. This will happen as soon as handle_cast completes (we’ve changed it to return a :noreply tuple instead of :stop, so the loop will continue).

We need to add the server to the child list:

lib/hello_world/application.ex

  ...
  def start(_type, _args) do
    children = [
      {HelloWorld.Counter, {:hello, 1}},
      {HelloWorld.Timer, {}},
      {HelloWorld, {}},
    ]
  ...

Let’s run it one last time:

$ iex -S mix
TICK
TOCK
TICK
TOCK
Hello World 1
TICK
TOCK
Hello World 2
TICK
TOCK
Hello World 3
Hello World 4
TICK
TOCK
TICK
TOCK
Hello World 5
TICK
TOCK
Hello World 6

Well that seems to have done the trick, and is probably more than enough GenServer for one post (sorry, it ended up much longer than I expected – thanks for sticking with it to the end!)