If you ask someone what the coolest thing about Elixir is, chances are they’ll start with pipelines (the |> operator even made it to the front cover of the PragProg book). But the next thing they mention will probably have something to do with OTP in general, and/or GenServer specifically, and these are arguably more powerful.
I’ve been using Elixir more and more lately, and have started to get my head round how to use GenServer (and friends) for a bunch of different things. I thought I’d write up some of those examples, in case it’s helpful to other Elixir learners (it takes a while to get your head round this different way of structuring code, and I’m definitely not there yet).
You can find the code used in the examples on GitHub, and the commit history should follow the stages in text.
Important caveat
As I said above, I’m quite new to all this, so take everything with a pinch of salt. There are probably better ways to solve all these problems, and better ways to structure GenServer applications. Also, in the interests of preventing an already far too long post getting longer, I’ve completely ignored things like testing, code organisation and so on.
Hello world
First of all, we’ll create a standard mix application:
$ mix new hello_world
* creating README.md
* creating .gitignore
* creating mix.exs
* creating config
* creating config/config.exs
* creating lib
* creating lib/hello_world.ex
* creating test
* creating test/test_helper.exs
* creating test/hello_world_test.exs
Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:
cd hello_world
mix test
Run "mix help" for more commands.
Mix gives us a skeleton top level module for the application – in this case HelloWorld. Let’s tweak that slightly to make a function that simply prints a message to the terminal:
lib/hello_world.ex:
defmodule HelloWorld do
def hello do
IO.puts "Hello World"
end
end
Nothing particularly interesting there, but we’ll just check that it works in iex:
$ iex -S mix Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Compiling 1 file (.ex) Generated hello_world app Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help) iex(1)> HelloWorld.hello Hello World :ok
That all looks OK then. From here on I’ll tidy up the output from iex sessions a bit for readability, removing the version info, compiler output etc.
Testing, 1 2 3
OK, on to our first task. We want to implement a counter, so that successive calls to HelloWorld.hello will print “Hello World 1”, “Hello World 2” etc. We’ll split this task into a few steps, starting with simply implementing a counter.
Now, the thing with functional languages is that they don’t have an obvious place to store mutable state between function calls. In Elixir’s case, the answer to this (borrowed from the underlying world of Erlang and OTP) is to maintain state in the event loops of processes. OTP provides a handy abstraction for this pattern in the form of gen_server, and Elixir wraps this up in GenServer, along with a sprinkling of metaprogramming to remove a lot of the tedious admin.
In fact Elixir has a more specific version of GenServer – Agent – whose only job is to hold state, but we’ll stick with the vanilla GenServer for the sake of this example.
Here’s our first stab at a counter server:
lib/hello_world/counter.ex
defmodule HelloWorld.Counter do
use GenServer
def handle_call(:next, _from, count) do
{:reply, count, count + 1}
end
end
There’s not a huge amount of code there (although admittedly more than there would be in a simple class or struct in an OO language), but let’s run through what there is.
First we use the GenServer module, which triggers some macro/metaprogramming magic to pull default implementations of all the callbacks required by the GenServer behaviour into the HelloWorld.Counter module. This saves a lot of boilerplate compared to the underlying gen_server from Erlang.
Now all we need to do is implement whatever custom behaviour we need – in this case storing an integer, and when asked returning its current value and incrementing the stored value. Our counter is a tiny server running in its own Erlang process, and at its centre is an event loop, which sits waiting until it receives a message, then handles the message appropriately. The event loop is initialised with some initial state (which could be anything, but here it’ll just be a simple number), and every time it handles a message it calls itself again with the new value for the state.
Thanks to GenServer, we don’t need to implement the event loop ourselves; we just need to write callbacks to handle the messages we expect to receive. The three main callbacks are handle_call (when the caller is expecting a response), handle_cast (when it isn’t), and handle_info (for messages that aren’t sent using call or cast). We need to pass back the next counter value when asked, so we’ve implemented the former.
The contract says that handle_call has to take three arguments: the message received (we pattern-match this to :next, which is the only message we care about), information about the sender (which we ignore), and the current state (in our case the counter’s value). There are several valid response tuples, but we’ve used the most common. This consists of the atom :reply (to indicate that we want to send a reply to the sender of the message), the reply itself (for us that’s the counter value), and the new state (the incremented count).
OK, let’s try that out in iex. First we’ll use GenServer.start_link to start a new process for our counter. We pass in our counter module, along with the initial state (the number we want to start counting from). We’ll pattern-match the reply to the expected success tuple, capturing the process ID (pid):
$ iex -S mix
iex(1)> {:ok, pid} = GenServer.start_link HelloWorld.Counter, 1
{:ok, #PID<0.206.0>}
Now let’s check that the counter works as expected, by repeatedly sending the :next message to the pid of the counter we started:
iex(2)> GenServer.call pid, :next 1 iex(3)> GenServer.call pid, :next 2 iex(4)> GenServer.call pid, :next 3
Good, we seem to have a working counter. There’s a small snag though: we have to keep hold of its pid (another piece of state) for as long as we want to use it, so we seem to be back where we started. Fortunately Elixir has us covered. We can specify a name when starting a server, then use that name instead of the pid when sending messages, and have Elixir find the correct process for us:
iex(1)> GenServer.start_link HelloWorld.Counter, 1, name: :hello_counter
{:ok, #PID<0.151.0>}
iex(2)> GenServer.call :hello_counter, :next
1
iex(3)> GenServer.call :hello_counter, :next
2
iex(4)> GenServer.call :hello_counter, :next
3
So far, so good. It seems a bit clunky to have to call GenServer functions directly from any code that wants to use the counter though, so let’s add a couple of wrappers in our own module. We’ll add our own start_link function, which takes a tuple of the counter name and the initial value, and a next function that wraps the Genserver.call:
lib/hello_world/counter.ex
defmodule HelloWorld.Counter do
use GenServer
def start_link({name, start_at}) do
GenServer.start_link __MODULE__, start_at, name: name
end
def next(name) do
GenServer.call name, :next
end
def handle_call(:next, _from, count) do
{:reply, count, count + 1}
end
end
Let’s give that a spin, and while we’re at it prove that counters with different names behave independently:
iex(1)> HelloWorld.Counter.start_link {:counter_1, 1}
{:ok, #PID<0.240.0>}
iex(2)> HelloWorld.Counter.start_link {:counter_2, 10}
{:ok, #PID<0.242.0>}
iex(3)> HelloWorld.Counter.next :counter_1
1
iex(4)> HelloWorld.Counter.next :counter_1
2
iex(5)> HelloWorld.Counter.next :counter_2
10
iex(6)> HelloWorld.Counter.next :counter_1
3
iex(7)> HelloWorld.Counter.next :counter_2
11
All looking good so far. All we need to do now is to start a counter with our application, so we can access it from HelloWorld without having to worry about whether it’s already running. For this, we’ll need a Supervisor. Normally you’d generate a supervised application by providing the --sup flag to mix new, which we didn’t do. No problem though: we can just step up a directory and re-run the command, making sure not to let it overwrite our modified HelloWorld module:
$ cd ..
$ mix new --sup hello_world
The directory "hello_world" already exists. Are you sure you want to continue? [Yn] y
* creating README.md
README.md already exists, overwrite? [Yn] y
* creating .gitignore
.gitignore already exists, overwrite? [Yn] y
* creating mix.exs
mix.exs already exists, overwrite? [Yn] y
* creating config
* creating config/config.exs
config/config.exs already exists, overwrite? [Yn] y
* creating lib
* creating lib/hello_world.ex
lib/hello_world.ex already exists, overwrite? [Yn] n
* creating lib/hello_world/application.ex
* creating test
* creating test/test_helper.exs
test/test_helper.exs already exists, overwrite? [Yn] y
* creating test/hello_world_test.exs
test/hello_world_test.exs already exists, overwrite? [Yn] y
Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:
cd hello_world
mix test
Run "mix help" for more commands.
$ cd -
This has created the module HelloWorld.Application (in lib/hello_world/application.ex) to describe our application, and also added an extra line to mix.exs setting this as the main module to run at startup. Here’s the default module as generated:
lib/hello_world/application.ex
defmodule HelloWorld.Application do
# See https://hexdocs.pm/elixir/Application.html
# for more information on OTP Applications
@moduledoc false
use Application
def start(_type, _args) do
# List all child processes to be supervised
children = [
# Starts a worker by calling: HelloWorld.Worker.start_link(arg)
# {HelloWorld.Worker, arg},
]
# See https://hexdocs.pm/elixir/Supervisor.html
# for other strategies and supported options
opts = [strategy: :one_for_one, name: HelloWorld.Supervisor]
Supervisor.start_link(children, opts)
end
end
All we need to do is add a worker for our counter. We supply the module name and the argument to pass to start_link:
lib/hello_world/application.ex
...
def start(_type, _args) do
children = [
{HelloWorld.Counter, {:hello, 1}},
]
...
Now we can use the counter in our original hello function, safe in the knowledge that it’ll have been started before our application runs:
lib/hello_world.ex
defmodule HelloWorld do
def hello do
IO.puts "Hello World #{HelloWorld.Counter.next :hello}"
end
end
Look! It works!
iex(1)> HelloWorld.hello Hello World 1 :ok iex(2)> HelloWorld.hello Hello World 2 :ok iex(3)> HelloWorld.hello Hello World 3 :ok
Tick, tock
So far, so good. It’s tedious having to keep asking the computer to greet us though – what we really want (bear with me here) is for it to keep doing so, once every second. How can we build a timer to make this dream a reality? The answer, I’m sure you’ll be unsurprised to hear, is with another GenServer. Here’s one way of doing it:
lib/hello_world/timer.ex
defmodule HelloWorld.Timer do
use GenServer
def start_link(args) do
GenServer.start_link __MODULE__, args
end
def init(_args) do
send self(), :tick
{:ok, nil}
end
def handle_info(:tick, state) do
IO.puts "TICK"
HelloWorld.Counter.next(:hello) |> HelloWorld.hello
Process.send_after self(), :tick, 1_000
IO.puts "TOCK"
{:noreply, state}
end
end
So, what’s going on here? Firstly, we implement start_link again. This time we’ve decided to make the timer specific to this one very important job, so there’ll only ever be one instance of it running. This means we don’t need to keep track of separate pids or names, so we’ll simply name the process after the current module (ie HelloWorld.Timer), which we obtain using the __MODULE__ macro. Our server doesn’t require any arguments, but we’ll dutifully pass whatever we’re given to GenServer.start_link anyway.
In the init callback, which is called on startup, we’ll just send ourselves (the call to self() simply returns the current pid) an initial :tick message, to start things going. With that message dispatched in the background, we return a tuple indicating that all is well, with nil as our initial (and, as it turns out, permanent) state.
The main work happens in handle_info, which is the callback that handles receipt of arbitrary messages. When a :tick message arrives, we grab the next counter value and pipe it into HelloWorld.hello (which we’re about to change to accept a number rather than accessing the counter itself). Once that’s done, we trigger the next clock tick by sending a delayed message, using Process.send_after. This isn’t part of GenServer, which is why we’re using plain messages instead of cast.
We also top and tail things by printing “TICK” and “TOCK”, purely so we can see the order that things are happening (this will become useful shortly).
We can add the timer as another child process:
lib/hello_world/application.ex
...
def start(_type, _args) do
children = [
{HelloWorld.Counter, {:hello, 1}},
{HelloWorld.Timer, {}},
]
...
Finally here’s our pared-down HelloWorld module:
lib/hello_world.ex
defmodule HelloWorld do
def hello(n) do
IO.puts "Hello World #{n}"
end
end
And this is what happens when we fire up iex again:
$ iex -S mix TICK Hello World 1 TOCK TICK Hello World 2 TOCK TICK Hello World 3 TOCK [ ... and so on ]
Everything but the kitchen async
This all seems to be working, but what if saying hello was a really slow operation? As things stand, the call to HelloWorld.hello is synchronous, so we’re holding up the timer’s event loop while we wait for it to complete. Let’s push the work into the background, by turning HelloWorld itself into yet another GenServer. This time we’ll make it a single shot fire-and-forget job (normally you’d probably want to do this with a Task – another of Elixir’s GenServer specialisations). We’ll add a random sleep, to simulate something that took some time to complete.
Here’s the code:
lib/hello_world.ex
defmodule HelloWorld do
use GenServer
def hello(n) do
GenServer.start HelloWorld, n
end
def init(n) do
GenServer.cast self(), :hello
{:ok, n}
end
def handle_cast(:hello, n) do
:rand.uniform * 1900 |> round |> Process.sleep
IO.puts "Hello World #{n}"
{:stop, :normal, n}
end
end
The hello function has been modified to start a server, with the number to append to the message supplied as an initial argument. In init, we send ourselves a :hello message, then return (with the number as the value to be used as the initial state), allowing the calling process to carry on with its business.
In handle_cast (with the message pattern-matched to :hello), we first sleep for up to two seconds, then print our traditional message, and finally instruct GenServer to cleanly terminate the process by returning :stop, with a reason of :normal and a value for the final state.
Let’s see what happens when we run the new asynchronous, sleepy hello program:
$ iex -S mix TICK TOCK Hello World 1 TICK TOCK Hello World 2 TICK TOCK Hello World 3 TICK TOCK TICK TOCK Hello World 5 Hello World 4
It all starts out promisingly, with the message appearing after the timer tick finishes, which means the hello job is successfully running in the background. The fourth and fifth calls go a bit wonky though, appearing out of order (clearly the fourth one decided to sleep for over a second longer then the fifth). Depending on what we’re doing, this might not be a problem, but sometimes it is. The real-life application that inspired this blog post involved periodically sending configuration to routers. Most of the time one set of commands finished before the next was sent, but occasionally a particularly large job could overrun, in which case the next one needed to wait its turn.
Form an orderly queue
Let’s assume that we don’t want one hello to start until the current one has finished. To accomplish this, we need some kind of queue. Erlang/OTP does actually have a queue module, but that might be overkill for our needs. At this point we remember that the mechanism that passes the messages into our GenServer event loop is itself a queue – maybe that’s all we need?
Let’s tweak our module slightly, making it run continuously as a single server, rather than firing up a new one every second (this is probably far better practice anyway!):
lib/hello_world.ex
defmodule HelloWorld do
use GenServer
def start_link(args) do
GenServer.start_link __MODULE__, args, name: __MODULE__
end
def hello(n) do
GenServer.cast __MODULE__, {:hello, n}
end
def handle_cast({:hello, n}, state) do
:rand.uniform * 1900 |> round |> Process.sleep
IO.puts "Hello World #{n}"
{:noreply, state}
end
end
Note how hello is no longer responsible for creating a new process, but is now just a wrapper round cast, sending ourselves a message asking for the hello operation to happen. A server can only handle one message at a time, so any new request while it’s sleeping will be queued up in the process’s inbox waiting for the next go round the event loop. This will happen as soon as handle_cast completes (we’ve changed it to return a :noreply tuple instead of :stop, so the loop will continue).
We need to add the server to the child list:
lib/hello_world/application.ex
...
def start(_type, _args) do
children = [
{HelloWorld.Counter, {:hello, 1}},
{HelloWorld.Timer, {}},
{HelloWorld, {}},
]
...
Let’s run it one last time:
$ iex -S mix TICK TOCK TICK TOCK Hello World 1 TICK TOCK Hello World 2 TICK TOCK Hello World 3 Hello World 4 TICK TOCK TICK TOCK Hello World 5 TICK TOCK Hello World 6
Well that seems to have done the trick, and is probably more than enough GenServer for one post (sorry, it ended up much longer than I expected – thanks for sticking with it to the end!)