For a specific high-performance workloads, I wanted to include a new and highly optimized endpoint onto Roundtrip.
If you don’t know what Roundtrip is yet, feel free to quickly check out the previous Roundtrip post and come back once you got the idea of what it does.
I had to select both a wire protocol and an actual transport that will be very efficient. To gain an even higher margin over HTTP, I knew I wanted it to be at least binary and not very chatty.
A good option for this would be Thrift, for example. However I wanted to go as low as I could, because I didn’t really need anything more than the bare simplest RPC mechanism.
However, going with straight up TCP wouldn’t gain me much because I typically hold development ease and maintainability as an additional value. There was only one thing I felt offering an awesome development model and being as close to (or even better than, on some occasions) TCP…
If you haven’t yet heard of ZeroMQ I urge you to check it out. The programming model is great, and the documentation is very accessible and interesting.
Installing it should be a breeze given that you pick the right
combination of library version and gem. It may vary based on your platform, but the end goal is to install a
binary library that
ffi-rzmq will bind to.
On Linux in this case, I chose to install the older (time wise, it isn’t THAT old) 2.2 series ZeroMQ variant.
The 2.2.0 ZeroMQ variant, from my experience, offered the smoothest
experience for me with
ffi-rzmq and everything just worked.
Before building from source, let’s make sure we have the proper dependencies:
Get the 2.2.0 sources,
and build using the typical
./configure && make && sudo make
Next, you should
gem install ffi-rzmq whether on MRI or JRuby. To interoperate with my existing Ruby code, I chose ffi-rzmq because it
offered the best compatibility with MRI and JRuby.
You should be aware though, that some Ruby examples in the ZeroMQ examples repository are not using
ffi-rzmq and may be incompatible with it to varying degrees. Despite of that, I still think
ffi-rzmq is the way to go.
Building a ZeroMQ Server
First, ZeroMQ is not a queue system, nor a queue broker.
ZeroMQ provides you with building blocks and messaging patterns that you can compose and form your own network topologies. The ZeroMQ Guide is the best place to learn about best-practices and networking patterns that you can use for your own projects.
REQuest / REPly
With ZeroMQ, REQ/REP stands for the Request / Reply messaging pattern. The simplest client-server topology you can probably think of. That is, the client makes a request, and the server replies in a synchronous, lockstep fashion.
Within Roundtrip’s ZeroMQ endpoint, the communication model would be very simple. The Client sends a Command, the Server performs it synchronously (well, Redis is quite fast) and replies with a JSON-serialized response.
The protocol is simple. A string describing a one-letter action and method parameters.
Is replied with
To understand how simple and powerful the ZeroMQ programming model is in the case of REQ/REP, here is the core of the request handling in the server code:
Things to note here are the symmetrical communication: after any
send. In addition, ZeroMQ handles all of TCP connections and
juggling for you; you can take the leap-of-faith and all you do is talk to a socket
sending a stream of messages.
At the other end, a client is responsible to send a request and handle the reply.
The trick with REQ/REP, or with any ZeroMQ messaging patterns for that matter, is to be able to address reliability concerns.
Even though the server code is simple, the client takes much of the complexity through handling reliability in the form of retries and timeouts:
Was it worth it?
In short, yes: using ZeroMQ to implement an RPC in this case was as close as using the code itself without a transport overhead at all.
Running 1000 transactions:
You can checkout the benchmarks here, and of course - take note that ZeroMQ benchmarking was done on a local machine, it doesn’t take into account network overhead (just the bare bones nuts and bolts of ZeroMQ as a transport).