Monolith vs Microservices: Where to start

There is a debate going around how to do the initial design of a new system in the context of Microservices. Should you start with a Monolithic approach initially and move to a Microservices later or use Microservices from the beginning?

Introduction

Martin Fowler recently wrote article called 'Monolith First' that talks about how to get started on a new Microservice project. Stefan Tilkov wrote a response Don't start with a monolith arguing the reverse. As you would expect from the authors involved, both articles are well thought out and full of great information.

Mr. Fowler posits that breaking things into services is error prone. You can't know ahead of time what the correct break out is so you should postpone that as late as possible. His implication is the main benifit of Microservices are their ability to scale. Mr. Tilkov hits on the point that the best time to architect a system is when you are starting out. Both authors, seem to miss the miss the point that these two options are not mutually exclusive.

The Modeling failure

These Engineers are thinking about Microservices as a unit of distribution rather then a unit of decomposition. Instead of thinking about Microservices as a way to structure their programs they should think of them as a modeling construct for those programs. Conflating Microservices-as-modeling-construct with Microservices-as-unit-of-distribution is like conflating CORBA with an Object System itself. It only makes sense in a very narrow context.

I have the good fortune of having spent a significant part of my career writing systems in Erlang. Erlang is a distributed, fault tolerant language that uses processes as the fundamental unit of decomposition in the same way that Objects are the fundamental unit of decomposition in languages like Java and C++. The difference is that processes are Microservices. They are completely independent and have an explicit API and life cycle. In fact, I have been describing programming in Erlang, as modeling using very fine grained SOA, for almost fifteen years. I wish I had coined the term 'Microservices' instead, but the idea is the same. In Erlang There is no other approach to modeling. So you build your systems based on these tiny communicating services. It allows you to build your system as a group of small services that provide and receive work from one another. Erlang has another feature that makes all this possible. That feature is Location Transparency. Location Transparency means that I don't have to worry about where a service is to be able to communicate with it. I only need some name, a unique identifier. I can use the system facilities to communicate with the service identified by than name and have some high expectation that the service will receive it.

With those two features Erlang has given me enough flexibility so that I can get all the benefits of composing my system in Micorservices and all the benefits of having the system be monolithic when it makes sense for it to be monolithic. Essentially, I don't have to worry about distribution during design and implementation at all. Erlang allows me to develop, test and do the initial deployments on a single node then, as the scale increases add new hardware to the system. This allows me to push out taking the additional conceptual load of distribution to as late a point as possible.

Doing It Right in Other Languages

This isn't really a Microservice vs Monolith debate. Its a reaction to the lack of certain fundamental properties in existing Microservice platforms for other languages. One of the most important properties is Location Transparency. Often DNS or other location metadata is hard coded into the system, forcing the implementer to make decisions at compile time that constraint what is deployed at run time. Locational Transparency allows you to design your system using Microservices as a modeling construct without worrying about those services will be deployed.

In total, there are four properties that a Microservice framework needs to support. These are:

  • Microservices are cheap to define.
  • Microservices are Locationally Transparent.
  • Service calls are context aware.
  • Groups of services can be declaratively described for deployment.

A framework that fully supports thees for properties allows us to use Microservices as a unit of modeling in any language. Lets deep-dive into each property individually.

Micro services are cheap to define

Using Microservices must be as efficient to design and implement as possible. In the best case it's at least as efficient to implement as the native module constructs in a language, usually an Object or a Module. You also can't force the definition of a microservice to be in some foreign syntax like XML or Json. There will be a lot of Microservices and the creation of those services needs to flow with the language itself. So, in Java, we might take use attributes to take an object and expose its methods to a service compiler. This compiler could be run at compile time or at runtime. The details don't matter. In C++ we could use templates. In OCaml, the solution would be camlp4. All of these solutions use the same approach. They give the developer the ability to mark a Microservice as a Microservice and expose it's API without significant development overhead.

Microservices are Locationally Transparent

Microservices must be named in some unique way. Those names must be propagated throughout the system. That may be a single node on a developers desktop or ten thousand nodes spread through-out data centers in cities around the world. Systems that have this property allow names to be used as a composition and allows for useful Locational Transperancy.

Service calls are context aware

Service calls are the ubiquitous form of work delivery in Microservice based systems. These calls must be as inexpensive as possible. The best way for to accomplish this goal is to make the call infrastructure context aware. When the Provider and the Consumer of the service are both on the same node, then the call must be made in the most efficient way possible. Preferably this would devolve into a simple function call. At the very least, serialization and deserialization should be avoided. With out this feature, the performance of a system that uses Microservices as a modeling approach will suffer.

Declaratively describe groups of services as nodes

It is important that we can easily change the composition of a node. That we can, through some declarative means change which services run where in our system. This must also be done without recompiling the system. In the best case, it would also be done without redeploying the system, though in many cases that is to much to ask for.

So there must be some configuration based, preferably declarative approach to describing nodes in the network that comprises the distributed system.

Conclusion

The debate about how to design the initial architecture of a new system, Monolith vs Microservice, is predicated on the false assumption that these two approaches are mutually exclusive. That Microservices are a means of distribution rather than an architectural choice. By questioning that assumption and creating a Microservice framework that supports the key properties that we have talked about in this article, we can build flexible, scalable systems that are also easy to design test and build.

Deploy Erlang Target System to Heroku

In this post these new tools will be used:

First, clone minasan and create the Heroku application on Cedar-14:

$ git clone https://github.com/tsloughter/minasan.git  
$ cd minasan  
$ heroku create --stack cedar-14
Now that Heroku has the cedar-14 stack if you are also running a recent Linux distro you can upload the target system created by _relx_ directly to your app, before now we would have to build it on Heroku or in a system with an older glibc to work on Heroku's Ubuntu 10.04. Since _minasan_ is using binary packages and a fork of _rebar_ be sure to use the _rebar_ included in the repo, same goes for _relx _so that including the Procfile in the tarball works. The first step will be to update the package index for _rebar_, then compiling and building the release tarball (with _erts_ included and _dev-mode_ off so the Erlang runtime is included):
$ ./rebar3 update  
$ ./rebar3 compile
$ ./rebar3 do release -i true --dev-mode false, tar
Using the new Slug API endpoint through _hk slug_ the tarball can be pushed directly as a slug to your app and then scale up the web process to at least 1:
$ hk slug _rel/minasan/minasan-0.0.1.tar.gz  
$ hk scale web=1  
$ hk open  

Your browser should now open to your new app.

A few things to note:

  • './rebar3 pkgs' will show you a list of available packages to use in rebar deps
  • Currently 'hk slug' only support sending a tarball that does not yet have the structure of a slug. So it is unpacked and repacked. I plan to support directories and properly formatted tarballs.

Designing for Actor Based Systems

Many people are intrigued and excited about Erlang style concurrency. Once they get the capability in their hands though they realize that they don’t know how to take advantage of the capabilities processes or actors provide. To do this we need to understand how to decompose systems with process based concurrency in mind. Keep in mind, that this material works equally well for actors in Scala or agents in F#. Differences between actors and processes don't much matter for the sake of this discussion. Before we dive into process based design it will be helpful to look at a more familiar approach so we can contrast the two.

If you come from an OO background your natural instinct is to design much like you do when decomposing a problem for OO programming. After all, processes are much like objects in that they send messages to one another and they hold state. It goes something like this

  1. Determine your use cases
  2. Create a narrative of what it is you are trying to design
  3. Run through the narrative and pull out the nouns as potential classes
  4. Do the same for the verbs acting on the nouns as potential methods on the classes
  5. Clean all this up getting consolidating any duplication
    For example, lets say you were trying to build software to run a vending machine. The use cases might be paying for and getting a soda. Another one might be paying too little and getting change back. So one of the narratives there might be

As a customer I put sufficient coins into the vending machine and then press the selection button for coke and then press the button to vend and the robotic arm fetches a coke and dumps it into the pickup tray. The coke is nice and cold because the cooling system keeps the air in the vending machine at 50 degrees.

Now we think about all the unique nouns in our narrative which are: customer, coins, vending machine, selection button, coke, vend button, robotic arm and cooling system. And we generally turn them into objects. Next we consider the verbs that act on those nouns and consider them for methods.

selection button : push

vend button : push

robotic arm : pickup (coke)

etcetera... After this you apply some lovely object oriented design principles and voila - you have a system that nicely models your narrative but does not take advantage of more than a single core on your system and is positively undistributable.

Oh come on you say, don’t be daft, substitute the word object for actor and you are good to go. Well as it turns out not quite. Do we really need a coin process, how about a vend button process and lets not forget the coke process?? Darnit, this makes no sense! Lets back up and see what we can do about this.

Designing for Process Based Concurrency

The first thing you must do before we move on is say this three times

“Processes are not threads. Processes are really cheap. Ohmmmm”

“Processes are not threads. Processes are really cheap. Ohmmmm”

“Processes are not threads. Processes are really cheap. Ohmmmm”

This trips up folks new to process bases systems. They want to be stingy with processes worrying that they will take a long time to create, have massive context switching times, pollute L1 cache, etc… Remember that in almost all such systems, certainly for Erlang, Scala and F# processes/actors/agents are green threads. This means they have their own schedulers built into the VM they are running in. You never have to swap out a thread to switch between running one process vs another. With Erlang based systems you usually configure one erlang scheduler per core on the system. These schedulers remain relatively constant.

With that in mind we solve some of the sticking points many new to process based systems have; not taking advantage of all the concurrency in the system or using complex “process pooling”.

One process for each truly concurrent *activity* in the system.

That is the rule. Going back to our vending machine what do we have that is really concurrent in that system. Coins? Not really. Slots? Not really. Buttons? Not really. Those are not activities they are things. What are the truly concurrent activities, the activities that do not have to happen in synchronous lock step?

  • Putting coins into the slot
  • Handling coins
  • Handling selections
  • Fetching the coke and putting it into the pickup tray
  • Cooling the soda
    We can use a process for all of these activities. We don’t If you want to name them for the nouns that perform the activities - but remember we are not making them processes because they are nouns. Notice how we go granular here, we did not just create a process for the customer and the vending machine. We created one for all the truly concurrent activities in our narrative - in this way we leverage more of the concurrency available to us. Now we know what processes we need, the next step is to organize them.

Organizing Processes

The various languages that use process based concurrency have differing levels of sophistication here. I am going to draw on the concepts from the Erlang language which have been used in the Akka system for Scala and which I have rolled successfully myself in F#.

Again, forget all of your OO modeling techniques. Processes are not objects, they are fundamental units of concurrency. Forget all your thread modeling techniques - it’s not even close. Share nothing copy everything changes the game. To get started think about which of your processes have to cooperate with one another. In this case what do we have.

  • Putting coins in a slot, cooperates with
  • Handling coins, cooperates with
  • Handling selections, cooperates with
  • Fetching the coke and putting it into the pickup tray
    and nothing cooperates with, Cooling the soda - it happens whether or not other processes are there to support it or not. Putting coins in the slot however makes no sense if there is no way to handle them, and handing them makes no sense of you can’t make a selection and making a selection… well you get the drift.

To model this we are going to use a tree of “Supervisors”. Supervisors create and watch over processes. Because of copy everything share nothing properties of actors one can’t corrupt another. So, a supervisor can watch over an actor and restart it when it blows up in the presence of some error. This means we get some incredible fault tolerance. But, that aside, lets talk about how to model these dependencies. We do so in a tree. First, we setup a supervisor at the top of the tree which models no dependencies between any of the processes it starts. In this layer we add the cooling system and then we add another supervisor which will start the group of dependent processes in the order in which they depend on one another. This supervisor will restart processes that die according to their dependencies. If a dependency dies the supervisor will kill and restart dependent processes so that everything starts in a known base state down the chain.

**
proctree

**

Now with things decomposed into processes, dependencies fleshed out and placed into a supervision hierarchy you are basically ready to go. Is there more to design for actor based concurrency - yes of course there is but here you have the fundamentals. Now it’s time to go and play with it and generate questions. Feel free to ask them here or on twitter at @martinjlogan.  I may do a second installment on some more advanced topics based on feedback.

If you want to learn more come to Erlang Camp Oct 10 and 11, 2014 in Austin!

**

**

Erlang Postgres Connection Pool with Episcina

Almost exactly a year ago I was looking to merge the many forks of Will Glozer's Postgres client for use in a project at Heroku. Instead Semiocast released their client, I gave it a try and never looked back. (But note that David Welton, a braver person than me, is working on merging the forks of epgsql at this time). I found Semiocast's client to be clean, stable and I liked the interface better.

At the same time I was in the need of a connection pooler. Many have relied on poolboy or pooler for this purpose but neither actually fits the use case of connection pooling that well. Luckily Eric and Jordan were in the need at the same time and created the Erlware project episcina, which they based off of Joseph Wecker's fork of Will Glozer's epgsql pool. Episcina differs in that it is purely for connection pooling, it is not for pooling workers and it is not for pooling generic processes.

Here I'll show how I combined the two in a simple example.

To start we have a sys.config file to configure episcina:

{episcina, [{pools, [{primary,  
                        [{size, 10},                          
                         {timeout, 10000},  
                         {connect_provider, {pp_db, open,  
                                             [[{host, "localhost"}  
                                              ,{database, "postgres_pool"}  
                                              ,{port, 5432}  
                                              ,{user, "postgres"}  
                                              ,{password, "password"}]]}},  
                         {close_provider, {pp_db, close, []}}]}]  
              }  
             ]  
}  

A key thing to note here is the connect and close providers are function calls to modules within the project and not the Postgres client. Episcina requires a return value of {ok, pid()} and the Semiocast client returns {pgsql_connection, pid()}, so we wrap the connection calls to get around that:

-spec get_connection(atom()) -> {pgsql_connection, pid()} | {error, timeout}.  
get_connection(Pool) ->  
    case episcina:get_connection(Pool) of  
        {ok, Pid} ->  
            {pgsql_connection, Pid};  
        {error, timeout} ->  
            {error, timeout}  
    end.  

-spec return_connection(atom(), {pgsql_connection, pid()}) -> ok.  
return_connection(Pool, {pgsql_connection, Pid}) ->  
    episcina:return_connection(Pool, Pid).  

-spec open(list()) -> {ok, pid()}.  
open(DBArgs) ->  
    {pgsql_connection, Pid} = pgsql_connection:open(DBArgs),  
    {ok, Pid}.  

-spec close(pid()) -> ok.  
close(Pid) ->  
    pgsql_connection:close({pgsql_connection, Pid}).  

And here is the query function to get a connection and return it to the pool after completion:

  
-spec query(string()) -> tuple().  
query(Query) ->  
    C = get_connection(primary),  
    try  
        pgsql_connection:simple_query(Query, [], infinity, C)  
    after  
        return_connection(primary, C)  
    end.  

This example project uses relx to build a release which will start episcina on boot:

{release, {postgres_pool, "0.0.1"},  
  [postgres_pool]}.  

{sys_config, "./config/sys.config"}.  
{dev_mode, true}.  

{include_erts, true}.  
{extended_start_script, true}.  

Boot the release to an interactive shell and play around:

  
λ _rel/bin/postgres_pool console  
([email protected])1> pp_db:query("SELECT 1").  
{{select,1},[{1}]}  

Some Thoughts on Go and Erlang

UPDATE: I'm seeing that I did not make the point of this post clear. I am not saying Go is wrong or should change because it isn't like Erlang. What I am attempting to show is the choices Go made that make it not an alternative to Erlang for backends where availability and low latency for high numbers of concurrent requests is a requirement. And notice I'm not writing this about a language like Julia. I have heard Go pitched as an alternative to Erlang for not only new projects but replacing old. No one would say the same for Julia, but Go and Node.js are seen by some as friendlier alternatives. And no, Erlang isn't the solution for everything! But this is specifically about where Erlang is appropriate and Go is lacking.

I'm going to attempt to leave out my subjective opinions for disliking parts of Go, such as syntax or lack of pattern matching, and explain objective reasons for the language and runtime not being fit for certain types of systems. But I'll start with the good.

Where Go Shines

Clients

As Rob Pike wrote, his biggest surprise was Go is mostly gaining developers from Python and Ruby, not C++. To me this trend has been great to see. No more slow clients installed through pip or gems! (Though for some reason Node.js for clients is growing, wtf Keybase?)

Go provides developers with a fast and easy to use high level, statically typed language with garbage collection and concurrency primitives. It would be great for C++ developers to move to Go as well, the programs that crash constantly on my machine are proprietary C++ that love to misuse memory -- Hipchat and Spotify. But Rob Pike pointed out that the C++ developers don't want the simplified, yet powerful, world of Go. Ruby and Python developers, rightly, do.

Tooling

Getting up and going with building executables depending on third party libraries can be done easily without depending on third party tools, it all comes with Go. While the tools aren't perfect, there are tools to fill in some gaps like Godep, it is still a huge win for the language.

Where Go Falls Short

Some of Go's design decisions are detrimental when it comes to writing low-latency fault-tolerant systems.

Concurrency

Yes, I listed concurrency primitives as a plus in the first section. It is in the case of replacing Ruby or Python or C++ for clients. But when it comes to complex backends that need to be fault-tolerant Go is as broken as any other language with shared state.

Pre-emptive Scheduling

Here Go has gotten much better. Go's pre-emptive scheduling was done on syscalls but now pre-emption is done when a goroutine checks the stack, which it does on every function call, and this may be marked to fail (causing pre-emption) if the goroutine has been running for longer than some time period. While this is an improvement it still lags behind Erlang's reduction counting and newly added dirty schedulers for improved integration with C.

Garbage Collection

In Go garbage collection is global mark and sweep. This pauses all goroutines during the sweep and this is terrible for latency. Again, low latency is hard, the more the runtime can do for you the better.

Error Handling

This isn't just about having no exceptions and the use of checking if a second return value is nil. Goroutines have no identity which means Go lacks the ability to link or monitor goroutines. No linking (instead using panic and defer) and no process isolation means you can not fall back on crashing and restarting in a stable state. There will be bugs in production and a lot of those bugs will be Heisenbugs, so being able to layout processes, isolated from each other but linked based on their dependencies, is key for fault tolerance.

And on top of these major omissions in dealing with faults Go has nil. How in 2014 this is considered OK I haven't wrapped my mind around yet. I'll just leave it at that, with a befuddled look.

Introspection

Not having a REPL is annoying for development, but no remote shell for running systems is a deal breaker. Erlang has impressive tracing capabilities and tools built on those capabilities like recon_trace. Erlang's introspection greatly improves development as well as maintenance of complex running systems.

Static Linking

Yes, another thing that was also in the positives but becomes a negative when used in systems that are expected to be long running. While having no linking does means slower execution it gives Erlang advantages in the case of code replacement on running systems. It is important to note that due to Erlang's scheduling and garbage collecting strategies many of these speed tradeoffs do not mean Erlang will be slower than an implementation in another language, especially if the Erlang implementation is the only one still running.

Code Organization

The OTP framework provides libraries for common patterns. OTP not only means less code to write and better abstractions but also improves readability. Following the OTP standards with applications, supervisors and workers (genserver, genfsm, gen_event) means a developer new to the program is able to work down through the tree of processes and how they interact. Go's channels, unidentifiable goroutines and lack of patterns to separate goroutines into separate modules leads to much harder code to reason about.

Can or Even Should Go Change?

Erlang has been around for decades and Go is the new kid on the block, so can Go improve in these areas? Some, yes, but most of it can not because of the choices made in the design of the language which were not fault tolerance and low latency.

This doesn't mean Go is "bad" or "wrong". It simply makes different choices and thus is better suited for different problems than a language like Erlang.