The Mythical Modular Monolith
What is it and how to build one?
We all know the Microservice trend that has been around for years now. Recently voices started to rise that maybe it is not a solve-all solution and there are other, better suited approaches. Even if, eventually, by evolution, we end up with a Microservice architecture, then there are other intermediate steps to safely get there. Most prominent, a bit controversial in some circles, is the modular monolith. If you follow tech trends you would have already seen a chart like this:
I will not get into details of this diagram, since there are other great resources that talk about it. The main idea is that if are at the bottom left (Big Ball of Mud) then we want to move up and right through the modular monolith instead of the distributed ball of mud. Ideally we want to start with modular monolith and if our product is successful enough potentially move towards Microservices.
The problem I found with those articles/talks is that they talk about the modular monolith but rarely go into details as to what that actually means, as if that was self-explanatory. In this piece I will outline some patterns that can help when building a modular monolith.
What is a modular monolith?
Before we can talk about how to build a modular monolith we need to answer this question. After all, what makes it different from a regular monolith? Is it just a “correctly” written monolith? That is correct but too vague. We need a better definition.
The key is in the word modular. What is a module? It is a collection of functionalities that have high cohesion in an isolated environment (lowly coupled with other functionality). There are various techniques that can be used to collect such functionalities and build a boundary around them, e.g. Domain Driven Design (DDD).
Breaking it down:
- Clear responsibility / high cohesion: Each module has a clearly defined business responsibility and handles its implementation from top to bottom. From DDD perspective: a single domain.
- Loosely coupled: there should be little to any coupling between modules. If I change something in one module it should affect other modules in a minimal way (or even better not at all).
- Encapsulation: the business logic and domain model should not be visible from outside of the module (linked with loose coupling).
A good rule of thumb to check if a module is well written is to analyse how difficult it would be for it to be extracted into a separate microservice. If it’s easy then the module is well written. If you need to make changes in multiple modules to do it, then it needs some work. A typical ball of mud might also have modules but they will break aforementioned guidelines.
Having defined what a module is, defining a Modular Monolith is straightforward. A Modular Monolith is a collection of modules that adhere to those rules. It differs from a Microservice architecture that all the modules are deployed in one deployment unit and often reside in one repository (aka mono-repo).
Integration Patterns
There are a number of integration patterns one can employ when building a modular monolith.
Each one has its strengths and weaknesses and should be used depending on the need. I have ranked them in the level of maturity.
Level 1: One compilation unit/module
The codebase has one compilation unit. The modules communicate between each other using exposed services or internal events. This approach is fastest to implement initially, however as the product grows it will become progressively harder to add new functionality as the coupling tends to be high in such systems. Same applies to ease of reasoning. At first it will be very easy to “understand” the system. As the time flows the number of cases needed to keep in mind will grow as it is very hard to determine what are the relations between domains. Benefit from this approach is that we can quickly deliver initial value while refining our development practices (especially new teams). From a practical point of view the build/test times of such a system will rise exponentially, slowing down development.
Recommendation: Use with a single small team (2–3 people). Ideal for Proof of Concept and MVPs.
Level 2: Multiple compilation units/modules
The codebase has multiple compilation units, e.g. multiple maven modules, one per domain. Each module exposes a clearly defined API. This approach allows for better encapsulation as there is a clear boundary between the modules. You can even split the team and distribute responsibility for each module, allowing for independent development. The readability also benefits from this approach since it is easy to determine what dependencies are between the modules. In addition we can only build and test the module that has been changed. This speeds up development significantly. Requires a little bit more fiddling with build tools but nothing a regular developer couldn’t handle.
Recommendation: Good for a typical product team. Team members can work fairly independently. Could work with 2–3 small teams. This approach will take you far as the implementation overhead is small, while it is very easy to maintain consistency through the code.
Level 3: Multiple compilation module groups
Each domain is split into 2+ modules. This is an expansion on the previous approach. This way we can extract an API module that other domains will be dependent on. This will further enforce encapsulation. You can even employ static analysis tools that ban other modules from being dependent on anything but the API modules. This approach could benefit from the Java Jigsaw Project.
Recommendation: This is ideal when moving from a medium sized product to a large product where 2+ full size product teams are needed. Each team will expose their API module that the others can ingest.
Level 4: Going web: Communicating through network
Same as Level 3 but modules are totally independent (no shared API module) and communicate using the network (REST/SOAP, queues, etc). This is an extreme step. One that should not be taken lightly. You lose compile time checking on the APIs and gain multiple problems related to networking. This allows a very high decoupling of the modules as there is no shared code (apart from some utils, etc). When deciding to take this step it means that we are nearing a Microservice architecture.
Option A: Having a single deployment unit. It might seem weird to call a REST API when everything is deployed in one unit but this approach does allow for better load distribution. This is especially possible when using a queue like Kafka for communication. However I agree that in most cases this is a redundant approach. It is a good stepping stone when moving to Option B.
Option B: Separate deployment units. This is pretty much the final move from a modular monolith to Microservice architecture.
Level 4.5: using separate repositories (aka moving away from monorepo), CI/CD pipelines, etc
Moving to full blown Microservice approach. Not going into detail as this article is not about Microservices.
Recommendation: Large/Multiple Products, many development teams, efficient DevOps culture, mature company.
Summary:
You might have noticed that Cost of Maintenance is never low. That is correct.
You can not hide complexity. You can only change its location.
Contracts and Testing
A very important aspect of modular monoliths is to treat the APIs or domain boundaries (call it as you wish) as Contracts between domains/modules that need to be respected. This might seem obvious but it is easy to fall into a trap in a monolith where the module APIs get treated as second class citizens. When designing and maintaining them we should think of them as if designing a REST API. Changes to them should be done carefully. They should have a proper set of tests. This is key when there are multiple teams cooperating.
One of the more common issue is no clear distinction between what team is responsible for which module. Responsibility for APIs becomes blurred and their quality drops rapidly. Each module should have one team responsible for it. Ownership is key. The API of this module should be a contract between that team and the teams that use it. That API MUST BE covered by tests. Only the team responsible for that module can introduce changes to that API. This does increase communication overhead and extends the time of introducing changes but keeps those factors constant instead of spinning out of control.
Conclusion
I hope those make the Modular Monolith a tiny bit less Mythical. I think we developers like to over-complicate things while in truth most software engineering comes down to a few basic principles. There are a few more topics I think would be worth discussing regarding the Modular Monolith (tooling, architecture as code) but I think this gives a good starting point. The most important takeaways are:
- Encapsulated modules with high cohesion and low coupling
- Ownership is key
Keep those in mind and you will get far.
