The Great Migration: Peculiar Complexities of Monolith to Microservices Migration

There are peculiar complexities that threaten the success of monolith to microservices migration projects by making them difficult and costly to undertake. 

Unfortunately, you cannot hide or ignore these complexities.

Even if, somehow, you manage to complete a migration project by evading the complexities, the party wouldn’t last long.

You will end up getting haunted by complex ghosts for years to come.

Let me explain why I am saying this?

Before I do that, let us set the context right. So that we are on the same page.

Let’s assume you want to migrate a monolithic ERP system to a microservices architecture.

• You aspire to utilize the cloud-native infrastructure.
• You hope to bring in latest infrastructure into play.
• You want to go serverless.
• You want to add more bells & whistles to microservices.
• You believe in the promise that your costs of operations will come down once you have created these new services.
• And that the new application will be easy to maintain & scale.

Fair expectations.

Nothing wrong with these.

But it is not the curse of the shiny things where your problems will start.

Not that the promise of microservices was oversold, it is just that you did not prepare for the real complexities that you were going to face to fulfill the promise.

Enough of the scare.

Let’s see how you would typically approach a monolith to microservices migration of an ERP system. And then we will see what exactly the complexities are?

The Typical Approach

Typically, if you have got it right, you will undertake the migration using what is called – strangler pattern.

You will start by analyzing the ERP system modules.

You will then design a logical breakdown of the modules into probable services.

Then you will identify the first batch of modules that can be moved out. The obvious choice will be the ones with the least dependencies.

You will rewrite the features of these modules as microservices. Then let microservices shadow the modules for some time.

And once the microservices are stabilized, strangulate/disconnect the original modules.

You will continue this process for other modules, till all of them have been moved to microservices.

Easy-peasy-lemon-squeezy! Right?

Not quite.

There are Complexities &…

All along you did not realize that sneaking behind this innocent approach of strangulation, there are monsters of complexity ready to blow your head off with nightmares and unending pain.

Here is what you need to know about these monsters.

• The feature transfer from old architecture to new distributed architecture is not as straightforward as you think. It is a paradigm shift monster. You will have to rewrite the old features using different event-driven patterns. While making sure in the process you are not losing the essence of the functionality. (actually, you can’t lose any functionality)
• If you design service boundaries wrong (which is quite likely), it will lead to very messy microservices. And they will be a source of constant pain for years to come. To save yourself from this future agony, you will have to spend a lot of time, effort, and attention on domain design. But isn’t the domain already designed in the ERP? Yes & No. Did you see this one coming?
• You will be swarmed with plumbing code for the new and old infrastructure making it challenging to move out from old infrastructure bindings. When regression issues appear, your costs start shooting through the roof.
• In the process of handling these complexities, you end up compromising on the time needed to rewrite features using scalable and extensible design patterns. Many a times you will be forced to copy old patterns & code from the legacy application. This will make you wonder if the entire process was worth it.

The crux of the matter is this – when you had told us that you will lead the way to microservices, knowingly or unknowingly you were promising us that you will provide a better architecture, better code & better design.

So, not fulfilling that promise makes you look bad. Very bad. Isn’t it?

Yes, there is this promised land of microservices. But you must make sure that you are ready to make the journey.

How the complexities play out during development

Nevertheless, you decide to jump into it – the blazing guns of glory. What happens next?

Let us go a little deeper to see how your Mad Max movie plays out in the real world of project deadlines.

• First off, you are bumped into spending a lot of time in domain design.

Because if you did not do that, you know you will not last long.

But the project sponsor loses patience when you tell them after a month that you have not written any useful line of code except for the design. You know what will happen next.

• You will be iterating the newly designed features a lot. You will be repeatedly designing, redesigning, and testing the new event-driven avatars of the old features. This will eat up your time.

“Are we there yet? Are we? When will it be done?” such questions will loom over your head every day. Then every few hours. And then they will get louder.

• You will be designing the architecture of the new application before writing any code. You better be. It cannot be an afterthought.

See, you are not writing an application from scratch. You are turning an already working healthy application into microservices, whose only sin in this world seems to be that it is a monolith.You cannot leave the “architecture design” for later consideration when the application has scaled to a large number of users, blah blah blah. Nope. You deserted that option long back.

• You will be writing a lot of plumbing code for the new event-driven features.

Soon you will run into standardization issues, naming problems, and more. You will pull your hair off when names start conflicting. Remember _customername_ can be _custname_, _cusname_, _custnm_ or _cusnm_?

And that is just domain names. Imagine this happening to your plumbing code. That’s the California wildfire.

• Your fascination for cloud-native infrastructure will make you write even more plumbing code and things will start getting weirder.

So here is the thing, while the above-mentioned work is doable, it seems like you will have to throw the budget, time & quality constraints out of the window.

Then the million-dollar question is:

Is it practical to undertake a monolith to microservices migration at all?

Yes, it is. And that is the reason I have written this article. This is not a doomsday article.

Here is what you can do to make it happen.

Proposed Solution

The problem with microservices migration is that there is a sheer lack of tooling that can help developers to solve these complexities elegantly.

It is like writing an ERP software using Assembly code.

So, what can you do?

This is where I would like to suggest an approach any team can follow,

• You should put together a distributed event-driven architecture design with the necessary infrastructure at the very outset. Make sure you have this design ready before you make any commitment on the migration.
• There should be some tooling where you can quickly iterate and test the event-driven avatar of the new features. And be able to redo the features correctly all over again without much hassle & delay.
• Work out design patterns for loose coupling of components for the event-driven architecture. What is the point of an event-driven architecture if you end up making tightly coupled components?

And because you will need these patterns again and again, find a way to templatize them. Even if somewhere you must bypass these patterns (due to project pressures of course), you should be able to refactor the code later using the templates.

• Set coding standards. Especially for the plumbing code. 

Or, even better, figure a way to auto-generate the plumbing code.

Let me give you a hint – if your architecture design is set, then you know that your plumbing code will follow a certain repetitive pattern. Therefore, around 80% to 90% of plumbing codes can be auto-generated.

• Introduce some common services and tech early on like, single sign-on, API gateway & containerization.

If you can do this, then what is left is the work that you really wanted to do in the project. That is following the strangler pattern and migrating to microservices.

Summary

I hope that I was able to help you identify the complexities.

And I hope this will help you execute your microservices migration projects carefully.

Hope you will be better prepared & avoid many pitfalls.

The solutions that I have mentioned above can be cooked up using exiting development tools and implementing tight controls in projects. Though it might be cumbersome, you will be able to complete projects.

But if you want to do it all elegantly, you might want to consider Flexbase.

Flexbase provides dev tools and an event-driven reference architecture for handling complexities in an elegant and proven way. You save time and money and get to do the real stuff. Your projects become a smooth sail.

For more details check out www.flexbase.in or connect with me on LinkedIn.