In the first part of the blog, we looked at Domain Driven Design (DDD) from a practitioner’s point of view.
- We considered how DDD helps break down complex systems into individual Bounded Contexts
- And then we discussed how Bounded Contexts are model as self-contained microservices
- Finally, we saw how microservices can communicate with each other using Domain Events and a messaging middleware
Using Domain Events as a means of communication helps our microservices remain self-contained and loosely coupled. However, many of our clients have long-running business processes that span multiple microservices.
Let’s consider the example of Billing System again. We have various microservices – Vendor Management, Billing, Approvals and Payment. Each microservice communicates significant business events that occur in their respective Bounded Contexts via Domain Events.
- The microservice’s responsibility ends once the Domain Event is emitted
- After emitting the Domain Event, microservice does not wait for any response, or worry about what happens next
- And one or more microservices subscribe to these Domain Events, and take action when they receive them
- For example, from our earlier solution, Approvals microservice will check for “New Bill Added” Domain Event and trigger an approval workflow
- And Payment microservice will look for “Bill Approved” Domain Event and initiate payment when it receives one
This is a good example of “Choregraphed Microservices.”
Now consider there is a new business requirement: any bill less than $1000 is automatically paid, and does not require any approval. To implement this requirement, we need to make changes to:
- Approvals Microservice, so it does not take those bills through approval workflow
- Payment Microservice, so it now starts to check for “New Bill Added” event, and initiate payment when it receives one that is less than $1000
While this is a simple example, imagine a complex system that has several hundreds or thousands of microservices. Implementing requirements such as these will result in modifications to multiple microservices. Above all, when a business process owner is looking at this, it becomes hard to understand the overall business process flow, and more difficult to make changes.
As many have described such systems in the past – when you build such a complex system with thousands of interconnected Microservices, you end up with a Microservice death-star.
“The danger is that it’s very easy to make nicely decoupled systems with event notification, without realizing that you’re losing sight of that larger-scale flow, and thus set yourself up for trouble in future years.” – Martin Fowler
We can overcome simple problems like the one described earlier using better designs. For example, have the Billing Microservice emit “New Bill under $1000” or “New Bill over $1000” event. And then, Payment Microservice can listen to “New Bill under $1000” event to initiate the payment. Now, the Approvals Microservice can listen to “New Bill over $1000” to initiate the approval workflow. However, such design approaches will be harder to implement and manage at scale. Add to this a requirement for what happens when the payment fails, who should retry etc.. These are some of the key challenges when it comes to designing distributed systems using Choreographed Microservices .
An alternative to above approach is “Orchestrating Microservices.” Having a microservice whose responsibility is orchestrating business process flow across various microservices helps address several of our requirements and concerns detailed above.
Orchestrating Microservices help with the following:
- Coordinate communication between various Microservices in the domain
- Localize changes to business process rules. Avoid unintended impact on multiple Microservices
- Provide end-to-end business process view
- Help maintain the single responsibility design principle
The above example can be modified to have a “Vendor Payment” microservice that coordinates between other microservices.
There are a number of options available to design and implement an Orchestrating Microservice such as the one above. Based on your use-case and the complexity of requirement, you can chose from a simple state machine to a workflow engine.
Common Pitfalls and Anti-Patterns
We need to address cautious against common pitfalls and anti-patterns while implementing Orchestrating Microservices:
- Avoid turning your system into a distributed monolith
- Concentrating business logic more than needed within the Orchestrating Microservice will result in anemic domain models
- Don’t build custom state machines / workflow engines unless there is a strong need
Key takeaways, and a checklist to determine the right solution option
- Start with choreography
- BPM Acid Test: BPM is suited only for applications with an essential sense of state or process—that is, applications that are process-oriented
- Persisted state
- Bursty, sleeps most of the time
- Orchestration of system and human communications
- Ownership of the workflow stays within the Bounded Context
- Polyglot state machine / workflow engine based on your requirement
- Choose from simple state machines to more sophisticated BPM Workflow engines based on the bounded context
- Decentralize state management
- Use Workflow Engine just to coordinate / orchestrate, and don’t build implementation logic into it
- Consider lightweight and embeddable state machines / workflow engines as first choice
- Stay compliant with latest BPMN 2.0, DMN, CMMN standards
- Choose state management / workflow solution that satisfy your organization’s non-functional requirements of
- DevOps Integration
- Performance and Scalability
Find the right balance
From my experience, leveraging the right blend of Choreography and Orchestration helps develop solutions that can be scaled, while providing end-to-end business process visibility. As Martin Fowler explained, you don’t want to be building a highly decoupled and autonomous microservices system that does not provide you the right business process visibility, making it harder to adapt to future requirements.
In Part 3 of the blog, will explore DDD Aggregates and how to avoid anemic Domain Models.
References, and additional reading:
- Microservices by Martin Fowler
- Complex event flows in distributed systems by Bernd Ruecker
- Essential Business Process Modeling
- Microservices Death-star