Chris Richardson of microservices.io and author of the book Microservices Patterns discuss microservices patterns which constitute a set of best practices and building-block solutions to problems inherent in building applications out of small coordinated services. Host Robert Blumen spoke with Richardson about the evolution of microservices, community adoption of best practices, patterns for inter-service communication (synchronous and asynchronous), async and high availability, the saga pattern and transactionality, transactional messaging, at most once delivery, and eventual consistency; the API gateway pattern and aggregation, latency and mobile devices, patterns for testing microservices, contract-driven testing, what does the Jenkins pipeline look like, production readiness patterns (log aggregation, metrics, distributed tracing), externalized configuration, and wrapped up with a few of Chris’s favorite anti-patterns.
Show Notes
Related Links
- microservices.io web site
- Cloud Foundry
- Microservices Patterns book by Chris Richardson
- Microservices Anti-Patterns slide deck by Chris Richardson
- O’Reilly keynote talk by Chris Richardson Potholes in the road from monolithic hell: Microservices adoption anti-patterns
- https://twitter.com/crichardson
- [email protected]
SER shows
- 213: James Lewis on Microservices
- 351: Bernd Rücker on Orchestrating Microservices with Workflow Management
- 337: Ben Sigelman on Distributed Tracing
- 210: Stefan Tilkov on Microservices and Architecture
- 219: Kafka with Jun Rao
- 215: The Gang of Four on Design Patterns
- 189: Eric Lubow on Polyglot Persistence
- 232: Mark Nottingham on HTTP/2
- 220: Jon Gifford on Log Aggregation
- 70: Gerard Meszaros on XUnit Test Patterns
Transcript
Transcript brought to you by IEEE Software
Robert Blumen 00:00:58 For software engineering radio. This is Robert Blumen today I have with me, Chris Richardson, Chris is an experienced software architect and consultant active in the field of microservices. His startup eventuate.io provides a platform for the development of transactional microservices. Chris is the publisher of the website, microservices.io, and the author of the recently published book Microservice Patterns, which we will be talking about today. Chris, welcome to software engineering radio.
Chris Richardson 00:01:34 Well, thanks. I’m pleased to be here,
Robert Blumen 00:01:37 Chris, would you like to tell the listeners anything about yourself that I didn’t cover in the bio
Chris Richardson 00:01:43 Just about covered it? I mean, I, these days I, as well as working on my startup, I basically travel the world, helping clients, um, adopt microservices through a combination of consulting and training.
Robert Blumen 00:01:57 Let’s get onto Microservice Patterns. We have done a number of other shows on microservices, including number two 13, uh, two 10 on architecture and micro services and three 51 on orchestrating microservices. So we’re not going to do an in depth review, but tell the listeners, what does micro services mean to you?
Chris Richardson 00:02:21 So today, you know, like software technology or the ability to deliver software rapidly, frequently, and reliably is becoming essential to most, if not all organizations, right. You know, the whole concept of software is eating the world. And so in order to deliver software rapidly, frequently and reliably, you need a combination of, um, should we say three things first is you need to embrace DevOps, right? Set of practices, essentially a software development. You need to structure your engineering organization into small autonomous teams. So cool. Two pizza teams. So though that’s somewhat of a imprecise metric, and then lastly you need an architecture that supports all of this. And on the one hand, if you’re building relatively small applications, you can probably get away with using the monolithic architecture. But if you’re, if you’ve got a large number of developers and you’re building a large complex application, chances are you actually need to use the microservice architecture. And essentially the microservice architecture breaks up. What would otherwise be a monolith into a set of relatively small, but not tiny services that can individually be developed and tested and deployed and scaled by, by the team that owns them,
Robert Blumen 00:04:01 Want to move on now and talk about design patterns or Microservice Patterns. When the book design patterns came out some 20 years ago, I believe the authors did not see themselves as really originating these ideas. They were more documenting solutions to problems that had been discovered and proven themselves over time with your work on Microservice Patterns. What is your assessment of the state of the community in terms of adoption? Is there an agreed upon set of best practices or is this more your view of how things should be done?
Chris Richardson 00:04:40 I would say my book is kind of a combination of both. I mean, you’re right. I mean, in many ways there’s no such thing as a, a new pattern. You know, it’s really a kind of essentially a labeling and a more formal description of something that has existed within whatever field you’re studying for some period of time. And it has demonstrated itself to be generally useful. So sort of, you know, so a lot of identifying patterns is really just mining what people are doing, perhaps in some cases, giving them a name and then kind of trying to distill down a description of the solution, but then also describe the benefits and the drawbacks of that particular approach. So it’s sort of a way of structuring something that has existed out there already.
Robert Blumen 00:05:50 Would it be fair to say then these patterns become a toolbox for architects to build their own microservice architecture?
Chris Richardson 00:05:59 Yeah, I think that that’s a good way of thinking about it. I mean, one of, one of, sort of, one of the key ideas with patterns in general and say, and sort of my book in particular is that when you apply a pattern, right, and the pattern, you know, the definition of a pattern is a reusable solution to a problem. You apply a pattern that actually solves some problem, you know, in the case of the microservice architecture, the problem you’re trying to solve is how do I structure my application in a way that requires rapid, reliable and frequent delivery of software, but at the same time that actually it invariably it creates sub-problems. Um, for instance, you know, with a microservice architecture, uh, one obvious sub problem is, well, how do you break up your system into a set of services? And then there’s also a whole bunch of patterns related to why you’re now building a distributed system.
Chris Richardson 00:07:04 And so there’s some various problems around that. And so it’s essentially your, you apply one pattern that creates sub-problems, which you then apply other patterns to solve and effectively you’re sort of, you know, you’re doing that recursively so to speak. And so these patterns form a web of interrelated patterns. And so when you are creating an architecture, you’re essentially traversing this graph, selecting patterns, and then selecting patterns to the sub-problems and so on and so forth. So it actually kind of provides a structure to this problem space and, and tells architects what problems they they’re going to, they’re going to encounter, but then also how to solve them,
Robert Blumen 00:08:03 Given that design patterns there, the solution to a problem. What are some of the problems posed by microservices that you don’t have with the model, if it require a new and different set of patterns to solve?
Chris Richardson 00:08:18 Yeah. So first problem is what services am I going to have? You know, it’s like, that’s the key question? You know, it’s not, Oh, you know, which deployment technology are we going to have? Should we use serverless versus dark or anything like that? In reality, a lot of those technical issues that people tend to focus on a much more like a detail. So number one is what services you’re going to have. And then once, and there are some patterns for that. And then once you’ve, you’ve up with a set of services, it’s like, Oh, well, one of the key, you know, one key aspect of the microservice architecture is that each services data is encapsulated within that service. And that immediately creates two sub-problems. How do you implement transactions, that span services, and then how you, how do you implement queries that span services? And so that then leads to the saga pattern, which is a way of managing transactions. And then there’s a couple of querying patterns, API composition, and CQRS that solve the querying part of it. So that’s sort of like, you know, there’s sort of, if you think of it as then, like why it’s kind of a graph or a tree that’s one major sort of sub branch that you have patterned solution problem solution pairs that you end up going down,
Robert Blumen 00:10:00 Summarize you’re drawing attention to, uh, going from a centralized data to distributed data, possibly multiple data stores as introducing a whole set of problems that the architect has to solve. Is is that correct? Yeah. Yeah. Okay, great. Just wanted to summarize that.
Chris Richardson 00:10:20 I would say a point of clarification is on the one hand services might actually have in a, in a very literal sense, their own database server, but quite often, all you need is sort of, um, kind of a logical separation of data. So like, uh, you could have a shared relational database server and then w on that server, each service can have its own database schema. So logical division. Now, obviously it all depends on the scale and the scale and complexity of your application, but I don’t want anyone who’s listening to go, Oh my God, I have to buy 10 X as many name, your database, vendor licenses in order to do microservices.
Robert Blumen 00:11:11 What I think you’re talking about is in a monolithic application, there could be one single schema. If it is a relational database, I’m free to write a query that joins the user table to the orders table, to the product table, to the product description table. And there’s no limit on how big of a SQL doing I can write as long as it will run, but in microservices at minimum, you might have user orders product and their own microservice, and they might still all be on the same server, but you’re not going to allow sequel to join across those boundaries because they’re a minimum, it’s a logical boundary. Is that where you’re going with that point?
Chris Richardson 00:11:56 Yeah, that’s exactly right. The, the, the, the tables of a service are essentially equivalent to the private fields of a class. They are, they are, it’s part of its implementation and they are hidden from the outside world. And they’re only accessible in directly through that services API.
Robert Blumen 00:12:18 Great. So we’ve been talking about a set of problems that are caused by going from more of a distributed or partitioned data model. I believe you had another set of problems in your graph of, of problems that you were going to highlight that that are specific to the microservice architecture.
Chris Richardson 00:12:40 Oh yeah. So another sort of class of technical problems are, well, I’ve got a bunch of services and how then, how do they communicate now? On the one hand interprocess communication is, is sort of happens in a monolithic system because it, a monolithic application has to communicate with other applications. But I would argue that the, sort of the degree of complexity around that is significantly higher in a, in a microservice architecture, because the part, the services that make up the application having to communicate with one another. So it’s sort of, interprocess communication is much more central to your application architecture. And so there, there’s this sort of pattern, basic patterns like, should you use asynchronous messaging versus synchronous communication mechanisms such as rest or GRPC. So that’s, that’s sort of one sort of sub problem that you have to address.
Robert Blumen 00:13:52 So let’s move on to talking about what some of the patterns are. And since we’re talking about communication, you do introduce these different families of communication patterns between microservices, synchronous and asynchronous break down. When, when do you choose one? When do you choose the other and what are the strengths and weaknesses?
Chris Richardson 00:14:15 Yeah. So on the one hand, let’s just say, use it rest. Is, is it in a sense the defacto communication mechanism, right? Yeah. Jason, or to me more precise, Jason, over HTTP, you know, that super familiar, everyone, you know, it obviously works with browsers, but then it’s just super familiar. So what, but on the one, so on the one hand that is kind of simple to use, but on the other hand that actually introduces tyke runtime coupling between the services. Yeah. So if, say the order service implements, arrest, rest API, and then it, while handling one of those requests, it has to make a synchronous call to the customer service and wait for response to come back. Then in that situation, the there’s temporal coupling between the two, the order service can’t send back a response unless the customer service can send back a response.
Chris Richardson 00:15:17 And suddenly the availability of that end point is the availability of one service to the power two. And if you end up with more complex architectures and more complex patterns of interaction, the availability of your system or overview or API starts to drop off dramatically. And so for those reasons, it’s, it’s actually preferable to rely on asynchronous communication as much as possible. And then in fact, a common anti-pattern is to go, well, we’re just going to break our system up into a bunch of services, and they’re going to communicate using HTTP right. Or rest. And, you know, there, you’ve basically built a system that’s tightly, coupled at runtime with multiple points of failure
Robert Blumen 00:16:10 In your book, in your book, you introduce a recurring example of the food to go software as a service. Could you briefly describe that example and talk about a couple of points in food to go that would use synchronous and another one or two that would use asynchronous and why you choose one or the other?
Chris Richardson 00:16:34 Oh yeah. So it’s an online food delivery application, you know, along the lines of Uber eats or door dash or deliver Ru. And, you know, my first book pose an action, which came out 13 years ago, had the same example, but it was, it was a monolith. So the scenario envisaged in microservices patterns is, well, 10 plus years have gone by and, and the, and the food to go application has outgrown its monstrous or kind of it now has a monstrous monolith and is having problems delivering software rapidly. I’m sure we can all relate to that. Yeah. It’s, it’s an incredibly common scenario. Right? So, so, so in the case of the microservice version of that application, we take a request like placing an order. Several things have to happen when an order is placed, you have to do some validation around can the consumer place orders is the order valid from the restaurant perspective is the restaurant kitchen willing to accept orders at this. And then you also have to charge the consumer’s credit card as well. So in the monolithic application, conceptually, all those happen within sight, the monolith, but in the microservice architecture, the order service ends up collaborating asynchronously with numerous services, such as the consumer service, the kitchen service and the accounting service. And it’s also subscribing to a stream of events that are coming out of the restaurant service so that it has a replica of the restaurants data in its own database. So it doesn’t actually have to interact with the restaurant service.
Robert Blumen 00:18:30 I’m going to not go through every single one of those, but I’m going to pick one. Why would the collaboration with the accounting service be asynchronous?
Chris Richardson 00:18:39 Well, one way to think about it is sort of the model is so there’s a rest it, so the order service has a rest API to create an order that API actually returns immediately saying essentially saying the request to create this order has been received. Check back later to see what the status of it is. And all of the processing required to basically fight will validate and finalize that order, um, happens asynchronously and including charging the consumer’s credit card. Now in the normal case that everything should happen within, let’s just say, tens of Millis, well, maybe a hundred within a few, couple of hundred milliseconds or something like that. So from the user’s perspective, the order could actually be processed instantaneously. But the nice thing is, is what one part of the system is temporarily down. Orders will still be accepted just that the processing of the might be delayed briefly.
Robert Blumen 00:19:49 Well, you’re driving out then is the decisions you make about what’s synchronous and asynchronous will have an impact on the availability of the system and its ability to do work and accept business requests from the customers.
Chris Richardson 00:20:05 Yeah, you end up with, with sort of asynchronous communication. The, the general availability of the system is higher. It can tolerate, it can still, in this example, accept requests to create orders. Even if part of the system is temporarily unavailable.
Robert Blumen 00:20:27 I can see out from this conversation, when you talk about one thing, it’s hard to completely separate that from everything else. While we’re talking about communication between microservices, you’re introducing issues around transactionality. And I do want to now explicitly move on and talk about transactionality and how it’s different in microservice application. You mentioned the saga pattern. Who’s what, what problem does the saga pattern solve? And, and how does it solve that?
Chris Richardson 00:21:01 Basically the saga pattern is a way of implementing transactions, that span services. So within a service, you have, you know, traditional, assuming you’re using a relational database, you have traditional acid transactions, but between databases, it’s sort of, you could say that using asset transactions in the form of two phase commit or X a has fallen out of fashion for a variety of reasons, including that two phase commit is a form of synchronous communication, which as mentioned earlier, impacts the availability of your system. So the idea with sagas is that instead of having a transaction that spans multiple services, um, to phase two, to PC transaction, that spans multiple services, you break it down into a series of local transactions. What, in each one of the services that contains data that needs to be updated. So in the KLF, we simple, simple example that I use is imagine like in order to create a, create an order, you have to check and reduce the customer’s available credit.
Chris Richardson 00:22:26 Yeah. Imagine it’s like, there’s a business rule that says the customer’s credit customer has a credit limit that can never be violated. So in this example, you need to create an order and reduce the available credit so that there’s up to those updates that need to be, need to happen in both services. And so the way that’s done with a saga is you could forget, for instance, you create an order in one service in the order service in the pending state, that then sends a message to the customer service to reserve credit. And then the customer service sends back a message to the order service saying that credit was approved was, was reserved, or maybe there was an insufficient credit, and then the order service can approve or reject that order. So, so you’ve broken it down into a set of steps that are local with messaging as the coordination mechanism between them. And so that’s sort of an event it’s essentially, that is an eventually consistent model for, for transactions.
Robert Blumen 00:23:38 If we were in a monolith, I could do some code and it might not look exactly like this. If you’re at the code, because you have all this propagation of transactions through different layers, but we’re still on the same process. So I could do begin transaction, save the user debit, the credit, insert the order, insert the charge, information, insert delivery, dress, and commit. And it’s very, it goes, well, it’s all going to be saved. The worst cases, it wasn’t saved, but we don’t have this problem of we debited the credit and we inserted the delivery dress, but no order was created. It’s at least we can say it’s all or nothing. Now in the microservice world, you’re talking about these things being in different services and some might fell some nine succeed, and you’re solving that with the saga pattern. It seems to me for this to work, you need to ensure that, uh, when you commit something that you’ve committed the message atomically along with the work that each microservice has done locally, and that you’re sure that the message will get delivered. Am I correct in going that direction?
Chris Richardson 00:24:45 Yeah. It’s, it’s funny. We’re like digging, getting deeper into this graph or problem problems solutions, right? We’ve gone from the microservice architecture to the database per service pattern to the saga pattern. And now we’re in this, what kind of sub problem area, which I call transactional messaging. And then now, interestingly, so yeah, as you identify this, each step of the saga needs to do two things. First, it needs to update the database, create an order or update the customer’s credit. And then it needs to send a message to say that it’s done that. And ironically, in a traditional application, you might use a distributed transaction for that, that span the database and your message broker, but that’s essentially the kind of technology that we’re trying to avoid. And so in order to atomically, update the database and publish a message, there’s two patterns that you can use. So one pattern is the transactional outbox pattern. So there’s this pattern as part of the asset transaction that updates the database, it inserts a message into an outbox table. So that’s just done atomically, right? And then there’s a second step that is retrieving the message from that outbox table and publishing it to the message broker. And that, that will give you the atomicity that you need along with the, and the guarantee that eventually that message will be published to the message broker
Robert Blumen 00:26:31 And the message broker is some technology like Kafka or rabbit MQ or something that handles the delivery of messages between the services. Correct? Yeah.
Chris Richardson 00:26:44 So the message broke a needs to provide certain guarantees. It needs to give you at least once delivery. It also needs to present a guarantee or give you ordered delivery. The, these sort of mass, these messages or events need to arrive in the order within which they were published. Otherwise things kick out really confusing. And so there are certain I, and that’s another problem where you need to somehow scale up consumers while also guaranteeing delivery. And so message brokers that provide all three out of the box include CAFCA and active MQ. And then to scale out the consumers with other, with a couple of other brokers, it was rabbit MQ, and also Reddis red, a stream specifically, you know, the open source product that, um, project that I work on eventuate, as you mentioned, eventuate.io, we ended up building an app, some extra code on top of those message brokers to enable scaling out consumers properly.
Chris Richardson 00:27:54 Digital ocean is the easiest cloud platform to deploy, manage, and scale applications of any size removing infrastructure, friction, and providing predictability. So developers and their teams can develop faster and focus on building software that customers love with thousands of in depth tutorials and an active community. We provide the support you need. Digital ocean stands out of the crowd due to its simplicity, high performance and no billing surprises, try DigitalOcean for free at dot co slash S E radio. We have a show coming up on messaging architectures, which it doesn’t have a show number yet because it hasn’t been published, but it will be published by the time people are listening to this. And it will go into to some of these issues in interest of time. We won’t by any means, be able to cover even all the major areas in your book. But I wanted to hit on some stuff that I thought was the most interesting. I wanted to move on to another area, which is introduced near the beginning of querying. You introduce the pattern of the API gateway. What is that pattern and what problem does it solve?
Chris Richardson 00:29:08 So the API gateway is one of the communication patterns and the area that it’s focused on is the, what I call external API. So in other words, what API does the application exposed to, to the outside, which might be literally the outside, outside of the firewall, or it could just be other parts of the organization.
Robert Blumen 00:29:35 I have, I have a hundred microservices. They may not all be public
Chris Richardson 00:29:39 Correct. So the idea with the API gateway is that it’s similar to the facade pattern and object oriented design. It defines the public interface of your application. So clients talk to the API gateway and then it’s responsible for implementing each say rest endpoint by invoking the appropriate services. It generally does three things. One, it will, we’ll just simply function as a reverse proxy. So routing, routing each incoming request, like create an order. It will route that to the order service. It could, it will also implement edge functions, such as security, potentially other things like rate limiting and so on. And then it might also implement API composition. So imagine that a client wants to know the status of an order. So that order status could actually be scattered across multiple services. So the order service would know some parts of the order status. The kitchen service would in the food to go application would know about the status of the order at the kitchen.
Chris Richardson 00:30:59 Like when is it in the middle of being prepared? Is it, you know, what, what’s the estimated ready for pickup time? And then the delivery service will know about the estimated delivery time. And then perhaps if it’s in the mail, if it’s actually, if the, if it’s in the process of being delivered where it is, and then say the accounting service would know other things. So it’s like the status is scattered across in that example for different services. So rather than the client making four different requests, it just says to the API gateway, give me the order status. And then the API gateway goes and makes requests to each of those services gathers back to the four responses and returns the response to the client. Um, so, and that has two benefits. One is a sort of performance or efficiency aspect, particularly if it’s like a mobile device trying to making a request over a low, lower performance, higher latency mobile network, but it also implements. It basically is encapsulating the internals of the, of the architecture. The, so the, so the clients are not aware of the individual services, which, which is a good thing because the architecture can evolve.
Robert Blumen 00:32:22 I think one of the things you’re talking about in this case is a issue. You identifying the book by the name chattiness yes. Go into more. What, what is chattiness?
Chris Richardson 00:32:33 So essentially there ends up being that it’s often a mismatch in granularity. Uh, the data that needs to the client wants to render on, on a page or a screen and the fine grain nature, uh, or relatively fine grain nature of your services. So in other words, especially on the, on a, with a desktop, you want user experience, you think about like the Amazon detail product detail page is displaying a heck of a lot of information, basic product information, delivery estimates. When you last ordered it, perhaps whole bunch of, you know, reviews two or three different kinds of recommendations, seller ranking, so on and so forth. Right? You could imagine that each of those individual pieces of data resides in a service. And so one page can easily correspond to potentially tens or more services. And so if the client had to get each one of those pieces of data individually, that results in a lot of round trips, which for certain types of clients, especially mobile devices would, would be generally not a good idea, right? It’s potentially poor user experience. There’s the risk of like, you know, draining the battery faster and so on. So the idea is with the API gateway pattern is that each client can be given its own optimized API, which in some cases, I mean, applications I’ve worked on, it’s almost like for each screen there isn’t, there is a dedicated API end point that so the client can just, so give me the data and it gets back a blob, you know, relatively large blob of data. That’s basically a joined from or composed by calling multiple services.
Robert Blumen 00:34:36 So in the model, if you would do a lot of this integration in sequel where you’d hit an end point and it would go and join a bunch of tables together, it sounds like what you’re doing here is you’re moving that joining function into this API gateway and you put the long haul hop in front of that. And then it’s going out in the data center and doing super short hops to pull all these chunks together and merge them.
Chris Richardson 00:35:04 Yeah, that’s essentially right. And so yeah, one place to do that is in the API gateway or the API gate rec gateway routes to a service that implements this composition.
Robert Blumen 00:35:16 Another issue you bring up is consistency, which you don’t have in the model. It’s because you’re always doing commits and leaving the database in a nice, consistent state. Do you have a problem where this API gateway is going and hitting all these services and it gets back things that don’t all quite make sense because they reflect things that are in different States of convergence toward a consistency.
Chris Richardson 00:35:45 So essentially, yeah, in a monolith, you retrieve data, you’re most likely going to do that within a transaction that will give you a transactionally consistent view of the data. Whereas in a microservice architecture with API composition, multiple transactions, one in each service are being executed. And so you theoretically could get sort of an inconsistent view of the data.
Robert Blumen 00:36:18 Is there a solution to that or is that a trade off that you need to make to get these other benefits?
Chris Richardson 00:36:26 I mean, fundamentally that that’s a consequence of having split the system up in the way that you have chosen to split it up. I mean, one of the interesting things is say for a given set of requirements, there are potentially multiple different ways of decomposing the system. So for instance, you know, in this class, I teach this three day class where the start with that, where students come up with a microservice architecture for given a set of requirements. I think that, you know, typically the class will come up with three or four different architectures and each one represents a sort of different set of tradeoffs in the sense that some queries might correspond to a single service, but other queries correspond will require gathering data from multiple services. And then that’s an architecture, a inversion B it’s different, it’s divided up differently in which impacts different queries. And so, so essentially if you find that because of the way that you’ve divided up your system into services, it results in a query that gives you unacceptable in consistencies. You, you might actually have to go back and revisit your decomposition and adjust the service boundaries so that you, you have the consistency that you need.
Robert Blumen 00:38:03 We did a show a little while back about the Postgres sequel query planner. I of to takeaways from that show is the query planner is brilliant at costing out different ways of getting information and doing something that’s pretty smart to get you all the information you want. And if you didn’t have that, you’d be looking at, Hmm, okay, I’ll grab all the users and then I’ll go out and get all these addresses one at a time. And he probably would do a worst job in the query planner. Are we taking a step backwards here when we decompose? And now we are test telling our developers to become a, essentially a SQL query optimizer, but maybe they’re not as good at it as the one in Postgres.
Chris Richardson 00:38:48 I see the analogy though, right. In the sense that I think basically it sounds like you just write some giant sequel query handed over to the query planner and it figures it out as opposed to breaking it up. I wouldn’t say it’s exactly the same problem, but there’s a certain kind of, there are some analogies here in the sense that essentially things like transaction management joins reference, maintaining referential, integrity, ends up being moved from the database where it’s just all automatic and you don’t think too hard about it up to the application level. So if you’re using the saga pattern, you see some aspects of what you’re doing a more complicated way. And then querying is more complicated and then say, ref, um, foreign key constraints. You’re now having to enforce those at the application level. So like when you create an order and it’s got a customer ID that that’s in the order service, and you can’t have a foreign key constraint to the customer table anymore.
Chris Richardson 00:40:06 So there’s a bunch of stuff that you have to do at the application level, which makes things harder. Right. But on the other hand, the assumption is that there are benefits to splitting up, right? And, and hopefully the, the, the benefits outweigh the drawbacks. So as you know, for instance, if you’re working on a service, hopefully the focus of your attention is on implementing, say the complex business logic within the service and the sort of the, the complicated stuff around sagas and quit and queering are relatively small part of what you have to worry about. Even though like my whole book has, you could say it’s own all this complicated stuff, but it’s just that the idea, hopefully the complicated stuff is sort of on the periphery of your services and the, the, the meaty stuff that’s implementing the business logic is actually the bulk of what you’re having to do.
Robert Blumen 00:41:20 Fair enough. I want to move on it in sure. We cover as much as we can. One of the areas, which is in my experience, extraordinarily more complicated when you go to microservices is testing. What are the major differences in testing? And I’ll say we’ve had a ton of shows on testing, almost all about testing program logic. What are the main differences from single program testing to microservice testing
Chris Richardson 00:41:52 Testing, and the microservice architecture is simultaneously I would argue easier and it’s harder. So it’s easier in the sense that a service is a relatively small amount of code. And so writing automated tests for it are a lot simpler and there are the simpler and faster running than they would be if you had some monstrous monolithic application. So there is this notion that testability is enhanced by the microservice architecture. The complication comes from the fact that services don’t offer often don’t exist in isolation, right? Let’s just say the order services having to collaborate with the customer service. So there’s a di there’s a coupling there. Say the order service is dependent upon the API of the customer service. Whether that API is there is literally a rest API, or perhaps it’s some, it’s an asynchronous kind of API. There’s still a dependency there. And so there’s sort of a bunch of problems around that.
Chris Richardson 00:43:14 Like, well, I want to test my order service, but it depends on the customer service. And then less, shall we say the customer service might depend on three other services and suddenly you’ve got this whole tree of transitive dependencies just cause you want to test one service. And that could actually get quite complicated quite quickly because you’re having in theory, you’re having to provision all of these services. So one, one, one important trick there is, is you want a way to test services in isolation while being confident that when you put them together, um, they can actually communicate successfully. So there’s a, there’s a few different patterns there. Um, one key idea is consumer driven contract testing, where the interaction between a consumer consumer, a client and a service is captured by a set of contracts, which are basically in the case of HTTP, um, example or request response pairs.
Chris Richardson 00:44:23 So it’s like specification by example. And those, those examples, or are used to test both the service and the consumer in isolation and provide. So if you’re testing both with the same contracts, then in theory, fingers cross, when you put them together, they can communicate. And then the other part of it is there’s component testing for services, which is where you test a service in isolation, but, but you use test doubles for any of its dependencies. So the order service, for instance, would be tested in isolation, um, with a test double for the customer service. And the test double would, if it was HTTP would be programmed to reply with canned HTTP responses for a given, you know, when it gets a request or if it’s, they’re communicating via a message, broker would be send back canned reply, pre-canned reply messages. Um, so there’s a bunch of these new, new attesting techniques that you have to use, but the whole, the end result is you’re able to test services in isolation while being confident that when you deploy them, they will actually communicate.
Robert Blumen 00:45:46 This sounds very much like the use of mocking, which I’m familiar with in Java or groovy. You have a library which will instantiate a runtime, a class that implements an interface, but you can script out what you want, that the methods on that class to do when you call them just for that test. And that could include returning a certain result or raising an exception. So you’ve abstracted that concept and moved it out to the HTTP server rather than the programming language. Is that correct?
Chris Richardson 00:46:23 Yeah. Yeah. So, so like what you’re describing, like with Mark objects, like say with the Makita framework in Java, that’s a particular form of test double, but you can have other forms such as a fake HTTP server. And then for instance, there’s the wire mock project that was an open source project. It’s basically why our mock, sorry, it’s mock Hito for HTTP. So that that’s pretty, pretty useful. And there’s other ones like that as well. And then you can do the same thing with messaging as well. So if you’ve got services that are communicating via a message broker, it’s pretty straightforward to have a testable that when it sees, when it receives message X, it sends back message. Y
Robert Blumen 00:47:17 When you were talking about using these contracts, my first thought was I am going to implement HTTP end point. And I think variable in Jason document has one name, but the client has a different idea of what the name of that field is. I think you’ve solved that by forcing both ends of the end point to use the same contract. And that should detect a lot of bugs that you might find at compile time, if you are compiling your whole thing into one big monolith.
Chris Richardson 00:47:51 Yeah. I mean, it’s very true that especially when you’re using a statically typed language like Java or any, any interface, mismatches manifest themselves as compile time errors, which is really good, right. But in a distributed system, you don’t have that kind of guarantee, especially over the other. The other key point is your system is not static services are constantly being updated. And so preventing API APIs from accidentally being made, um, like introducing, breaking changes accidentally in your API APIs, it’s really critical that you prevent that from happening. And so consumer driven contract testing is, is, uh, is, does exactly that it prevents the, uh, the service team from accidentally changing the API in a way that would break any of the, that the consumers.
Robert Blumen 00:48:57 We did a show a little while ago about Jenkins. We had some fairly general discussions about the concept of a pipeline. Jenkins can execute any kind of pipeline. You have some views in your book about what the deployment pipeline would look like in a microservice architecture. Talk about that. What, what is the pipeline that you recommend?
Chris Richardson 00:49:22 So, so in general, I mean, a deployment pipeline is essentially a, a series of automated test suites. And the idea is, as you go from left to right, the test suites potentially get, gets slower and more sophisticated and slower, but get more production like, so maybe the first stage of your, your deployment pipeline might just execute unit tests. So they, they execute blindingly fast, but, but essentially don’t catch a whole class of errors. And so then you have the next phase execute more, more sophisticated, yet more expensive kinds of tests. And so a lot of so key idea there is that, yeah, you can go implement the deployment pipeline using pick your favorite CGI server or cloud or cloud provider. Um, doesn’t really matter too much, which one you do. But the important idea is that essentially each service has its own deployment pipeline. So it’s being built and tested independently of all of the other services.
Chris Richardson 00:50:41 And then I suppose I should say they are ideally the final stage of the deployment pipeline actually pushes that service into production. I mean, in theory, it could all, all of these different pers service deployment pipelines could feed into some kind of end to end testing environment, like what you would commonly call a staging environment. But, you know, there’s a lot of thinking around that is that instead of having a staging environment, which is only a, a rough approximation of production, and also the fact that end to end tests tend to be kind of slow and kind of brittle Pratt, it can be much more effective to just deploy into production in a very disciplined way using like Canary deployments or Bluegreen deployments and with careful monitoring so that if there are problems, you can quickly roll back,
Robert Blumen 00:51:42 Moving on to another area you provide patterns is productionizing or production readiness. How do you define production readiness? Well, there’s all of this.
Chris Richardson 00:51:55 What should we say? You know, there’s one key, absolutely key part of what a service does is implement is, is the business logic, right? I mean, that that’s like the whole reason for it existing, but then in order to be sort of deployable in a real application that does a whole bunch of other stuff that you have to deal with, for instance, security, or sort of, it’s the one really critical thing, right? Is services need to be secure. So that’s one part of being production ready. That’s separate from the business logic. And then another big part that apart there is observability in a sense, well, you always have to deal with sort of observability or monitor ability, even side of monolithic, except that it’s a bit more complex in a microservice architecture because when requests come in, they can bounce around between services and so troubleshooting and, and like figuring out where the latency is, is being, you know, as high, as much more complex in a monolith.
Chris Richardson 00:53:05 I suppose you just pro you run a profile or of some kind, and you can track it down. But if requests bouncing around between a whole bunch of services, it’s not so obvious. And so there’s a whole bunch of observability patterns that you have to implement ranging from simple ones, like having a health check end point that the runtime environment compare biotically paying, like make a get request, HTTP, get request to, to verify that your service instance is healthy and is ready and can handle production. Traffic does a, you know, another pattern is log aggregation. So the logs from all of the services are being sucked up into a log aggregator where they’re stored and they can be searched another critical pattern, this distributed tracing. So, you know, each inbound request gets a request ID or a trace ideas. It’s cool. That gets propagated from one service to the next ends up in log messages.
Chris Richardson 00:54:10 And then that interacts nicely with log aggregation, because then you can search for that request ID and see all of the relevant logs. Plus the services report, what they’re doing to a distributed tracing server, like open Zipkin, which can then display traces of, and you can get the time break down for a sample request, and you can see where the latency is being spent, you know, is occurring and so on. Um, and then there’s other patterns like application metrics by gathering metrics that the app or that sort of business level metrics about what your services are doing, and then feeding them into a metrics database where they can be aggregated, visualized and alerted
Robert Blumen 00:55:01 One that you didn’t mention that I was going to ask you about is the externalized configuration pattern. Oh yeah. Talk about that one.
Chris Richardson 00:55:10 So the basic idea with that, which is, which is really one that that’s, you you’d have to deal with in a monolithic application as well, is the application should just be built once I buy the first stage of the deployment pipeline, and then you can run it in multiple environments. But what that means is that say the configuration properties of external services, like the database server, can’t be hardwired into the source code because they are different in each environment. And so they need to be passed in at runtime somehow. And so you need an externalized configuration mechanism for doing that, which might be something as simple as setting environment variables that works quite nicely in a Docker environment where it’s easier to, whether you like say, if you’re using Kubernetes, right, you have a config map that defined some, some property values that then get injected into the container as environment variables. And then like if you’re using spring boot, spring, boot will just pick them up, make them available inside the spring application context. But there’s other mechanisms that you can use while conflict files that get injected in, or you can have, so those are sort of the push model, or there’s the pull model where the service can go and request the it’s configuration from a config server. Let me, so you’ve got couple of different options there.
Robert Blumen 00:56:50 What all these things, what ties in together that we’ve been discussing this last few minutes is enable you to manage, understand and troubleshoot the application when you run it in the production environment. Is, is that a fair summary?
Chris Richardson 00:57:08 Yeah, well, yeah. Like, yeah, I suppose manage as in like yeah. Configure it and figure out what the heck it’s doing.
Robert Blumen 00:57:17 Okay. We’re getting pretty close to the end of our time. I had section of questions about anti-patterns and maybe I’ll ask you, what is one of the most pernicious or most common anti-patterns that you observe?
Chris Richardson 00:57:35 There’s two. I mean, so one is technical, which is where you, you create an architecture where you have a set of services that are communicating via HTTP, and perhaps with the S with the assumption that networking is free. So you’ve taken a monolithic design, modularized it, and then given each modular rest API. So they you’re likely to have performance problems because there’s lots of network round trips. So latency will be horrible. Also you’ve basically built a tightly coupled system with single points of well, multiple points of failure. And then if you’ve really done a bad job of modularization, you’ve basically ended up with building a distributed monolith where changes to one service just ripple throughout the, the sort of all the chain of dependencies. And you end up having to make multiple changes to multiple services in lockstep. So you’ve, you’ve basically got a monolith that’s made of services, which is like the West best of the worst of both worlds.
Chris Richardson 00:58:57 Um, so there’s sort of technical anti-patterns like that, but then, but then maybe the more challenging ones are more sort of organizational political process policy oriented, which I gave it a keynote gave this talk pot, hot pot holes in the road from monolithic health conference. And that, you know, an example of those, if one of them is called the red flag law and that, and its name comes from the fact that when automobiles or self driving vehicles first came out in sort of the early 19th century, that some jurisdictions passed the law where a pedestrian had to walk in front of the car holding a red flag. So you kind of had, I presumably that the vehicle could actually go faster than the pedestrian. Right. But, but you slowed it down because of the law. And I’ve seen this inside some organizations where it’s like, yeah, we’re going to have microservices, but then they don’t change the policies or their processes.
Chris Richardson 01:00:03 Right. So for instance, they might just stick with manual testing, like automated, like many organizations. One of the biggest weaknesses is actually the lack of automated testing. So you’ve got this architecture that gives you the potential to deliver software really quickly, but then you basically have someone walking in front of it with a red flag, but because you’ve got manual testing, or maybe you’ve got policies that say no deployments during business hours, and you can only deploy on the fourth Saturday of each month, that midnight. So it’s sort of like, you know, in a sense you’re missing the point of the microservice architecture and you’re not going to properly benefit from it.
Robert Blumen 01:00:54 Thank you for that. And we’re getting pretty close to the end to wrap up. Well, you mentioned your company, eventuate.io, microservice, Dido, which has many patterns, your book, where can listeners find your book?
Chris Richardson 01:01:10 Oh yeah. So if you go to microservices.io/book, you’ll find a description of the book and, and links to where, where you can buy it.
Robert Blumen 01:01:23 Do you have any upcoming conference talks that listeners might want to attend?
Chris Richardson 01:01:30 And in September I will, you know, the full conference season I’ll be, be at some conference or the other.
Robert Blumen 01:01:39 Okay. Well, Chris, thank you very much for speaking to software engineering radio, right? Thank you. This has been Robert Blumen. Thank you for listening.
[End of Audio]
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected].
SE Radio theme: “Broken Reality” by Kevin MacLeod (incompetech.com — Licensed under Creative Commons: By Attribution 3.0)
