Ashley Peacock, author of the book Creating Software with Modern Diagramming Techniques, speaks with SE Radio host Akshay Manchale about diagrams in software engineering. They discuss the power of diagramming and some reasons we don’t fully use it as often as we should. Ashley contrasts historical use of UML diagrams versus modern diagrams, which don’t have hard rules about representations. The episode examines different types of diagrams through an example application and how it could be built with modern tools such as Streamy to simplify the building, versioning, and maintenance of diagrams.
Show Notes
Related Episodes
- 226: Eric Evans on Domain-Driven Design at 10 Years
- 228: Software Architecture Sketches with Simon Brown
- 495: Vaughn Vernon on Strategic Monoliths and Microservices
- 539: Adam Dymitruk on Event Modeling
Links
- Creating Software with Modern Diagramming Techniques
- Mermaid Diagramming and Charting Tool
- www.plantuml.com – open-source tool that uses textual descriptions to draw UML diagrams
Transcript
Transcript brought to you by IEEE Software magazine and IEEE Computer Society.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Akshay Manchale 00:00:16 Welcome to Software Engineering Radio. I’m your host, Akshay Manchale. My guest today is Ashley Peacock and we’re going to talk about diagramming and software engineering. Ashley is a staff engineer and architect working in the UK tech industry with over 10 years of experience. He has experience across the tech stack with particular focus on backend technologies. He’s an avid user of diagrams and a huge advocate for the power and conveying ideas, documenting architectures and whiteboarding problems. He’s the author of the book Creating Software with Modern Diagramming Techniques. Ashley, welcome to the show.
Ashley Peacock 00:00:48 Thank you very much for having me. Glad to be here.
Akshay Manchale 00:00:50 To get started, I want to get a sense of why you think diagramming is important in other forms of engineering. Diagramming maybe a requirement if you are a civil engineer, maybe you know, you’re adding diagrams as a requirement by law, but in software it’s not so much so that it’s required. It’s nice to have, but I know you’re a huge advocate for diagramming. So why do you think it’s important and why don’t we use it more often?
Ashley Peacock 00:01:18 I think the power from a diagram comes from the ease with which it can make information digestible. So I’m sure you’ve experienced someone sends you a document to read and you’ve opened it up and it’s just a huge like wall of text and the chances are you either left in the tab and thought, I’ll read that later because you don’t want to read it or you read it and you skim read it, you don’t read it properly. But it can just be really hard to digest, just a wall of text or if you’re a conference there’s a reason that you have slides for a conference speaker. It’s because there’s something on the screen for you to read. Something you can digest that’s easy for you to digest. Now if you talk about software engineering, documenting your architecture or explaining something to someone with some sort of visual diagram that can really help them to understand the problem, understand your architecture, that kind of stuff.
Ashley Peacock 00:02:12 In the company that I work for, it’s got to the point now where, I hate to use the word famous but like I’m slightly famous within the company that I work for, for diagramming to the point that people find it so useful and so much easier to understand problems and information. I would go to meetings, there wouldn’t be my meetings and they would be like, oh I’m sure Ashley’s got a diagram that can explain this to us. And obviously I did not have a diagram available to them because it wasn’t my meeting. But it kind of shows the point of they weren’t necessarily used to diagrams but once they saw the power in them they were kind of, they were much more open to them. And another example, my CTO at the time, he kind of shouted out one of the diagrams I used because he wanted people to use diagrams. He’d seen the power of diagrams and I’d create a rather large system design kind of architecture document. And it really kind of pushed my career forward because I was able to express myself through diagrams, my ideas, my proposals and I can make them digestible for anyone that wanted to read those documents.
Akshay Manchale 00:03:11 Yeah, you make a great point that it’s good for communicating ideas and I think it makes it easier for the readers. From a writerís perspective, at what points do you use diagrams in general software engineering life cycles?
Ashley Peacock 00:03:25 So I split diagrams into two types of diagram, I call them kind of short-lived diagrams. The more ad hoc and long-lived diagrams that you want to typically commit along with your source code. So the short-lived ones are used during a brainstorming session or during pairing to help explain something to someone. So if we are pairing together and I’m trying to explain to you the problem we’re trying to solve or a piece of code for example, I can just quickly whip up a diagram to help explain that. If you’re not quite understanding what I’m saying with my words, I can try with a visual approach instead. It’s not always required, it’s more, it’s something you can reach for like an extra tool in your tool set that you can help explain things to others easily. And the long-lived side is a stuff you put along with your source code in your repository and that will be things like your architecture. So anyone who wants to understand about a given systemís architecture, there should be a place they can go, they can read that document and understand at a glance rather than reading a whole a huge rule of text or relying on asking someone who’s got information in their brain they can just go read it. That’s a really common one.
Akshay Manchale 00:04:35 I want to get a sense of why you think people don’t use it more often. There’s a clear usefulness to it when you are the reader and you’re the consumer for any sort of software architecture that’s represented in the diagram makes it simpler, easier to understand but we don’t really use it as often. So what’s the friction here?
Ashley Peacock 00:04:54 I think there’s a few different factors that go into this one. I think some of it is there are older technologies or older standards, one of them being UML which stands for the Unified Modeling Language. It was kind of traced in the nineties as a way to standardize creating software diagrams. And I think it’s kind of got a bad reputation. They made some maybe choice decisions. I think they tried to make it almost too standard and or too restrictive. They want things done in a certain way rather than focusing on the power the diagram could bring. They’re focused on the notation for example, which is not so important. So I think people have got used to hearing from someone oh, old it’s not good anymore or they’ve had, someone’s had a bad experience with it and it kind of almost like Chinese whispers of this thing is not valuable because someone they’ve heard someone who doesn’t like this thing.
Ashley Peacock 00:05:41 And the other thing is in recent years there’s been lots of developments in the tooling for crane diagrams. So maybe 10 years ago you would’ve had to manually draw this diagram using boxes and lines and it would take you ages and if someone was to ask you to change that diagram after you’ve drawn 15 boxes and they want to add two boxes in the middle, it’s going to take you half an hour to redraw all your boxes. So if you’d gone for that experience, it’s quite painful. I used to do that and if someone would request change to my diagram, a part of me inside would cry because I knew it was going to take me hours to redraw this diagram. And I guess the other one is education. Maybe people aren’t aware of it or they don’t have to get started with it and if there’s not that many people that are doing it and I think it’s got some popularity but it’s not widespread, it can be hard to know who to speak to, to get a better understanding of diagramming, where do get started, the types of diagram, that kind of stuff.
Akshay Manchale 00:06:38 Yeah, the UML model is definitely something that comes to mind when you talk about diagrams and maybe it’s a little too popular for its own good. In the past with the traditional software development cycles. Do you think that the UML diagram was in itself reviewed? Is that something that had to conform to standards and was that just for standards sake or was it something that they could take and convert to code or convert to something? Was there any particular reason that you see why there was so much rigidity in the UML style diagrams?
Ashley Peacock 00:07:16 So I think with any standard, the whole point is, it is standard. So you see a class diagram for example, that represents classes in your codebase that you create. And if I create one of my codebase and you put them side by side, the class names will be different cause they’re different classes. But the actual structure of them and the way the lines are drawn and so on, those are standard in their model. And to a point that makes sense because if you didn’t have a blueprint of what to go from, there wouldn’t be so much value because it’s people wouldn’t know what to do, right? Like all the diagrams would just, they’d all be their own flavor of diagrams without the standard but they went very heavy on the kind of notation and I don’t think they focused so much on the kind of use cases which I think for me I care less about what type of diagram someone uses and more about the information and the use case they’re trying to convey.
Ashley Peacock 00:08:15 Kind of classic example is the other day I work with a colleague and he was modeling classes. And if you were to follow the letter, he would’ve drawn a class diagram but he drew what’s called an entity relationship diagram, which is used more for database schemers. But all he cared about was the names and how they related to each other. He didn’t care about methods or properties or in the case of a database, the columns. He just cared about the relationships. So for that use case, it didn’t matter which one he used. So there was no real point in ruining that like hard work that he’d put in. You can obviously point out that you know maybe you in future, but for him he got out of it what he needed and the type of diagram was kind of less important and the use case was, and the information was the important bit. I think that’s more got lost a little bit.
Akshay Manchale 00:09:04 Yeah, that makes sense. In the diagramming world there’s UML with its agility. Are there modern diagramming types that you are an advocate for or you see people using? Is there, are there names associated with it for specific types of ideas that you want to convey?
Ashley Peacock 00:09:22 So there’s UML diagrams that we’ve talked about a little bit and there’s from UML there’s 12 maybe 15 diagram types in total. I generally use two or three tops, because I think most common use cases can be covered by maybe four types of diagrams and that’ll cover like 95% of your use cases. The two from UML one we talked about briefly, which is class diagrams. So those are used for understanding classes and their dependencies. So you could draw class A, B, and C, you could show how they, one depends on the other and you can show all the methods and properties on each class and you can use those I use in the two ways. I would say one it’s just understanding. So I’m sure we’ve all been in there, we’re working on let’s say monolith and you’re trying to make code change and you’re in a bit of code that you’ve not worked on before and you go into your ID you find, let’s just say the controller because it’s like a, you know HTB request and you’re clicking through the files and before you know it you’ve got like 10 tabs open in your IDE and you’re trying to remember, you know, which class talks to which class and all the dependencies.
Ashley Peacock 00:10:34 And sometimes it’s easier to step back and just extract that information rather than trying to keep it in your head and just draw it out. And especially if you’re pairing, you then have the same understanding and you can draw that diagram together, you can validate that, it makes sense and then if you want to, you can go one step further. And once you’ve drawn that diagram, if you are making a code change that is not small, you know if you’re doing, I donít know change some text or TA changing tiny behavior, I wouldn’t do this. But if you’re doing a larger refactor you can perhaps draw the class diagram as you’d like it to be when you’re finished your refactor and you can almost prototype using the diagrams and see if that makes sense cause it’s a lot cheaper than doing the refactor and then realizing, oh I’ve now got like a circular dependency or something because you haven’t thought ahead.
Ashley Peacock 00:11:18 So it’s about planning and understanding the class diagrams. The second one from your UML is, sequence diagrams and these, I would say this is my favorite diagram, if I was to pick a favorite because I think it’s just, it’s so versatile for so many use cases. There are two things that I visualize with it. One of them are like user flows. So understanding how a user uses your system. So if you have a login flow or a signup flow and if you’re in a, let’s say a microservices architecture, you need to understand the steps that that user’s going to take and which systems it’s going to touch. So they’re going to, I donít know, fill out the form, click a button, some things are going to happen and you can understand either A, how that currently works if it all exists, or B, if you’re thinking about architecture you can draw the use cases of how the users are going to use your system so that you make sure you build the architecture, the fits what your users are actually going to do. And the second one is for code flows. So you can have steps in the sequence that are for classes, sending messages to other classes. And again that’s an understanding thing. I wouldn’t use this design but if you are looking at the example we talked about earlier in the monolith, you could also use a sequence diagram to understand the messages being passed from cast class. Cause obviously the class diagram gives you the dependency graph but that’s very different to the order of execution in the sequence diagram.
Akshay Manchale 00:12:44 Yeah that’s a common thing when you’re reading through code you are trying to build this mental model of A calls B, B calls C and then you have some state that’s in C that gets passed onto Z. So if I’m hearing you right, sequence diagrams are one way to represent that in a more concise and easy to understand way. Is there anything else to it around sequence diagrams that you think aids understanding?
Ashley Peacock 00:13:09 The only other ones maybe are you can also model almost like happy and sad parts in your system. So we talked a little bit about use cases and how people use your system. You can do the same for code as well that are happy and sad paths. So you can kind of think about all the possible ways people can interact with your system and then how it responds. And similarly that can be useful for when it comes to doing testing for example. So it can be really easy to dive in, code your feature and then you to test it works, right? So you might write immuno test but also quite commonly you might do some manual testing or depending where you work you might actually have like a test engineer that does that for you. If you’ve drawn out these happy and sad paths, you can use that in that kind of conversation about look this is how the, we think it’s going to be used and the test engineer might think of more scenarios but the very least they’re going to appreciate that you’ve put some thought into the testing and you’ve got something for them to kind of start with rather than then having to work it all out.
Akshay Manchale 00:14:06 I know talking about diagrams in an audio only format is a little bit challenging and in the book you use an example company or an example service to aid understanding throughout the book in various chapters. So I think you talk about streaming as a service. Can you just give an overview maybe of that and then we’ll use that as an example going forward in some of the more detailed descriptions of diagramming techniques and tools, et cetera.
Ashley Peacock 00:14:30 So streaming is a video streaming company. You can think of it like a Netflix and they’re kind of just getting started as a company. They’re, you know, they’re not three engineers in a garage but let’s just say they’re 20 engineers for example. So they’re just get going, they’ve got some funding and they are going to build the next big video stream platform. And to do that, we’re going to go through the journey that streaming would go through in order to build I guess their MVP so that the minimal viable product, the smallest thing they can build. But the different steps of how I would approach that kind of greenfield brand new project and the diagrams I would use to help build the architecture that streaming needs.
Akshay Manchale 00:15:14 Okay. Awesome. So starting with that, let’s talk a little bit about more modern diagramming techniques. So you’re in that room, how do you get started? Like do you start with a diagram, maybe you’re collaborating on a whiteboard, how do you make use of these diagrams to convey ideas and come up with that architecture for this next generation streamy streaming platform? Can you talk a little bit about that?
Ashley Peacock 00:15:38 Yeah, so I’ll give a quick overview of the different tools that are available, there’s, quite a few. Before modern diagram came along there were, well everyone would’ve used a UI tool. So something like Microsoft Visio for example, that you have to pay for, there are Open-Source alternatives like Diagrams.net or Draw.io is probably the one that everyone knows; it’s free, it’s open-source, it’s already on your browser, it looks very pretty. You basically drag and drop everything around on their kind of interface. But it’s very, it’s very slow as we talked about earlier, if some, if you know, if you need to update that diagram or change that diagram, it can be very time consuming and it’s a maintenance nightmare. There are other problems with it. It’ll save in an unreadable format for a human. So if you ever used Draw.io and if you went and looked at the save file, it’s a huge blob of like XML from memory, you have no idea what’s going on because it’s written for a machine which means you can’t diff it, and you could commit it to your project if you wanted to.
Ashley Peacock 00:16:39 But the next time someone comes to edit that diagram, they’re going to load into their Google Cloud, they’re going to change the diagram and they’re probably going to forget to update it. So now you’ve got a mismatch between what’s committed in your repository, it’s on your Google Cloud or my Google Cloud and they’re all out sync. It’s a bit of a nightmare. And if you want to put, let’s say an architecture diagram in your readme, you need to export the image out of that program and store that in the repository, which now means your source of truth is disconnected from the kind of final artifact that you want people to read. Classic example is someone leaves the company and that diagram for your architecture is in their Google drive. And I don’t know all companies but where I work there’s a choice when someone leaves of whether you keep their Google Cloud and it’s given to someone else or you just delete it forever and once it’s deleted, it’s gone, you’ll never get it back.
Ashley Peacock 00:17:31 So if that’s your architecture or your documentation for your architecture and that person leaves, your documentation is now effectively gone and someone’s have to redraw that diagram. So as you can probably tell I’m not a huge fan of the manual tools. They have places who are drawing a very complex diagram or let’s say you are presenting to executives and it needs to look really, really pretty then sure draw it in there as a one-off. But generally speaking for the diagrams are mainly going to talk about building software architecture, I would stay away from those. The tool that I think started the movement towards more automated tools and the movement or the practices kind of called diagrams as code is PlantUML. And that’s a tooling in Java, I think it was around the 2009ish I think it came out. And it basically allowed you to write markup that’s similar to markdown.
Ashley Peacock 00:18:24 So you might have A, and then like a hyphen and an arrow to B and and you can put like a label on it with A, with a code or something and give it a label for that line and it would draw you a box of A to B with an arrow and it’ll just render that for you. So you’d have to draw anything and it’s super simple to use. Problem with it though is that it’s because they’re in a Java, you can’t run that anywhere. You have to run, you know, a Java server, you can obviously run it on your Mac or your PC or whatever, but it’s, it’s a bit of a nightmare to kind of get set up with. So it has plugins that you can enhance its capabilities with. So out the box it offers you the u UML diagrams I talked about, plus all the ones that we haven’t touched on and you can enhance it with other types of diagrams that have come around since.
Ashley Peacock 00:19:11 But the new kid on the block is Mermaid and the book focuses on Mermaid and is written in JavaScript, which is the main difference. The markup is remarkably similar. If you looked at the plant UML and dynamics syntax, for the most part they’re pretty identical because there’s only so much kind of innovation you can have in drawing these types of diagram because they are mostly UML diagrams so there are standards associated with them. Because it’s written JavaScript, that means it runs in the browser and because it can run the browser, that means it’s been widely adopted, which is the big, big, big selling point. I think PlatUML is awesome, Mermaid’s also awesome but Mermaid, the big one for me is the adoption. So if you’re using your fusion Jira confluence notion, if you’re using the big one GitHub, you can put your Mermaid markup in a markdown file on GitHub and with no work whatsoever when someone used that markdown file in their browser, GitHub will render that diagram for you.
Ashley Peacock 00:20:10 So that means your source for your diagram is committed in your project and wants to update it. They can just go in, edit the markdown, commit it and it’s updated. So your single source truth for let’s say your architecture is now in your project, it’s stored, there’s no getting out sync and the rendering is just done for you. And if you had external knowledge centers, for example, let’s say you export that content and you host your own knowledge type for example, that’s easy to do in the build, right? You could just use the tooling that nano gives you, it has a command interface and you can just render those, those diagrams in the markdown file before you render those has art pages for example.
Akshay Manchale 00:20:49 And you get all of the versioning for free it sounds like, because I was wondering how do you do versioning and diagrams and how do you keep track of old versions maybe? So I guess this gives you a very nice way to just version inside of your repository itself.
Ashley Peacock 00:21:06 Yeah, exactly. You can basically just rely on the version control that everyone’s got used to these days that will version your diagrams and you can look at the old versions, you can’t lose this diagram, it’s well. And this’s GitHub was to you know, erase what the hard drives and whatnot. Then we hope that doesn’t happen but yeah that diagram’s not going anywhere.
Akshay Manchale 00:21:24 Nice. Can we take something from the Streamy service, some flow, some interaction and kind of describe what this Mermaid or plant your mouth syntax kind of looks like in order to visualize that? How hard is it? How easy is it? Can you shed some light on that?
Ashley Peacock 00:21:41 Yeah, I can do that and I’m going to start where I would logically start. If I was Streamy and I was doing this in the real world and I’d been given this greenfield project to do and that’s understanding the domain that Streamy is working in. For that I would lean on domain-driven design and create a domain model that we can do using ADE.
Akshay Manchale 00:22:06 For listeners, we have a full episode on domain-driven design and I’ll leave a link to that in the description.
Ashley Peacock 00:22:14 So the way I approach domain-driven design, and I would listen to the episode for more detail because I’m not going to be able to cover it in this in this space, but I would think about the main entities that Iím working at Streamy care about. So the things we talk about when we’re in meetings that the customers interact with, what are those? And we need to get this down and we need to get the names agreed cause one of the main kind of tenants of domain-driven design is everyone speaks the same language. So if we’re talking about, let’s say a title, if you go onto a video stream website, you watch titles, something might call them videos, it doesn’t really matter what you call them, there’s no right and wrong what you call something as long as it fits your domain. There are obviously some ways to do it wrong.
Ashley Peacock 00:22:58 If you call it a desk it’s probably going to be very confusing but as long as you give it a sensible name, that’s fine and your models also might look different. If you’ve got two sets of people and they both modeled Streamy, they’ll probably come up with completely different things. So in your head now you might be thinking about what streaming looks like. It probably looks very different to what I’m going to describe. So we have our title, we’re going to use a class diagram for this because think it’s the most appropriate diagram to represent the domain model, although they’re not, there’s no properties, there’s no methods, it’s just a domain model, they’re just names but it’s the closest thing in diagrams that represent a domain model because when we convert this domain model into code, all these entities should exist as classes in our code base.
Ashley Peacock 00:23:44 So we have our title that’ll be a box and connected. So with arrows going to this title you might have different types. In UML we call this generalization. In programming you think of it like interface or inheritance and you have types of TV show or short or film and those are types of title. So they’re extending title if you like. And from title, a title might have seasons and episodes for example. But where domain modeling and UML can help us is you need to understand how many of one thing relate to another thing. So a title, for example, if it was a TV show, it might have one season, it might have two seasons, so on; if it’s a film they’ll have zero seasons because it’s, it’s a film and therefore in the diagram you have things called multiplicity. And what this effectively does is tells you how many of one thing relate to another thing. So, in the case of title and season you can say a title has zero to many seasons and a season belongs to just one title. Because, you know, season one of Game of Thrones can obviously only relate to Game of Thrones. It can’t relate to, I don’t know, Dexter or something. It doesn’t make any sense.
Akshay Manchale 00:24:56 And how do you use this in Mermaid syntax and PlantUML syntax to represent?
Ashley Peacock 00:25:01 So Mermaid syntax, each diagram has a type, so the first line in any Mermaid is the type of diagram. So our case is a class diagram. The sequence if you seek the diagram so forth and for a class diagram is probably the simplest of syntax and markup that you can probably have of any diagram because there’s no methods and no properties. We just have the names and how they relate to each other. So it’s super, super simple. So first line class diagram and then let’s say we want to describe the relationship between a title and its seasons. You would put the word title and then you would put, let’s just say two hyphens and then another space and season. And then if it was to render it you would just get title and you get season with the line between it. And that’s as simple as it possibly could be. And if you then white add in an enrich with the multiplicity either side of those two hyphens, you would just put multiplicities in speech marks. So one and zero to many for example. And manual we would take that and it would just render these box in these lines.
Akshay Manchale 00:26:05 It’s interesting how this renders things from text because sometimes when I’m creating it with UI tools I’m kind of confused about how big certain boxes should be. Should my line be bold, should it be really thin? Does it look okay? Does Mermaid take care of that with like default settings? And is there a way for you to change that maybe to make it visually more aligned to maybe your taste or the way that you want something to sign up?
Ashley Peacock 00:26:33 Yeah, certainly. Depending on the environment you are in, you have different levels of control. So out the box Mermaid has its own rendering engine so it will render everything in its default way and it’ll look nice. But if you wanted to change certain things you can do that. So there’s a type in Mermaid called a flow chart and it’s more like, you know if you’ve ever seen a flow chart, you have circles and diamonds and it could be like, you know, decisions and things, but it’s their kind of generic catch-all for maybe things that don’t fit into anywhere else. And in that example you can change the direction, for example: rather than being top to bottom, it could be left to right. In the case of class diagrams and most other types of diagrams, you can use CSS styling to change how things look, limitedly.
Ashley Peacock 00:27:22 And if you’re using online mode, it has a live editor that I use a lot, that lets you quickly create the diagrams and it gives you kind of two panes, the left you put your Mermaid markup and the right will just automatically refresh every time you change. So on that one you can do these kind of simple things if you were rendering them on your own website, you can do whatever you want because there are Javascript kind of hooks for you to use, you can inject your own styling but even with the kind of default and minimal more locked-down version of Mermaid, you can still change, you know the colors, the borders, all that kind of stuff. So you can kind of make it branded to your company if you wanted to do that. And Mermaid also comes with themes out the box. So it has a default kind of all the boxes will be purple for example, but there are themes that make it blue or black or you know, dark mode that everyone likes these days, that kind of stuff. So it’s quite versatile and because it’s Open-Source, new things are always being added. I think you know a new release comes out every couple of weeks with new features and new things you can do.
Akshay Manchale 00:28:22 Yeah it’s really powerful. I think this is very similar to what Markdown does where you have the syntax and you have the extend Markdown syntax and then depending on the rendering engine you might even have other syntax types. Like I know different rendering engines have different ways to render code blocks that I like. Some of them have syntax highlighting, some of them don’t. So this is somewhat akin to that. And I guess what you’re saying is that you can also override it similar to how you can with uh, a markdown rendering engine. That’s very cool. Can you draw sequence diagrams that you mentioned earlier? Is it possible to use Mermaid for something like sequence diagrams or maybe even just UML itself or is it just no UML but we’ll take some concepts you have, the boxes and the lines without all of the rigidity. Is that the idea or can you also get all of you know, UML sequence diagrams and just plain simpler representation of diagrams?
Ashley Peacock 00:29:18 Yeah, so I think you can do both depending on what you’re trying to do. So if, if you are trying to build or understand a sequence, I would always just use the sequence diagram because it’s going to handle the layout and the rendering for you. So in a sequence diagram you have what are called lifelines and those are the individual nodes that messages are passing between. So let’s say you had a flow for a user being able to sign up to streaming and create their account so they can view titles. You might have a little stick figure. So in Mermaid you have what called actors on participants and that all that means really is it changes the little kind of icon from a box which represents normally like a system or something or a class and a stick figure which reference a person. So you might have the browser and you have a line from the browser to the signup service that is retrieving the HTML for the signup page and then you’d have a dotted line back because in sequence diagrams commonly you have a dotted line to show responses and a solid line to show requests.
Ashley Peacock 00:30:24 So you can easily differentiate between the two and that will return some HTML to the browser. And that’s kind of like the first interaction in the sequence diagram. Now the user is going to submit that to submit their details to the SAM service, which will be a post request. So on the lines going between, you can put very simple kind of notation to explain what’s going on in these interactions. But you definitely don’t want to be putting, your JSON payloads in there. These need to be like nice and high level, you know it’s post request and it’s hitting the same point. That’s simple. Then when it gets to the signup service in my, you know, you want to call out some of the stuff it does. So let’s say you’re working with microservices, you have your site service and let’s say you have a user service that stores your user data, the signup service might make a post request to the user service.
Ashley Peacock 00:31:08 So then you have another line to another kind of lifeline and that might return a 201 to the site service and then that might return a 301 to redirect the login page to the browser. So then you have a series of interactions between different participants and one of the really nice things you can do in Mermaid is you can add numbers, which sounds not that powerful but if we are talking about a sequence that has, the one I described for example in the book has nine, 10 different interactions on it rather than someone trying to describe all at this point they can just say, and number six, what’s happening here, can you explain this? I don’t understand this. It’s really quite powerful tool just to aid general collaboration for example make it easier to collaborate. As for diagrams outside of UML, Mermaid has a kind of catchall that I mentioned earlier called flow charts and those are kind of super versatile, not UML, just general flow charts where you can draw pretty much whatever you like and you can change the shapes of the boxes.
Ashley Peacock 00:32:15 So you can have, you know, square boxes if you’re doing a service, if it’s, you need to represent a database, you can have like the little cylinder that’s commonly used on diagrams for a database. If you are maybe drawing almost like a business rule company used to work at, it was a loan company so they gave loans to people, there was a decision engine with hundreds of rules and each rule could be defined as a flow chart, which is you take a data point, you might have a little, an arrow to a diamond that says, you know, if this data point’s greater than this then go on this path if it’s less than this, go on this path. And it’s a really nice way of using a simple flow chart just to visualize business rules. And if you were building this decision engine, it’s got hundreds of rules as an engineer you can just go and look at the flow chart for the rule that you are currently coding and it’s all there for you. All the use cases, all the business in that flow chart. One of the other ones that I don’t think we’ve touched on yet is if you want to document your architecture, I always reach for something called the C4 model.
Akshay Manchale 00:33:17 Yeah, let’s dig into the c4. Can you describe what that is and then we’ll come back to the moment how to represent there?
Ashley Peacock 00:33:25 So the C4 model is a way to model your software architecture. It was created by a guy called Simon Brown. You may have seen him speak if you go to conference and things, he’s quite kind of common on the calendar of conferences, you’ve spoken to one you’ve been to and you can use a design document model your architecture. And it’s one of those long-lived diagrams that I, in every system that I work on, it will have a C4 model in it. And what it does is it takes what I call a focus system. So the system that you are responsible for at the team and it helps you document and understand the architecture for the system. So in the middle you’ll have your system that you own, typically above it you’ll have the dependence and these can be systems or people and below it you’ll have the dependencies.
Ashley Peacock 00:34:16 So any downstream systems that you use within c4, the name comes from four diagrams, so c4, four diagrams and they’ll mostly begin the C. First one, which is the one I’ve just described is the system context diagram. And that is the kind of 50,000 foot view of your system and it just does, it’s completely non-technical. There’s no protocols, there’s no databases, it’s purely describing almost like the business interactions between the different systems. So in the case of Streamy, you might have a premium member for example, and you might have, let’s say a service because we’re going to be a microservices for example. There’s the listing service and that listing service contains all the data and all the information for all of the titles that stream offers. So anything in your architecture that wants to get information about a title, he reached out to the listing service.
Ashley Peacock 00:35:17 But obviously the person, the customer also kind of interacts with this, right? Because they’re going to want to view the titles. So maybe as a front end component to it, they renders the titles to the user. Now that listing service to do its job might break down and call other systems. Let’s say for example, reviews, let’s say your review titles, maybe you want to store your reviews separately for some reason in a review service or you want to search the titles. And because searching is very kind of bespoke, let’s just say you want to use elastic search, you might have a search service, right? That’s completely separate for searching the titles for example. Now you can draw that on your C4 like I just described and you can, I’ve shown this to you know, product managers, stakeholders, completely non-technical people and they’re able to digest understand what it is that you’re building and that can be, that can be really useful to help explain what you’re doing.
Ashley Peacock 00:36:13 And you know the classic question of why did this take so long or why is this taking so long? You can kind of explain to them why you’re building it this way from a completely non-technical view. I won’t go through all of the layers of the C model because I actually mainly focus on the first two and the second one here’s what we call the container view and this is then getting into the technical layer so you have your system context for the high level view and then if you want to zoom in, so engineers typically will want a bit more information than just a box that says this is the service. They might want to know how this service is put together, what containers it has. So let’s say in the listing service you can almost think about on the first diagram, double clicking the listing service and it zooms in and it kind of splits out all the single deployable units that make up your service.
Ashley Peacock 00:37:05 So when you push the master, it deploys what do you need to run, let’s say as part of the service you have a mobile application and also you have a web application so people can access your listing service via mobile, via web and for whatever reason you deploy those together from the same repository. And maybe because calls to the get titles is expensive, maybe you have it in memory cache or something where you hold these titles in in Redis for example. And these would be three separate containers almost in like a boundary of your system that you could document and you still have your downstream services. That’s still cool but it’s just, you can almost imagine in the middle you have like a box with dash lines around that show the boundary of your system and anything in that boundary belongs to this service and you can dig into it further and further and further.
Ashley Peacock 00:37:57 And as, as you go through the model it digs further and further. But I’ve never really needed the next two, which briefly are components and code with components being the high-level modules that make up each container and one level deeper is the classes in those components or modules. But those, even if you hear Simon speak, he will generally say those are there to enable the build to do that. But generally the first two are almost people need. Especially if you’re building microservices, I would say components maybe if you’re in a monolith because it better represents your architecture.
Akshay Manchale 00:38:36 You may be able to represent different modules within a monolith as a container perhaps. Is there a difference between the container and component in terms of the representation? Because once you zoom in one level deep as it just boxes that encapsulate you know what it’s doing, describes it and then have connectors to other similar containers of competence.
Ashley Peacock 00:38:58 So if I was doing the C4 model and I was working in a monolith, what you’d basically have on the system context is you’d probably have a stick figure or whatever of a human and they are you know, doing all of their interactions with the monolith wherever they’re doing in the case of Streamy, they’re viewing titles, they’re reviewing titles, they’re searching titles, you know updating their account, they’re doing everything. You would probably have to find a general way of describing this because you can’t list everything they’re doing otherwise there’s this line, it’s going to be, it’s going to be a long line. But then if you know, like I said again I think of like double clicking, zooming in on that monolith and you would look at the container, you would perhaps maybe have a single web app and maybe it’s got a database hanging off it, but I don’t think it fairly or accurately represents your architecture if it is a monolith just to have that one container because you might think, huh, this architecture is super simple, they’ve got a web app and they’ve got a database.
Ashley Peacock 00:39:52 But when you look at that monolith, if it’s you know, a 10 year old monolith and depending how that monolith is arranged, because it might be the classic big ball of mud, it’s not arranged in any logical way, then you’re probably, in terms of diagramming, I don’t think you’re going to be able to do it, to be honest, I don’t think mostly because it’s going to change too much, because it’s the monolith. And second of all there might not even be clear components, it might be all over the place. But the likes of Shopify are pushing and promoting modular monoliths where or majestic monoliths, depending which one you prefer. Where within a single monolith you have separate segments. They kept different names for them. But in in rails, which what they use is Rubian rails, they’re often engines and those are akin to modules. So they’re self-contained segments of the codebase within the monolith.
Ashley Peacock 00:40:46 So you might have a payment module, you might have a title module reviewing the titles and managing the data for the titles. You might have a user module for example. And if you have that monolith and that nice structure, I think you can make a component diagram that shows up the high-level components that make up your monolith and that’s akin to, you know, each module if you were in microservices would probably be, each module would probably be a microservice. So I think using the component diagram for a monolith gives you a fair representation of your architecture versus if you are a microservices the container level, it’s just enough detail because when you zoom in, assuming you’re doing microservices, you know nicely sized services correctly, all of the code in there should be kind of described by your service and the kind of underlying components probably aren’t too important at least to have in your readme.
Akshay Manchale 00:41:42 Makes sense. Going back to Mermaid as a tool for representation, how do you do C4 with Mermaid? Does it give you that visual aspect of zooming into a particular box that expands into a more detailed representation? Is it that way or is it just different diagrams?
Ashley Peacock 00:41:58 That would be nice. I have a feeling you could possibly, if you had your own website where you would point to like an internal knowledge base, you could probably write some JavaScript that allowed you when you clicked on it, it maybe, you know, change the image or something fancy or scroll to another point in the page. But if you are using GitHub, because I just like GitHub to do all the work for me really, I would just have you know, an architecture markdown document and at the top I would you know, write a brief introduction of high level view of the architecture and what it does. Because I think diagrams on their own super powerful but some words are also, you know, to supplement or also helpful and I would have the system context diagram, I would put a link to the C4 model. So if people want to read what the C4 model is, I’d put a link to the C4 model, there’s a website for it, it’s very good.
Ashley Peacock 00:42:41 And then below that I would put the container diagram so they’re both one below the other and people could just both view both together. I think it’d be really cool if it did what you said and you could zoom in them. But yeah, Mermaid hasn’t quite got there yet, maybe it will. As for creating them in Mermaid, I create them personally in flow charts they’re really used to make, because they’re quite simple diagrams for the container diagram. When you are making the kind of boundary for your system within flow chart you have sub graphs which are like diagrams within diagrams almost. And using that you can then make your nice little box for your service. And then anything inside the box is what you deploy as part of your service. It has experimental support for the C4 model I believe at the moment it only does the system context.
Ashley Peacock 00:43:27 I don’t think it does all of them yet. They’re looking to add it and it uses the same syntax, the PlantUML plugin for C4 users. So they’ve just imported it. I’m not a huge fan of the syntax, I think it’s quite ugly so I don’t really enjoy working with it. I think it’s a bit too verbose so I’m going to stick with the flow charts now at least until maybe Mermaid picks their own syntax for it because it’s so experimental. So we’ll see where that goes. But for now I use the flow charts and it works. It works super well.
Akshay Manchale 00:43:54 I want to switch gears a little bit into more informal diagramming techniques. Maybe just whiteboarding, do you have any particular approach that you take towards whiteboarding and diagrams as you would with respect to something that you’re building on a PC, putting that into a document?
Ashley Peacock 00:44:13 I think the main one, the popular one at the moment is a tool called Mirror. That’s why I use generally for any kind of white boarding. Most recently I did a session on event storming, which is probably maybe going to be covered in your duties talk, I donít know if they’re going to talk about event storming, but it comes from domain-driven design. And before, before COVID and before everyone, or not everyone but before remote working was everywhere when people were in offices event storming would be done with sticky notes. So, and I’ll explain too much about what event storming is because we’ll be here for a long time. But it’s a way of understanding the kind of key events that happen in your domain and you would each have stickies and you would round your sticky, you’d stick it on a whiteboard and you’d basically have, by the end of the first kind of section of event storming, you would have a big line, there’s a chronological line of all the events that happen in your system.
Ashley Peacock 00:45:12 Obviously in a remote world that doesn’t really work. I’ve, I’ve seen it where people have kind of put the camera and the people in the room have put the stickies up and then the remote people talk to the people in the room about, you know, hey you put this sticky here and it’s a nightmare. So most recently I did it on Mirror, which is basically just an online whiteboard. You can draw all sorts of things in Mirror but because it’s visual, it’s not obviously suitable for the diagrams we’ve talked about with adding them into GitHub and so forth. But for collaboration and brainstorming and coming together and collaborating on something like event storming, I think Mirror is probably the best tool now and you can see what people’s courses are, you can see what people are editing, it’s quite powerful.
Akshay Manchale 00:45:50 Nice. And with respect to co-generation, I want to see if there are tools and techniques to generate code or some artifacts starting with diagrams. I think in the UX word people use wireframing tools where you draw something on a tool maybe and you might get a static website or a static page that represents it. So are there techniques like that for generating maybe components, modules for a service, maybe it’s a monolith or something and then you know, you can draw out something in Mermaid or some other tool that generates the required stubs for what you’re trying to build.
Ashley Peacock 00:46:31 I think we’re not quite there yet for most things like that. So there’s the reverse. So if you are working a statically typed language, a really common thing you can do in let’s say Java or C sharp is you can export a class diagram from your code base. If you’re working in a monolith, you’re probably not going to be able to export it cause you’re probably not going to have enough memory. But you can export segments of it perhaps and it will generate your class diagram, not a moment or anything, it’ll just give you an image typically. So I think that’s been around for quite a while. I think the one that is probably, I don’t know if you can do it here, maybe you can, but there’s a diagram called an entity relationship diagram that we talked about earlier which helps you represent database schemers and on an entity relationship diagram or an ERD for short, you have the tables, how the tables kind of relate to each other and the kind of columns, keys and so on.
Ashley Peacock 00:47:27 So if you were designing, let’s say Streamyís underlying you know data, which is often very different to your kind of co classes, you might want to do some upfront design with any ID so that everyone understands the kind of the database and you can visualize it and make sure all the data is, you know, if you want to normalize it for example, you could see if there was any duplication or anything you could improve. It’s easy to get feedback on the design of that database from you know, maybe an architect or whoever you know you can get some feedback on the diagram before you build it. You know, once you’ve deployed it and you’ve got data running through, it’s really hard potentially to then go back and change that and it’s also a huge waste of time. But because the way you write EIDs, I could easily see them once you’ve agreed to design, converting that into SQL.
Ashley Peacock 00:48:15 Because the way you write EIDs is very close to the SQL definitions of when you actually create them in your code. So I could see that one definitely being a thing if it’s not already a thing. I donít know if it is, maybe there are tools out there. The other cool one I heard about recently, and I’m not sure how long it’s been around, but in AWS you can, I think it’s called cloud formation designer or something along those lines. And you can basically draw diagrams in AWS console using the products. So you could draw a C2 instance, an S3 bucket, Postgres database, whatever you want and the diagrams and it will generate you the Cloud formation for that architecture. So you could collaboratively design or design, convert your C4 model into, you know, infrastructure’s code if you’re using Cloud formation or Terraform, obviously in this case it’s a S so they’re going to give you Cloud formation but you could draw them in that console and it would give you the code, the underly Cloud formation that you could then just put straight into your deployment pipeline and deploy it. So that’s another example of turning a diagram into code. I think that’s probably the area where diagrams has some area for innovation, I guess.
Akshay Manchale 00:49:28 Yeah, going in the reverse order is maybe the last thing I want to cover. It’s starting with a complicated piece of software, a code base, maybe I join a new company and there is this massive code base that I want to get familiar with. Starting with the diagram would be great, but more often than not they don’t necessarily have a diagram or maybe it might be stale. So starting from the code, is that a good way to generate some sort of a diagram? Are there any tools to learn and onboard into a new project or a code base?
Ashley Peacock 00:49:59 So I think the ones we’ve talked about already, assuming you can commit them and the, you know, when we talk about GitHub automatically rendering Mermaid diagrams in Markdown, the two key ones for me that I have in any repository is the domain model. And if you don’t understand the problem space you’re working in, you’re probably going to hit some problems and you can put that, you know, in your Readme or it can be in a Markdown off your readme and the C4 model is the second one. I would always have any new joiner that joins. You can just point them to those resources and you can enrich the diagram with some, you know, with words it’s not like, you know, I’m against words, definitely put some words down alongside your diagrams, but that’s a really good way for them to get up to speed with your problem space you’re working in.
Ashley Peacock 00:50:45 And the architecture, the code is a trickier one because it really is going to depend on monolith versus microservices versus kind of where I am where I work at the moment, which is kind of, it’s the classic of everyone hated monoliths, they went to very far on microservices and then they realized, ah, this has got some other problems. So you know, we’re kind of working our way back to somewhere in the middle. People like to call them like macro services or whatever you want to call them. So commonly I might have like a component diagram, for example, from the C4 model that lists the high-level modules for that kind of medium sized service for example. And then at least they understand the high-level modules of how that code is arranged. The, the trickiest thing with code is documenting it is, I wouldn’t do it too much because it’s so fluid and so easy to change.
Ashley Peacock 00:51:37 Like you could draw the diagram and it could be out of update the next week if someone you know comes in and rehash everything, or even just rehashes part of it, refactor, sorry. And there’s nothing worse than out of date documentation that that’s worse than having no documentation at all. Almost because they’ve gone and read this documentation, they’ve spent some time reading it, and then they’ve gone to look at the, actually know, they’ve checked out the code, cloned it, and it’s completely different and they’ve just wasted loads of time reading this document that wasn’t useful. So for code itself, I would be cautious about documenting it, which is why also I talked about short-lived versus long lived, where the long-lived diagrams, the architecture, the domain model, those I commit because they will change. They are living documents, they should be updated, but they don’t update. You know, how often do you change significantly change your domain model or your architecture.
Ashley Peacock 00:52:25 It might be once a quarter or once a month, but it’s not going to be so much, it becomes a burden or at time sync versus the short-lived, which is class diagrams or secret diagrams. They’re used to understand something in that moment, but they’re probably rapidly going to become out of date. I think the only overlap or the one that’s applicable in both is perhaps sequence diagrams. If I was doing, let’s say an architecture decision record, which is a document you can write to rationalize why you are making an architecture decision, I think putting high level sequence diagram of the flow that you are building for this architecture super useful and it’s probably unlikely to change again. So that’s one that does both. But yeah.
Akshay Manchale 00:53:09 It’s great advice and I think with that we’ll end the show. Actually thank you so much for coming and talking about diagramming and some of the tools to really build your diagrams and make it easier to maintain them even. Thank you so much.
Ashley Peacock 00:53:22 No worries. Thank you for having me. If you want to learn more about my book, Creating Software with Modern Diagramming Techniques, on how you can use it to build better software, it’s available from all the main retailers, Amazon so feel free to check it out. Thank you.
Akshay Manchale 00:53:51 I’ll leave a link to Ashley’s book in the links so you can learn more about diagramming. Thank you so much for coming. This is Akshay Manchala, for Software Engineering Radio. Thank you for listening.
[End of Audio]
Thanks so much for this episode! It’s prompted me to give Mermaid a try for diagramming!