Search
Search
Nick Tune and Jean-Georges Perrin SE Radio Guests
October 17, 2024

SE Radio 638: Nick Tune and Jean-Georges Perrin on Architecture Modernization

Nick Tune and Jean-Georges Perrin join host Giovanni Asproni to talk about their proposed approach to modernizing legacy systems. The episode starts with some high-level perspective to set context for the approach described in their book, Architecture Modernization (Manning, 2024). From there, the discussion turns to important details, including criteria for deciding which aspects to revisit; some of the activities, processes, and tools; and the importance of data engineering in modernization efforts. Nick and Jean-Georges describe how to successfully implement an architecture-modernization effort, and how to fit that work with the teams’ other priorities. The episode finishes with some warnings about the typical risks associated with modernizing a legacy system, and suggestions on how to mitigate them.

This episode is sponsored by QA Wolf.
QA Wolf logo

Show Notes

Articles and Resources

Related Episodes

Transcript

Transcript brought to you by IEEE Software magazine and IEEE Computer Society. This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number.

Giovanni Asproni 00:00:51 Welcome to Software Engineering Radio. I’m your host Giovanni Asproni and today I’ll be discussing Architecture Modernization with Nick Tune and Jean-Georges Perrin. Nick works with the product and technology leaders to map strategy, model domains, design, architecture, and build continuous delivery teams. He’s the author of Principles and Practices of Domain Driven Design and co-author with Jean-Georges Perrin of Architecture Modernization. Jean-Georges is JGP for short, is the Chief Innovation Officer at ABI Data. The chair of the open data contract standard is a co-founder of the IDA user group and author of multiple books including † Implementing Data Mesh Sparking Action, 2nd edition , and of course † Architecture Modernization with Nick. Nick and Jean-Georges, welcome to Software Engineering Radio. Is there anything I missed that you’d like to add?

Jean-Georges Perrin 00:01:41 Well, thank you. Thank you for having us. Just to show off a little bit, I’m also a lifetime IBM Champion. I’m a PayPal Champion and recently I’ve been data mesh MVP — and MVP stands for Most Valuable Player, not minimum viable product here.

Giovanni Asproni 00:01:57 Nick, anything to add?

Nick Tune 00:02:00 I think I came like third in a hundred meters at primary school, at Sports Day once.

Giovanni Asproni 00:02:05 .

Jean-Georges Perrin 00:02:06 You beat me.

Giovanni Asproni 00:02:07 So let’s talk now about the subject of this podcast, so about architecture modernization. So let’s start with, some context for our listeners. So my first question is, what is architecture modernization as you propose it? Is it a set of activities, is a process, is a methodology or something else?

Nick Tune 00:02:26 So the general concept is as we build software systems, they get older and the world around us changes, new technologies become available and new patterns and practices and ways of doing architecture become available. Our company’s business model changes and so we have a software system that is quite behind what’s possible in the modern day. So modernization is really removing those disadvantages of the old systems by using modern practices, telling and thinking I would say. So it’s really a topic or a theme. I wouldn’t say it’s a process. I wouldn’t say it’s a technology. I wouldn’t say it’s a very specific thing, it’s just the process or the act of doing something in a general sense.

Giovanni Asproni 00:03:08 Okay. So is it aim at large scale changes or incremental improvements? Because from reading the book the way it is described there is also the concept of creating a group of people that will supervise these changes and will help the teams in doing this. So reading it seems to be a large scale and they already something that is a kind of a big thing or is it more incremental improvements in day-to-day work?

Nick Tune 00:03:34 Well, I think it depends on the context, but I mean for a lot of companies, nobody wants to do modernization. Nobody wants to spend three, five years fixing their legacy systems. But at the same time, business leaders want to be able to build new products to expand to new customer segments and new countries. And when the legacy system doesn’t allow you to do that, sometimes you have to do large scale changes. Obviously, we would all prefer to do smaller day-to-day changes and if you have good discipline and good practices and you don’t build up technical debt, yeah you can do this on a more gradual ongoing basis and not need to do big projects.

Giovanni Asproni 00:04:10 Okay. And it’s also kind of one of activity or an ongoing one or a bit of both. What I mean is does it have a kind of a start and an ending and then you say now we have a modern architecture or it’s something that you continuously do to keep the architecture somehow relevant?

Nick Tune 00:04:31 I mean there might be periods where we’re doing more or less modernization. The more technical debt we build up, the more difficult our legacy systems we come to work with, the more we have to invest in those. So there might be a period where the company’s talking about modernizing and then when the system becomes less of a bottleneck, less of a blocker to the business kind of starts to, we stop talking about that. So there’s not really an end date, not really a start date either, but there are exceptions. One of the case studies in the book is OpenTable. They did a big modernization around 2012, around that time period they basically stopped all product development work, did this big modernization project for nine months and then carried on again as normal. So you can do it that way, but most of the time it’s more of a fuzzy thing with big peaks and then starts to taper out maybe. And there’s always this ongoing, continuing to add product features versus doing modernization work. That’s usually a difficult topic. That’s always a challenge, especially when it comes to OKRs and roadmaps and everything.

Giovanni Asproni 00:05:43 Yeah. Because I guess everybody wants to have a system with more features that serves more customer needs and so spending time on modernization seems to be kind of maybe a cost more than something that will enable future business. I would imagine at least this is the perception that some have.

Nick Tune 00:06:01 Yes. An investment. Yeah.

Giovanni Asproni 00:06:03 Yes. So this looks to me that is kind of a bit of both a one-off activity and ongoing one and this depends very much on the current context of the system. So some kind of periods of big modernization efforts then work as usual, maybe the team trying to keep the technical debt that under control and then again potentially another big effort and so on and so forth.

Nick Tune 00:06:28 Yeah, I think it’s a business question, what does your business want to achieve? Are you looking to expand into new countries? How difficult, how expensive would it be to do that? With your current systems it might not be possible. So that would be the driver of the scope. What are the business goals and how does the current system constrain those goals?

Giovanni Asproni 00:06:48 Yeah, okay. And another aspect, you say that modern software architecture is social technical, so involves both technological and social aspects. How does that affect the architecture modernization choices and decisions? What I mean is how these social and technology aspects work with each other?

Nick Tune 00:07:08 Yeah, so at the end of the day we have to make decisions about which team own which parts of the system and that can actually have an impact on how we design the system. Sometimes there are different ways to slice your architecture and the way to think about that is if we slice it up in one way and we had teams owning this bit and this bit versus slicing it another way and teams owning different bits, what will allow teams to work more independently so that they’re not blocked or having to coordinate their work. And we can also think from a reliability perspective, which way of slicing the architecture and the teams is likely to result in the fewest bugs. So that might involve doing some domain driven design and looking at what do we consider transactions to be, which bits of data do we need to update atomically that will shape our architecture boundaries and our team boundaries.

Giovanni Asproni 00:08:06 In terms of these two aspects, is there one that is more important than the other? Like are social aspects more or less important than the technology one or they are at the same level? What would you say in your experience?

Nick Tune 00:08:20 It’s hard to really say that one’s more important than the other. I think both need to be done really well. I wouldn’t like to say either one’s easy, so no, I wouldn’t pick either one. I would say both are difficult challenges and the actual problem is how to do a joint optimization to find a compromise that spans across both the organization and the software.

Giovanni Asproni 00:08:41 Hmm, okay.

Jean-Georges Perrin 00:08:42 If I may add to that a little bit, it also depends on where you are in the project because the socio aspect to your project or to your modernization, whether it’s software or data, you may feel it differently at different time of your project. Okay. So first at the very beginning, it’s a business decision as Nick said, and you’ve got to go through acceptance of that part. So there’s already a bit of socio going on there. And then as you roll out, your architecture methods are changing and then when you use a magic change word, okay, you’ve got to coach people with this change. And that’s also where the socio aspect is really important. And in my experience I think it’s often undermined.

Giovanni Asproni 00:09:31 Yeah, and talking about these things, I’d like to know if you, from your experience, so now of course when we talk about social aspects, there is always Conwayís Law that comes in play. So for our listeners, is basically Conwayís Law says that the shape of the team structure follows the shape of the system being built. Now usually when we talk about Conwayís Law, we talk from a system perspective, say this is the architecture now this is how the teams should be structured to create, to implement the system according to this design. Have you seen in some companies, maybe you can have some examples of doing the other thing, like this is a team structure we have that we cannot change and so we need to match the architecture to what we’ve got in terms of structure. Have you ever seen anything like that?

Nick Tune 00:10:23 Yeah, so I have worked in the UK governments and they were doing a digital transformation and the digital bit was being built by this new digital organization and they had their own CTO and the internal facing stuff owning a lot of the legacy systems and databases that was owned by the enterprise IT team who had their own CTO. And so when you’re building a digital service, and digital usually means customer facing UI website front end. When you are implementing a new feature, your data has to come from somewhere. When you want to store data, you have to store it somewhere. And so a feature runs all the way from UI, from front office to back office. You’ve got the integration in between the databases I talked about. So yeah, I was on this project, we had this problem, I gave some talks about this almost 10 years ago now actually.

Nick Tune 00:11:21 And we proposed to change the architecture so that we didn’t have this dependency like this front and back on the front we are all doing stuff in-house, sitting together in an agile way. The backend bits, they were outsourcing lots of it to different vendors using different technology stacks. So very difficult to collaborate. When a feature crossed this organizational boundary, it was very difficult to even help. A lot of coordination was needed. So we proposed a restructuring, but that would’ve meant one of these CTOs would have to give something to the other one. And they were both fighting to become the global overall CTO. So neither of them wanted to obviously give away anything. It was all about power structure, very dominated by politics.

Giovanni Asproni 00:12:09 That’s interesting. This is where the social aspects actually had the really were the most important thing in this case. So they, even if maybe a system designed differently changing the structure, would they be more efficient to use, maybe easier to support still the way that there was this structure there that nobody wanted to change.

Nick Tune 00:12:28 Some bits were flexible, but some bits were not. And the two different departments were going in completely different directions that would make it harder to change at different tech stacks, different ways of working. It was, it wouldn’t have been possible to bring it together later on either.

Giovanni Asproni 00:12:44 How did you manage to deliver this project? Must have been really hard.

Nick Tune 00:12:49 Yeah, it was quite difficult. There were situations where we were building this new UI and we wanted to change the user experience. Like we had user researchers, they were going out every week talking to citizens of the UK, they were involving developers, they were presenting these videos of all the user research sessions, and we were like, oh, it’s amazing. So many things we can improve and make our citizens happy. And it would be like, let’s add a new bit of data on this page, let’s add some more data, let’s collect a bit of information over here. And then we started to hit these blockers and it’s like, well we have this legacy database which sits in A DMZ owned by a different vendor and the X in our schemas here. And it passes through these different systems. So if you want to change a scheme and you have to update it in four places, you need to pay a vendor like tens of thousands of pounds just to give you a quote for how much it will fix. So yeah, you just can’t improve the product. And I think people started to call this lipstick on a pig. You can do these digital services, but if you can’t actually change the business rules and the data and make those deep improvements, you’re just putting a new website on an old legacy system. Which has some benefits but not as much as it could be having.

Giovanni Asproni 00:13:57 Yeah, I think we’ll talk about this maybe later also when we talk a bit how to go with implementing an architecture modernization program because these kinds of problems seem to be potential showstoppers. Now a question about the architecture modernization approach you propose in the book. Now there are other also books about improving legacy systems, modernizing architectures in a way. What is different in what you propose compared to what was already there?

Nick Tune 00:14:28 I don’t really think there was much there, to be honest. I don’t really think there’s a lot of content out there on this topic. There are books that talk about refactoring like Michael Featherís Working with Legacy Code , but this book really isn’t about that on a tactical implementation level. It’s more on a, what are all of the dots you need to put together to do modernization successfully? So I was writing blog posts about this for a number of years, the different aspects, thinking about how this all fits together. I didn’t really see that much out there. And the book doesn’t really talk about an approach or a framework. The book covers the different topics you need to think about to do modernization well from a strategic level to an architecture level, to a migration level, but doesn’t say a specific way of doing things. It recommends techniques like wordly mapping and event storming and DDD, but you don’t have to use those and there’s no certification. It’s not a step-by-step process, it’s more tools and how you can fit them together.

Giovanni Asproni 00:15:28 And I guess the way I see it is also basically doesn’t stop simply what you need to do at the code base. It’s more general what you need to do at the system, broadly defined also what you’ve got around your software system or the organization, how it works, how people work together and skills and everything else. So it seems to be more general than the typical things you read about refactoring systems on improving architecture.

Nick Tune 00:15:58 Yeah, it’s not a refactoring book, it’s not many technical patterns in there. There is some stuff on DDD, there is some stuff around how you can migrate from your old to your new architecture. Typically you are going to be using some form of the strangler fig with an incremental migration. So you’ve got your new system and your old system running in parallel and you’re moving bits across and you have to ask questions like, do we migrate the res or the rights first? So we can take a bit out of the legacy. Maybe it’s a part of your website and it’s presenting some data, but the information still comes into your system from the legacy. So you have to have some synchronization between those two. So touches on those migration patterns. It shows how you can do things like software design, event storming and what that would look like in your code. But yeah, it’s not a detailed book about refactoring patterns.

Giovanni Asproni 00:16:51 Yeah. Okay. And now if we go into a bit more detail. So basically as we said, there is looks at all aspects of the system. And so as I understand it, this approach is about potentially revisiting pretty much everything around the old system. Technologies, the design, the features, the team structure, the team skills as opposed to doing the same things but better somehow. So it’s kind of revisiting things and having a better look at what we are doing with our system and see how we can have a better one, more than in many respects. But now I have a question about what kind of criteria we can use to decide in our system that if you want to modernize it, which aspects to revisit and which aspects to keep the same.

Nick Tune 00:17:38 Yeah, so some of the aspects we might want to revisit are the UI for example, do we just fix the code or do we make some UI improvements as well? Like the government example I talked about. Something else we might ask is do we make some improvements to the domain model and the data model? The simplest or the lowest version might be you just rebuild the old system in a new tech. But if your code’s tightly coupled and poorly designed, you’re going to bring that into the new version. So you might also desire to improve your domain model. Domain model is how you represent business concepts in your code. It’s the language you use. As systems get older, very often the way people talk about the products doesn’t match the words used in the software. Very generically we might talk about things like a customer, but in the code we might use words like a client.

Nick Tune 00:18:26 That’s a simple example. But over time we get these mismatches. So that’s one of the areas we can invest in the domain model. And for me the question is always what’s the business potential? We need to understand if we could completely modernize the UX, completely refactor our code and make it a perfect domain model, what would that be worth? How much would it cost? And then we can do that on a case-by-case basis for different parts of our system and work out what’s the best ROI for each area. In some cases it might just be lift and shift to the cloud and in some cases, it might be a complete revamp from the code infrastructure, UX domain model.

Giovanni Asproni 00:19:09 What about team structure, team skills and all these kinds of aspects as well? How do you decide that? Does it come after you decide, well ideally the domain model and potentially the architecture? Is this the ideal situation?

Nick Tune 00:19:25 Yeah, so as we are thinking about the new architecture and what the new boundaries will be, we have to start thinking how do we break up our business and how, what might we structure our teams around the architecture? So those three things are always the same equation.

Giovanni Asproni 00:19:40 Okay. So you mentioned UI, user experience, domain model. So of all these many aspects, are there some that are special in the sense that you always want to revisit them?

Nick Tune 00:19:53 Probably not. No. I don’t think anything is always. Sometimes you might keep your existing software and you might just change the code, you might fix some of the most complex bits in your legacy. Sometimes you might change the tech and not much change in the actual design of the software. So I think each of those bits is independently changeable. Obviously, it depends. Sometimes you might have a legacy system where the UI is very coupled to the code, and we can’t change one without the other. Or sometimes a legacy code might be so difficult to work with that if we want to change a tech, it’ll be so expensive we may as well fix it the main model as well. So the things can be changed separately, but we have to look at how coupled they are and how difficult it is to change each part independently.

Giovanni Asproni 00:20:40 Okay. Have you got a kind of real-life example showing how to apply some of these criteria how to think about what we need to revisit and how to proceed. Have you got any real project example you can give us? Of course, you know, without naming names.

Nick Tune 00:20:57 Yeah. So we could start with the UK governments. That was 10 years ago so probably not too many secrets there anymore. In that example, the government system had existed for decades. The business rules had existed for someone told me maybe even more than a hundred years. What was happening was when businesses were submitting their property tax assessments, the government works out how much tax you should pay and legally you are allowed to dispute that and complain. So everyone was taking their legal rights and disputing how much tax they pay. Government was getting a lot of these cases building up, they didn’t have enough support workers to process all the cases. And so businesses were taking the government to court. The government wasn’t even turning up at court because they didn’t have enough people to be there. They were losing a lot of money. Bad reputation doesn’t look good.

Nick Tune 00:21:51 So obviously when we’ve got a crisis like that, it’s very easy to make a case for modernizing and modernizing properly. I think the other aspect of the government was there was an initiative on a whole government level to do some modernization work and the government set global standards on what’s acceptable in terms of how the UX works, in terms of technology practices. So we have the individual business areas that have varying needs and then we might have global policies in our company that dictate where we have constraints where we must do things to a certain standard for example.

Giovanni Asproni 00:22:29 Hmm. In this case the government was setting some expectations and some standards at least for parts of the user experience in the systems stockholder?

Nick Tune 00:22:38 Oh yeah. They were setting a lot of standards. You can see online, um, GDS government digital service in the UK they had a list of like 10 or 11 points and if you were modernizing some systems, you had to go to an assessment at their offices in London. And if you didn’t meet all the criteria, they would say you’re not moving to the next phase. You’re not allowed to open your service to the public. Okay. If you couldn’t demonstrate for example, that you were iterating with real user feedback. So we had all those user research sessions and we were able to show them that and they were, , very happy with that. We were using the government design style kits, we were measuring our service, how it was being used. We were using things like continuous integration, which were minimum standards. We were putting our code open, so we ticked a lot of these boxes that were minimum requirements. Okay.

Giovanni Asproni 00:23:30 So in a way, in this case you didn’t have to think too hard to see what to revisit because there were some precise requirements in many respects in this

Nick Tune 00:23:39 Case. Yeah, on a global level there were some requirements about the minimum level, the minimum requirements in each area. And so that make a lot of decisions for you. We also are using this government platform that allowed us to spin up microservices, new front-end applications in a very conventional way with a paved road. So again, that already gave us a lot of the foundations and constraints we needed. So we didn’t have to think a lot there either. The technology and the infrastructure, those things were taken care of and decided for us.

Giovanni Asproni 00:24:10 And on another aspect, so you mentioned this before as well, that in the book list set of activities, tools, things that can be used to think about modernization and do that, but there is a lot of them. So of course there is mention of congress law, then there is event storming, worldly maps, tin topologies, there is behavioral code analysis. I mean you don’t quote it with the name but is you mention, code scene from Adam Thornhill and more as well. So do we need to use them all or how are we supposed to choose among them to proceed with our modernization efforts?

Nick Tune 00:24:48 Well, I’ll answer the question. Maybe JG has some thoughts on this as well, but I’ll give you my response first and then I’ll stop talking for a bit I guess. So if you are modernizing, you have to do a lot of things. You have to make a business case. What is the business trying to gain? How does the business plan to grow in the future? How is our current system preventing or making it too expensive to achieve those business goals? So things like wardley mapping to talk about your strategy. You cannot do strategy but you’ll probably make a lot of bad decisions so that, you can’t really negotiate that. Then you have to map out how your system currently works. You have to think about your current business processes. Think about do we change the business processes or do we keep them the same? You have to look at your existing code base and work out how do we refactor it; how do we start to break it apart? So the book does list a lot of techniques like that. But if you’re modernizing, these are the things that you have to do. Strategy, current state, future state business process, decoupling your architecture, migrating from the current state to the future state. So you don’t have to use all the techniques I recommend, but you have to answer all those questions. You can’t avoid them really.

Giovanni Asproni 00:25:58 And how can people choose among those techniques? Is there, some simple criteria they can use if there is one?

Nick Tune 00:26:06 I mean it depends on what problem you’re solving first. I think we always go to the left, let’s say with the why and then we move right to the how. So if someone says to me, what do we do next? I have to understand where are we in the process? Do we have a clearly defined business strategy and problem we’re solving? Well we need to start there first. Okay, if we have that, then we can spend some time thinking about which are the most important areas of our business to focus on. Then we can start thinking about which parts of the system would most need to change to improve those parts of the business. Then we can start thinking about how do we start breaking up the system, mapping out the current state in detail, making a future plan. And if we have that plan, then we can start thinking how do we execute, how do we actually put work in our backlog and start doing the work?

Giovanni Asproni 00:26:57 Okay. Have you got an example? Again from a real project in doing this, I mean at least showing how you go from, I donít know from the strategy to the rest of the work, maybe mentioning some of the tools just to put them in some kind of place to give people some mental model on how to think about these things.

Nick Tune 00:27:18 I think different clients I’ve worked with have been at different stages. For example, worked with a company does electric vehicle charging infrastructure. They already had a very clear business strategy. They wanted to grow, open up some new, I think they were calling them sites where you can go and actually charge your car where the electric vehicle charging stations are installed, for example. Now they had a very ambitious business plan, expanding it across multiple countries. And they were now wondering, okay, the business goals are clear. How do we get there on a technical and an organizational level? How do we start taking the current systems and having more clearly defined areas, different business domains, and how do we start organizing our team so that they can each work independently in different areas of the company and help us grow as quickly as possible. So the question was in that case, where do we start?

Nick Tune 00:28:13 What’s the right starting point? People I worked at the company, the architects and the CTO and the CPO, they identified some candidate areas where they thought it might make sense to start, but they had three of them and they weren’t sure which is the right domain to start with. If we start here, it will benefit in this way, but if we start over here, we can do things more customer facing. So the question here was all about how do we start. So in this case, me and my colleague Eduardo, we worked in an enabling role. We formed an architecture modernization enabling team and the goal of that team was to think about this modernization journey and to organize some workshops to decide what would be the right starting point, what would be the correct domain to start with? And then the next step would be to have this in-person workshop where we do events storming a map out the domain, identify the sub-domains and the team structure and actually build that roadmap to let’s start modernizing this part of the business. That answer the question? You want me to dig into any now is a bit more detail?

Giovanni Asproni 00:29:12 Yeah, I answered the questions. Yeah. So you said of course the strategy was already clear. Maybe if the strategy hadn’t been clear, I would imagine potentially some workshops potentially also using some wardley mapping to decide which direction.

Nick Tune 00:29:26 Exactly. If the strategy wasn’t clear, we would’ve had to go more in that space. There are some caveats around this. If that parts of your system need to be modernized and you want to demonstrate the foundations and put those in place, well you can start without a clear strategy. So that those things could happen in parallel.

Giovanni Asproni 00:29:46 Yeah.

Giovanni Asproni 00:30:02 And now a question I think this one probably is Jean-Georges, because I see that in your approach you give a significant role to data engineering, at least reading the book, there is an entire chapter dedicated to it. Data engineering data measures. First question, what is the relationship between data and architecture?

Jean-Georges Perrin 00:30:21 So I think you can’t do anything without data. So you can do all the architecture you want, all the nice design, all the modernization you want. If you don’t have data, you are not going to go very far

Giovanni Asproni 00:30:34 In modern systems. I see that now everybody talks about data. Yeah. So well at least in the system, enterprise systems that I see there is always very often at least data engineering team. So probably everybody wants to capitalize from the knowledge. Right. Now when in a modernization effort, what is the typical aspects related to data that maybe in a typical project, in a typical system people want to take care of that maybe we’re not there. I can imagine, I donít know, when I started working data was mostly database queries to satisfy user stuff. But now there is more analyzing interactions trying to get inside. So maybe you can tell us a bit more about that.

Jean-Georges Perrin 00:31:15 Yeah, sure. So I think data has significantly changed and the data engineering practice has not, and this results in creating very centralized team. And I completely relate to what Nick was saying about the two the two CTOs fighting a little bit because in big organization you will very often have a data organization and you will have some enterprise or software organization, I mean enterprise architecture, software organization. And they don’t often get along. And this is also due to these different ways of working. Okay. Technically you can call it socio, the socio aspect of the platform or the socio aspect of working, but it’s really about the ways of working. In my experience, I’ve seen very often centralized data team that were just growing, I wouldn’t say exponentially, but they were growing very fast. But just to cope with what was needed to maintain and the maintenance cost prevented them to actually go to some modernization.

Jean-Georges Perrin 00:32:27 And so every time I’m not trying to throw the stone or throw the ball towards the centralized data team, but a lot of what we’ve seen, and for me the book is also the conclusion, probably not the conclusion, but a good step of where we’ve been going in modernizing since I would say early 2000 to now, okay. And you’ve seen the advent of agile methodologies, you’ve seen the advent of all this scrum, this product thinking, et cetera, which in software and of course then in architecture has been very strong but has not been the case in data. Okay? Data has always started to focus on I’m going to do my job. Okay. So when you’re thinking about, when I’m discussing with a lot of data engineering teams or data engineering organization is they say, yes, we are agile, we are doing sprints, okay.

Jean-Georges Perrin 00:33:22 And basically their sprints are mini waterfalls of two weeks and it’s not working. So when I say it’s not working, it’s not scalable. Okay? So, so all the lessons we’ve learned in software and that mostly Nick put in the book, we are not using that in the world of data enough. Okay? So, and I still think very strongly that data mesh is one of the solutions. Probably one of the ideal you want to go towards. Okay? So if you listen to Gartner or some people they say, oh, data mesh is already dead or, but if you look also at Gartner, the same Gartner, and they say you look at the component of what data mesh is and the four principles that are translated directly into things that are on the rise within the Gartner environment for example, I think that’s, that’s where we are, okay? We want to modernize architecture for all the good reasons that Nick said and, and you as well Giovanni and the thing is right now, but the data needs to follow up. Okay? So there’s no way we can have a state of a architecture or a modernized, whichever level of maturity you want to give it without also modernizing data.

Giovanni Asproni 00:34:33 Have you worked in any projects in any efforts where actually data, modernizing data was the major driver for an architecture modernization effort?

Jean-Georges Perrin 00:34:43 I’ve seen a few, typically two days when companies are going from on-prem to the cloud, okay and large companies, not everybody is fully migrated and some are actually coming back. But the thing is, and you see the, you see the pitfalls as well when you are part of this kind of project is that I’m trying to do a lift and shift, okay? But for data, having a lift and shift is much more complicated. Let’s say I am living on-prem data warehouse like an ExaData or Teradata and I’m going to the cloud and doing a Redshift or BigQuery for example. Because usually you also change the technology of the tools you’re using. That is very complicated for as part of the architecture redesign. Because easy tendency is to say I’m going to have direct pipelines that are mimicking what I have on prem to what I’m going to do on in the cloud.

Jean-Georges Perrin 00:35:34 Okay? And it’s very complicated because first the technologies are different, second the expectations are different, the billing is different, and the performance is different. So I’ve seen a lot of projects where things were optimized for on-prem. Okay? So even using stuff like a SQL server completely on the biggest machines they could have and using SQL server as a data warehouse technology on-prem and then you go to the cloud and then you say, okay, I’m going to do Redshift. But all the optimizations they put in place for as SQL servers, the way the structure, because at this point you are tweaking the architecture, you’re tweaking it to, you still follow the guidelines of the architecture, but the implementation is so close to the engine itself that you are not benefiting from what for example, a Redshift could give you, right? You’re still having the same models that you would have in SQL server.

Giovanni Asproni 00:36:31 If I understand correctly. Let’s see if I’m understanding correctly your point, basically you’re saying when you have a modernization effort that involves data. So for example, shifting the system, putting that into a cloud from on premises, trying to keep pretty much the same shape of the system but in the cloud actually is a problem. And probably a better way to do that would be to revisit the way data is managed and maybe you need to reduce some things differently because if you have stuff like say on the premises SQL server optimized for SQL server, then you move, you said Redshift, it’s like well now we need to do something different with the data otherwise will be suboptimal.

Jean-Georges Perrin 00:37:10 You’re exactly on point. And that’s why I think Dan and I guess that Nick agrees on that as well is that’s why I think like data product thinking and data contract as well help us in the data engineering world to go there. Because what you’re actually giving your customer is disagreement this proposal around data management.

Giovanni Asproni 00:37:35 Okay, so now I’d like to talk about the implementation aspects, yeah? So the first question is, we may be thinking that we may need to modernize the architecture, yeah? Maybe we have a gut feel or something that seems to be not working well, features that maybe are a bit slow to be developed or some aspects like this. But what are some criteria, some kind of crisp criteria if you like, that we can use to decide that an architecture is worth modernizing?

Jean-Georges Perrin 00:38:08 On the data side, it’s mainly, for example, I would say it’s the main driver is the change of philosophy versus on-prem to going to the cloud or new laws that come in effect. Okay, I live in the US but I travel a lot to Europe and I see the impact of what GDPR has done on data and a consequential higher maturity when it comes to governance of data in Europe versus the US, there’s also a difference. Okay, so that seems like it’s a very positive difference towards Europe, but there’s also a lot of varied positive difference towards the US. But the thing is that’s probably not the topic for today. I think that what I’m seeing is that those two are the drivers, right? There’s always, there’s kind of three main drivers when you’re looking at modernizing a project for me is, either you want to save money, you want to make more money or because the regulator pushes you to go in a different way.

Jean-Georges Perrin 00:39:06 Okay? So that’s, I think that’s the main three drivers for me and very often for data it’s a regulation. Okay? So I work in the financial sector in the US I work also in healthcare, regulation on data here is very big and pushes a lot of those aspects of modernization. Okay, you’ve got to build more reports, you’ve got to have this regulatory implementation et cetera. That’s what the kind of the driver are. And in terms of work, the one sequence of that is that you either goes a traditional way and you’ve got this mini data engineering project that are going there or you’ve got a more global picture and you say, okay, well now I’m done with that and I want to do a modern data engineering approach with once more data contract data product or data mesh.

Giovanni Asproni 00:39:53 Okay, so we understand from the data perspective what about other aspects? So well as we said, data is a lot of regulatory things. It could be even saving money or make more money. The other aspects. So, in addition to data, so anything else that you need, modernizing what other criteria can we use there or are the same or are the criteria the same?

Nick Tune 00:40:17 I think it can be various things. Sometimes it can be around costs, sometimes it can be around support costs. So you might have lots of very manual support processes. You might have customer support teams or call centers with hundreds or maybe even a thousand people and you’re like, what if our software didn’t cause so many errors or we could fix problems more automated, we could save millions over the course of a few years. So that might be one around cost savings. I think most often it’s around growth opportunities. Like what are the things that we could do as a company that currently aren’t possible or are too expensive. New products moving into new markets currently working at pay fit, a French company for example, being very successful in payroll now has other big ambitions companies grown. They want to build newer products grow the company so they can build more capabilities, become more profitable as a company.

Nick Tune 00:41:14 So often a lot of these factors are happening at the same time as the company’s growing. The system that was fine before, that worked for one scale for building one product for a certain kind of customer suddenly as you want to scale the company and the organization. Now the current system is maybe a bit too coupled in places it wasn’t a problem before, but it is now. Or as you try and do more things and support costs grow. So yeah, it can become expensive to do things. A sign to look out for is when maybe you got some product manager or some salesperson who says could we build this new API for this partner? It’s a really strategic partner. All they need is a search API that works across three bits of data.

Giovanni Asproni 00:41:57 They need easy. That sounds easy.

Nick Tune 00:41:59 That’s all they need. And you are like, well those three bits of data live in three different legacy systems. We have different teams that own different parts of it. Those systems are currently very unreliable. We deploy them once every three months and you are like, oh my gosh, if I work for any sane company with a good architecture, we can implement that in a day’s worth of work here it’s going to take three or six months. Giovanni we have to say no to things that should be very easy and should be very valuable. So that’s something to look out for definitely, when things are too expensive and we’re saying no to things that could have a decent impact for the business.

Giovanni Asproni 00:42:35 Actually I’ve got now a different question that is, have you ever come across a situation where you actually decide that, it’s not worth modernizing, let’s keep it this way. So people were considering about modernizing the system then had a deeper look and say actually it’s not worth the effort and the cost. Have you got any example of this?

Nick Tune 00:42:57 As a consultant that happens all the time. Yeah, they call you in, they’ve got these big ambitious business goals. They want you to tell them how they can modernize their systems very easily and you tell them, well it’s going to take three years. You’re going to have to put some projects on hold while you do the modernization work. You can still do some feature work, but you have to balance modernization versus product work. And they’re like, we can’t justify to the CEO. And one client I worked for, I was talking to a Chief Finance Officer, and she was saying, why are my developers talking about microservices? Why do I need to sign off this budget to do some modernization work to move to microservices? Yeah. So this happens for a variety of reasons.

Jean-Georges Perrin 00:43:39 I would even add to that, I was probably a little bit more in the enterprise side as well is that instead of modernizing its buy something off the shelf. Okay. And I’ve seen this scenario as well. So oh you start or you’re thinking that you’re going to work on a project which is going to be built a new feature or a new feature set and then at the end of it, here are actually going to do an integration project between your CRM and your loyalty program for example. Okay, so there’s also this where it’s not worse modernizing the existing or just hey let’s get something off the shelf.

Giovanni Asproni 00:44:14 That’s true. Sometimes just buying something new is the best idea. And I think this is part of the strategic work you do at the beginning of a modernization effort. So maybe with wardley mapping that you decide what is called to the business, what you can buy and take some decisions in this respect. Am I correct?

Nick Tune 00:44:34 Yeah, definitely. I was in New Zealand a while ago, earlier this year and on day one I did some events only with this company and they mapped out this current system that needed lots of improvements. They were going to fix every different aspect of it. When we did the wardley mapping, they were like, we could fix all of these things and modernize this legacy system, but if we do that, we’ll have no time to work on all of this new AI stuff, we want to do. Whatever you think about AI,let’s just put that on hold. All this other new stuff we wanted to do. Well we only have a limited amount of people to work on this. And so that pushed them towards, yeah, we could buy something off the shelf here and that would free us up to work on these more interesting AI things that we want to work on.

Giovanni Asproni 00:45:17 Okay. And now another question that I suspect I know the answer but I’m not quite sure. So if you actually come across any systems that were designed and managed in such a way to be evergreen therefore needing no modernization and often, we talk, and I talk as well about evolutionary architecture and this kind of systems that ideally should allow you to create a system that follows the business needs. It may be with a lesser amount of technical data accumulated. So have you come across such a system in any of your projects?

Nick Tune 00:45:58 Yeah, I’ve worked over the last 15 years probably with two or three companies that fit that criterion. And I’ll tell you what was consistent about those companies. They had teams that were autonomous, those teams were doing extreme programming practices like TDD and pair programming. They were very focused on continuously improving their work. They were talking about refactoring all the time. They were always trying to learn new stuff. They had like training caterers during work hours and stuff. People might say, ah, these are all just geeks who are going crazy about craftsmanship and everything and all extreme programming. But those were the companies I’ve seen that didn’t need to do this big modernization work because they built quality into everything they did, and it was an ongoing topic.

Giovanni Asproni 00:46:46 And the fact that it was ongoing was I guess a concern also for the senior leadership to keep this quality high I would imagine. So it was not simply the teams deciding to do that.

Nick Tune 00:46:57 It was a mix of having a good CTO who understands the value of these practices. I think having a good CTO is consistent in all those experiences. A CTO who understands quality and having a CEO who trusts the CTO and people used to say things like, we’re not a tech company, we’re a music company, we’re so focused on tech here. But the CEO could see we had multiple teams. I think at the start there were six or seven teams and that grew over the years, deploying to production every day. When the business can see results like that, yeah that buys you some space to you don’t have to justify anything. When a customer raises a bug and you fix it in an hour deployed to production, well that kind of buys yourself some credibility and a lot of companies, business leaders can see that, okay, they talk a lot about tech, a lot about refactoring things like DDD extreme programming, but they can see clear results from that.

Giovanni Asproni 00:47:54 Yeah, that’s quite interesting.

Jean-Georges Perrin 00:47:56 Nick, if I have a question for you, you mentioned that there were three companies over your last 15 years. What’s the percentage? What’s the three-company represented? The percentage of company you’ve dealt with?

Nick Tune 00:48:08 Ah, it’s probably about 3%. Yeah, very small number and even sometimes it’s not even the whole company, it’s when you’ve got a large company you might have pockets. So I worked with one of the UK’s big supermarkets chains a few years ago. They had some teams who were doing amazing work and other teams that are building up a lot of technical debt. So yeah, in a big company, also like when I worked at Salesforce, some teams are doing great work, continuously improving, having high quality and others were just building bigger and bigger legacy systems every day.

Giovanni Asproni 00:48:43 And now a question for I guess both of you is have you got any recommendations for fitting the modernization work with other priorities of the business? Because I think we mentioned this a few times before, that this is one of the contentious points is like, well if we spend time modernizing, we cannot really spend that time adding new features. But are there any suggestions you have to give people on how to talk about the need of modernization, how to negotiate with the other priorities? Of course I said both of you because we talk about more some of the software aspects but for data as well and data being particularly sensitive and complicated too.

Nick Tune 00:49:30 So I would say a few things are always important or very helpful. I would say the first one is having a consistent message at the C-level. So are the CEO, the CTO and the CPO all giving a consistent message or is the CTO talking about modernization and the Chief Product Officers talking about lots of new features? Well if you’re giving mixed signals at the top of the organization like that, you’re already off to a bad start. People in the middle managers, the directors above them, the people working in individual teams like developers and product managers, they’ll have to decide modernization or new features. We’re getting different messages and most of the time people will just default to building new features because that’s visible. That’s what they’ve always done and that’s what they’ll get rewarded for. So you need to have a very consistent message that modernization work is important in this company and a very clear message of what it’s worth for the company of why not building a feature now is worth 10 or 100 times X more value in one- or two-yearsí time.

Nick Tune 00:50:32 So a clear connection between modernization and business goals, consistent message, and a consistent message from the different leaders at different levels of the company as well. So there’s no doubt, I would say those are the fundamentals. And then when you’re building your roadmaps for the year or for the quarter or for the semester, again that’s another chance where I have to make more fine grain decisions. I think building different possibilities is key. So build three different roadmaps. I would say. What’s your lots of modernization, not much product work, what’s an equal balance and then what’s mostly product and not modernization. So you can talk about the different trade-offs and you can ask different stakeholders to give inputs on which one they value the most.

Jean-Georges Perrin 00:51:20 I would say in data all that applies. But one thing which is difficult in the data world is to measure the RI of data. Okay, so what’s my re return on investment when it comes to data project? One of my recent experiences was in the risk division of a major FinTech and how do you measure that you didn’t get fined; you didn’t get any money stolen or it’s very difficult okay to determine that. But you’ve got to find this KPIs, and you’ve got to stick to them. And as Nick was saying, you’ve got to report that to your CPO, CTO and CEO. Okay you’ve got to report to your leadership all you are doing these KPIs and before you start the project you’ve got to do that. And I think that if you keep that in mind, as Nick was saying for the greenfield is lucky three greenfield project, you can get almost anything. Okay. So then you get the credibility, you are constantly on your KPIs, you’re constantly delivering value to the organization. And then it’s easier to say, okay, well now I want to start this project. But it’s always going back to, it’s starts with a business decision.

Giovanni Asproni 00:52:38 Okay. And now let’s try to end with a bang. So a good question to end with a negative note. How often do architecture modernization programs succeed or fail in your experience?

Nick Tune 00:52:50 I’ve been burned a lot of times by things I didn’t work out. Like I’ve been involved in projects where they say things like, yeah, we promise we’ll let you focus on that this year. And then before you’ve even really got started, it’s like, can we build this one new thing, this big new product feature? It might take a couple of months, but once this is done then we’ll get to this deeper modernization work. And then it just carries on. And you actually don’t do any modernization work.

Giovanni Asproni 00:53:17 How often does this happen? Is it a common thing? Is it something you found in many places? This kind of attitude?

Nick Tune 00:53:25 Yeah, it’s quite frequent I would say. Ones that work the best are ones like the UK government where you have this minimum level. When you have a very strong problem or a crisis like the government had, that’s always a very strong motivator that helps you to modernize. Because there’s a clear need, it’s hard to avoid it. And when you have those standards, those minimum acceptable levels, those global policies, that means you have a need, and you have some standards. So you’re going to modernize and you’re going to do it to a good standard not taking shortcuts because that’s not allowed. So when those recipes are there, yeah, it normally does work out. Maybe not everything goes perfectly of course, but less than half the time, maybe 25% of the time I would say things grow really well. Okay. Over the course of two or three years.

Giovanni Asproni 00:54:11 And so if you would sum up let’s say the most three or four common causes of failure, what would they be in your experience?

Nick Tune 00:54:19 Yeah, the first one is, like I said before, not having a strong enough reason and a compelling reason and sticking to that reason and not just defaulting back to product work. That would be the first one. Then I think it’s around having the expertise in the company to do modernization work. Sometimes, yeah, the top of the company, the CTO might have a big vision, but you look at how the teams are working, they don’t understand why it’s important to decouple different parts of the system. They don’t understand the concepts like DDD. They don’t understand why it’s important to decouple your business logic, your application layer logic and your UI. Like we’ve always just built these things tightly intermingled. What would be the benefit of doing that? So I think the other one’s having the skills to do that. Yeah, so the vision, the skills.

Nick Tune 00:55:09 And then the third one is probably things normally get stuck. You can get stuck in a halfway phase where you start modernizing and don’t finish and you’ve got the new bits of a new system and bits of an old system. So I think really thinking through the migration plan of how you get from A to B does it need to be fully defined upfront, but you need to really think ahead to what can stop us finishing the migration and what can we do to predict and anticipate as many of those things as possible so that we don’t end up in a state where we’re stuck halfway and the system’s more complex than it was before. And I would say one of the examples that I see a lot there is when companies have some kind of platform or some patterns where the new system can easily talk to the old system. Maybe you have an event driven architecture, and you can easily publish an event from a legacy which is consumed by the new system or the new system can publish in events and it can be handled by the legacy. So having those things able to talk to each other easily reduces a lot of the friction there I would say. But it’s not the only thing, but one of the things to think about.

Giovanni Asproni 00:56:16 So it seems to me that the last point maybe is kind of lack of appropriate planning really. It’s like when you say when you talk about not being stopped halfway finding big risks that we have not anticipated. So seems that people don’t take the time to actually plan.

Nick Tune 00:56:36 It’s partially planning, but it’s also around discipline. Once we start this thing, if someone tempts us to build a new product feature, we have to be really disciplined and say we’re halfway through this migration from the old to the new. We currently have a lot of complexity. It might be even more complex. And before we started, because we have a new and old data in different places, this UI shows one price. This UI over here shows a different price. We want to avoid this synchronization issues which can happen when we mid migration. So it can be planning but it can also be, as I was saying, discipline to not diverge from the plan or to not make too many concessions.

Giovanni Asproni 00:57:20 Okay. And how can we increase the chances of success then?

Jean-Georges Perrin 00:57:25 Going back over your series of questions here, I’ve been lucky, I would say enough that I’ve been in projects where modernization was always a success, but it was not always the expected goal we set at the beginning. Okay. But as Nick was saying, you’ve got to be very careful and not to having like two systems that runs in parallel. But my experience, maybe I was lucky enough, is that we always managed to bring incremental value even through modernization. Okay. So, and that’s I think something to keep in mind because yes you can have this big planning and I agree that for some project you don’t have a choice having this major planning. But if you are good at Agile, you don’t always have this, you’ve got a roadmap, but you don’t have a second-by-second planning. Right. So I think that here you can still combine that with incremental added value to your modernization like for any project.

Giovanni Asproni 00:58:25 Yeah. Nick, anything to add to this?

Nick Tune 00:58:29 I agree. I think it’s always going to be an ongoing balancing act. You just need to make sure that it doesn’t balance too much in the way of new product features and you don’t do any modernization work and you’re stuck in this halfway state. I work with people a lot and we often talk about modernization and they’re going to get something out of this, but they don’t want to do it. It might be an engineer or a customer support person and they’re like, yeah because the last one didn’t finish. Now I have to use three systems and not two systems. So not finishing can have big consequences. So, the key thing is to make sure we do keep making progress. Either we don’t do it or we finish it, but we don’t want the worst thing, which is to be stuck halfway and have this Frankenstein system.

Giovanni Asproni 00:59:15 I’ve worked in several of those systems to help fixing them. I’ve experienced with, especially big banks at these things like that one system. And then they said that they needed to modernize them, create a completely new one, and then there had two systems to manage and then they said the second one was not good enough and create the third one. And then they have three systems in production. Okay guys, so now I think we’ll it’s time to wrap up. I think we’ve done a quite a good job introducing architecture modernization. So thank you very much. Was there anything we missed that you’d like to mention?

Nick Tune 00:59:34 Exactly, exactly that.

Jean-Georges Perrin 00:59:54 Don’t forget the data.

Giovanni Asproni 00:59:56 We wonít Thank you, Nick and Jean-Georges for coming to the show. It’s been a great pleasure for me. And this is Giovanni Asproni for Software Engineering Radio. Thank you for listening.

Nick Tune 1:00:00 Thank you.

Jean-Georges Perrin 1:00:09 Thank you, Giovanni.

[End of Audio]

Tagged as:
Join the discussion

More from this show

Menu

Recent posts