Artie Shevchenko, author of Code Health Guardian, speaks with host Jeff Doolittle about the crucial role of human programmers in the AI era, emphasizing that humans must excel at managing code complexity. Shevchenko discusses these concepts and key takeaways from his book, including the three problems caused by complexity: change amplification, cognitive load, and the most severe, unknown unknowns. He suggests that maintaining code health should be viewed pragmatically as a productivity question, requiring an ownership mentality and product focus to balance short-term delivery with long-term maintainability. The episode also covers vital processes such as using design documents for upfront analysis and code reviews, highlighting four goals: high code quality, knowledge sharing, delivery speed, and — most important for team productivity — psychological safety.
This episode is sponsored by Monday Dev
Show Notes
Related Episodes
- Episode 633: Itamar Friedman on Automated Testing with Generative AI
- Episode 603: Rishi Singh on Using GenAI for Test Code Generation
- Episode 626: Ipek Ozkaya on Gen AI for Software Architecture
- Episode 520: John Osterhout on A Philosophy of Software Design
- Episode 662: Vlad Khononov on Balancing Coupling in Software Design
- Episode 574: Chad Michel on Software as an Engineering Discipline
Other References
- LinkedIn: @artie-shevchenko-67845a4b
- Complexity in Managing Modularization by Poul Kyvsgaard Hansen and Hongyi Sun (Computer Society Digital Library)
- Measuring Enterprise Architecture Complexity by Maria Eugenia Iacob et al. (Computer Society Digital Library)
- Managing Design-Time Uncertainty by Michalis Famelis and Marsha Chechik (Computer Society Digital Library)
- Managing Technical Debt Under Uncertainty by Suwichak Fungprasertkul and Rami Bahsoon (Computer Society Digital Library)
- Cognitive Complexity — An Overview and Evaluation by G. Ann Campbell (Computer Society Digital Library)
Transcript
Transcript brought to you by IEEE Software magazine.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Jeff Dolittle 00:00:18 Welcome to Software Engineering Radio. I’m your host, Jeff Doolittle. This episode will explore the role of human programmers in the AI era. Topics around machine learning and AI have come up frequently in previous episodes. Some examples are Episode 633 with Itamar Friedman on Automated Testing with Generative AI. Episode 603 hosted Rishi Singh on using Gen AI for Test Code Generation. And in Episode 626, Ipek Ozkaya joined Software Engineering Radio again to discuss Gen AI for Software Architecture. Today we’re hosting Artie Shevchenko, who is a former Google software engineer, ex-CEO, and founder of a startup. Currently, he’s a software engineer at Canva and a lecturer at ITMO University, which is famous for winning more programming world cups than any other university in the world. He is also the author of the recently published book Code Health Guardian, the Old New Role of Human Programmers in the AI Era. Artie, welcome to the show.
Artie Shevchenko 00:01:22 Thanks for having me.
Jeff Dolittle 00:01:24 Glad you’re here. The subtitle of your book, as I said, is the Old New Role of Human Programmers in the AI Era. In your view, what does it take for programmers to be successful in the era of AI?
Artie Shevchenko 00:01:37 I think there are three things you need to be great at. One is collaborating with AI. Another is delegating to AI. These two are pretty much on the hype right now. Everybody talks about Copilot and Cursor I guess. But there is also one more thing there. You really want to be better than AI at something. And if we were talking about software engineers, I would say your best bet is to become an expert in managing code complexity, just because complex reasoning is hard for AI and in software engineering, that’s probably the hardest problem we have. How do we keep our code bases reasonably simple? So yeah, honestly, I wouldn’t overfocus on learning these AI tools like Cursor and Copilot today and really spend some time to make sure that you’re ready for this new role of human software engineers, which is Code Health Guardian. That’s how I call it. And that’s the title of the book.
Jeff Dolittle 00:02:39 So you mentioned the idea of complexity in case it’s unclear to our listeners, help us connect. How does complexity relate to this idea that you describe about programmers as Code Health Guardians?
Artie Shevchenko 00:02:50 Yeah, definitely. When I say code health, I basically mean code that is simple enough, code that is reasonable, simple or maintainable. So if that’s the hardest problem in software engineering to keep our code bases simple enough, then it’s likely that we will continue to be better than AI in this for at least some time, maybe years, maybe even decades. So that’s basically the link there.
Jeff Dolittle 00:03:20 Why does AI struggle to keep things simple in your experience?
Artie Shevchenko 00:03:24 Complex reasoning is hard for AI, so it looks simple on the surface often. I think that’s what AI is very good at. It gives you an impression that it understands things, right? And it gives you an impression that, okay, that’s super simple. It’s like 1, 2, 3, here you go. But sometimes the real world has very significant challenges that need to be deeply understood, and that’s what humans still do better than AI at this point.
Jeff Dolittle 00:03:56 So would you say sometimes the AI comes up with simplistic solutions rather than simple ones? Okay. So your contention is that humans still need to maintain the sense of complexity management and the way we do it is by guarding the health of our code?
Artie Shevchenko 00:04:08 Yeah, exactly.
Jeff Dolittle 00:04:09 And you mentioned maintainability. Are there other aspects to code health, you mentioned simplicity and maintainability. How else would you characterize code that is healthy?
Artie Shevchenko 00:04:19 That’s basically the definition. If we’re talking about definitions, I just use the term health as an alternative to saying that is reasonably simple or pragmatically simple. But I like to think about code health as a productivity question. I think it is ultimately a productivity question. So if you think about human productivity or programmers productivity, the first thing that comes to mind is team productivity, right? There are a bunch of factors that make team either productive or not productive, like psychological safety, reliability, clear goals, clear responsibilities, impact, all these kind of things. But if we zoom in a little bit into an individual in a team, still in a reasonably productive team, you will have individuals which perform really great and in individuals that perform not that great. And historically, if you’re an engineering manager dealing with this issue, the question you’re asking yourself is, well, is it a skill problem or is it a will problem?
Artie Shevchenko 00:05:28 Right? But I find that often the answer to this question is no. It may be all good with skills and it may be all good with the will. So for me, I don’t think I ever had any major issues with the will problems or skill problems. For me I definitely had some issues with finding right balance between long-term and short-term productivity. And that’s what I call term productivity. It’s pretty hard to talk without any visuals, but if you can imagine having a spectrum where on the left side it is short-term productivity and on the right side it’s long-term productivity, if you’re too focused on the long-term productivity, it’s kind of obvious why you’re not productive because you’re often planning to be productive in the future. Right?
Jeff Dolittle 00:06:19 Someday I’ll be productive. Right?
Artie Shevchenko 00:06:21 Yeah and if you’re focused too much on the short term, you become too tactical. You create too much complexity along the way, and that’s what AI agents could do for us too. And you really want to have the balance there. You want to care about user’s needs right now, to deliver now, but also care about user needs like a year from now so that you don’t create too much complexity so that you cannot execute in a year from now.
Jeff Dolittle 00:06:49 Right. So in a few interesting insights from that one, you didn’t seem to say that long term or short term is better. There’s a balance between long and short-term productivity. Did I understand that correctly?
Artie Shevchenko 00:07:00 Yeah, that’s correct. And definitely there are times when you need to focus on the short term for several weeks or maybe even months and just deliver. There are times when you need to focus more on long-term, but in general, you need to find the healthy balance between these two extremes and be somewhere in the middle.
Jeff Dolittle 00:07:19 Okay. And then the other aspect I think you’re implying as well is that it’s human’s job to ensure long-term productivity that AI might make a short-term productive, but that at least as it currently stands, it’s not going to solve the problem for us of long-term productivity. Is that correct?
Artie Shevchenko 00:07:38 Yeah, exactly.
Jeff Dolittle 00:07:39 Okay, great. Well, that might lead really well into my next question, which is what motivated you to write Code Health Guardian?
Artie Shevchenko 00:07:45 Well, I guess I was always passionate about knowledge sharing. So even starting with my uni years, I remember we were doing classes for high school students and even primary school students a little bit. And right now I am teaching in university, doing university classes. So yeah, this passion for knowledge sharing was pretty much always there. But in terms of this particular book, I guess there are three key events there apart from doing university classes, which definitely prepared me for writing this book because they inspired, the content of those classes, inspired the content of the book very, very much. But there are two other key events I would mention. The first one is doing good code reviewer internal classes at Google. So the story there is that when I was at Google, I was pretty much every day I was just spending a little bit of time reading random documentation. At some point I started realizing that there is so much information about how to do code reviews on our internet, and I just wanted to consolidate all of that. So that’s what I did. I just consolidated all that, created a single slide deck for that, and at some point that slide deck became pretty popular and I even started getting requests from other Googlers asking whether there are any tech talks about this, if there are any classes about this. So I started doing the good code reviewer classes internally at Google, and that was pretty successful. I think in just one year we had over a thousand Googlers software engineers who went through those classes. We got like astonishing 4.5 rating based on over a hundred reviews. And that’s really golden standard at Google for tech talks. Like, you know, when we’re doing onboarding, that’s roughly the rating you get for those onboarding classes.
Artie Shevchenko 00:09:58 And like if you can, if you can reach 4.5, that’s kind of, that means you’ve done a good job. So yeah, that was the first key event doing those classes. It gave me quite a lot of confidence, I guess, to start university courses later. The second key event was John Osterhout book presentation in Google’s Sunnyvale office. So I just randomly ended up being there. I had no idea what to expect. I guess it was just advertised by some internal channels. And I ended up being in that room just knowing that some guy’s going to present on his software engineering book. And when he was talking about software engineering and complexity and complexity causes and complexity symptoms as he called them, that really resonated with me and was like, oh yeah, that really makes sense. And yeah, then when I was starting the university classes, I think to a large extent, maybe up to 50%, the initial content of those university courses was actually based on John’s book philosophy of software design. Also, of course my own experience at Google and a bunch of other books like Pragmatic Programmer, but John’s book was very, very key.
Jeff Dolittle 00:10:02 That’s great. And I’ll put a link in the show notes to Software Engineering Radio Episode 520 where I interviewed John Osterhout about his book, A Philosophy of Software Design. So that’ll be good for listeners to be able to dig into.
Jeff Dolittle 00:11:27 Can you share a real-world example of a project or a team that neglected code health and it led to significant productivity issues?
Artie Shevchenko 00:11:36 Yeah, sure. I think actually these stories is what made me truly pragmatic and truly care about code health to be like strategic about code complexity. Not on one of the extremes on the term productivity spectrum. And that’s stories from my startup, from the company I was running. I think that was really, really motivating. Not only because I as A CEO cared about products so much and productivity, but also that because it was a self-funded startup, so I was actually paying engineers. To do the work. So if you see having productivity issues, you have really good incentives to fix it. And we had problems on both sides of the spectrum. We definitely had problems on the to clean code approach, like more kind of a long-term, and investing in a long-term productivity when both on the backend I was spending too much time on stuff that did matter that much, especially in the startup world where you need to iterate so fast.
Artie Shevchenko 00:13:41 But on the frontend side, I also saw frontend engineers building some sort of frameworks and reinventing wheels and to being kind of strategic about what they’re doing, but that’s not what I wanted. And we needed to iterate, we needed to get feedback from customers very quickly. So yeah, that’s on that side of the spectrum. But interestingly, we had problems on the other side of the spectrum as well. So sometimes they would just iterate and deliver stuff. But after a year, I think, or so frontend engineers came to me and said like one of the key pages I wanted changes on, they just couldn’t do that. They just couldn’t deliver that. They were just telling me, you know, like, it’s too complicated. I’m doing something, I’m breaking something and I don’t really understand what I’m doing anymore. And we had to start from the beginning. So we had problems on each side of the spectrum, which was pretty interesting.
Jeff Dolittle 00:17:32 Yeah, that’s fascinating. So you’re basically saying there was sometimes too much long-term thinking and too much short-term thinking at the same time.
Artie Shevchenko 00:17:40 Yeah, I guess in different parts, in different projects.
Jeff Dolittle 00:17:44 In different parts.
Artie Shevchenko 00:17:45 In different parts, yeah.
Jeff Dolittle 00:17:46 And so there’s a balance you have to strike. You need some long-term strategic thinking, you need some short-term iterating, but you have to apply it appropriately in the appropriate context.
Artie Shevchenko 00:17:57 Yeah. You need to.
Jeff Dolittle 00:17:58 That’s a tricky balance.
Artie Shevchenko 00:17:59 It is a tricky balance.
Jeff Dolittle 00:18:00 How do you strike that balance?
Artie Shevchenko 00:18:01 I believe the easiest way and how it worked for me is the ownership mentality, honestly.
Jeff Dolittle 00:18:09 Ah, well that’s the other thing I was going to highlight that you said it was your money.
Artie Shevchenko 00:18:12 Yeah, exactly.
Jeff Dolittle 00:18:14 So tell us more about that. That’s interesting.
Artie Shevchenko 00:18:16 What exactly do you want to hear?
Jeff Dolittle 00:18:18 Well, you know, I had a mentor of mine tell me that if you want to increase your engineering outcomes, you should treat the company’s money like it’s your own money. And it sounds like you’re basically saying the same thing. So how did it change things for you when it was your money?
Artie Shevchenko 00:18:34 Yeah, as I mentioned, you care about productivity not only because of the product, you start caring about productivity also because you’re actually spending money on this. I don’t think you can apply this to many software engineers because they’re not paying their money for what they’re doing. But I think what is realistic to expect is still to have this ownership mentality, like be kind of a product owners. I think that’s the only way to truly strike the balance there. Because if we’re saying, okay, and I’ve read it somewhere too, like it’s a product manager’s job to care about the product and its software engineer’s job to care about code quality so that you can continue trading in the future, right? But I don’t think it’s a perfectly valid statement because once you do that, it’s kind of a distributed responsibility and you have this push and pull and depends on who pushes harder. And it’s like, it’s tricky because product managers wouldn’t have half of the story, right? They’re not exposed to the code base; they cannot reason about the tech adapt and all that stuff. So I think the only good solution here is for engineers to be more product focused, to be more like product owners and engineers have much higher chances of having a full picture.
Jeff Dolittle 00:19:53 Do you have an example of an engineer who did a good job of learning how to think more like a product manager and how that helped them be more successful?
Artie Shevchenko 00:20:01 I had a colleague at Google who had an eight by four paper printed in front of her and above her desk there was a paper saying, user problem is bigger than engineering problem. And honestly, I think every engineer should have such poster, at least in their head and always remember that ultimately what we’re doing, the ultimate cause for this is to build products and to help our users. If you can remind yourself about this regularly, I think inevitably you will be more productive, you’ll be more product focused and will be motivated to find the right balance on that term productivity spectrum.
Jeff Dolittle 00:20:51 Great. So I think we have a general sense of the answer to this next question, but maybe give us some more specifics on how you selected the topics that you cover in the book and then we’ll dive more into some of those topics.
Artie Shevchenko 00:21:02 Yeah, it happened very organically. As I mentioned before, the book is a story of a healthy code base. So every chapter of the book is an important part of this story. So when I was about to start the book, I already knew pretty much the name of every chapter and more or less the content of every chapter. And there are two parts, if you look in the first part, it is structured around code complexity model about three complexity problems, change amplification, cognitive loads,
Jeff Dolittle 00:21:36 We’ll talk, we’ll talk about them. But, yes.
Artie Shevchenko 00:21:40 Yeah, I’ll just mention them here. So, unknown unknowns and there are seven complexity causes and then there are multiple chapters, all the other chapters from part one, address one or two of those complexity causes. So Chapter three is mostly designed by contract is mostly about two interfaces. Chapter four is about simple interfaces. Chapter five is about having too many dependencies and application. Chapter six, also about too many dependencies. Chapter seven is mostly about obscurity. So that’s basically how the first part is structured. And then there is also the second part which is about code related processes, the key code related processes, it’s about design docs, about trade off analysis, about code review and build versus by decisions. And the reason there is this second part is that code health starts outside of code. And if you want to tell the story of a healthy code base, you need to talk about these code related processes as well.
Artie Shevchenko 00:22:45 There’s an interesting related story actually from Google. I didn’t invent the term code health; it is actually the name of a working group internally at Google. And I’ve heard that when they were discussing how to name that group, they were thinking about naming it something like code quality or something like this. But the reason for choosing the name code health was that a lot of the responsibilities of that working group was actually about these key code related processes like code review. And for the very same reason, there is a second part in this book because if we want to talk about code health, we need to talk about code related processes as well. Code health starts outside of code.
Jeff Dolittle 00:23:29 So to summarize a couple things, you said it, it sounds like a lot of what we’re talking about here is how do we get productivity towards solving user problems and how do we do it in a sustainable way? So if we’re too shortsighted, we can get productivity quickly, but then we can slow down and if we’re too long-term thinking, we may never get the productivity we want in the first place. We have to balance those forces. Is that a good summary of what we’ve discussed so far?
Artie Shevchenko 00:23:53 Yeah, that’s a good summary. I think that’s one of the qualities of this book that is very pragmatic in some ways, maybe even more pragmatic than pragmatic programmer, that we really want to strike a perfect balance. It took me a long time to be balanced myself and to be more pragmatic and I just wanted to share my experience about that.
Jeff Dolittle 00:24:16 Okay. Well then let’s dive in. You mentioned as we, you know, start moving through some of the topics in the book, the three problems of complexity, and you mentioned the before, high cognitive load, change amplification and unknown unknowns. Can you describe what those mean and then give the listeners some concrete examples of what happens when we don’t address those three problems in real software projects?
Artie Shevchenko 00:24:38 Yeah, sure. So you probably have heard about these complexity problems. If you’re familiar with John Osterhout’s work. So these are change amplification, cognitive load and unknown unknowns. He calls them complexity symptoms, but actual thing, they’re actual practical problems we’re dealing with. So the first one I started with the least severe problem, it is change amplification and it’s basically when your small change, which you think will be a small change, is amplified and it actually results in a huge _____(?) like hundreds of lines of code across dozens of files. Something like that.
Jeff Dolittle 00:25:21 Say that again. So it affects hundreds of lines of code across dozens of files.
Artie Shevchenko 00:25:25 Yeah, it’s kind of a, you expected it to be simple. And simple change in a single file or in a couple of files, but actually and
Jeff Dolittle 00:25:34 It just ripples throughout the entire code base.
Artie Shevchenko 00:25:36 Exactly. Yeah.
Jeff Dolittle 00:25:37 Okay. So that’s change amplification.
Artie Shevchenko 00:25:39 It is change amplification, but it is the least severe of the complexity problems. There is a very nice diagram in the book in Chapter two, which connects complexity problems to productivity problems.
Jeff Dolittle 00:25:53 Right now least severe doesn’t mean it’s not severe, it just means it’s less severe than the other two?
Artie Shevchenko 00:25:57 Yeah, exactly.
Jeff Dolittle 00:25:58 Okay.
Artie Shevchenko 00:26:00 Exactly. And it depends on the scale, right? If you have like two duplicates and there is no obscurity around that they duplicate each other, you’re probably fine. It can change it in two places, three places. It just affects the development speed. But development speed is just in front of another productivity problem, there are more productivity problems which are probably more important likeÖ
Jeff Doolittle 00:26:21 Alright, before we talk about those real quick, I will put a link in the show notes to a previous episode with Vlad Khononov where we talked about Balancing Coupling and Software Design, and he talks a lot about how you would handle this kind of change amplification. So weíll include that in the show notes. So change amplification is the less severe, does not mean itís not severe, whatís the next most severe of those three problems of complexity?
Artie Shevchenko 00:26:43 Yes, so just let me just refer to the previous one.
Jeff Doolittle 00:26:48 Ah yes, thatís right. Sure, no problem.
Artie Shevchenko 00:26:50 It is the less severe because it only affects development speed but there is also another productivity problem which is bug in product. We don’t have any bugs in a product. Another productivity problem is developer exhaustion. So the problem with these problems is that if you look at the graph, they’re all very much interconnected with each other. Developer exhaustion, bugs and code complexity and there are a bunch of vicious cycles there. So it’s kind of reinforces itself. And if we’re talking about change amplification, itís kind of isolated in this productivity diagram. It doesn’t participate in these productivity regression vicious cycles. That’s my understanding why it’s the least severe problem. And there are nice visualizations in the book about it. I think I agree with John Osterhout here, so we’re on the same page. I’m pretty sure he says the same thing like change amplification is the least severe problem.
Jeff Dolittle 00:27:45 Again, it’s not that it isn’t severe, it’s just not the most severe.
Artie Shevchenko 00:27:50 Yeah, it depends on the scale.
Jeff Doolittle 00:27:51 Right, right. Exactly. Okay.
Artie Shevchenko 00:27:54 Then the next one is cognitive load. And that’s something you typically think of when you hear code complexity. So that’s what you’re thinking. Okay, that takes too much effort to understand this code to actually digest what is happening here. That’s basically cognitive load.
Jeff Doolittle 00:28:15 Well that could probably be subject to competing forces as well, couldn’t it? Like maybe, I mean what could that look like? 10,000 lines in a single function? Would you call that cognitive load perhaps?
Artie Shevchenko 00:28:27 Yeah, definitely. Definitely. Yeah, because you need to understand all the dependencies between lines there, I guess.
Jeff Doolittle 00:28:34 But then what about if I split that up into like 20 different services and then I inject them all into one thing that could be the opposite end of the spectrum where I’ve maybe over decomposed and that would also increase cognitive load.
Artie Shevchenko 00:28:46 Yeah, there are definitely different ways in which you can achieve this. You can make things complicated in many different ways.
Jeff Dolittle 00:28:53 In many different ways. Well as Neil Ford has said, developers are drawn to complexity like moths to a flame often with the same outcome. So this is how it goes.
Artie Shevchenko 00:29:03 Yeah. But it is really bad having high cognitive load. You may need to spend many hours to actually understand what’s going on. Sometimes you may not be able to understand it yourself, but probably you will find somebody who can understand hopefully. So yeah, that’s kind of a attributes of this cognitive load problem is bad, but it’s not the worst.
Jeff Dolittle 00:29:29 So the worst is coming. Yeah, yeah.
Artie Shevchenko 00:29:32 Yeah, the worst is still coming. The worst is unknown unknowns. And that’s basically when you do not have some information that you really need to accomplish a task at hand. So if you want to do something but you do not know where to start, or even worse, you don’t know what you need to know to do this correctly. So there is some obscurity involved here.
Jeff Doolittle 00:29:58 So can you give me an example of you just said for example, there’s something I need to know, but I don’t know that I need to know. Has this happened to you or then team members maybe that you’ve worked with in your startup where you discovered later that the actual issue that you were struggling with was unknown unknowns?
Artie Shevchenko 00:30:11 Yeah, there was an interesting story that happened to us in Google Mass. My first team at Google at some point, we had a very senior engineer joined our team and he actually joined us as an engineering manager. But I guess at some point he was bored because it was mostly a maintenance project that we were working on. And I guess because of that, he was looking through the code and he deleted some configuration for our batch jobs. And the comment on that PR (?) was literally something like, you don’t need all this stuff like this should all be done automatically. It was about choosing the right data centers for masters and for slaves for our batch shops, which are basically improved map producers, if you can think about it like this. And what happened is one morning, literally all of our batch jobs, like 20 of them started sending us an alert saying that we cannot start, the job cannot be started.
Artie Shevchenko 00:31:12 These are daily jobs and we cannot start because the jobs for yesterday are still running. And yeah, getting these alerts, I quickly realized what’s going on. The choices for masters and slaves for their jobs were all very, very poor. And all those jobs were in like previous stages just choosing one node in North America, another node in Europe, and like trying to send all the data between them. And that’s because automation didn’t work. He was correct that ideally if those jobs were written from scratch, it all should have happened automatically. But these were very old jobs that were written like five years ago or something like that. And at that point there was no automation and it required manual choices and it was really fancy logic. It was really complicated. It took me quite a lot of time to really dig down into like how it worked.
Artie Shevchenko 00:32:11 And I actually wrote a pretty long page internal that was explaining how these configuration parameters are working, which became pretty popular actually. But he just went ahead and deleted that part on configuration. And so there are two learnings there. One is that you shouldn’t delete something that you don’t understand. Why is it there in the first place? And that’s what is called Chester in pants. And we can talk about this, but there is also another problem there that there were no comments in those configuration files explaining why do you need all of this configuration? How does it work? So I can definitely relate to him being confused about it and trying to do something about it.
Jeff Doolittle 00:33:00 Okay.
Jeff Dolittle 00:33:31 So next in the book you explore the seven causes of complexity. So describe for listeners what these seven causes are and then how these tend to show up in a code base. Because the goal here, as you’ve said, is to create people who are guardians of code health. And to do that we need to understand complexity. So what are these sources and what would we see in a code base where we’re not properly considering these seven causes of complexity?
Artie Shevchenko 00:33:57 Yeah, sure. So let’s go over them in the order of decreasing severity roughly. So the first one is obscurity. And we already started talking about this, the previous example . . .
Jeff Doolittle 00:34:11 Unknown unknowns. Yeah.
Artie Shevchenko 00:34:17 Yeah, unknown stuff. The previous example is missing documentation, missing comments. That’s a very common example of obscurity, but obscurity is so general, it manifests in so many different ways. It can be obscure naming, that’s another very popular manifestation of obscurity. But also there are things that are really hard to avoid. And even it, it can be uh, necessary trade off obscurity. For example, if, if you’re thinking about event driven design, it necessarily has some obscurity in it. If you’re trying to debug stuff and you’re trying to reconstruct the chain of events that caused this thing, like that’s, that’s not an easy thing to do, right? There is some inherent obscurity there and sometimes you intentionally agree to having obscurity, but as much as possible, I guess it’s better to avoid it. The next one would probably be complex interfaces. Complex interfaces probably don’t need any in description, right? It’s kind of intuitive. But the problem with them is that it actually cause all three complexity problems. So if we’re thinking about change amplification or cognitive load, unknown unknowns, all that can be caused by having complex interfaces.
Jeff Dolittle 00:35:36 This may be an interface with lots of operations or maybe the operations don’t, they’re not cohesive, they don’t belong together, these kinds of things?
Artie Shevchenko 00:35:44 yeah anything like having too many parameters in an interface is not
Jeff Doolittle 00:35:48 Too many parameters. Yeah, okay.
Artie Shevchenko 00:35:51 Like confusing name and quite a lot of different things can go wrong with interfaces, but there are interfaces that appear simple. They can be simple to read and understand, but they still can be complicated. They can be hard to use. For example, if you think about assembler, assembler is most of the comments, the comments, they’re pretty simple, right? It’s easy to understand what they’re doing, but if you want to do something assembler, even device drivers, they’re not being done as simpler these days, right? Because that’s too complicated. You need to learn all those patterns, how to use these simple interfaces to do anything useful. So yeah, simple interfaces are about not being simple to read and understand, but also about simple to use as well. Yeah, we had an interesting story internally at Google. We were very passionate about implementing a complete crowd API at some point.
Artie Shevchenko 00:36:49 And we were talking about this like delete, update, create all those methods and spent quite a lot of time doing this. And then later realized that actually the teams, two teams that were using them, they didn’t need this granular APIs. They actually wanted more specific APIs that would address their use cases. But we were just going forward with this clean approach to provide a full API, which doesn’t make sense in hindsight. So I’m really, really cautious about building something that would solve any problem, just use it however you want to use it. This kind of arguments are not the best you can end up with having very flexible API that doesn’t really help. Sometimes you need that, right? If you’re writing a public API and truly do not understand what it will be used for and you want to give developers some freedom to experiment and build whatever comes to their mind, but often that’s not the case. Often when we’re building an API, we already know what are the internal use cases for that and we just need to spend some time to make it not just appear simple and be easy to understand, but also be easy to use as well.
Jeff Dolittle 00:38:027 Yeah, that’s a good call out to a previous episode I did with Yuval Lowey on his book writing software and everything about his methodology is about driving things via the use cases. So we’ll put a link to that in the show notes as well because you called out use cases a couple times. So that made me, think about that as well. So we’ve talked about obscurity, we talked about complex interfaces, we don’t necessarily need to go through all seven causes of complexity because listeners can get the book and then we can continue on. But were there any other causes of complexity that you wanted to highlight where maybe they affected a team that you were working with or maybe a team successfully addressed one of these causes and it was noteworthy?
Artie Shevchenko 00:38:39 Yeah, I think one of the causes that I see being addressed really well at Canva, that’s where I’m working right now is unfamiliarity. So just historically Canva was very good about keeping things consistent, especially when camera was smaller. Right now it’s a bit more challenging and that kind of makes sense too because if you think about consistency, the first thing you need to understand is whether it’s consistency about coastal or consistent about code quality. If we’re talking about coastals, it’s simple, right? You just have a coastal, you have a proper formatter used by everyone and it just works. It’s great value for money. But if we’re talking about code quality, it’s a bit more complicated. The next question is whether we’re talking about consistency across like the whole organization, the whole code base, or are we talking about consistency specifically within a team? So if your organization is smaller, it’s more like a team consistency than organization and it’s much easier to communicate those global rules and make sure they are consistently applied across the code base.
Artie Shevchenko 00:39:55 But in general, the value, especially as the company grows, the value kind of decreases because you don’t work in other teamís code bases as often. And also it’s so hard to communicate those rules and be consistent about them. So across the company it becomes a bit challenging and the value for money is probably not that good. But if we’re talking about consistency within a team, I think that’s very, very important because the value is much higher. You’re actually working on the same code base and the cost is much lower because you have just a few folks you need to agree with and make sure that you communicated those rules and you just be consistent about how you do things in your code base. One global rule that we have at Canva and which I personally now benefit a lot because I work on the authorization project, is that access checkers are very well structured. There is a well-defined structure of access checks across the whole company. So it means that we can actually improve this over time because it’s consistent for different teams. If it wasn’t, if every team was doing access checks in their own way, it just wouldn’t be possible.
Jeff Dolittle 00:41:13 I think a lot of people are doing access checks in their own way, so that might be something that they could learn from. Let’s move on to discuss some design principles for code health. So you know, there’s a lot of books written about principles and practices and you mentioned specifically some design principles. So let’s talk about what those are and then how those can affect code health.
Artie Shevchenko 00:41:32 Your opening a Pandora box, honestly because. . .
Jeff Dolittle 00:41:34
Artie Shevchenko 00:41:36 I guess I could answer with half of the book’s content.
Jeff Dolittle 00:41:39 Yes. Each one of these could be an episode unto its own, but just give listeners a taste like designed by contract for example, what does that mean and how does that help with code health?
Artie Shevchenko 00:41:49 Yeah, sure. Designed by contract is very crucial because you need to trust interfaces if you want to keep your code reasonably simple. If you do not trust interfaces, if there is not enough information in the interface itself or if you cannot trust it, like you end up adding quite a lot of complexity into a code to just get around it and you know, just have those safety nets. So it’s really cost a thing not to have interfaces that you can trust. And I think the only way to solve this trust problem is to follow the design by contract paradigm. It’s a pretty old one. Right. And I probably don’t need to explain what it is.
Jeff Dolittle 00:42:37 You might actually, because you know, every year there’s new people in the software industry, so you know, some of our listeners who are new to the industry may not be familiar with it. So if you don’t mind, give us just a brief, you know, you mentioned interfaces and contracts. How do those relate? What is designed by contract?
Artie Shevchenko 00:42:50 I think it all starts with modular design. And I will use the term model as John Osterhout use it. Basically anything that has an interface and implementation, I think it goes all the way back to Parnasís definition.
Jeff Dolittle 00:43:05 It does a module is something with an interface, inputs, outputs, and inside it should be a black box. So it could be a function and it could be a system and anything in between.
Artie Shevchenko 00:43:15 Yeah. But then if you want to modules to be useful, if you’re on your decomposition, don’t look like FU which calls FU one, FU two, FU three, FU four, like you really need to decompose it meaningfully and then have clear contracts for the other modules that you’re calling from the implementation. So that’s where design by contract really comes into play. It’s kind of obvious things, but it’s about being explicit about what the module does, documenting whatever is missing from the code, just writing down as documentation and then internally enforcing those contracts internally in the module. So if you provide a clause that has an interface, it is your responsibility to actually test that implementation that it actually does what the interface promises to do. So that’s basically the crux of the design by contract, you document what this module is doing, and you internally enforce it.
Artie Shevchenko 00:44:22 It’s only, not only about tests. Assertions also is a huge part of it. They’re especially useful for enforcing preconditions. Design by contract as a term was coined by Bertrand Meyer when he was working on his AFI (?)language. And he used the terms precondition post-condition in cost and variance I believe. So basically the way he was putting it is that if all the preconditions are met, then the module guarantees that all the post conditions will be met after the call and that all the in variance would still hold. So basically that’s I think roughly the definition he was given. And when we’re talking about assertions, they in particular help with validating the preconditions in around time. I think pragmatic programmer makes a very good point about assertions saying that it is naive to rely on tests only because yes, you need good test coverage and that’s definitely the main contract enforcement mechanism. But apart from tests, there are billions of different states your program will go through in production in runtime. So having assertions in runtime actually is complimentary to having good test coverage.
Jeff Dolittle 00:48:42 That’s interesting. You said there could be billions of states your system could be in. So effectively it sounds like you’re making the case that assertions are another strategy for handling complexity.
Artie Shevchenko 00:45:52 Yeah, exactly. Let’s say it’s a strategy for enforcing contracts that at least all the preconditions are really met in your production system.
Jeff Dolittle 00:46:01 Right and reducing the number of possible states that your system can be into ones that you are aware of.
Artie Shevchenko 00:46:07 Yeah. If you do not do that, you will end up with having extra logic handling those edge cases. But so often you just need to say, hey, this code wasn’t intended to be used with such an input. So you just fail fast and that’s the best thing you can do. You simplify the code, you don’t handle those edge cases and you make sure that the system, if it is alive, it is in a correct state.
Jeff Dolittle 00:46:33 Yeah, it’s interesting. I think young software engineers often try to swallow exceptions, for example, and pretend they didn’t happen, but more mature engineers want an exception to bring the system down. And that’s essentially what you just described there.
Artie Shevchenko 00:46:47 Yeah, there are definitely cases when you want a safety net. Like in particular if you’re having an important operation that is followed by some sort of a cleanup or optional operation that can be handled later. Maybe you want to wrap it in a try catch, but in general, yeah, I am 100% agree that you don’t want to mask errors.
Jeff Dolittle 00:47:10 Let’s talk about dry. We talk about one of the principles for code health. You have some interesting takes on by dry, I mean don’t repeat yourself principle. And I think some people maybe take this to the extreme, but I think have a more nuanced perspective on this. So talk about the dry principle, how it relates to code health and how to balance just how dry you should actually seek to be in your code base.
Artie Shevchenko 00:47:32 Yeah, that’s one of the takeaways I hope from this book is that complexity is not a single problem. There are actually three different problems change amplification, cognitive load and unknown unknowns. And they all have different severity. So if we’re talking about dry principle violations, the worst problem there is not change amplification, even though it’s the most obvious one, the worst one is unknown unknowns. So when it becomes really, really bad is when you do not really know that there is a duplicate somewhere else. It may look differently, it may be different code, but it’s kind of a still a knowledge duplicate, right? And if you do not know that there is a duplicate, then you’re in a bad state because these duplicates can over time they can start diverging and that’s the way towards driving your system crazy, having diverged duplicates, which are not really duplicates anymore.
Artie Shevchenko 00:48:36 So if you can get rid of the unknown unknowns problems, I think we are already good enough. So we had this tooling practice at Google and tooling for marking the duplicates in code. Often it would be different languages like frontend and backend, basically saying if you change this then change this. And if you have such comments in quote, then basically you’ve addressed the unknown unknowns problem. There is no obscurity anymore around the fact that there is duplicate. And if there are not too many of them, it’s probably not a big deal too. So you just need to change it in a couple of places. That’s probably okay. But if you can, if you can avoid duplicates, definitely avoid them because duplicates is often an overhead.
Jeff Dolittle 00:49:19 Yeah. And I’ll reference again the previous episode with Vlad Khononov on Balancing Coupling in Software Design because he references a lot of this in that book and listeners who want to deep dive into that could check that out as well. So we’ve talked a lot about the elements of complexity and how to understand it. We’ve talked about some design principles as it relates to code health, but there’s also processes that you discuss in the book about helping to maintain code health. Can you describe what some of those elements are of process or what those processes are that help teams to either improve their code health or maintain their code health?
Artie Shevchenko 00:49:54 Yeah, definitely. So I think one process that I don’t see being used a lot outside of Google and can also use as design docs a lot, but I’ve seen a lot of companies which do not, seen a lot of companies which kind of ignore this upfront design and even has a negative connotation, right? When you say upfront design, like it feels uncomfortable. Like are we not agile anymore?
Jeff Dolittle 00:50:19 Well, there’s big upfront design and then there’s a little bit of upfront or just enough, right? And I think something tells me you’re advocating for just enough.
Artie Shevchenko 00:50:26 Yeah. And that’s the case. So I would argue, and that’s what Google practices actually suggest, that a bit of upfront design actually helps you be more productive and it actually makes the process a bit more delightful too because you don’t need to delete as much code as you would otherwise. So yeah, having a bit of an upfront design is definitely helpful and I would argue that it’s the best way to do tradeoff analysis. And the best alternative to a design doc in most cases is a one pager, which is basically a small design doc.
Jeff Dolittle 00:51:05 It’s a small design doc. Well, yeah, I mean you talk about tradeoffs once you have an implementation, nobody wants to talk tradeoffs anymore, especially once it’s merged.
Artie Shevchenko 00:51:13 Yeah, exactly. But sometimes like the product would push you in that direction, you would need to talk about tradeoffs. They would become obvious.
Jeff Dolittle 00:51:23 Yeah. Interesting. Okay, so what differentiates a design doc from, I’m going to use the W word, you know, when people think of waterfall, they think of a massive specification with every possible thing. But since that’s not what you’re describing, you talk about a one pager, but you said that’s a very small design doc, but what is a design doc? What goes in it?
Artie Shevchenko 00:51:44 Well, there are always pros and cons, right? If we’re talking about longer design docs or shortage design docs. It depends on like it won’t catch as many issues if it’s a short design doc, right? But at the same time, the return on investment may be the best if we’re talking about short design doc?
Jeff Dolittle 00:52:03 Right, but if I’m a listener and I’m saying, okay, I agree with you that I should do some kind of design docs, maybe not big design upfront, but some design upfront, then what should I put in these design docs?
Artie Shevchenko 00:52:13 So there are pretty standard templates there. You’re just describing some background about the project you’re about to do. Goals, maybe non-goals, you provide a quick overview of the solution and maybe talk about some details. Depends on whether it’s a big design doc or not that big design docs. But I would argue that every design doc should have at least some tradeoff analysis in it. I think that’s the very heart of design docs that you’re making decisions, right? Where will you make that decision? Will you make it in a design doc or will you make it in some meetings? Or will you talk to stakeholders one-on-one and then get out of sync with each other over time? Or you just to design like in your head or on the whiteboard before coding. All these alternatives have so many downsides that it becomes pretty obvious once you write this down. And there is a tradeoff table in the Design Docs chapter in the book, which clearly shows that design docs are absolutely the winner there.
Jeff Dolittle 00:53:21 Great. And I’ll put a link in the show notes to the episode that I hosted Chad Michelle talking about Lean Software Systems Engineering for Developers. And one of the key concepts in that book is the idea of gaining and maintaining a shared understanding throughout the project. And that seems like that dovetails really well with what you’re describing with design documentations. They help us gain and maintain that shared understanding. Another aspect you mentioned of process is code reviews. A lot of people do code reviews, but maybe you have some nuanced perspective on what exactly good code review entails. Why should we do code reviews to ensure code health?
Artie Shevchenko 00:53:59 There are four goals for doing code reviews and interestingly, when I first was doing good code reviewer classes, internally at Google, we had just two. And that’s one thing that was really surprising when I was writing this book. Then I actually discovered two more top level goals. I can tell the story about this, but let’s go over the first ones too. The first, the most obvious goal is high code quality, right? You basically just need the second pair of eyes. Two heads are better than one. You’ve done something, you definitely need to review it yourself, but you also need somebody else with a fresh eye to take a look and catch something. If they have a different expertise, if they’re more senior than you, that’s helpful, but that’s not necessary because fresh eye actually helps a lot here. So many problems even junior developer can catch just because they didn’t see this code before then do not have all the context you have in your head as an author.
Artie Shevchenko 00:55:05 The second reason for doing code reviews is knowledge sharing. And that’s huge because in our industry we so often don’t have the same context. If you go to meetings and start talking about solutions high level, you experience all the time, right? People try to understand what you’re talking about and just give you a bunch of garbage advice just because they don’t have any details and they’re trying to help you, but they really cannot. But code reviews are very different because when you’re doing code reviews, you have all this contacts in front of your eyes. So you have all the code, all the code is there, and if something is missing, you have an author there who is happy to help you provide you with all the context that you need. So having the same context being on the same page is very helpful for knowledge sharing because now you can, having the same context, you can actually share what you think, share what you know, share your previous experiences and all this stuff.
Artie Shevchenko 00:56:03 And that actually helps you grow so much faster. But there are two other goals there, which we didn’t have originally in the good coach reviewer classes internal at Google. And how I got to those goals is really funny because the original presentation, a good coach reviewer at Google, it had two parts, three parts, but it had part for the goals and it had a huge part for the practical tips and tricks, which is basically like 10 practical tips and tricks. Google has learned over two decades doing like millions of reviews. And when I was writing the book, I was trying to do an experiment, just try to connect those practical tips and tricks to the goals. So you have two goals. You have around 10 practical tips and tricks, and you try to see what goals do they help with. And what I’ve found out is that nearly half of the practical tips and tricks didn’t address any of those two goals.
Artie Shevchenko 00:57:07 And that was something really surprising. And if you see this, then you can do two things, right? You can start questioning the tips and tricks, which probably wouldn’t be wise because that’s a lot of wisdom there like that Google has accumulated over the decades. Another option is to question the goals. And that’s what I did and that’s how I discovered two additional goals. And the third one is psychological safety. And psychological safety is actually let’s, let’s do the fourth one first because it’s simpler. The fourth one is delivery speed. It’s kind of obvious you want to optimize your code reviews for delivery speed. You don’t want to end up with this analysis pers thing, or you don’t want to do like perfect PR out of this. There’s no such thing as a perfect PR. So yeah, delivery speed is definitely another top-level objective or top level goal there.
Artie Shevchenko 00:57:58 But there is also one more that is not that obvious, which is psychological safety. And it all goes back to another project that was done internally at Google. You can find a lot of information about it online, which is named Project Aristotle. It was basically a research on Google teams. I think over a hundred teams participated or something like this, trying to figure out what makes teams at Google productive. And there were a bunch of expectations what would come out from this research. Things like what about complimentary skill sets? Or maybe very good communication, like whether you have extroverts in your teamwork versus introverts. But interestingly, if you look at the results of the research, nothing like that came up. It doesn’t affect team productivity much. But what does affect team productivity much is psychological safety first of all. Then the second predictor in the list is reliability, but it’s a distant second.
Artie Shevchenko 00:58:59 So psychological safety by far exceeds any other thing if we’re talking about team productivity. So you really need to have good psychological safety climate in your team if you want your team to be productive. And how it relates to code reviews. Code reviews is, oh wait a moment, let me explain psychological safety a little bit. Psychological safety is basically when you feel safe to take risks in front of your team, if you feel safe to look stupid in front of your team, like ask you stupid questions or just bounce off weird stupid ideas. Well looking, looking stupid ideas, right? So that’s kind of a psychological safety definition. And code reviews affect the psychological safety climate a lot because code reviews are where you’re faced with your errors more often than anywhere else, I guess. Maybe design docs is another thing, but design docs are like, they’re more high level. You can argue you. You can talk about things like as long as saying like, talk is cheap, right? Show me code. So in code, that’s where you will see the actual errors and that’s where you will either deal with them professionally and we’ll be open to discuss it, acknowledge them, and just build up the climate of psychological safety. Or you’re going to be defensive, not willing to talk about stuff and just destroy psychological safety climate in your team. So that’s another third or fourth goal of code reviews.
Jeff Dolittle 01:00:30 Well, as we wrap things up, were there any surprising revelations or lessons during the writing process?
Artie Shevchenko 01:00:37 So I just talked about code review, and that was surprising for me to find out that there are two more high level goals because I was doing code review presentation, code review tech talks, probably more than any other tech talks in my life. And the fact that I only found out that there are two more high level goals only when writing the book, that was pretty surprising, very unexpected. And yeah, by the way, I’m referring to internal Google presentations a lot here, but they’re based mostly on internal documentation that was open sourced recently by Mexican and Alexander. So I’m not whistleblower here, so I’m sharing something that is already present in line. I’m just like structuring it in a slightly different way. So yeah, I was really surprised that I found two more high level goals there. But a couple more things also surprised me a lot.
Artie Shevchenko 01:01:36 The first thing is that I was holding so tight to John Osterhoutís definitions of complexity causes and complexity problems for such a long time when I was working on a, I mean, it’s kind of organic because my university course was based on John’s work in a large extent. But as I was it iterating on the content, these seven alternatives, seven complexity causes were already circulating in the content of the university courses. But I wasn’t calling it complexity causes because it was so much in my head that there are two complexity causes, dependencies and obscurity, and I must stick to it. But that’s not the case, right? Because they’re quite abstract. If you think about dependencies, and that’s so abstract in calling it, how is it helpful saying that dependencies are a complexity cause even though I agree, mostly agree that you can track it down to dependencies and obscurity, but pragmatically speaking, if you want it to be practical, you want to be more specific here.
Artie Shevchenko 01:02:37 And I think seven complexity causes are actually much better if we’re talking about obscurity, untrue, complex interfaces, unfamiliarity, too many dependencies, duplication and unstable dependencies is much more practical and pragmatically better than having just obscurity and dependencies. So that’s one thing that struck me that I held tightly to those two dependencies, or sorry, complexity causes for such a long time. And another thing is, I guess about deep interfaces, if you remember John’s work, he actually introduced the term deep modules. And if you look through the Google group discussions about this, you’ll find Jimmy, I forgot his surname, some Jimmy was critiquing John for not being specific by what he actually means by deep modules. And they had a very long discussion there and they didn’t resolve it. And after some time, I just jumped in and we’re talking with John about it last year.
Artie Shevchenko 01:03:44 And I think what we’ve found out is that John by himself is not sure what he means by deep modules, because he’s not sure. Whether we’re talking about implementation there or not talking about implementation. And I had this gut feeling when I was starting the book that this need to be clarified. But I didn’t know what exactly would come from it. And actually redefining it and saying, okay, let’s talk about deep interfaces and leave everything else to saying like, okay, we don’t want modules to be dysfunctional. Let’s a separate topic. Let’s just decouple it a little bit and just focus on interfaces when we’re talking about depth. That was a pretty big revelation.
Jeff Doolittle 01:04:25 So when you say deep interface versus a shallow interface, what exactly are you describing?
Artie Shevchenko 01:04:32 So that’s basically again, value for money. How much functionality does the interface promise in turn for the complexity of the interface? So you can think of it as like, functionality divided by complexity of the interface.
Jeff Doolittle 01:04:45 So a narrow interface that does a lot for you versus a broad interface that does very little for you. Exactly.
Artie Shevchenko 01:04:51 Exactly.
Jeff Doolittle 01:04:53 Okay, great. Well this has been a great exploration of your book, Artie. If listeners want to learn more about you in the book, is there a website they can go to?
Artie Shevchenko 01:05:00 Yeah, definitely. You can go to codehealthguardian.com.
Jeff Doolittle 01:05:03 Artie, thank you so much for joining me on the show.
Artie Shevchenko 01:05:06 Thank you very much. Thanks for having me.
Jeff Doolittle 01:05:08 This is Jeff Doolittle for Software Engineering Radio. Thanks for listening.
[End of Audio]
