Search
Search
Kent Beck
May 10, 2024

SE Radio 615: Kent Beck on “Tidy First?”

Kent Beck, Chief Scientist at Mechanical Orchard, and inventor of Extreme Programming and Test-Driven Development, joins host Giovanni Asproni for a conversation on software design based on his latest book “Tidy First?”. The episode starts with exploring the reasons for writing the book, and introducing the concepts of tidying, cohesion, and coupling. It continues with a conversation about software design, and the impact of tidyings. Then Kent and Giovanni discuss how to balance design and code quality decisions with cost, value delivered, and other important aspects. The episode ends with some considerations on the impact of Artificial Intelligence on the software developer’s job. Brought to you by IEEE Software and IEEE Computer Society.

Show Notes

Related Episodes

Links And References

Transcript

Transcript brought to you by IEEE Software magazine and IEEE Computer Society. This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number.

Giovanni Asproni 00:00:18 Welcome to Software Engineering Radio. I’m your host Giovanni Asproni. And today I’m joined by Kent Beck for a conversation on software design based on his latest book, Tidy First? with a question mark at the end. Kent is the creator of Extreme Programming, a pioneer of software patterns and a co-author of Uni. He rediscovered test-driven development and is an observer of three Xís, Explore, Expand, Extract. He’s a signatory of the German manifesto and currently is the Chief Scientist at Mechanical Orchard. He authored several books on extreme programming test-driven development and patterns, and of course Tidy First?. Kent, welcome to Software Engineering Radio. Is there anything I missed that you’d like to add?

Kent Beck 00:01:02 Ciao Gio, it’s good to be with you this San Francisco morning. No, I think that about covers it as far as the computing world is concerned.

Giovanni Asproni 00:01:10 Okay, then let’s start. So first of all, let’s start about the reasons for writing the book. Yeah? So reading your book, you wrote there that your aim was to bring to today’s audience the contents of the book Structured Design by Yourdon and Constantine, which you call the bible of software engineering. Can you tell us a bit more about that?

Kent Beck 00:01:37 When I was in college, I had this brand new book as a textbook called Structured Design. It’s a green cover. If you had it in college, you fear this book as well (oh you have a copy, fantastic). Because it was a college textbook, of course, I only read the assigned parts and I read them as quickly as necessary to pass the test. So I didn’t really absorb everything that was, or much of what was in the book. And then in 2005, I was invited to be on a panel with Ed, (rest in peace) and Larry to discuss the 25th anniversary of the book. So I thought, well, I better read it. And I read it cover to cover and it was an absolute page turner to me. And what I discovered as I was reading it was there laid out were the basic fundamental laws like Newton’s Laws of Physics, but for software engineering, the challenge is, the examples are ancient.

Kent Beck 00:02:42 , there’s a discussion of paper tape, there’s a storage medium, there’s a discussion of the big debate between assembly language and higher-level languages. It’s just really hard for a modern reader to read it. And so I said, well Iím going to translate, but the information was absolutely fundamental. Coupling, cohesion, all laid out there why software is so damn expensive and what you can do about it. They understood all this stuff and our understanding of those basic principles has sort of been handed down from one person to the next to the next and gotten very diffused. So that was the point at which I said, okay, I really do need to write a book that brings this information to a modern readership.

Giovanni Asproni 00:03:31 The decision to write the book. So I read it cover to cover. I also have to tell you that I was using some markers in the book to say, okay, what can I ask you in the interview? I ended up marking everything. So I had to cut down content. But one thing that I read as well, which I always do in books, is also the references. And in the references he wrote something about a book of A Philosophy of Software Design by John Osterhout, which I paraphrase. You said that reading that book that prompted you to write Tidy First? because he makes some good points, points well taken, but they are presented dogmatically. Can you explain as what you mean?

Kent Beck 00:04:13 So it’s deeply satisfying as a programmer to take some messy code and clean it up. That activity just feels good to somebody with a programming kind of mindset and philosophy of software design says, okay, well if how to clean up the software, always clean it up. And that just seems to me to be not the whole story. That there are pressures, there’s legitimate pressures that cause you to leave software in a messy state. And part of the skill of being a designer is knowing how to sense those pressures and when to give into them, but also sense those pressures and knowing when to buck them. So somebody says you finish a feature, they say they want the next feature. You say, not quite. Before I implement this feature, I’m going to tidy first. And that’s a design decision. The decision whether or not to design is as much a design decision as the decision of how to arrange the modules and services and functions and statements and the functions and so on and so forth. So that was my objection to that book. I love it’s a slim little book, which you’ve been kind enough not to mention Tidy First?, there’s not a lot of pages in there. I don’t apologize for that. I only wrote about the things that I felt that I understood well, and that would be most important. It’s not every important topic about software design, but I saved you a bunch of hours of reading through stuff that wasn’t important. So you’re welcome.

Giovanni Asproni 00:05:56 In the book you state also another thing that you say that much design advice just flat out contradicts available evidence. So can you expand on that maybe with some examples as well?

Kent Beck 00:06:07 Sure. The simplest example is when people talk about averages of metrics that don’t converge. So for example, what is the distribution of the number of lines per function? Somebody might say, well functions should be no more than seven plus or minus two. Well, which is just some magic number pulled out of one context and put into a different context. But if you look at the actual distribution of number of lines per function, it doesn’t look like that at all in any software. Even if you write it with that rule in mind, it’s still going to be distributed in something that looks more like a power law. And one of the characteristics of power loss is that they don’t converge the central limit theorem doesn’t hold, there’s no mean the length of functions, the more functions you have, the larger the mean is going to be.

Kent Beck 00:07:08 And that’s just a natural law. It’s like asking what’s the average size of avalanches or what’s the average size of hurricanes? The more hurricanes that you measure, the higher the average intensity is going to be. That’s just natural law. And so that’s something I wanted to bring to the conversation about software design is, this is actually a natural process and you and I are instruments of the expression of that natural process. We have influence over it, but we don’t have control over, we can’t just design the software any way we want because there are these natural forces impinging on the decisions that we make and we can’t escape those. So if anybody says, , the maximum size, whatever, the maximum number of functions per module and maximum number of parameters per function or something like that, they don’t understand that what they’re dealing with is a natural process and that mean of the number of parameters or the max of the number of parameters. The bigger the system is, the bigger both of those numbers are going to be.

Giovanni Asproni 00:08:18 Yeah, the power loss in software seem to be pretty much everywhere.

Kent Beck 00:08:22 Yes. The runtime of tests, the number of times a test fails, just over and over again. I’ve been pursuing these distributions for 20, 25 years just because I’m fascinated by them. And , I’m not sure it’s entirely a healthy or sane approach, but there you go.

Giovanni Asproni 00:08:43 And now let’s go a bit more into, let’s say the main subject, if you like, of the book or the main concept. So let’s start from the three main concepts, I think around which the book evolves. There is, cohesion, coupling and tidyings. And then of course that is everything that comes out, design and everything else. But let’s start with this. So first question from me is, can you briefly give our audience a refresher on cohesion and coupling what they are?

Kent Beck 00:09:10 Sure. And these words have come to mean much fuzzier concepts over time. So I’m returning to the original definitions from Yourdon and Constantine and from Yourdon and Constantine. Coupling is, coupling with respect to a change that you want to make is if I change one element, I have to change the other element. If so, then those two elements are coupling. So for example, if one function calls another, they are coupled with respect to changes to the name of the function that’s called. If I change the declaration of the function, I have to go change all the callers too. So a calling function is coupled to the called function with respect to changes of the name. And vice versa, if I want to change the name at the call site, I’m going to have to go change the name at the implementation too. So those two elements, and I say elements because this is fractal, it goes all the way from two expressions in a statement, two statements in a function, two functions in a class, two classes in a module, two modules in a service. It’s up and down and up and down. Coupling is a relationship between elements where changing one of them requires that you change another one.

Giovanni Asproni 00:10:37 And cohesion.

Kent Beck 00:10:39 Cohesion, okay, so this is what took me 18 years to write this book. And it was being able to explain cohesion that I just, I’m very stubborn so I kept going and kept going until I could actually explain it. So inside of an element, so let’s say inside of a file, you have a bunch of functions. That file is cohesive, if the functions are coupled. If I change one of the functions, I have to change all the other ones, then the file is cohesive. So imagine you have a function with a hundred lines and you always change the same two lines in that function, but you don’t change the other 98 lines. That function is not cohesive because there’s a bunch of stuff that doesn’t change and a couple of statements that do change. So you might say, well that’s bad, we don’t want coupling because it’s expensive.

Kent Beck 00:11:37 And it sets up these cascading changes, which is what drives the cost. So what you would do if you wanted to improve the design of that system is you take the two lines that change all the time and you would extract them as their own little helper function. And then when you need to go make changes, you just have to change those two lines. You don’t have to change two out of a hundred lines. You might say, well, but I don’t want to have to change those two lines together. And the answer is, well, sometimes there’s coupling and you can’t get rid of all of it. Because you can’t predict what all changes you’re going to want to make. And remember that coupling works with respect to particular changes that you want to make. So you can’t get rid of it, but you can sweep it all into one pile. And that’s what cohesion is about. It’s about limiting the potential spread of coupling.

Giovanni Asproni 00:12:32 Okay. And now the concept that gives the title to the book, so a tidying, the question is what is a tidying? I mean, in the book also list 15 of them so you can tell us what it is and maybe give us some examples of tidying.

Kent Beck 00:12:47 Sure. So this is a pedagogical choice on my part. This is not the last word on software design. This is the first of three proposed books. And pedagogically I decided, let me start at the smallest scale of design so people can get lots and lots of practice working design into their regular workflows. So the that phrase tidy first is a situation that we all encounter 10 times a day as a programmer. I need to change the behavior of some code, but it’s messy and it’s going to be hard to change because I can’t see what the control flow is or the names are wrong or just badly formatted or there’s something, , it’s producing the numbers that it’s producing, but it’s structured in a way that’s hard to change. And so you’re left with this dilemma. Should I tidy first? And that’s why the question mark in the title, which if you’re an author, I’m going to warn you, please never put punctuation in the title of the book. It’s turned out to be a disaster. But there you go. So I need to change this code, it’s messy. Should I tidy first? And the answer is maybe sometimes yes and sometimes no. That’s the kind of answer that the philosophy of software design doesn’t give you to go back to an earlier point. But you have to decide, okay, well should I tidy this up first? And there are programmers who do it by habit. There are programmers who never do it. And I don’t think either of those is the optimal solution.

Giovanni Asproni 00:14:26 Yeah. An example of a tidying, just to give an idea of people, , because we are talking about cleaning up things tidying up.

Kent Beck 00:14:33 Rationalizing a name, you see a name of a variable and it doesn’t make any sense to you. And then you understand the code and a tidying would be to just fix that name.

Giovanni Asproni 00:14:43 Yeah. So

Kent Beck 00:14:44 Now you’ve changed the structure of the code, but you haven’t changed its behavior at all. It’s going to compute exactly the same outputs for the same inputs, but now it’s less messy, it’s easier for you to understand.

Giovanni Asproni 00:14:57 Yeah. So, now that you just mentioned the behavior and the structure, so tidyings are purely structural in correct what they achieve, so they don’t change any behavior at all.

Kent Beck 00:15:09 Yeah, yeah. Absolutely not. And that’s part of the thesis of the book or the whole style that I’m advocating here is, you make a strong distinction between the changes to the structure of the system and changes to the behavior of the system. And you never mix the two together. You might alternate every three minutes, but you’re never changing a name. Adding an if statement, swapping the sense of test of something all at the same time. There’s too many chances for errors and errors sap your confidence and you lose the confidence of your teammates and down it goes.

Giovanni Asproni 00:15:52 So tidyings also, I think you say this in the book, are kind of small refactorings?

Kent Beck 00:15:58 Correct. Unfortunately, refactoring is drifted a long ways from its original definition. People will say, oh well I need to refactor. And they’re changing the structure and the behavior of the system in sometimes unpredictable ways. And I absolutely don’t want to do that. If I’m working on the structure, if I got the structure hat on, I’m just working on the structure.

Giovanni Asproni 00:16:20 I’ve got a question that is maybe out of my personal curiosity more than anything else. So to me, tidyings look a lot like patterns in the way you describe them. It’s like, you can do this and then you tell people, but watch out on when to do them. You may consider that. So I’ve got this impression that, , also the work you did on patterns somehow as an influence here.

Kent Beck 00:16:43 Oh, a hundred percent. Once I started to see repeating decisions with consistent sets of, posing constraints and that’s what a pattern is, I can’t unsee it. So just in seeing the world and seeing my work, I’m seeing it in terms of those sorts of patterns. And yeah, it turns out that there really are those kinds of repeating patterns and if you pay close attention to them, you can do a much better job of making those repeating decisions if you look at them in terms of the conflicting sets of conflicting forces that act on them.

Giovanni Asproni 00:17:22 Going back to the design. So I’m interested in understanding, well maybe in giving maybe the audience some examples of the actual practical effects of tidyings because they are small changes. Yeah. Tiny, often very tiny, , renaming a variable, maybe groups uncalled differently. The break a white line in between two groups of statements and to clarify the meaning or things like this. But in practice, if I understand it correctly, they can have quite dramatic effects when you compound them in terms of code quality. Can you tell us a bit more about it and maybe some examples from real software world?

Kent Beck 00:18:01 Sure. And so that is part of the underlying assumptions of this, I call it empirical software design. This empirical style is that small changes compound and every once in a while, so this is an effect. I think most people who’ve tried to design in the style have noticed. Most of the time you make a little change to the structure and it has a small effect and you make another little change. It has a small effect, and then you make a little change. Seems just like another little change. And then it has a small effect. But then you see, oh, but that lets me do this and then that lets me do that and then I can do those two things and then I can get rid of all this whole blah, blah, blah, blah. But what triggered that was just that one little snowflake flopping over and then you end up with an avalanche.

Kent Beck 00:18:53 When I say let’s make large changes in small safe steps, that’s the kind of effect that I’m taking advantage of. So an example, there was a feature that was introduced late in J unit four called, now I’m going to, it’s been a while since I worked on that code where you could wrap tests in an object. And this was good for pre-condition, post-condition. It was good people of rules, they’re called rules. Anyway, that feature, which was a, a big deal at the time it was released, was the result of a bunch of teeny tiny changes. We had this really complicated chain of functions that called each other, and we just started to turn each one into its own object. And then we nested the objects and then we said, oh, it would be so easy to add another object to this chain. And it was a big feature, but the actual implementation of it was small.

Kent Beck 00:19:55 And we had set ourselves up for that kind of like Jenga, , where you pull the, , you set yourself up, you pull out the things that keep you from implementing this function. Part of the philosophy is this phrase I came up with for each hard change, make the change easy, then make the easy change as opposed to sweating and straining to make really difficult changes, which inexperienced programmers tend to do. So you pull out these Jenga blocks and then you pull out the last one, the whole thing comes tumbling down. But you didn’t bring in a bulldozer and knock the whole edifice over. You just made small changes, small changes and then you made a small change that had a very large effect.

Giovanni Asproni 00:20:43 Yeah. And another thing is making all these changes one after the other, I guess probably has an effect of the power loss we deal with.

Kent Beck 00:20:51 Exactly right.

Giovanni Asproni 00:20:53 It’s also, you end up touching, I think you mentioned this, the code you keep touching all the time is pretty much clean. So the code that is not clean is the one that you don’t touch simply because of this power loss, you keep working pretty much on the same area most of the time.

Kent Beck 00:21:09 Yeah. St. Pareto comes to our rescue once again in this. So programmers, some programmers have this fairy tale that they tell themselves about how those whole system could be tight, clean and not messy at all and completely tighten and just, no software is like that. There’s always parts that are better and parts that are worse. And there’s parts that are improving and there are parts that are getting worse. That’s the nature of it. It’s more like a garden than it is like some perfect sculpture. And because of that, if you haven’t been tidying, when you start tidying, you’re overwhelmed by all the messes you start to see. And that’s part of that question tidy first is realizing, okay, well I’m going to tidy around here because I need to make changes here and I see a bunch more tidying and I’m going to choose not to do those just yet. But, Pareto says 80% of the changes are going to happen in 20% of the code. If you adopt this kind of tidying in your local area, most of the code that you change most of the time is going to be well structured. And yeah, there’s parts of it that you still might be embarrassed about or ashamed of or would be difficult to change if only you had to change it. But you’re probably statistically speaking, not going to have to change it so it doesn’t have any practical effect.

Giovanni Asproni 00:22:42 Yeah. And now maybe a bit of a, you may see that as a provocative question if you like, why not use big refactorings instead?

Kent Beck 00:22:51 Well, the first line I wrote in this book was software design is an exercise in human relationships. What I wrote that and I thought, that doesn’t make any sense. It’s about coupling and cohesion all about power loss. But the more I thought about it, the more I realized if the human relationships that affect software development aren’t healthy, the software’s not going to be healthy. And what’s fundamentally wrong with these pause refactorings, I don’t have a good name for these, but I’ll work on it. These where you say, oh, we need a month to get this cleaned up, is the effect that it has on the human relationships. That might be the most efficient way to do the, to make the changes that you want to make. But if your users or your managers or your product people say, oh, those programmers, they can’t be trusted.

Kent Beck 00:23:53 They just go away for a month. They say they’re working really hard, but when they come back, the system does exactly the best-case scenario. The system does exactly what it did before. Why did I wait a month for that? I could have had a bunch of features and that damages the relationship. So part of the skill of empirical software design is this being able to make, you can’t, if you’re on radio, you can’t see me lacing my fingers together. But to make the structure changes and the behavior changes in small enough steps so that from the outside it looks like you’re just working on features or you’re just working on design. But inside you’re going back and forth and back and forth and back and forth and back and forth.

Giovanni Asproni 00:24:45 Yeah. And I think another problem that I’ve seen at least, with the big refactorings is that sometimes they are done in arbitrary places in code. Code that just looks ugly, but it’s not necessarily the code that they are going to touch later to implement new functionality.

Kent Beck 00:25:01 Correct. So it’s, it’s like insisting that I’m going to go down to my basement and there’s a pile of stuff in the corner that just doesn’t bother me. No, but I need to go down there and change it. Now I’m a little OCD and I understand the satisfaction of cleaning up the corner of the garage. But practically speaking, if you’re getting paid for the work, it’s probably not the highest priority.

Giovanni Asproni 00:25:28 Yes. Going back to structural and behavioral changes. So tidyings, we said you are refactoring, so there are structural changes. How do they relate with behavioral changes? I think we hinted at that when we were talking about composing them. And so when you gave the example of JUnit 4, can we expand a bit on this?

Kent Beck 00:25:47 Well, if you look at some code, let’s say we have to change two functions in a file and one is at the beginning of the file and the other is at the end of the file. So the two functions are coupled according to our earlier definition, but the file is not cohesive because I only have to change these two functions and the functions aren’t even next to each other, to other functions in between them. I can just change function A and then change function B and I’m all done. But I’m going to have to navigate between the two. Maybe I have to bring them up side by side in two editors so that I can see how the two are changing at the same time. So I could do that and it would take me a certain amount of time and I’d have a certain probability of making a mistake in the process.

Kent Beck 00:26:39 And so first and the tidy first in there, just the, the simplest, the littlest change that’s going to make the behavior change we want to make easier is to take the two functions and put them together. Maybe the two go together at the top, maybe the two go together at the bottom, maybe they go someplace in the middle, I don’t care. But if the first thing I do is reorder the functions in the file, that’s the tidying, I’m done, I’m finished. I could deploy that to production is perfectly safe. If it’s not perfectly safe, do it in smaller steps. So now I’ve done that tidying and now I’ve made the change easier. Now I make the slightly easier change. So every time you’re going to make a behavior change, if it feels harder than it really needs to be, I want to ask myself, should I tidy first? And the answer is sometimes yes and sometimes no. Now sometimes you’ll make a behavior change and then that leaves things kind of messy and then you tidy afterwards and that’s fine to do a little bit of it. And then sometimes you don’t have anything pressing and there is a corner of the, of the garage and or the basement and you just go and tidy it just because, but only a little bit and realize that you’re indulging yourself as you’re doing.

Giovanni Asproni 00:28:05 Hmm. But also I guess that could be the case also. Sometimes some tidyings let you see things that you are not seeing before. I don’t know, something you extract the helper or Ö

Kent Beck 00:28:15 Yeah, yeah, a hundred percent you change some names and now you realize, oh this code here is exactly like that code there. I couldn’t see the similarity because the names were gratuitously different. And then I fix that. Now I see, oh that’s really doing exactly the same thing and now I can unify the two and I can reduce coupling.

Giovanni Asproni 00:28:36 Yeah. Another thing that I find interesting is this difference between the structural and behavioral changes, the reversibility aspect. So you say that be structural changes are generally reversible. I mean most of the time, maybe depending on the side, but behavioral ones are often not reversible. So can you give us some examples?

Kent Beck 00:29:00 Sure. Jeff Bezos talks about the type one and type two decisions. It’s the same, the same concept. And when you need to make a decision, one thing to notice is whether you can undo that decision easily. If you can undo a decision easily, you don’t have to carefully analyze, should I do this? People spend more time analyzing the potential effects of a decision, than they will actually making the decision. If it’s reversible, that makes no sense, just do it. So I think that these four lines would make a good helper function, just extract it, just see. And if you have proper tooling, that’s going to be a snap. It’ll have zero chance of breaking anything. Just do it. Because if you don’t like it, you can undo it. You inline the function because of course your ID has really good refactoring support.

Kent Beck 00:29:57 Some do and some don’t. And so those changes are easily reversible. So they don’t need to be carefully analyzed and thought through before you execute them. Behavior changes, on the other hand, have effects in the outside world. You make it so new users can’t sign up or you report the wrong numbers to the tax authority or you make payouts incorrectly or you ship things to the wrong address. All those are effects in the world that aren’t easily reversible. And so you should take extra care when making those irreversible decisions. The mistake people make is they think, well we, the structure’s like this and we’d like it like that. And so we’re going to sit and think really carefully about whether we should do that. No, start moving the structure from how it is to how you think it should be. If you discover that’s the wrong direction, you can easily reverse it. So don’t treat those decisions the same way that you treat the decisions that have irreversible effects in the outside world where you want unit tests and functional tests and you want to double check with the users and you want to roll it out a little bit at a time and all that stuff. That’s all to prevent irreversible effects.

Giovanni Asproni 00:31:21 There are also some tidyings or structural changes that can be irreversible or very difficult to revert. Things that come to my mind, I don’t know anything. If you rename an API, you don’t change the behavior but give it a different name or I don’t know, change DRL of an https call or arrest call or things like this. So also with structural changes, there is a chance that depending on the change, even if the behavior is the same, the change might be very difficult to reverse. Am I correct in this?

Kent Beck 00:31:56 Yes. So working on J-unit, we spent more energy on backwards compatibility than we did on new features just in terms of engineering hours. And that’s because there were billions, tens of billions of dollarsí worth of tests out there that that relied on one set of APIs and we just didn’t want to break those. So the key trick there is to have two versions in parallel at the same time and this is the standard deprecation thing where says, okay, you’re calling this function, you should call this function instead, but we’re still going to make sure that the software works while we’re in this transition period. That makes that decision to a degree reversible because if you decide then the new way is bad, then you may be able to go back to the old way. Now you can get stuck with one foot on the dock and one foot in the boat where you have the old API and the new API and you have to maintain them both and they’re getting further and further apart.

Kent Beck 00:33:05 And that’s quite uncomfortable. But you try and avoid that by working in small safe steps to the degree that you can. But yes, those decisions that leak out into the outside world, those design decisions that leak out into the outside world, those become irreversible. And so you need to take a different process thinking about them than you do about the ones where that are cohesive, where you can control how the coupling goes. So that http call is coupled with respect to the names of parameters and the name in the URL. If you want to change the URL, you’d have to change all those callers and there might be millions of them and you don’t have any control over them. So you have coupling there that constraints the kind of changes that you can make. So you’d like to take that coupling and make it in as small an area as possible.

Giovanni Asproni 00:34:01 What about the architecture of a system? Tidyings are small things. Yeah, small changes, tiny things. Then we talk about how they compose and can actually enable bigger changes. So what is the relationship with the architecture of a system and the tidyings?

Kent Beck 00:34:19 So to me, design is fractal. It’s about you have these elements, they have relationships, the relationships benefit the elements, not the elements. Coffee has not quite kicked in yet. The relationship if I have a function and it can be simpler because I can call this helper function. Now that there’s benefit in that relationship between those two functions. I don’t see any reason to differentiate from the perspective of principle between elements that are like two services in a big system and two expressions that are in the same statement. I don’t see those as fundamentally different things. The same principles, coupling and cohesion apply reverse their ability and irreversibility power loss apply at all those different scales. That’s hence that word scale free. You want to make changes in small safe steps. It’s an exercise in human relationships, like all the principles still apply, it’s just the scale at which you work is different.

Giovanni Asproni 00:35:26 And what about the relationship with design? How much do we need to design, say upfront?

Kent Beck 00:35:32 Okay, you’re trying to hook me in here. So the more comfortable you are making changes to the structure of a running system, the less need you have to design upfront. And the problem with designing upfront is you are making decisions in conditions of maximum uncertainty and minimum of knowledge. So in this empirical style, you want to minimize the decisions that you make when they’re really likely to be wrong. Now, I don’t, people disagree with this and God bless them, but I don’t understand why you would insist on making the maximum number of decisions in conditions of maximum uncertainty and minimum knowledge. I just, don’t get it. There is this fear, okay, we’re going to get this system into production, it’s going to have a horrible design and we’ll never be able to fix it. And I don’t accept that premise. I think that if you take good care of the relationships between designers and users and managers and investors, and so if you take good care of those relationships, you will have time to do the design. The whole thing is not going to be beautiful and tidy. Yeah, exactly right. But all software that’s making money is messy. And if it’s not making money or not yielding a return for open-source projects, then who cares.

Giovanni Asproni 00:37:07 Also, imagine that if you put enough safety net around you maybe being a bit disciplined with having test automation, automation for deployments and things like this, making changes to a system that is in production becomes a bit easier. I’m not saying that is easy, but easier than if you lack this discipline.

Kent Beck 00:37:28 Right. And the practices that you just listed are all part of making the change easy.

Giovanni Asproni 00:37:34 Now let’s go to discussing when to tidy, that tidy the question mark.

Kent Beck 00:37:40 Correct.

Giovanni Asproni 00:37:41 Yeah. So one thing that I find interesting and I can relate to I have to say, is that your focus not only on the quality of the code, yeah? Which seems to be the case with many software design books focus on the code and the structure of the system and how thing cohesion and coupling. But a bit in a, in a context free situation if you like. And so you said, well the answer to tidy first or after or later or never depends on several forces. So you mentioned cost, revenue, coupling and cohesion. I think these are the main ones you mentioned. Can you tell us more about that?

Kent Beck 00:38:28 Sure. So one of the reasons that strategy of designing upfront runs into a lot of resistance. I want to be careful how I critique it. One of the reasons that is that that upfront design runs contrary to the economic forces that are driving your software. And the two big economic forces that push against designing upfront. One is just the survivability of the system. The sooner you can get some features in there, creating revenue, getting people using it, the more likely your software is to survive. That’s one factor. And if you say, well, but I’m going to spend three months designing it before I implement the first feature, software might not survive three months with no features. So yeah, it might be nice to go off and get all the design worked out before you start implementing stuff, but if the project’s canceled as a result, why did you do that?

Kent Beck 00:39:33 The second one though is this time value of money, the net present value. And if you’re a programmer and you don’t know what NPV is, I strongly urge you to look it up and then play with it. So this happened to me probably 25 or 30 years ago. I’d had these discussions where I had perfectly good engineering reasons for wanting to do what I want to do and the business people would just like, what are you even talking about? Makes no sense. Somebody mentioned about the time value of money. Oh this was probably when I was working on insurance products and I built spreadsheets and I started playing with the net present value and my eyes were open. And this is the idea that, money sooner is the same as more money. So if you, uh, revenue, revenue sooner is the same as more revenue cost later is the same as lower cost.

Kent Beck 00:40:36 And right now we have relatively high interest rates compared to the last 10 years. And so this effect is amplified and this is the idea that if I can get revenue sooner, I can earn interest on the money that I made. So that’s the same as if I earned more money, but I earned it later. The design upfront is cost, it’s going to have future benefits, we promise, but they’re going to happen later. So we have to make a lot more money later than we made now in order to make up for the delay in achieving that revenue, it’s also costs that happen sooner. So those costs are amplified in today’s perspective compared to if we took that same amount of investment and we spread it in payments overtime. So when somebody says, well I want to, I want to design the everything right up front, I think one you are increasing the chance of their project’s just going to die. And two, you are working against the time value of money which the business people understand. And I as a programmer had never encountered and I didn’t understand it. So even though you’re, that has a kind of intuitive appeal to me as an engineer, it just goes against the context in which I’m working and that context isn’t going to change. If I adapt my style of working to the context that I’m actually working in, things are going to go more smoothly. I’m going to have an easier time doing what I’m doing.

Giovanni Asproni 00:42:17 This applies to the tidyings as well then. So the decision of basically tidying first later or never I guess is in this context as well.

Kent Beck 00:42:27 Sometimes, it’s a no-brainer. I have some messy code, it would take me a half an hour to figure it out and make some changes. If I spend 10 minutes tidying, then I can spend 10 minutes making those same changes. Okay, no brainer. I should definitely tidy first for sure. Sometimes it’s kind of a, I can tidy for 15 minutes and then spend 15 minutes making the change or can spend 30 minutes making changes to the messy code. Probably in that case it’s still in your best interest to tidy it because somebody else is likely to change that same code in the future and they’ll reap the benefits of the tidying that you make. It’s when the potential benefits of the tidying push out further and further. Like well over the next three years we’re going to recover the cost. And that’s why the bias in tidy first is I’m going to make the changes to the structure right where I want to make the changes to the behavior and I’m going to be increasingly suspicious of making the changes that ripple from there.

Giovanni Asproni 00:43:28 Yeah, makes perfect sense. One thing that also I found interesting is the Constantine’s Equivalence that you came up with, which basically state that cost, the cost of software and coupling are essentially the same thing. Now this is interesting because also from what is said and all the things we discussed also there is also costs in decoupling as well, which seems to be a bit of a contradiction now. How do we solve this one?

Kent Beck 00:43:58 Sure. So Constantine(?) equivalent says the cost of software is mostly the cost in changing the software. This upfront cost is a small fraction of the money that you’re going to spend on the software overall, but all those changes aren’t the same because of the power loss we’ve talked about. There’s going to be a few big changes that really dominate the complete cost. Just as you have a few gigantic hurricanes that dominate the effects of hurricanes all around the world. Those big changes come because there are changes that ripple through the system. I change this, ooh, that means I have to change those two things. Ooh, because I changed this other thing and then I have to change five more things, blah, blah. And then you’ve got, even though the initial trigger was just let add one more button to this screen, the ripple effect went through the whole system and became extremely expensive.

Kent Beck 00:44:56 So if you count-up the cost of those changes that set off an avalanche, that’s a big proportion of the total cost of changing the system. So the cost of the software is roughly equivalent to the cost of those big changes. And those big changes come about because of coupling. That’s exactly the description of coupling. I change this so that I have to change that too. So if you squish all those together, the cost of software is approximately the coupling in the system, but there’s also costs involved in decoupling the system. Because we’re going to have to make those structural changes to get rid of the coupling. So a common move like this is I’ve got a server and a client and I have some protocol on the wire. I write an interface definition language used to be if I changed the bytes that went across the wire, it’d have to change both sides.

Kent Beck 00:45:56 So they’re coupled with respect to changes to the protocol. If I introduce an interface description language, now I only have to change the IDL and the client and the server automatically change. So I have decoupled those two, but I’ve had to define the IDL, I’ve had to change the tooling. I maybe I changed the deployment, da da da. So there’s some cost to eliminating that coupling and now you’re in this trade off space where you have the cost that you’re paying for coupling and the cost that you’re paying for decoupling. And it’s the sum of those two things. That’s the cost of the software, which is why software design has such a centralized role in the business of software because you are juggling those two factors. How coupled is the system? How much would it cost to decouple it and how are we going to main maintain that balance over time?

Giovanni Asproni 00:46:55 Talking about the more implementation details, let’s say that some people in our audiences say, okay, I want to look more carefully when I tie the up code and maybe follow an approach closer to what Ken recommends. Yeah, maybe a bit more disciplined with refactoring structural changes and behavioral changes. Now one of the things that of course you, you wrote about is how to push those changes in the repository. So especially for teams that use good requests and maybe code reviews.

Kent Beck 00:47:32 Whole another topic. Yeah, but go ahead.

Giovanni Asproni 00:47:34 I won’t ask about rank based development. I’ll just ask about this.

Kent Beck 00:47:39 Okay, good.

Giovanni Asproni 00:47:40 I’ll ask you the difficult questions. So what are your suggestions here? And maybe we experimented with it a bit and see if you have any recommendations?

Kent Beck 00:47:50 Sure. So one recommendation is if you have to use pull requests, as I said, I have strong opinions about that, which I won’t get into here. If you’re using that style, if you’re working in that kind of workflow, try as an experiment. I’m not saying this is the right thing to do. I’m saying this is a worthwhile experiment. Try only having structure changes or only having behavior changes in a single poll request. The structure changes should be really easy to review. I extract some helper method, I’m done. Yeah, the system’s a little bit better. That’s an easy one to review. You might not require review at all once you get some experience doing this. If I have behavior changes, don’t mix structure changes in with them. So my behavior changes going to be, here’s a new test and then here’s some changes to the logic of the system without which the test would fail.

Kent Beck 00:48:49 With which the test succeeds. Okay, so that’s again a relatively easy change to review. It requires more review than the structure changes because you have to ask yourself questions like are there other corner cases that are aren’t being covered here? Is the test sufficient to give me confidence that this is going to work, but only have one kind of change per PR. Or if you want to, you can have multiple commits in a PR and each commit could only have behavior or structure changes. You work it out, but strictly separate them. Now sometimes when you’re programming, you don’t know exactly how you want to implement or you can implement something so you’re kind of fooling around and you change some logic and then you see the structure’s a little wrong. So you change the structure and then you can change some more logic easily and then back and forth and back and forth.

Kent Beck 00:49:48 And what you end up with at the end of a few hours is a, a big jumble of changes. And part of this experiment that I’m suggesting is either you throw away what you’ve done and you start over. And in this SB style where you’re only making structure or only making behavior changes at a given time, or you look at all of the changes that you made in that three or four hours of wandering around and you pull it apart, you retroactively apply this rationalizing model where there’s a beautiful Parnas paper called a rational design method and how and why to fake it. That’s it’s exactly this, where you say, okay, well we wandered around and we were all confused and it was ambiguous and finally we got the system that we wanted. Now go tell the story as if we went directly from point A to point B. And people will be able to understand that a whole lot better than trying to recount all the wanderings that you did. You can do that or you can just throw everything away and then redo that four hours, it’ll take you half an hour and it’ll be a clear set of, we change the structure in these kind of ways, which makes it easy to change the behavior in these kinds of ways. Try that as an experiment. If it doesn’t change anything, it doesn’t change anything, but who knows?

Giovanni Asproni 00:51:13 And now, I guess the regular question about AI, because you should have seen these many said programmers among other roles are going to disappear. They say all sorts of things about AI, but I know that you are currently working for a startup which offers legacy modernization, which makes use of AI to provide its services. So you at least have some contact with this.

Kent Beck 00:51:36 I also have a personal project called rent-a-Kent. So we can talk about both of those.

Giovanni Asproni 00:51:42 Okay. And the thing is you of course wrote the book 31st for humans, not AI systems. So what is your view on the role of AI and software development? Will programmers become a role of the past? Will AI enable everybody to become a programmer or something else?

Kent Beck 00:52:02 Everybody doesn’t want to be a programmer. That’s the key point. So as an old person, I have seen this story told over and over and over and over and over again. And that’s what Cobalt, there were these mainframes and you program them in assembly language and the business people wanted the computers to do things, but they couldn’t get the programmers to make the computers do the thing. And then along came COBOL and now regular business people will be able to write their own systems. Well it turns out that there’s more to writing a system and kind of knowing what you want it to do. It’s an actual discipline. There’s skills involved if you don’t have those skills, disaster results. And then that same, I wasn’t around for the Cobalt thing because that’s 1960 I was just born. But that same impulse in today’s, it was no code two years ago, a year ago it was no code.

Kent Beck 00:53:02 Now we don’t need programmers anymore. Anybody can write their own software. We have no code. Well, AI is, it’s just the next level up of abstraction. There’s some fundamental amount of information you have to give the system. So you can’t just say to an AI, write me a profitable application. There’s not enough information in that prompt. So somebody has to say what the system has to do. And as soon as somebody has to say what the system has to do, your programming, whether it looks exactly like programming or not, and I am using various LLMs right now and it feels like programming, but it’s kind of programming with chopsticks in a blackened room, wearing ski gloves. You’re like, oh, I don’t like, you know here’s this somebody, asked my system the question and I, this came back a terrible answer. How do I fix that? And I’m like, oh, I don’t know. Do I change the prompt? Do I, blah, blah, blah. But it’s still programming.

Giovanni Asproni 00:54:07 And then some final questions. These are questions I’m really curious about. So the first one is that, you have brought lots of ideas to the software industry. So the most famous ones, test driven development, extreme programming patterns, and then there was the three X, Explore, Expand, Extract. And then now there is Tidy First. Do you see all of them as part of overarching method or theory of software development?

Kent Beck 00:54:37 So I wouldn’t say theory in a mathematical kind of sense. I would say there’s a very human thing that I have been trying to achieve my whole career, which is to help geeks feel safe in the world. People with this kind of geeky mindset oftentimes feel very uncomfortable, unsafe in perfectly ordinary situations. Like at a cocktail party, oh God, I hate being at cocktail parties because I’m always like, am I talking too much? Am I not talking enough or should I talk to somebody else? Well, turns out that that’s a skill. Learning to feel safe in that environment is a skill and it’s a learnable skill and I’m around people on a regular basis for whom this doesn’t even seem to be a problem, but it is for me. So that sense of I want to feel safe when I’m actually safe and I want to feel unsafe when I’m actually unsafe, say I’m making some change in the system and thereís some side effect to this that I donít know?

Kent Beck 00:55:42 Well alright, I want to have new skills so that I actually am safe. I’m working in a safe way so that I can feel safe so I can go home on the weekends and feel good and I can feel a sense of confidence and accomplishment when I go home at night. I want to have those feelings. So sometimes it’s the skills of learning to actually engage in situations that felt unsafe, but were unsafe. And sometimes it’s learning skills to be safe, like separating behavior and structure changes. Ugh, life changer. Just the amount of anxiety goes down so much when I’m willing to take that extra step. So I would say my unifying theory of software development is that programmers are people too.

Giovanni Asproni 00:56:36 I noticed reading your writings on the web and experiences, the thing you blog about and how that you’re writing on your sub-stack account, it seems to me that you are one of those people that keeps re-evaluating what he used to think and see. Try to see those in a new light somehow and maybe improve.

Kent Beck 00:57:00 Yeah, it’s an obsession. It’s not necessarily healthy, but I can’t stop it, so I may as well try and get the leverage out of it that I can.

Giovanni Asproni 00:57:08 Based on that, have you found anything that you thought or wrote in the past that you now consider to be wrong?

Kent Beck 00:57:15 Wrong in the sense of I wouldn’t give this advice again, let me give you a concrete example. I’m a big believer in small batches. So if you’re going to do continuous integration, the ultimate of small batches is to take one pull request at a time and run it through the integration process. I have a team that I worked with, I’ve worked with for 20 years and 25 years in Switzerland, and their integration process took 15 or 20 minutes and diffs were piling up throughout the day. And so they, if the principle was well, the small batches, you’d have to do one diff at a time. It turns out that they started batching their diffs and the latency, the waiting time went way, way down for everybody in the whole company started working better. So that’s counter to the principle of flow of small batches. But practically speaking, I mean the long-term solution is to make sure that the integration process doesn’t take 15 minutes, which is a longer term more difficult decision. So was I wrong to recommend the principal of low to that team? I don’t think so, but that’s only one principal and there are other principles involved too. So I would say there, there are times when I’ve missed context like that for sure. Yeah, but, but just like flat out, oh, that’s a bad idea. I don’t, I don’t, I don’t run across those.

Giovanni Asproni 00:58:54 And my last question, this is about your Canon TDD. So I think you published that in your sub-Stack as well. Yes. Now looking at that, at the graph you created there, there is no tidying first mentioned. So why is that?

Kent Beck 00:59:12 Because that’s not part of the TDD workflow. TDD is very specific and that was the point of that article was people would complain about TDD has become this political hot button. I don’t understand. You can’t make me write tests, man. I’m not trying to make you write tests. I’m saying I’m predicting your experience if you work in this particular workflow, and if that’s not your experience and I’m wrong in my prediction, that’s fine, then do something else. I don’t care. But many of the criticisms were just of some other workflow. So people would say, well I don’t, I don’t like having to write a hundred tests before I can write any code. I’m like, well that’s not what it says. So that’s why I tried to lay it out. I’m not saying this is how you should program. I’m saying if you’re criticizing TDD and you’re criticizing something that isn’t this, then you’re criticizing something else entirely.

Giovanni Asproni 01:00:14 Okay. Kent, I think we reached the end of this conversation. Thank you very much. And is there anything that we missed that you would like to mention?

Kent Beck 01:00:25 So subscribe to the newsletter [email protected]. I’m working on a large language model, which takes all of my writings, which is millions of tokens and fine tunes models based on those called, Rent-a-Kent.

Kent Beck 01:00:44 So Rent-a-Kent account AI to check out on progress on that and there’s some interesting early returns for that. Thank you so much for the very prepared interview, Giovanni. I really appreciate that.

Giovanni Asproni 01:00:57 Thank you Kent for actually coming to the show. It has been a real pleasure for me and we came to the end. This is Giovanni Asproni for Software Engineering Radio. Thank you for listening.

[End of Audio]

Tagged as:
Join the discussion

More from this show

Menu

Recent posts