Juval Löwy, Software Legend and Founder of IDesign discusses his recently published book, Righting Software, with host Jeff Doolittle. This episode focuses on Löwy’s belief that the software industry is in a deep crisis, evident from the numerous projects that fail to deliver on time, on budget and on quality. He discusses his belief that the software industry is actually in the dark ages, rather than experiencing a renaissance. Elements of human behavior, history and psychology are discussed as they pertain to the crisis. Ultimately, Löwy expresses his ultimate goal, and reason for writing the book, which is to help “right” the software industry. His book describes Löwy’s method for system and project design, which he offers as a major breakthrough in addressing the crisis.
Show Notes
Related Links
- Introduction to Righting Software
- Juval Löwy on The Path of the Architect
- The Evolution of Services, Dot Net Rocks Episode
- IDesignIncTV on YouTube
- SE-Radio Episode 332: John Doran on Fixing a Broken Development Process
- SE-Radio Episode 331: Kevin Goldsmith on Architecture and Organizational Design
- SE-Radio Episode 308: Gregor Hohpe on It Architecture and IT Transformation
- SE-Radio Episode 273: Steve McConnell on Software Estimation
- Episode 132: Top 10 Architecture Mistakes with Eoin Woods
Transcript
Transcript brought to you by IEEE Software
Jeff Doolittle 00:53 Welcome to software engineering radio. I’m your host Jeff Doolittle. In this episode, I am pleased to welcome world-renowned software architect and author. Juval Löwy to discuss his recently published book, writing software. Juval, founder of I design is a master software architect, specializing in system and project design. He has helped countless companies around the world deliver quality software on schedule and on budget recognized by Microsoft as one of the world’s top experts and industry leaders. He participated in internal strategic design reviews for C-sharp WCF and related technologies and was named a software legend. He has published several bestsellers and numerous articles on almost every aspect of modern software development. Louie speaks frequently at major international software development conferences and conducts master classes around the world. Teaching thousands of professionals, the skills required of modern software architects and how to take an active role as design process and technology leaders.
Jeff Doolittle 02:03 Welcome to software engineering radio. Juval my distinct pleasure. Jeff, your book is titled writing software, but it’s spelled R I G H T I N G rather than w R I T I N G. Can you explain the wordplay and the reason for the title? And so the aim of the book is to white, the broken industry that we have the software industry, as I liked in the put fish for the book, almost every aspect of software development is actually broken and well, the aim in the book is not just by the industry, but hopefully there’s a launching pad or bring about a soulful development. The Renaissance that reminds me of a claim that you make in the beginning of the book. When you say specifically the software industry as a whole is in a deep crisis. So perhaps you can expound little bit on that.
Juval Löwy 03:00 Nature and causes of this crisis as you put it. Well, it’s easy to explain the elements of the crisis. It’s deeper to explain the causes. So if you look at how we develop software today, every aspect, every element of software development is broken. Let’s look at costs, software costs too much. Now, when you say something cost too much, it, so the elective term with respect to what we’re the only way to quantify it is with respect to the benefit to the customer. And we’re past the point of diminishing attorney software development. For example, almost everybody that uses office three 65 or 2019 would have been just fine with office 2010 or 2013. Why? Because it was just good enough, but 2017 costs the same as this other than 10 and 2016 costs the same as 2010 and so on. So you’re just sending money for nothing for these vendors.
Juval Löwy 03:53 When in fact they all software, if it was done wide, it’s just good enough. In addition, the cost of software is not just the cost of writing the bits. It’s the overall cost of ownership. And if you went to actually account for it correctly, of course, seven, 10 years and multiple versions, the cost would actually be prohibitively expensive. In addition, the cost of doing changes to software is so much that every once in awhile, when the customer manager asked developer to do something, they saw the hand in the air and said, we cannot do that. Now cannot is a statement of visibility. Well, in fact, if it’s just code, you can write different code. When they’re saying we cannot do that. They actually doing a statement of cost because the thing of course we can do it, but the cost of doing it is going to be so prohibitive.
Juval Löwy 04:41 It’s going to be cheaper to just wipe the slate clean and do this whole thing again from scratch and do this one thing you’re trying to do here, which makes the cost of what they’re trying to do here is expensive. And then there’s time. Time means nothing for software developers, deadlines for most developers are these useless things, wishing buyers they’re chained to the desks. In fact, you laugh. It’s actually sad. If developers meet the deadline, everybody’s going to be surprised because the expectation is for them to fail. And I can you own the business and be viable in the world with time and money, do not matter. I mean, you couldn’t operate the hotdog stand this way. Then there’s the climates developers solve the own problems. In fact, developers routinely fail to communicate with customers. If customers give developers the collect requirements, everybody’s going to be surprised again, which on its own is a whole other, uh, enigma.
Juval Löwy 05:40 Because if it’s other suppliers, the customers give you the only requirements. Why does everybody fail to accommodate the failure, to communicate with the customer, which is a whole separate question, whether the filter communicate with the customer and side by side, with unmet requirements and disappointment there’s requirements of a perfectly communicated, but developers simply decide not to do because it’s too expensive or it takes too long, or they just don’t know? And it’s really puzzling. And nobody takes into effect how the introduction of the system would actually change the requirements against which it’s written in the first place. Then the staffing often not the same people that write the code, maintain it over years. And the people that maintain it simply cannot make sense of the software. Most software teams have crossed a social world of being able to make sense of the complexity of their own system
.
Juval Löwy 06:32 And so the system is completely unmaintainable and they try and solve problems or introduce and improvement and introduce new problems and new defects. And it’s of course now it’s some kind of a runaway reaction where you do one problem. It’s all one problem introduced two additional ones and so on, and that just grinds people to nothing. And then we have a high attrition and burnout, and this just heaved on people to the point that the stress level is so high. You see developers actually getting sick from work. And then there’s of course quality. I mean, in software, the word bug is synonymous with software. It’s it’s, it’s almost with, Hey, I have a little bike over here. No, it’s a hideous monster defect. I mean, even the little bug in a sense, or can take down the whole jumble jet, apparently, there are no little bugs.
Juval Löwy 07:22 I mean, what’s nonsense. It is to try and classify bugs. And if I were to tell the listeners that every project I was ever involved in, I shipped on schedule on budget with zero defects. I’m not quite sure which part of that statement would offend them more. But I suspect it’s the fact that I shipped was zero defects because in their mind it is absolutely impossible to have impeccable software and to make it even worse. None of these problems is actually new people now spend their entire career in software land, not seeing software done right, even once. And so now something else has happened. They think that it’s just impossible to have any other level of existence. And so if you try and actually even improve things or suggest things, if they’ve already concluded, it’s impossible to do any better in not start insulting their own intelligence.
Juval Löwy 08:13 They start fighting you as they try and actually improve things. So the status quo must be the way it has to be the headway. It has to be. It’s a, that’s why I’m saying that this is a crisis. Every aspect of software development is actually broken and it’s not just missing that. Go to Google and type a software projects, successful failure rate. And you’re going to see horrendous. So when the circus statistics, most of the projects actually fail. They fail to meet the objective. The commitments, the ratio of a project of failure is higher than projects that succeed. And even that is a misleading statement because this situation is so bad. Companies, organization had to change the definition of success, because if you can’t clean up your act, either you clean up your act or you lower the bar. And so success in software today is defined as anything that doesn’t think of the company right now. You can find it later. Your mistakes can destroy lives 10 years from now, but if some code leaves the building missing a leg with a black eye, well, that’s actually okay, that’s considered success. And so it is next to impossible to find projects today that are done correctly, start to finish. So how would you
Jeff Doolittle 09:27 Respond to the person who might say, I agree that there are elements of a crisis, but the company that I’m working for is actually doing quite well and making significant margins. And we aren’t experiencing the crisis the way that you describe it, how might you approach someone like that? And maybe what would identify the fact that they actually are in crisis, perhaps they don’t realize it, which I think is kind of what you’re,
Juval Löwy 09:54 What’s called Adams. This is what Scott Adams, uh, recently, uh, dubbed loser sync, meaning in one word, it’s the thinking of a loser. It’s he coined that term and there isn’t a slew. The thing is because first of all, he can say, I’m doing well. You could have done much better. And you lost a lot of money. That’s, that’s something that is probably implied sort of the what’s the, what is seen and what isn’t unseen the door that didn’t bark and all of that. But it’s more fun than, than we do software today is if it’s the dark ages, meaning if you look at the dark ages, most people had the ugly boots and fancy short lives, but you know what? They still had a good time and that is family and at parties and they laughed. Right, right now, even in the dark ages, you had people that were way better off than other people.
Juval Löwy 10:41 I mean, look at the warlord in the 10th century with the feast and the servants and all of that, right? You can say I’m at the top of the world. I am the King, I’m the best. And the problem is it’s all true by the way. And you’re very successful. The problem is you’re still a 10th-century wall. Lord, if you have to choose between being just a, nobody in a 21st century or a 10th-century warlord, I think you would choose the warm bus once a day and Instagram than being a 10th-century warlord. So saying that you’re the top of the world and you’re successful in a pitiful environment is absolutely the statement of a loser.
Jeff Doolittle 11:19 And maybe somebody who’s not really caring about the state of the industry in general, but as content to take, what’s good for them perhaps.
Juval Löwy 11:28 Well, I’m very much in the Adam Smith camp that people should be a joystick and an egocentric. And if everybody, the act two wants to own interest, eventually his invisible hand, make sure that there’s prosperity for all. Okay. So I’m not, I’m not that much that people say, okay, let’s just fix the world, especially you can’t just find to fix up the world and like the whole industry, if you want me to be messy, right? I mean, to me, it’s just hypocrisy people saying, we should do this and we should do that. Then society should do this. In fact, the house is a big mess, you know, cleaning up your home before you pontificate for others, how to do their job. If it isn’t working at home, don’t export it for example yet. So that’s, I think a good summary of the nature of the crisis.
Juval Löwy 12:12 Well, hopefully, there’s an answer for how the crisis can be addressed. And I have a feeling you have some opinions about what those solutions might be. So if you have slacked away the specifics of quality and schedule and budget and the ability and so on at the root cause is a poor design. It’s a poor design of the software system. It’s a full design of the project of was used to build it. It’s a poor design of the team, of their location, of the people, of the way you build the system. All of that is fully designed. And what I propose in the book is a way or methodology of fixing it. Now it’s not a silver bullet because there are no silver bullet. In fact, all that is in the book, quality, a lot of practice, and a lot of devotion to actually get, right.
Juval Löwy 13:00 But it’s also not just the one thing that I’m proposing in the book. And the reason is if you have a multidimensional crisis, you need a multidimensional solution. You can’t just fix one aspect. You can’t just fix it, the design of the system, and then everything else would be fine. And it doesn’t work this way. Even if you have the best system design and architecture, if the project doesn’t have enough time or money to build it, or developers cannot build it, you will fail. Right? So everything has to be aligned for success. What you’re recommending then sounds like an integrated and holistic approach to the software crisis. The word integration and holistic is, is, is a good step. I look at it as a formula, meaning to fix the software industry, you need to fix both the system aspects and the project aspect. So he’s designed system, it’s designed projects and along the way with also design your own career and your teams and all of that would fall into place as well.
Juval Löwy 13:58 I want to focus the remainder of our conversation specifically on system design so that we can give our listeners a sense of what your proposals are for designing systems. Where would you like to start that exploration? What’s the most important element? I guess the most important thing is to discuss what not to do. Okay. And the reason we have to start there is because the, what not to do is the prevailing model upon day off designing software systems. Today I’ll be blunt unless somebody has spent some time with me, they’re doing it wrong. Almost every system I’ve seen in the wild is designed incorrectly. And so I guess the most important thing that we should discuss is that you should avoid what I call functional decomposition. So let’s just explain those three words, architecture in general. And he’s an act of decomposition. You take a big nebulous idea and you break it down into smaller building blocks, components, services, whatever the methodology or the obstruction level is that makes sense.
Juval Löwy 15:05 Most people do function on the composition. Functional. The composition means you look at the required functionality of the system and you need compose accordingly. You reflect in the architecture, the required functionality. So if I give requirements, spec the bleeds, 1.2, one CSUSM should do a and blown 5.6. So you should do B you wouldn’t have an Abe block B block, same block. That is the kiss of death. If you did that, the project has already failed past tense. The project has already failed before anybody wrote the first line of code. And why is that? So how much time do we have? So I’ll give you, I’ll give you a few reasons why this is bad. If every one of them is literally, and then in the coffin to kill it, but I’ll save the best snare for last. So we can discuss it in the light of what is the right way of doing things.
Juval Löwy 15:55 So first you’re not going to get just three things to do a decent size software system. Today may have a hundred, 200 things to do. If you do true functional decomposition, you’re going to have an architecture with say a hundred blocks. Now Fiona blocks is really, really the epitome of a bad design. And there is only is while each block is small doable, understandable. The integration cost of such an architecture is astronomical. Absolutely. And so we’re talking about level of complexity that defined the mind. Now you could do something else. You can have a component to a service and pumping to it, a and the B and the C and the D. And you have these ugly dumping ground functionalities, and you get these crazy mandalas of dooms. You can have two or three of them side by side sometimes. And the poem we start wanting don’t have the integration cost or from the explosion field things.
Juval Löwy 16:52 The internal complexity of those big things is also astronomical because complexity is not any other size, something twice as big as not quite as complex. It’s four times as complex, 20 times, it’s complex and so on. So the functional composition always lands you in these extreme areas of complexity that nobody can understand the intricate relationship between the inside and the course components of your architecture. And if you don’t understand your architecture, nobody can maintain it, extended reuse it. The other problem is functional. The composition is that some of these to set up the system, do functionality, a do B, do billing, invoicing, shipping, whatever it is. And that’s some of that is the client of the system. But the notion that you need to do a and then B and then C is business logic. So now all of a sudden the client is a fuse infuses business logic.
Juval Löwy 17:40 The client has to know that you can do billing. If you haven’t done invoicing. Now, every developer knows that you should never pollute. The client code is business logic, and yet they’re all doing it. And there isn’t these functional decomposition forces them to pollute. The client has business logic. So now two things happen. First. The last thing you want is to we down the business logic in two places in the system and in the clients, since you have to put it in the client, you tend to beef up the clients and dumb down the system. So now you end up with the clients being the system and some kind of a poor excuse or hand-waving for so-called service makes sense. The other poem is what does it take to add a new client? So marketing waltzes into the office. And so we need to add a mobile device. The problem is it’s not just adding another client, it’s a duplication of the system and knife to maintain the business logic in two places. And it’s not going to be exactly the same business logic because of slight differences between the clients. And then after the third type and the force type of for client at some point, developers hold the hand in the air and say, we cannot take it anymore. No more clients.
Juval Löwy 18:43 The other problem is that because of the interaction between these areas of functionality, suppose I have another system that needs to do just a particular functional entity. Just be just billing. If you lift the beef from one system, you can have a drop it in another system because the a and the C is hanging off it because before the B or do an a and after the B a do a C, and the reason these functional decomposition is always timed the composition as well. You decompose based on a particular order as a result, reuse does not exist in the function decomposed system. Another problem is that as you enter the system to do the, a back to the client, B back to the client, see back to the client, you’ve entered the system in three places, and that means sleep places to about security and sleep places do all, but scalability, performance, availability, responsiveness, dependency, swoop performance.
Juval Löwy 19:32 Everybody knows that a good design ideally would have a single point of entry, but should certainly minimize the number of points of entry. Functional composition always tends to maximize the point of entry and visit of course complexity, because now you start smearing security and scalability across all places in your architecture. So points of entry are maximized. Coupling sounds like it’s maximized and business logic gets spread out across the system, particularly in the clients. And complexity is maximized because of the interaction and the internal complexity. But you know what, you know, if I can prove to you that functional the is the own way of designing system without using a single so far. Okay, great. I’m going to use the first law of thermodynamics, the first law of some of the simply States that you cannot add value without sweating. Now we don’t know quite why this is the first of them with dynamics.
Juval Löwy 20:32 There’s nothing in your tone in physics that says it has to be so, but it is, but it is. We know at least that on this side of the big bang, that’s the way things are. We’re not quite sure what happened before the big bang, but on this side of the big bang, the first law of someone dynamics holds the universe. And everybody knows that you cannot have a free lunch. I mean, that’s another way of saying those. No Wolf adding value was out sweating. Now, if you look at architecture, architecture is a high value activity. Everybody’s listening to us instead of doing something else, because the value architecture, in theory, being able to have a system that’s maintainable for a long period of time with minimizes total cost of ownership that is extensible, usable, secure these all high added value aspects. Now function, the composition is fairly easy.
Juval Löwy 21:24 If you have a list, either in requirements, spec on a scrum board all day, quite functional nineties, the architecture is simple. No sweat. You say I have an AB block and B block a C block, but that’s the boredom. You said, no sweat. Now you try and cheat the first law of thermodynamics because on one hand, you endeavor to add value. On the other hand, you endeavor to do it without sweating. And according to the nature of our universe, this is just not going to happen. And so, yeah, I just proved to you that without a single software argument, you should never do function on the composition. And yet that is exactly what almost everybody is doing. So, as you’re describing this, this sounds like a horrible way of doing things. So why would anyone do things this way? If it causes so many problems, the answer is very simple.
Juval Löwy 22:14 Human nature, human beings cannot resist the allure of the free lunch. And this has nothing to do with software, right? I mean, let’s look at the simple example. Let’s look at the alchemy for thousands of years, people tried to turn lead into gold. It didn’t work even once. And yet they commanded Kings Kingsland. Some, they were the smartest people in every kingdom and every Alchemist had their own unique way of mixing the potions and saying the income stations and so on zero results. And for thousands of years, the best practice was to keep trying this why the law of the free lunch it’s in our nature to do this. And unfortunately this has nothing to do with intelligence. The goodness outcome, mr. Fulltime was Isaac Newton, and most people are less modern. Isaac Newton, Isaac Newton was an Alchemist. He kept trying to find the philosopher’s stone and so on.
Juval Löwy 23:11 And so it takes iron wheel to overcome the desire to do functional decomposition. And that’s actually the challenge in doing it. I need doing what we’re going to discuss, hopefully how the right way of doing it, because everybody around you are going to be hell-bent on doing it incorrectly. But, you know, given the nature of the crisis in software, you basically be fighting in Sonata here because the smartest man that ever lived said that the definition of insanity is doing things more of the same with expecting better results. And I, the customer and the boss to want you to do things better. I mean, arguably, if you’re doing a new system there, isn’t doing it is because the old system didn’t work. Cause you should be able to maintain your systems for decades. And yet you can’t and therefore you need to do something else, which is hopefully better, but how could you better do better by doing more of the same? Right? And so Einstein said, that’s the definition of insanity. And so you will have to fight the lunatics, trying again, to do function on the composition.
Jeff Doolittle 24:15 Yeah. And that’s interesting when you mentioned it’s not necessarily a measure of intelligence, it’s just a measure of human behavior to do things the same way over and over again. And that being said, there must be a solution to this problem. And I have a feeling you have one in mind yesterday.
Juval Löwy 24:35 What I propose architects do is decompose based on volatility. The composed based on velocity means architects should identify areas of potential change thing that could change in the system. And those you encapsulate in building blocks of the system. And then you provide the client behavior of the system as interaction between these encapsulated is of volatility. Decomposing based on volatility is literally the essence of a good design or system around us, our design using this principle. And the reason is when you start thinking about Lindsay, basically composition, you start envisioning your system as a series of volts. Each vault encapsulate some area of volatility. And when the change happen, the change is very dangerous. It’s like your hand on it. And the auntie opened up the door of the opiate vault. Plus the handgun that inside and closed the vault and the vault does boof than whatever was inside.
Juval Löwy 25:34 The vault may be complicit, completely destroyed. And yet there’s no sharpness flying all over the place, destroying everything else in the system and put differently. You’ve contained the change. Now, if you look at functional, the compensation function, the composition would maximize the impact of such a change because it decomposed based on functionality. But if initially the change can never be in one place because your building blocks are associated with functionality, not with change of volatility. So now when the change happens, it is like those things haven’t gone that into your system and everything is torn apart. And that is the real reason why function. The composition is so deadly. All the other things I said about it is just the word, your appetite, the inability to handle change, ease the case of this for any software system. Could you summarize
Jeff Doolittle 26:25 By saying that things that change together should live together in your system?
Juval Löwy 26:30 There’s, there’s another layer here, which we can get into, which is if you need to design software systems, you can ask yourself are the common areas of volatility in difficult office systems. And if you can recognize that in advance and you come up with a nice starting point for the design effort, but you can take it even a step further. And you can say are the typical interaction and the relationship between these areas of volatility. And if you bring that into your design effort, then you already have some structure and you’ll have to reinvent the wheel every time. And that is actually the nature of, uh, all good design or go and good architectures that there is this very high level of recurring of self-similarity. So I wouldn’t actually say lift together. I would say that you should actually look for high degrees of symmetry and high degrees of recurring architecture.
Juval Löwy 27:25 Now we’re not talking about the actual details, for example, a mouse and an elephant have the same architecture. You agree about that. Yes, but this is a totally different. And so, and yet, if you know that you need to have a kidney or liver, two lungs and so on, then you’re already very much on your way to designing a small little animal or a big animal. Right? And so that’s another thing that I discussed actually in the book I discussed the structure. You can actually put on top of software systems, but it’s all driven by the idea of decomposing based on volatility that you identify the volatilities. And then if you identify certain types of alternatives, they have this kind of structures in your architecture, and they’re allowed to do these set of interactions.
Juval Löwy 28:43 What are some concrete volatilities that might help our listeners have a better understanding of what that term means? So most of the systems implement some sets of required behaviors and not at the end say requirements or functionality. As I said, they quiet behaviors. It turns out that even the way most developers and marketing people phrase the climate is actually wrong because they use functional requirements, functional requirements. Talk about the quiet functionality of the system. Well, in fact, what developers should really receive from the customer or marketing is the acquired behavior of the system, how the system is required to behave. And there isn’t, you want to focus on the quiet behavior is because required behaviors are less open for interpretation, like quite functional entities, like one functional auntie. There’s so many ways of doing any particular functionality, but their behaviors are much more close here.
Juval Löwy 29:41 You have to do it like this, like this, like this and like this. So you can think of any quiet behavior as some kind of a sequence of activities. Once you understand that all the climates and mix first, the sequence of activities that live in the name for it, it’s called a use case. Now, if all use cases are sequences of activities, then there’s actually inherent volatility in it because the sequence could actually change. If I ha, if I have C activities, ABC, I can do it. Sequentially, ABC. I can do them all concurrently in Parlin. I’m still doing the same activities, ABC. And yet there’s enormous volatility in the sequence, or maybe I’m doing a first and then BNC sequentially, or maybe I’m doing a and M based on a condition doing B or C, right? So the sequence can behind highly volatile independently from the activities.
Juval Löwy 30:33 And so whenever you have this kind of volatility, I recommend you use a type of a service. I call a manager, which encapsulate the volatility in the sequence. Now, even if you keep the sequence fixed, you can still change the activities because I could have a sequence of a, and then be an NCF and have a sequence of a and then D and then C. So I can change the activities independently from the sequence. And so volatility in activities, meaning all the ways of doing a particular activity is completely independent from the volatility of the sequence. And so once you understand that you start looking for, well, the theaters and activities, and I call those things in my structure and taxonomy engines. Now all the business logic in the world typically operates on some kind of either state that you maintained or interacts with other systems that maintain your state.
Juval Löwy 31:27 And so there’s actually two additional facilitators here, which is how you access those resources. I mean, how many ways they all fixing another base? 17? I mean, maybe more encountering, right? Right. And even if, um, you fix the, um, the way you access something, what you access can actually change because I could access a file or cash or a database, and that can be local or remote or cloud base. And so the resource that you access can be a cue, can be another base. It can be a cash is also volatile. And so the whole set of volatility is existing resources. And also in how you access those resources, seven 48 is existing clients. Different clients can have different lending technology. If they present information to users or different SDKs that they expose and different technologies that they use to expose the functionality of the underlying system.
Juval Löwy 32:22 So that also voltage you need to encapsulate a whole family of varieties is something I call utilities. Every different software system has infrastructure that it requires to survive things like security message, bus queuing, the agnostic instrumentation, profiling, logging how many ways they are doing logging, how many ways they are doing security. And so the last thing you want to do is say the security volatility. At course, all your components, because when the security requirements will change, all your system will detonate. And so you need to have utilities and encapsulate the infrastructure volatility, which has nothing to do with why behavior of the system hardly any requirement, spec contains specific requirements against the way we’re supposed to do logging, right? That’s internal to the system, but you have to do it along with the agnostic and instrumentation. And so that makes sense. And so once you understand that there’s is occurring patterns of volatility in software system, the next thing you can say, okay, I’ll have a difficult interaction.
Juval Löwy 33:26 Just six as typical volatilities are the typical interactions. Could you compile a list of do’s at least of doesn’t, at least of all ways of how these things interact with each other? Now it’s fairly abstract as, as I’m doing it like this, but at the end of the day, these are universal design principles. If you open your eyes, you will see everything in the world of army was actually designed this way. For example, we are recording this webcast right now by integrating areas of encapsulated volatility. For example, the laptop I’m using is very different than your laptop. In fact, it is as unique as a fingerprint. There isn’t a single laptop like it on the face of the planet. And yet all of our data is encapsulated behind the microphone, Jack, and then the HDMI, uh, and the video card, as long as it sends the video card to the screen and the audio signal down the wire to you, I don’t really care what happens beyond it.
Juval Löwy 34:20 It’s, it’s completely different than any other lifter, but it’s encapsulated. I could not conduct this webcast and podcast. If I were to care about the specifics of these things, right? If I look at, uh, my heart, my heart encapsulates an enormous amount of volatility of pumping blood, you can have high pulse, low pals wanting, standing, walking high adrenaline, noradrenaline is low and blood pressure, high blood pressure. And yet I don’t care because I’d voted is encapsulated. I could not do any of my day to day activities. If I cared about the volatility of pumping blood, and when you’re driving a car, I can drive your car sooner because even though your car is very different than mine, that voted is encapsulated behind, say the steering wheel and the, a gas pedal and so on. And by the way, it used to be that these things were not encapsulated.
Juval Löwy 35:13 And then it was really, really difficult to operate these machines. Now, the nice thing about it is that I can actually change my laptop with no ill effects to my heart or my car. And I know it sounds strange because most of our system that’s completely impossible. And yet, as long as I encapsulate the theater of my laptop, my car and my heart, I actually unaffected. Maybe my wallet is, but not my heart in my car. Okay. Sure. And so all good design decomposed based on qualitative, meaning the Incapsula didn’t hand volatility of what is actually going on. You mentioned
Jeff Doolittle 35:46 Podcasting before, and the fact that our laptops are not particularly made for podcasting, and yet here we are podcasting. So can you explain the relationship between this concept of encapsulating volatility and how real-world interactions actually emerge?
Juval Löwy 36:09 So this is actually, uh, one of the key concepts I have in writing software. It’s an idea. I call a composable design and let me walk you back. So this kind of a mental machinery of what this is actually all about the mission of the architect is to design a system that addresses all requirements by all requirement. I actually mean all the requirements. I really mean all requirements that sounds daunting, present, and future known and unknown, nothing less will do. You can’t leave some requirements and say, well, I don’t have it because I didn’t have enough time. You have to satisfy all requirements. That’s where the body’s said, nothing less will do. If you don’t do it. Then some point in the future, the system is going to implode. Now, traditionally people tried to do it using some variation of functional decomposition now by doing so, what they were actually doing, they were designing against the requirements.
Juval Löwy 37:07 When was only against the requirements. The one thing we know the clients do is that they actually change. And that’s actually a good thing because of requirement wouldn’t change. Nobody here would actually have a job. None of our listeners and other you are normally by the way, would have a job because somebody somewhere would lie to somewhere the software. And that wouldn’t be good enough. We’re here because of climate change. And I can to be, it’s actually a good thing because there’s so few of us and so many of them. So the more quantum change, the better off we’re all going to be. The other problem is that the reason you’re laughing is because if you design against the requirements and the requirements actually change, then that inflicts a horrendous pain and the, when this cost, because now design have to change and once fuck them. Once you already have developed the system and deploy it, if you change the design, that is the most expensive thing you can possibly.
Jeff Doolittle 37:57 Yeah. That sounds very painful. Every new requirement means a significant change across existing components or possibly even the creation of new components. Every time there’s a new requirement is that basically what you’re saying.
Juval Löwy 38:10 That’s the typical situation where you designed against the requirements. And so developers, managers, and so on, have literally learned nowadays to resent the change to the requirements. Because I think flick spending, nobody likes pain. Now the solution is so simple. It is a little, most the entire color, which is Don designed against the requirements. Now it sounds so absurd when I say it and he actually laughed, but I’m dead serious, right? You’d like, if any, if you’d like Ben to keep doing it, keep doing it. That’s just fine. So the question is what should do? And so the mission of the architect is to identify the smallest set of building blocks that you can put together to satisfy all the requirements. That’s the mission. Now, I really mean all when I’m saying all, all requirements known and unknown present and future and the curveball, anybody could ever slow it to you.
Juval Löwy 39:01 You can handle, now this all must be confined within some sort of parameters, correct? No, no, no. It’s all. Or it’s all, all death, all, all for a particular business or all for a second. So now w now we’re getting to the justice. So if you look at most systems, most, some don’t just need to do three things. They need to do a hundred things. Two things, see other things, right? Lots and lots of requirements and requirements can be exposed functionally or behaving in the form of use cases. But there’s a plurality of them, right? It’s not, it’s not one or two asleep. So if you have flea on the things to do, we’ll see on that, the requirements, it turns out that most acquirements most functionalities. I actually not distinct and unique. Most requirements of variation of other requirements, most use cases of nation, of other use cases.
Juval Löwy 39:50 We have the happy case, the set case in incomplete case, the gets with just doing club custom over little paid. Excellent. So let’s actually define two types of use cases. There’s call use cases and fluff call use cases that present the essence of the business. That’s what brings the customer to the door. Fluff is up. It’s all the other things. It turns out that all business systems have a very, very small number of core use cases. I encourage now the listeners to do a mental exercise and going count in the head, the number of truly these things, things the system is supposed to do. And most of it will come up with one, two, maybe sleep, and we find a number two or sleep to be very typical. We find a number four and five to be very high. And I’ve only seen six once.
Juval Löwy 40:41 And what I’m saying, I’ve seen see six once. I mean, it was a system designed for one of the largest companies in the world. The company that every one of the listeners have its product at home for a system that is actually a system of systems supporting the entire it infrastructure at had 40 services. And it was supporting 120,000 users, 24 by seven in five continents. Okay. Do you know what? Then you get six core use cases. Most systems have a very low number of core use cases. I’ve lost count of how many system we designed it, iDesign it’s in the many hundreds, we have never seen more than six. And like I said, six, and it’s only happened once. Typically the number is two or see maybe four. So it’s not a very large number. What the architect needs to do is identify the smallest set of building blocks.
Juval Löwy 41:28 And again, those building blocks encapsulate some areas of volatility that you can put to satisfy all of the core use cases, since all use cases of variation of the core use cases. What they actually present is therefore a different interaction between your building blocks, not a different, the composition they’re off. So now when the acquirements change, you’ll design those nodes, and this is a fundamental observation, and I call this whole approach, composable design. You never designed against the requirements. You want to come up with a smaller set of building blocks that you can put together this fine. All of the use cases. Now remember the core use case represent the essence of the business, the essence of the business hardly ever changes. And if it does change, it changes the ketonic pace. Meaning it’s really, really slow. Federal express is in the business of shipping packages.
Juval Löwy 42:24 It could mean move into ballistic missile control, but not instantly and maybe never possible, but not plausible. And if possible, incredibly unlikely and very slowly. And so by focusing on addressing the core use cases and have the smallest set of building blocks, that you can satisfy all the core use cases ways. Now, when they climate’s change, what changes is the fluff, it’s not the core. And therefore all it expresses itself is a different interaction between your building blocks, not a different, the composition they’re off. So nominate requirements change. Your design does not. Okay. That is fundamental observation about the design of things. There’s plenty of examples. For example, the design of the human body was complete 200,000 years ago. Meaning homosapiens appeared 200,000 years ago. Now I’m pretty sure that being a software architect, wasn’t part of the spec at the time. And yet I’m quite capable of being a software architect now for most developers and architects, that would be in contradiction because how could you possibly have exactly the same design as a Hunter gumbo and yet survives an architect.
Juval Löwy 43:29 Most people will do functional the composition. Then we’ll design against the requirements. If they were to design somebody like me, there would be a box in me for everything I need to do. There’ll be a box conduct, a conference session, write the book, drive to the airport, mow the lawn, kiss the wife, all the things I need to do. And yet I don’t have a box in me called BSF architect or write the book. And yet I’m using exactly the same component as a Hunter gatherer. And there isn’t these while I’m using exactly the same components as a Hunter together, I’m putting them together in a different way. And he said, definitely the core use case hasn’t changed. And it was just one core use case, which is survive. That’s the only use case that I need to actually support everything else is fluff. You look at your immune system, there’s near infinite possible pathogens, bacteria, and viruses that could infect you and kill you.
Juval Löwy 44:21 And your body cannot have an infinite number of antigens to actually go and fight it. So others it’s actually work. Your immune system has a small number of it recognizes the invasion, composers quickly, the correct response to it and treats it. Everything is done this way. Now, when I have actually said before, I said that you need to come up with a smaller set of building blocks. That’s actually a very key observation. They have to come up with a smaller set. And the reason it tends to be small is because in general, less is more. You’re not in the business of doing more work in the business of doing less work. You want to get away with the smallest number of building blocks, most number of lines of code smallish, number of everything. Now, if I give you a requirement spec with building blocks in it, I could have a design that has on the blocks in it.
Juval Löwy 45:11 Well that’s, as we discussed before, we need bad design and that can also come up with the design, had just one giant God, the block that does everything well, we’ve learned the hard way. That’s not a good design, just somewhere between one and 300 is the smallest possible number. It turns out that, that number, if we try and guesstimate, what the number is, we can use all the of manga is more like one 10, a hundred a thousand. And I think everybody recognized that number is more like 10. It is like a hundred. So in orders of magnitude, it’s more like 10. That could be 12. It could be 20, it could be 13, but it’s closer to 10. It is to a hundred. And there is an inch closer to 10 is because if the were supposed to provide the quiet behavior of any system, not just software is by the interaction between those building blocks.
Juval Löwy 45:58 Then fundamentally, you’re looking at how many possible combinations you have now. The number of possible combination of 10 components is on the order of 10 factorial, which is buzzing magazine, a bazillion by the time you’re talking about 50 factorial, there’s enough combinations here. Look for every grain of sand on the planet. And this is even without discussing partial sets and repetitions. If I’m allowing that the number is truly astronomical. Number of molecules in the universe and the number of new molecules in the universe and such. And so Leo five component is not enough because the factorial of Leo five is not high enough. Well, the time you have about 10 components, the number of combination is astronomical. And now anything you can slow at it present and future known and unknown. You’re good to go. And this number of about 10 is another fundamental observation about the nature of things.
Juval Löwy 46:51 For example, if I were to count how many components are in my body and not talking about sales, I’m talking about components, liver, kidneys, and so on. It’s about 10. And how many components are in your laptop? Well, I’m not accounting transistors, I’m counting memory CPU, GFI card network card. Well, it’s about 10. And how many components in your car engine, block, gearbox, water pump, battery, whatever it is, 10 everywhere you look, you’re going to find that number 10. And the reason these all system of converge to that number because no system, even from an evolutionary standpoint, is any incentive of having frivolous components. It doesn’t actually need, you have the smallest possible number of components that you can compose together, despite all possible requirements. Okay? That’s the fundamental observation about how you design complex systems. You actually have very simple components so that you can have near infinite possible interactions.
Juval Löwy 47:45 And all of that leads to somewhere else. If you have a small number of components themselves, fairly simple, then suppose something changes. Suppose something changes in the business domain. Well, all you have to do is you don’t change the components, just change the way you put them together and you respond quickly to change. But you have to be asking you a question. Isn’t there an ability to quickly respond to changes in the marketplace and in the business domain, the essence of agility. Yes. And it sounds like what every software company would wish for. And every business for that matter, every software company, which is for every business is craving. And yet almost every software system is failing to deliver because of design against the requirements. And so it is almost diabolical. They try and respond quick to the requirements by doing function and the competition here’s the a here’s the B his billing is invoicing.
Juval Löwy 48:36 And by doing so, they absolutely guarantee the lack of agility because when the change happened, they did on it. It’s diabolical. So how do our listeners begin to turn the requirements that they’re given, which I imagine from my experience and theirs as well are often suboptimal. How do they begin to turn those requirements into what you describe as core use cases? So you may have heard something. I never said, I said, don’t design against the requirements. I didn’t say you should ignore the requirements. Absolutely. You need the requirements of variables as you need them is where you identify the errors of volatility and the coal use cases. Yes. So you have to invest in the climate analysis as a result. This of course takes longer than functional the competition for functional compensation show, just Skype it to a user story, tack it on the Kanban board and the way you’re going.
Juval Löwy 49:29 I’m not saying you should do that. Do that eventually. Absolutely. If you want to do the fastest artifacts, then should do it, but you’re doing it using the building blocks of composable design, which encapsulate volatility, right? Everything has to be in place before you can actually even do the first attempt of doing it. And it includes of course, looking at the requirements. And in the book, I describe a set of analysis tools of identifying these areas of volatility. And I describe a set of, um, techniques and postures, even that you can actually put yourself in to make it easier to come up with these areas of volatility, because remember, nobody would ever have a conversation with you and the climate that sounds like this, Oh, we’re going to change these. We’re never going to change the app. We’re definitely going to change this. Now. We’re not going to change stuff.
Juval Löwy 50:20 They’re only going to talk to you about functionality and features. That’s the only conversation you’re ever going to have. And so what you need to do is take all of that input in it’s very important, but then start discerning and trying out of it, the areas of volatility and the core use cases, the downside is that it takes longer than functional decomposition. And you’re going to have to do it on the fire because nobody understand these things. Everybody wants to get the features out. You’re literally going to have to be fighting in Santi, because again, they’re all going to be hell bent on doing it wrong. Guaranteeing the pain. It has never worked before. They’re going to hold hell, bent on doing it again. Okay. Just like alchemy. And so the trick is I fought not just doing it. That’s not good enough because if it takes you years to do it, right, you’re not going to get yours.
Juval Löwy 51:06 The trick is actually to do it effectively and efficiently quickly. And in order to do that, you have to have a lot of practice doing it. So you still don’t violate the first off some of the Nomics, because if the sweat enormously to get to that level of proficiency and professionalism, so you can do it quickly. Okay? So you don’t violate the law. You just don’t sweat when it’s time to actually do it. And that’s the essence of true mastery, you know, and they’re the saying that undefiled everybody’s sink to their level of training. If you’re trying to figure out these things on the fire, you’re just not going to be very good. Sun shoe said that winners win. And then they go and fight. And that’s the position you need to be at. So that’s, I wish, I wish I could promise the listeners just a magic bullet. Oh, just want to sleep and do it. My way is a being the boom. You’ll be fine. It doesn’t work this way. This in fact is the hallmark of quacks. Quacks promise you an easy win. No, there’s no easy wins here.
Jeff Doolittle 52:08 And you mentioned before that you’re not proposing a silver bullet, but if anything, it sounds like this is a rigorous engineering discipline where you’re breaking down the problem into a composable design where each element of the design, it encapsulates those areas of significant change or volatility. And it’s putting them together that causes the features and the requirements to be
Juval Löwy 52:32 Addressed. Exactly. Right, exactly. Right. But you know, is it instant to be a good engineer? It takes training. It takes education, takes practice, practice, right? Is it, is it a basic expectation of professionals? Be it engineers or doctors or lawyers or pilots, is that well versed? They know the trade by heart and that they’ve factors limitlessly to be very good. Right. The last thing you want to do is be patient number one of a doctor. Okay. And nobody would really want to fly with the pilots, uh, that just learn how to maneuver correctly. I, that’s not a good idea. Right? You won’t fly with somebody that’ll test stencils on hours. Okay. That’s a good place to ask.
Jeff Doolittle 53:11 Absolutely. And that reminds me of something that I’ve heard of before called the 2% problem, which is you can never be good at something that you spend 2% or less of your time on.
Juval Löwy 53:20 That’s something that like I coined the 2% public coin that you’re kidding me. Of course.
Jeff Doolittle 53:28 Oh, that’s funny. That’s hilarious. So now the architect has designed these components and each of these components is encapsulating some area of volatility, but I can almost hear the product managers and the other managers saying, where are my features?
Juval Löwy 53:50 So it turns out that features are always an everywhere aspect of integration, not implementation. It’s a universal design rule. And it is a fundamental observation about the nature of the universe again, which is what makes it the universal design rules. So, so what I mean by that, if you look at the podcast right now, if you need to provide a feature called the podcast, where’s the podcast, is it in the microphone? Is it in the keyboard? In the screen, in the laptop, you cannot point at any one thing and say, that’s what the podcast is. You have to integrate the laptop, the microphone, the, uh, streaming devices, the servers, all of that has to be integrated to get the podcast. There never a box you can say, huh? That’s what the podcast is. You have to integrate things. Okay. So now let’s look, say at my laptops, which I’m using to call this podcast, the laptop actually doesn’t know that there is a podcast.
Juval Löwy 54:51 The laptop only needs to send to your server. Now, some kind of audio signal. So as far as the laptop is concerned, there’s one feature here which is send an audio signal. That’s it. However, there is nothing in the laptop itself that actually provides that I have to integrate the CPU when, uh, the calculation is done. They have to integrate some files for running the operating system and have to integrate the, uh, Sam card and all these things to be integrated, to send you the sound. There is no one books in the left of that says send sound. Okay. So I mentioned say the hard drive, the hard drive is storing a bunch of data required for it, not the hard drive doesn’t actually know about sending files and sending sound. The S the hard drive is providing a feature off storing stuff. So as far as the Hunter is concerned, the feature is stalling.
Juval Löwy 55:49 Okay. So if I look at how the hardrive is done, is there any one thing in the hard drive that’s doing storing? And the answer is no, because the hydro itself has little memory chips, and it has a small box that has little schools. That’s holding some kind of farm media. It has its own power regulators and so on. Okay. So if I look at the little school that’s holding the flame of the hard drive together, it doesn’t really think about storing files. As far as the squares is concerned, the squares providing one feature, which is fastening things, but it’s the one thing in the school that’s doing the fashioning. No, you have to integrate the STEM of the school and the friends of the slew and the head of the screw and the talk to get the future of fact fascinating. Now you can drill this way all the way down to the quarks, and you will never see a feature.
Juval Löwy 56:38 Interesting. So basically what you’re saying is, well, actually, what that brings to mind is a fractal that selves, all the layers. It is exactly a fact of wherever we land. In a fact, I’ll provide each feature as an aspect of integration, but never implementation because you can never actually implement the feature. You always offer the features aspect of integration, not implementation. So the screw seems like a really helpful concrete example there because the screw itself does not have at the feature called fascinating. And yet the nature of the way the screw is designed in conjunction with the threading that you screw it into. It’s that integration of those two parts that provides you the feature of fascinating two things together. Is that correct? That way? You’re saying yes. And again, zooming out. If you look at the hard drive, there’s the single thing in the Hodge of doing the storage I have to get at the school, the media, the power, Julito the box, all of that is to mitigate that storage.
Juval Löwy 57:39 And if we look at the level of the laptop, there is the single box and lift, I’m doing presentation or sending an audio file. If you integrate the GFI, Cobb, the CPU, the hard drive the bus, the memory to do that, and you keep going like this. Hmm. So you mentioned before that idea of all to summarize it and call it the law of 10, that in every system, there tends to be about an order of magnitude of 10 components, but as we’re drilling down, and maybe this was a question and listeners minds before, as we’re drilling down, we’re probably finding another layer within each of those individual components of 10 order of magnitude of 10 at the next level. And then if you drill down into one of those, that’s fine. And at some point you HD stomach model of clocks. We just lost all my check is 22 things.
Juval Löwy 58:24 And this which is also often, okay, absolutely. Now, as architects, we don’t actually deal with squawks or with screws. And as architects, we actually deal with overall system architecture. Okay. Well, these things are far more concrete and dishonorable and not as granular as individual schools, right? So I’m just showing you that everything in the world is actually put together this way. Yeah. That makes me can go up or down and you will never actually see a feature. Well, this is some heavy stuff. And I think our listeners are going to have a lot of questions and possibly desire to explore more deeply. So as we’re wrapping things up, is there anything else we missed that maybe you’d like to mention in the time we have left? Yes, what’s coming next. So as soon as you design the system, without losing a heartbeat, you must design the project to build it.
Juval Löwy 59:20 It doesn’t matter how good is your design. If you don’t even have time or money, or if the public is too whisky, you will fail, deliver the system. So just as they have to design the system, you have to design the project to build it. And it may come as a complete shock to you to learn that people are doing it wrong. And you can make exactly the same argument about system design and project design that you should never do what they’re doing today in project, but instead you should do something else. And we can have a whole different podcast on how to design projects.
Jeff Doolittle 59:50 And I imagine that just how you had some opinions about how to improve on system design, there are also approaches and techniques for learning how to correctly do project design. That is very true. As you said, as we’re in the interest of time, we’ll have to move on and leave that at that. But thank you so much for your time. And if people want to find out more about what you’re up to, where some places they can get more information,
Juval Löwy 00:15 They can always go to lighting software.org and learn more about the book or buy the book. And hopefully at some point we also want to start, because like you said, these ideas of how to come to them was hard to wrap your head around. And so we may be, we’re considering launching a series of book clubs, which are going to be guided reading by us at iDesign of the ideas in the book. Okay.
Jeff Doolittle 00:41 So that’s writing software.org, spelled R I G H T I N G putting right writing software.org, the industry. Exactly what? Okay, well, Juval. Thank you very much for joining me to discuss your book, writing software. It’s been a fascinating discussion. Thanks to Jeff. This is Jeff. Do little for software engineering radio. Thanks for listening.
[End of Audio]
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected].
SE Radio theme: “Broken Reality” by Kevin MacLeod (incompetech.com — Licensed under Creative Commons: By Attribution 3.0)
