Brenden Matthews, a seasoned software engineer, entrepreneur, and author of the Idiomatic Rust and Code Like a Pro in Rust books (both from Manning), speaks with SE Radio host Gavin Henry about Idiomatic Rust. They start with a look at what “idiomatic” means, and then discuss generics, traits, common design patterns you’ll see in well written Rust code, and anti-patterns to avoid. Matthews suggests some tools that can help you immediately write idiomatic Rust, as well as what building blocks can also help. This episode examines what generics are and how they compare to other languages, as well as what traits are, how macros help, what a fluent interface is, and why unwrap() is bad. They also discuss what code smells to look out for, Clone, Copy, and a really nice place to go read real-world Idiomatic Rust code.
Brought to you by IEEE Computer Society and IEEE Software magazine.
Show Notes
Related Episodes
- SE Radio 562: Bastian Gruber on Rust Web Development
- SE Radio 490: Tim McNamara on Rust 2021 Edition
- SE Radio 644: Tim McNamara on Error Handling in Rust
Other References
- In Rust We Trust – A Transpiler from Unsafe C to Safer Rust
- Unleashing the Power of Clippy in Real-World Rust Projects
- brndnmtthws – Overview
- https://www.linkedin.com/in/brndnmtthws/
- Brenden’s Blog
- Idiomatic Rust
- Code Like a Pro in Rust
- GitHub – brndnmtthws/conky: Light-weight system monitor for X, Wayland, and other things, too
Transcript
Transcript brought to you by IEEE Software magazine and IEEE Computer Society. This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number.
Gavin Henry 00:00:18 Welcome to Software Engineering Radio. I’m your host Gavin Henry. And today my guest is Brenden Matthews. Brendan is a seasoned software engineer, entrepreneur and active open-source contributor with over 20 years of industry experience. He has been working with Rust since his early days contributing to various Rust tools and open-source projects and is the author of two books, Idiomatic Rust and Code Like a Pro in Rust , both from Manning. Brenden is also the creator of Conkey, a popular system monitoring tool and a longtime member of the Apache Software Foundation. Brendan, welcome to Software Engineering Radio. Is there anything I missed in your bio that you’d like to add?
Brenden Matthews 00:00:57 No, that’s pretty good. Nice work.
Gavin Henry 00:00:59 Two books under your belt’s, pretty impressive. One’s impressive enough. To lay the foundation I’d like to start with an overview of what Idiomatic Rust is. So what does this mean?
Brenden Matthews 00:01:11 It’s a good question. So I suppose, I mean forget Rust for a second, to write any software in a way that is idiomatic is to do it such that the way you’re using it is in line with the best practices and patterns that are well known commonly used generally for whatever that language or tooling is. And a lot of that is pretty subjective I would say. Right? And these things are also like context dependent. Just as an example, every different company tends to have their own ideas about how to do like coding style and so on. And some companies will say like we want all our variable names Camel Case and others might prefer a snake case or whatever. And often it’s also just like code-based dependent. Like if you’re doing open-source stuff, generally you follow whatever pattern that project that you’re contributing to already uses.
Brenden Matthews 00:02:13 And so anyway, in the case of Rust, Rust has its own idioms and patterns that make sense in a Rust world. Rust specifically because it introduces some new ideas about how to do programming. I would say it means there are some new idioms and patterns that you would not have seen in any other programming languages unless you’ve used ones that have very similar features to what you have in Rust, which other languages have the features that Rust has like at least nowadays it’s not like Rust is the only language in the world that has traits or whatever, but if you’re coming from say Python to Rust, then it’s probably a wildly different world from what you’re used to. So how do you get up to speed very quickly such that you can write Rust that makes sense in a Rust world I guess.
Gavin Henry 00:03:09 And you mentioned sort of standards and styles there within companies. Is idiomatic in relation to any language the same as a code style or code standards that you’ve set or are they different?
Brenden Matthews 00:03:22 It’s a good question. So I spend some time talking about this in my book actually. So one thing is defining the difference between a pattern or a design pattern and an idiom. And generally speaking, at least the way that I define it is that idioms relate to things like code formatting more like stylistic things. An example like I already used this, how do you format your variable names to use snake case or camel case in the case of Rust, Rust has like a bit of a quirky pattern or idiom that it uses, which is that generally most like variables are snake case whereas names of structures or enums or whatever their definitions are going to be. I guess it’s not Camel case but whatever is the casing where like the first character is uppercase, maybe that is just Camel case. Although when I think of Camel case I usually think like the first character is lowercase and in any case, and then there are like few other ones like if you define a global variable in Rust, then that’s supposed to be all uppercase separated by underscores, whatever that’s called.
Brenden Matthews 00:04:37 And so these idioms like if you think about it actually doesn’t really matter how you format your variable names in real life, right? Like once the code gets run through the compiler and comes out the other end, like these things don’t really matter. So in that sense it’s just purely stylistic, right? That’s totally different from something like design pattern which design patterns relate more to how you structure the code in like more of an architectural sense and that can affect actual outcomes like performance and algorithm runtimes and stuff probably trivially, but it can have some effect to a certain degree, especially if you’re talking about getting into like CPU caches and stuff.
Gavin Henry 00:05:24 Yeah, we’re going to touch on some of the patterns that you mentioned in your book in the next couple of sections you touched upon talking about what should be the right way to do things like the cases or the style and when in fact it doesn’t really matter. And that’s something that I’ve come across myself because I’ve been learning Rust for a while now and you just have this inbuilt feeling that you want to do it right, what’s the right way to do it. So are we bad programmers if we don’t write idiomatic Rust?
Brenden Matthews 00:05:53 Well this is a tricky question and again I think this is something I talk about in my book, but the way I see it is that you’re generally writing software for two different audiences right? Like one is the computer or the compiler wherever the execution environment for the code and the way that computers handle the code doesn’t really matter at all from a stylistic perspective, right? And if you’re working entirely by yourself on private code in your own little proprietary world, I suppose then this stylistic stuff really doesn’t matter at all outside of your own preferences, right? Like you can do whatever you want like you don’t even have to write your variable names in English if you don’t want to, even though English seems to be kind of like the standard for most software code generally, when you start working on anything that involves working with other people, collaboration, then they’re like, there’s sort of like two layers to the language, right?
Brenden Matthews 00:06:54 The first layer is the layer that is written for the machine or the compiler like the actual instructions that tell the computer what to do. That’s one layer. And then the second layer is the layer that involves how you’re communicating to other people who have to read your code, how the code works. And this is where these idioms and patterns matter because language is essentially just a system of like patterns and grammars and different structures that contain words that, words are basically just symbols that represent different ideas and things and people and places and so on. And without like a sensible structure that anyone can understand then you’re kind of just like inventing your own language. And so if everyone is speaking their own language then nobody’s talking to anyone because nobody understands what anyone else is saying and keep in mind like there needs to be a certain amount of overlap between different dialects and so on in English for example, some people speak in different accents and you can generally understand what people are saying across different accents. And if you think of an accent as an analogous to like different programming coding styles, then yeah there are variations but the actual underlying language, like if you’re speaking English needs to be relatively in order for you to like communicate and understand what someone else is saying and so on. You need to follow the idiomatic English so to speak, to communicate effectively.
Gavin Henry 00:08:30 What I’m understanding you saying there is the fundamental underlying Rust language when other people are trying to read what you’ve written, you don’t want to get in the way of them trying to understand what you’re actually suggesting the code should do. So if you’re not following those IDMs or patterns then they’ve got to spend time trying to break through like your analogy of the accent to understand your intention.
Brenden Matthews 00:08:53 Right. Not just that but like the way that the human brain works is that it’s just constantly doing pattern matching. And so when you’re looking at someone else’s code, the faster you can recognize and understand what the patterns and the structures are and what they mean, then the faster you can understand the code and the more it aligns with sort of the well-known, well understood idiomatic things about that language’s practices, then the faster you can get up to speed on that code and as someone writing the code, the easier your code is for other people to understand, the more valuable it is. And as someone reading the code then the quicker you’re able to like recognize the patterns and understand what’s happening without having to necessarily walk through each line of code and trying to understand what is happening here, then if you can understand what’s happening that code very quickly without having to go through everything like line by line and figure out where each byte is going and what’s happening, then you save yourself a lot of time. And so a shortcut to making code easy to write and easily understand for different people is to follow, stick to well understood patterns and idioms and so on.
Gavin Henry 00:10:07 Yeah, that helps me with your analogy because you can wouldn’t necessarily be classed as a bad programmer if it’s just for yourself because no one’s else going to see it. But as soon as you get there then they might just think that, so what tools are out there are in the Rust ecosystem that can help us identify things that would need to be reworked or things that we’re missing to make our code look how others expect it to look?
Brenden Matthews 00:10:31 Yeah, that’s a good question. So nice thing about Rust is that Rust makes this pretty easy and so for Rust there are two tools that are fairly critical if you want to write idiomatic Rust, one is Rust Format and so that strictly handles formatting of code like white space and all that. Basically deciding like where to put each character within the source file. And then there’s another tool, Clippy and Clippy, there’s a lesson about formatting and is more about, I’d say it’s sort of like halfway between design patterns and code formatting.
Gavin Henry 00:11:11 Okay. And is Rust format the same as cargo format? Because I’ve used that.
Brenden Matthews 00:11:15 So yeah, when you type cargo format that’s running Rust format, which is like the name of the formatting tool.
Gavin Henry 00:11:22 And it’s FMT you type isn’t it? I think.
Brenden Matthews 00:11:24 Yeah so the Rust format, like the binary name is Rust, R-U-S-T-F-M-T, just all one word.
Gavin Henry 00:11:32 Cool. I’ve used Clippy a lot myself as part of IDEs or on the command line and one thing I noticed the other day is it even helps you identify new ways that things have done because I’d either referenced something, read something or took a suggestion from Copilot or whatever it was and it’d give me an old way to do Rust and then Clippy picked up and said no you don’t need to do define a future this way anymore. You just put async in front of it. So it must have been pretty old.
Brenden Matthews 00:11:59 Yeah, Clippy is pretty useful. I think it can be annoying for example, if you took Clippy and you ran it with like an older Rust code base that hadn’t been updated in a while, then it’s going to generate a bunch of warnings and that might be annoying. It’s a bit like C or C++ compiler warnings and I think not so much anymore, but I remember, back in the day, maybe like 20 years ago, it wasn’t unusual at least in an open-source world to have these projects where when you compile them the compiler would just spit out like hundreds, maybe even thousands of warnings. And I think it was pretty common for people to just like some people used to just kind of ignore all those warnings. And then I think more recently the culture kind of changed where people were like, well if compiler’s making this warning maybe I should address it.
Brenden Matthews 00:12:45 Even if the code works and compiles, maybe we should just fix that warning. And so at least anecdotally what I’ve noticed is that most projects, at least in like C and C++ world don’t have like tons of warnings and so Clippy is sort of like that compiler warnings except it’s like at least like C and C++ compiler warnings generally relate more to like the machine’s interpretation of the code. Whereas Clippy is somewhere on the edge of like some aspects of it or like have to do with how the code actually affects the end result, the machine interpretation of it and some aspects relate to just more like stylistic and structural aspects of like how you’re writing your code.
Gavin Henry 00:13:31 So to close off our introduction, both Rust format or running on the command line cargo space FMT format can help enforce things and Clippy you can reach for which is cargo space Clippy as well I think?
Brenden Matthews 00:13:46 Yes, just cargo Clippy.
Gavin Henry 00:13:48 And that will help identify at least get you on the path of getting into idiomatic Rust and community specs.
Brenden Matthews 00:13:55 I think if you use these tools they get you like 90 plus percent of the way there and it’s actually really easy to use these tools nowadays because if you set up like your IDE VS code or whatever and you use the Rust analyzer extension, then I think for the most part it just turns these on by default for you and then you can dial it up if you want and make it basically like increase the level of Clippy and Rust format enforcement by tweaking their configs. Like for example, if you’re using something like VS code generally I think by default with the Rust analyzer extension when you save your code and you have like the format and unsave where you have like the compiler stuff set up, then I think the command it runs by default to check your code is cargo check. But you can go into the settings and just change that from cargo check to cargo Clippy and now instead of just running the compiler normally on your code to check if it compiles, it’ll actually run Clippy which will not only check if it compiles but also check all your compliance with like Clippy’s set of rules.
Brenden Matthews 00:15:03 And so now in your ID you have not only like the compiler errors and whatnot but also the Clippy output.
Gavin Henry 00:15:10 That’s the first thing that I do on JetBrains, Rust Rover, and a new one I switched to called ZED but because I’m in the FFI Foreign Function Interface between C and Rust, I have to switch off a lot of the kit Clippy stuff because it’s mainly for Rust whereas bring in some C stuff it does a lot of lowercase things that Clippy goes no you shouldn’t be doing that so you’ve got to switch off.
Brenden Matthews 00:15:33 Yeah, generally I think for most projects maybe early on you can start using Clippy and Rust format and you just kind of stick with the defaults but in practice it’ll probably reach a point where you have to go in and tweak the configuration either to like, on this file, ignore this check or whatever or just, adjust it to whatever your preferences are. But certainly for like a pure Rust project where you’re not doing anything too unusual I would say then the defaults should be pretty good.
Gavin Henry 00:16:08 I’d like to now move us on to our sort of main section spend the next 15 minutes or so talking about the main items in your book. I would say generics and traits because you state they’re the main building blocks for idiomatic Rust, so shall we explore both?
Brenden Matthews 00:16:26 Sure.
Gavin Henry 00:16:27 So I’ll start with generics. What are generics?
Brenden Matthews 00:16:30 Good question. So without trying to like look it up on Wikipedia.
Gavin Henry 00:16:35 Yeah just a one-liner because I’ve got about seven bullet points per generics and traits. So a one-liner will be good.
Brenden Matthews 00:16:42 First of all, generics are not unique or new to Rust, it’s an idea that it’s exists around for a while but the general concept is that you have some kind of structure and you want to be able to build a structure in a generic sense such that that structure can operate on different kinds of underlying data. So you can create an instance instantiate an instance of your structure and you can put, you can have an integer as its internal data or you can have a string or you can have some other structure inside that and these generic structures can then be applied, you can reuse the same structure for a bunch of different kinds of data. So like the classic example of this I suppose in Rust would be like you have a vector which is essentially just like a list of things and or an array and you might want to store a string or an integer or some other structure within your vector and that vector is generic in the sense that it can hold any different kind of thing provided all of those items within that sequence are of the same type.
Brenden Matthews 00:17:54 Generics generally only matter in languages that are statically typed because you have to know what the types are ahead of time I would say.
Gavin Henry 00:18:04 How does Rust provide them? Is it because of its strong types or
Brenden Matthews 00:18:09 Yeah, so Rust is very strict in terms of its typing discipline. So generally speaking all types need to be known at compile time that’s totally different from any sort of dynamically typed language where you don’t have to think about types when you’re writing the code, and the environment will figure out types at runtime. Problem with dynamic typing is of course like things can blow up at runtime if you don’t handle all the cases that you need to handle when your code is executing and gets data that is not what you expect and then if you’re doing any sort of static typing you have to handle all of the cases that compile time. The advantage of that is that you don’t have type errors at runtime. The disadvantage is that you have like more work or higher cognitive load at compile time when you’re writing the code in order to think about different types and so on.
Gavin Henry 00:19:10 You gave us an example where a vector holds all the same type on a structure or generic, we might want to hold different things. Would that be a good case of when we know we should reach for generics and we’ve got this model that sort of different types of data, is that correct?
Brenden Matthews 00:19:28 Yeah, so I think generally the default use case for generics is that you want to make something that some structure that holds other things and you don’t necessarily know what the other things are that you want to hold, right? This is a little different like if you think about like kind of two main types of structures, right? One is where you already know all the things that are in the structure, like it’s going to hold this count and it’s going to hold this string which is a name of a thing and it has this other thing and so on, right? That’s sort of like the not generic structure, it’s just, it’s always this well-known thing and it’s like a fixed shape. The other kind of structure is something where you’re defining, you’re like architecting a data structure that has certain properties but you don’t know what is going to be inside that data structure or maybe you do know but you want to have it hold something hold a string in some cases and hold an integer in other cases in which case you would want to use generics or you may just want to build a structure and use generics so that perhaps in the future you can apply that structure to like other problems or other data types.
Gavin Henry 00:20:40 Well with the pros and cons of starting out with a generic structure so you could take a string and do something if it’s a string or do something different with an integer versus just going for a normal structure.
Brenden Matthews 00:20:53 At a basic level the advantage using generics is again like you can increase code reuse. So rather than having to write a vector for strings and a vector for integers and a vector for whatever, you just have one vector and you can make it generic for most types that satisfy whatever the requirements are to be in that structure. So that’s nice. But the main trade off, like the reason why generics might kind of suck in some cases is that well there are two problems, they introduce more complexity generally when it comes to writing code. Like it’s harder to write structures for generic data types for any kind of complicated cases, right? And when you start to introduce generics you end up having to as start adding in more and more other abstractions to sort of work around the fact that you’ve added this new layer of complexity.
Brenden Matthews 00:21:48 So that’s a disadvantage. Another disadvantage that generics is just that in a language like Rust, it’s very very hard to do certain things generically in practice, right? Like yes you can make a structure that holds both string and in strings and integers but there are certain operations that just can’t be done generically. For example, you can sum two integer but like you can’t really sum two strings. It’s just, I mean you could define your own definition for what it means to sum two strings but like strictly speaking there’s no such thing as summing strings, right? Like they’re not numbers, it can’t be added to each other. And so if you wanted to have a structure and like it had a like a summing operation in it, you’re going to have to handle those special cases somehow in your code. So like even when you do these generic things, you might often end up having to actually like add special cases where they’re strictly not generic.
Brenden Matthews 00:22:39 So like, you have a structure and if there’s an integer in there then everything’s fine. But then in the special case where like oh I have a string now I have to do something different. And so it’s a little funny in that sense because you strictly speaking you’re adding a lot of more complexity by using generics and so you have to balance that trade off. Do you want it to be, do you want to have this like additional flexibility with the downside of having more complexity or do you want to just have like a very basic structure and just make sure everything is known without any generics and then when you don’t use generics a lot of things do become much easier generally when it comes to writing the code, which is something you would see very quickly if you use Rust, I guess.
Gavin Henry 00:23:24 Does Rust figure out what the types are going to be of the generic structure at compile time or does it do some of it at runtime or is it?
Brenden Matthews 00:23:32 Well generally yeah it’s all going to be happening at compile time but like the Rust language itself doesn’t do any of that at runtime for the most part with a couple of minor exceptions. But if you wanted though, you could introduce like your own layer of dynamic typing sort of at runtime within your own code. They’re also like crates that effectively do this and the way that you do that is that you have generic containers that wrap some other type of data and so you, you introduce like a new layer of abstraction or interaction in between your data type.
Gavin Henry 00:24:08 Yeah, no I didn’t want to do it, I was just interested where, it looks everything. How do generics compare to other languages — the Rust version — compare to others and then I’ll move us on to traits because we’ve got quite a bit still to go over.
Brenden Matthews 00:24:22 So I would say Rust generics are very similar to what you would find in say C++ templates, but they’re a little different overall from anything else. Mainly because unlike a language like C++, C++ has classes and Rust doesn’t have a concept of classes, it instead uses traits which talk about. But, off the top of my head I can’t really think of like languages that are really super similar to Rust in terms of how it handles generics and C++ is fairly strict, relatively strict and it’s like templates is what would be the equivalent in C++. And then languages like Java for example have generics but Java’s generics I think are like a little more flexible than like a C++ and they’re Java’s a bit easier to use generally.
Gavin Henry 00:25:15 Which do you think influenced Rust the most out of what you’ve seen in your life?
Brenden Matthews 00:25:19 I think personally at least with regard to the generics in Rust, I would say it’s a combination of like multiple languages. I think C++ is a big influence.
Gavin Henry 00:25:29 We’ll come back to that we’ll say C++. So the other big part you said there that Rust doesn’t have classes, it has traits. So what are traits or what is a trait and how does Rust provide them with a nice example so we can understand that would perfect.
Brenden Matthews 00:25:44 So traits are a way of describing behaviors or features that apply to different types of data, right? So a trait itself is strictly a definition of some behavior and when I say behavior I mean like generally a trait, it’s like some method or a set of methods that apply to some type. So a trait itself is not an implementation of anything, it’s just like an interface or description. In fact interface is the word used for traits in some other languages like Scala for example. And so your interface just describes how you can interact with the data type and the trait is sort of like a generic useful language that you can use to say this object has these features essentially.
Gavin Henry 00:26:35 Yeah, I understand it as a shared behavior between types is what I always think when I read the word trait and then I think of where you put a derived display or something like that on something. I think that was a trait, I’m not sure, that’s why I’ve got an expert on.
Brenden Matthews 00:26:53 So the thing with traits is that you need to implement them for any given structure or type, right? So if you define your own structure or an enum or something and you want it to have certain behaviors, then you need to implement that trait. So like let’s use an example of a trait that’s like something you would very commonly implement for things in Rust and that would be the Clone trait. Now Clone is generic enough that you don’t actually have to write the implementation, you don’t have to do like import Clone for your structure. You can derive it using a macro and that works because most of the basic types in Rust have these traits implemented, right? So like a string has a Clone implemented already, integers already have a Clone and so on and so forth. So if your structure is made up of all these basic types that already have a trade implemented, then you can just automatically derive a Clone implementation for your type using a macro because all the macro has to do is just call Clone for all of the types contained within your type. And so it’s relatively easy to just automatically derive those very generic traits. But for anything more complicated than that then you’re going to have to like write the actual implementation manually for whatever it is or, if you want to get fancy you can actually just write those derived macros yourself for like if you want to make a crate and define as some trait and make it so other people can just say derive this trait for any type, then you can create your own macro and do that.
Gavin Henry 00:28:38 Thanks. So I’m going to move us on to the patterns part in your book for the last bit of the show and then anti-patterns. But just to summarize traits there and some terminology we introduce for those that aren’t familiar, we implement things on a type, so this is my understanding so pull that apart. The macros are a way to inject some other code above something or in the Rust to say put this bit of Rust code here and the copy trait allows you to deal with the ownership of a variable that is a string or something so you can copy that string to another owner rather than borrow or fetch. And then you said traits are similar to interfaces in other languages or is that all just rubbish?
Brenden Matthews 00:29:23 That’s mostly right except I should point out that in Rust, Clone and Copy mean different things. It’s like a subtle difference there. And usually when people say Copy in Rust, what they mean is Clone. Copy has its own separate nuanced definition which strictly means at least if we’re talking about the copy trait copy strictly means that you can duplicate two objects by making a literal byte copy, right? So an example of like something that you can copy as an integer, right? You have one integer, you want to make another copy of it, you can literally just copy the byte straight over but that doesn’t work for a lot of objects because internally their representation might include pointers and you can’t just copy a pointer necessarily. I mean unless that’s your intent but usually what you have to do is you have to copy the actual contents of what that pointer is pointing to and then move it over. And so you can’t do a direct byte copy and so Rust disambiguate that by using the word Clone instead, which implies that you just want to make a new copy of that object with the same contents but it’s not strictly like you can’t just copy the pointer and have it point to the old data. You need to have a new copy of that internal data. You might call it a deep copy I guess.
Gavin Henry 00:30:46 I understand. And could you think of a trait like a base class in that object orientated language? So if you inherit from that base class you’ve got this shared behavior or is that the wrong way to think about it?
Brenden Matthews 00:30:59 Yeah, it’s difficult to draw direct analogies between traits and classes because I think if you try to do that, you’re just confusing yourself.
Gavin Henry 00:31:08 Yeah, just learn the new way.
Brenden Matthews 00:31:09 Yeah, I think it’s just better to say like it’s not the same as classes and just learn traits and forget about classes.
Gavin Henry 00:31:17 Cool. So the next section is, Patterns and then Anti-patterns after that. I’ve pulled out six patterns in your book, but I don’t think we’re going to have time to go over those. So if I list these six and you want to pick your three favorites, but you see more often than not would that work?
Brenden Matthews 00:31:35 Sure.
Gavin Henry 00:31:36 So we’ve got macro pattern, builder pattern, fluent interface, observer, command and new type. I’ve seen you type around, but I’ve not looked into it so if we could do that one that would be good. And then another two that you think would be good for the listeners.
Brenden Matthews 00:31:52 Yeah, so I think probably the ones that you might see most often or maybe another way I’d put it is like patterns that you would get the most benefit toolings that you get the most benefit out of properly understanding is probably, yeah the top two are the builder pattern is very useful and once you get into using that it’s like hard to stop using it. Affluent interfaces are also very useful. You will, if you’re not familiar with them, you haven’t encountered them before, then you’ll very quickly like see those used elsewhere. And again, once you like get into using them then it’s hard not to use them because they’re fun to use and it’s a nice abstraction and I wouldn’t describe macros as like a whole pattern but learning to use some of the basic macro patterns is super valuable because there’s sort of like two levels of writing software. The beginner level is you go and you write, you actually write the code out, like you type it out or you copy it from Chat GPT or whatever, right?
Gavin Henry 00:32:56 Yeah that’s me.
Brenden Matthews 00:32:58 That’s level one. And then level two is that you rather than writing code, you get to the point where like you’re writing code to write the code.
Gavin Henry 00:33:06 Code generators.
Brenden Matthews 00:33:08 Yeah, right. Essentially macros, right? Or rather macros are one way to do that. And in Rust there are two different types of macros. There are declarative macros which is kind of like the, I don’t want to say easy but maybe like less complicated way of doing macros. And then the other type of macro is a procedural macro and in Rust a procedural macro is where you essentially write Rust code that generates Rust code by directly manipulating the term in Rust is, well this is not Rust specifically, but the term is Abstract Syntax Tree, AST and that’s just a representation or like a data structure representation of the syntax of the source code that you can alter programmatically. But you can also just write code that generates source code like strings and essentially spit that out of your macro. You don’t have to use macros but every once in a while you’ll come upon a case where it’s like well it would be great if I could do this without just like copying and pasting a bunch of code or maybe writing a Python script that spits out my code or things that I’ve done in the past and aren’t really the worst thing in the world are for example, like you might have a YAML file that contains a bunch of data and then you write like a python script with like Jinja tube template or something and you like read in the YAML and then feed that into your template that spits out say Rust code or something else.
Brenden Matthews 00:34:30 That’s not really the worst thing in the world but to like a level beyond that is getting into learning using Rust macros directly, which you don’t have to do but if you want to be in like a hundred percent pure Rust world then that would be the,
Gavin Henry 00:34:44 So that’s using Rust to generate Rust code before it gets compiled.
Brenden Matthews 00:34:48 Yeah, that’s the other thing about macros is, they execute before the code is actually fed into the compiler. So it’s a step before a compilation. It’s basically a step after you write the code and before it gets compiled.
Gavin Henry 00:35:01 Yeah. Like a pre-processor in C is that?
Brenden Matthews 00:35:03 It’s similar to pre-processor in C but you can’t really compare it directly to that because C pre-processor are pretty naive I would say, they let you do things that you probably shouldn’t do and they don’t properly handle like typing and so on and so forth.
Gavin Henry 00:35:19 So we would reach for macros when we can’t be bothered typing anymore or we want some extra stuff created.
Brenden Matthews 00:35:26 When you can’t be bothered typing or if you’re like typing out the exact same structure or code or boilerplate over and over again, then you might want to be like well, I could just write a macro for this.
Gavin Henry 00:35:38 And is that normally found in a library that you’d publish to, we mentioned a crate and things before this, which is crates to Io and a way to share code between Rust programs, is that correct?
Brenden Matthews 00:35:52 Yeah, crates are just Rust packages that it could be a library or a program.
Gavin Henry 00:35:58 And would you use macros in a binary project or a library project or a crate or just wherever it makes sense?
Brenden Matthews 00:36:04 Oh well you can use them in all of the above but if you want to write a macro as part of a library you can absolutely do that. So you can create a crate that exports some macro and reuse that elsewhere. That’s a pretty common thing in Rust.
Gavin Henry 00:36:18 And you mentioned fluid in interface pattern would be something that we’d see quite a lot.
Brenden Matthews 00:36:23 Yes. So probably a good example of this in Rust that is essentially part of the core language would be iterators, which, well you can use iteration in multiple ways but one way is to use them using a fluent pattern where you essentially like chain a set of operations together which would be like map and filter and whatever and you can stack these operations and operate on some sequence of items by breaking your operations down into like distinct generally like functions that apply some operation or whatever to the data and then return some result. That would be like pretty common way to interact in Rust that way.
Gavin Henry 00:37:09 So we’re thinking of interface like we spoke about a trait or is it, do we not think about it like that? Is it just an interface to a function?
Brenden Matthews 00:37:16 So no it, it’s not similar. You might use traits within your implementation of that interface but generally speaking like if you’re building a fluid interface it’s going to be made up of methods that you write as part of a structure but the method itself returns something where you can then apply the next operation or there’s another method in that thing that is returned that you can then apply some operation two. That makes sense.
Gavin Henry 00:37:45 Yeah. So is that kind of like functional programming where everything function and you can turn that and act on it or?
Brenden Matthews 00:37:52 So when I was talking about iterators and their fluent interface, that is the way that you use that with iterators that would be a good example like a functional pattern. But fluent interfaces don’t necessarily have to be functional in the sense that with iterators you’re usually passing a function as the argument into, whatever step you’re applying to the items in the sequence or whatever. But your fluent interface doesn’t have to take functions as parameters. It could be fairly non-functional and still be a fluent interface I suppose.
Gavin Henry 00:38:23 That makes sense. I mean I’ve seen that in Rust code, I’ve seen it in other languages where you kind of pass a closure to the function to go and do something once it’s found something.
Brenden Matthews 00:38:35 I should just define that strictly speaking like a fluent interface is just an interface that uses method chaining, right? So you chain multiple method calls together effectively.
Gavin Henry 00:38:45 And the thing that you return from the first method call has to be an object
Brenden Matthews 00:38:50 Generally. Like you might have a method call that’s like terminal in the sense that like if you call this, that ends the chain basically. But generally what the fluid interface returns is some object that has more calls that you can make and you can, chain them together basically.
Gavin Henry 00:39:06 Okay well we’ve only managed to do two out of the six there so I’ll put a link into the book so people can dig in deeper if they like. But there are two common ones that we would see to move you from level one Rust to being elite I suppose. So we’ve got patterns we discussed two of six that was macros and flow interface you spoke about build our observer command and new type but we’ll leave those for now. And you also speak, which I like that you have this in your book which is why I want to talk about it for the last bit of the show, Anti-patterns. So we’ve come along, we’re into Rust, we’ve reached for cargo format, we’ve reached for Clippy, we’re getting all the help that the fantastic Rust ecosystem gives us because it’s vetted and checked to compile time. So we get all these lovely, lovely things but we need to think about things or code smells or stuff that we see around to keep an eye out for. So is that what you define an anti-pattern?
Brenden Matthews 00:40:04 Yeah, so anti-patterns are things you shouldn’t do and I suppose in a perfectly designed language there would be no anti-patterns because you wouldn’t be allowed to do anything you shouldn’t do. But in practice that’s pretty hard.
Gavin Henry 00:40:18 But if the compiler lets you do it then it can’t be bad.
Brenden Matthews 00:40:21 Well yeah most of the time right? Like the best example of this in Rust is of course the unsafe keyword and as the name implies it’s sort of like this escape hatch that lets you get around Rusts strict rules about how you handle memory and so on. Yeah
Gavin Henry 00:40:40 I use that every day.
Brenden Matthews 00:40:41 So generally I would call like unsafe and anti-pattern but it’s, it’s not very black and white, right? There are still no matter what going to be certain cases where you have no choice but to use this thing and some of the Rust purists might say like well I don’t want to use any code that uses unsafe because I don’t want my code to be unsafe. And I’m not saying that if that’s your philosophy that’s fine but I think the word unsafe, I think like when the Rust language designers chose the term unsafe, that was maybe not the best word because you can still do things that are safe but require unsafe code And you can see examples of this just by looking through the Rust standard library where there are certain operations that are totally safe but they still require unsafe code because you just can’t do them in under Rust set of rules without having them happen with an unsafe code?
Gavin Henry 00:41:37 An unsafe marks a block or a function, sorry you used the key word unsafe one word to mark a block of code or a function to say if you see a pointer or anything that is normally banned, I know what I’m doing so maybe it should actually be called I know what I’m doing .
Brenden Matthews 00:41:55 Yeah, something like that. Yeah, you’re basically on the right track. It’s basically like you’re telling the compiler, I want you to temporarily suspend your strict rules, I want you to set the borrow checker or on vacation for a moment while I move this data around or like, move these pointers around or something because I just have to do this to make my code work, right? Like it just has to be done and there’s no other way due to the limitations of computers basically sometimes you just have to do that. A good example of like where you might have to write unsafe code that is generally perfectly safe would be like certain types of system calls for example. Yeah.
Gavin Henry 00:42:35 Or using C libraries.
Brenden Matthews 00:42:37 Yeah C libraries. The reality of computers and software and operating systems is that no matter what you’re going to have to interact with the outside world to do anything useful, right? Do any kind of Io and in order to do those things you’re going to have to make call like do some type of function call or whatever that is strictly speaking unsafe and Rust terms and there’s just no other way to do it, right?
Gavin Henry 00:43:02 Yeah because it can’t reason about the memory or the ownership or anything like that so.
Brenden Matthews 00:43:07 Right, right. Exactly.
Gavin Henry 00:43:09 One that jumped out as a beginner, although I’ve written a lot of Rust code now it’s still level one or level two that surprised me in your book was where you said an anti-patent would be seeing the unwrap () keyword everywhere. Which if you could just define what unwrap does and then why if you see it in a program everywhere you class as bad that’d be great.
Brenden Matthews 00:43:31 Yeah. So calling unwrap any pattern might be controversial because unwrap in a way it’s a bit like unsafe, right? You’re telling the compiler or whatever that like I know better than you, I know that this call is not going to fail so I don’t need to check it. Right? And to explain what unwrap is for anyone that doesn’t know there are certain structures which are designed to like hold something that may or may not exist, right? Like the two best examples of this are result and option which when you write Rust you end up using a lot. So in the case of an option, an option, it’s a structure that holds something that either is or isn’t, I guess it’s just binary, right? So, you might have a string and you either have the string or you don’t. And so if you have a string then you can just get the string from the option.
Brenden Matthews 00:44:25 If you don’t have the string then the option contains a type called none, right? And an option is just an enum and it just contains two possible things. One being sum is what it is like some, right? Like something and then none. And with an option it takes a certain amount of boilerplate to like always handle both cases, right? So in your code, you might want to occasionally just like skip a handling both the cases in particular if for sure that it’s not going to contain none then it doesn’t matter, right? So an example of this might be you have some code where you construct that option and you put something into it immediately and then on the next line of code you don’t actually need to like go through the process of like checking is it none do handle like that’s an error or something like you don’t really need to go through that because you already know that option is always going to contain something because you literally just put something in there on the previous line of code so you could just call unwrap and unwrap just returns what that option or result or whatever contains while like skipping the whole bit about like handling the like error or none case and that’s fine.
Brenden Matthews 00:45:35 But if you start to use that in context where you should be checking if your option or result actually holds the thing that you want, then now your code is going to blow up at runtime. So if you think of like unwrap as this escape hatch that lets you get around having to handle the like the none case or the error case in the case of a result. So none in case of option and error in the case of a result then I would say it’s a little bit lazy, right? Especially if you’re using that where you shouldn’t use it and it’s very easy to do.
Gavin Henry 00:46:10 And I’ll try and think of another example as well and where I’ve used it is say for example I’m trying to load a TLS certificate on a command line tool for example at startup if you just used un Rapid and say to the user that’s loading it, oh I can’t find this, would you like to create one? Or something like that, right? If you’re deploying the binary it just might never start and you’re not handling that situation slickly, you just kind of say, oh well I don’t care if this crashes. But then in the same context you might not care if it crashes because it’s just something you’ve written for yourself you are always running it.
Brenden Matthews 00:46:46 It’s all super contextual. You got to ask yourself like does it matter? But I do make one suggestion in the book, which is that in cases where you’re using unwrap because you’re being lazy, which is not a bad thing necessarily, my suggestion is that instead of using unwrap use expect, which is it’s essentially the same thing as unwrap but with one difference which is that you can put a message in. So it’s kind of like using assert for example, right? Like you already are very confident that this thing is what it should be. So on the exceptional case where it’s not what it should be, then now you can just type a little error message. And the nice thing about using Expect is then if your program blows up, then it will at least give you an indication of why it blew up. Provided your message is helpful. So that’s one suggestion. But also if you’re using unwrap without really understanding why you’re using it, if you think about it or even expect for that matter, it’s just like, oh this is just how you get the value out of an option. If that’s how you think about it, then you’re, that’s definitely an anti-pattern. Like you definitely need to understand like what the correct way is to use that. In what context?
Gavin Henry 00:47:56 When I first started I just thought unwrap was how you write Rust and everything had unwrap on it. Every example I ever seen on any website or book it was unwrap and then yeah, I learned more and then I went to expect if it was someone using the program like myself and I was like what just unwrapped and why did this crash? And then oh you expect, right? Because you can say this is why I did this. So I suppose it’s a journey.
Brenden Matthews 00:48:18 Yeah it is.
Gavin Henry 00:48:19 Okay, let’s finish off this section because I can’t believe we’re almost done already. Are there any favorites, anti-patterns of yours that we should watch out for or code smells that you often see in programs that we should make people aware of?
Brenden Matthews 00:48:33 That’s a tough question. I mean I think like unwrap is a good one. Unsafe is another good one. If you see people using those a lot in normal Rust code, then that indicates that whoever’s writing that code either doesn’t really have a good understanding of Rust and how to use it or they’re being like just very lazy about the way they write Rust. Those are good ones. I think beyond those, there aren’t really a lot of great major anti-patterns in Rust I would say because it’s so strict about how you do things. Like it starts to all get I guess very like stylistic and wishy-washy. But an example though I would say is that like some people in the Rust world really don’t like for example, using Clone too often, right?
Gavin Henry 00:49:21 Yeah, I’ve got some notes here and Clone is in there.
Brenden Matthews 00:49:24 Yeah, people sometimes again it’s sort of like the unwrap or unsafe, like people who are new to Rust sometimes will start to use Clone too often because again it gives you a way to sort of get around Rust’s strictness. Like if you’re trying to pass an object around and you want to use references, then handling references means you have to deal with the borrow checker and that makes things a little more tricky in Rust. And so one easy way to get around that is just make a copy, right? Like you Clone your thing like a string that you’re passing around or whatever and that’s not necessarily bad but like Rust purists would frown upon that because now you’re adding this additional like overhead and increasing your memory usage and wasting a bit of time like allocating more memory in and so on.
Gavin Henry 00:50:10 And more importantly it’s not idiomatic is it?
Brenden Matthews 00:50:13 So personally I think like Clone is nitpicking on Clone is kind of a bad example because it’s just so context dependent and its sort of like sliding scale of like how strict do you want to be? Sometimes there are cases where it’s just going to be way easier to just Clone this, just make a copy of it, right? You can take this even farther to getting into the idea of making all data structures immutable. And when you do that, you always make copies of things, right? Because you don’t modify memory. So if you want to change something you have to copy it anyway.
Gavin Henry 00:50:42 Yeah, that’s what they do in Elixir and Erlang.
Brenden Matthews 00:50:44 Right? And so you can’t be super rigid about these because at the end of the day it’s like Rust is just a tool, and you can use it in a lot of different ways and it’s not necessarily right or wrong to do a specific thing. It’s all just very context dependent. And it’s also I think answering whether or not you should do one of these things is hard when you don’t have a lot of experience using these. But over time like the more you use them you start to get a sense of like does it make sense to actually use Clone here? But you just have to practice and write code and you start to learn and get a sense for it. I
Gavin Henry 00:51:16 Think the first thing personally as a beginner is just getting it to compile and shut up all the tools that are really helpful.
Brenden Matthews 00:51:23 Yeah, yeah.
Gavin Henry 00:51:24 Okay. Well I want to sneak in one last question if we’ve got time and then wrap up. So I think I asked you this on our soundcheck. Do you have a good list or a Rust project that we could read or look at on any open-source thing that has a lot of what we’ve discussed in your Go-to one I say go and look at this?
Brenden Matthews 00:51:43 Yeah, well the default answer to this is actually just the Rust Standard Library. And the nice thing about Rust’s built-in documentation tool, which is called Rust Doc, is that when it generates the docs it gives you a link to the source code for everything. So like one of the most common data structures that you would use in Rust whenever you’re using Rust is the vec() — like vector. And so, if you want like very quick introduction to looking at idiomatic Rust code, then I would just like Google the Rust vec() docs and open them up and just start clicking on the little source link on the different methods and see what’s going on inside the Rust code.
Gavin Henry 00:52:25 Excellent. I’ll put the link to that in the show notes. Okay. I’m really glad we did this show and I hope it helps others understand what idiomatic Rust means. I certainly learned a lot. Was there anything we missed that you’d like to mention before we finish off?
Brenden Matthews 00:52:38 No, not really. I hope people enjoy using Rust.
Gavin Henry 00:52:40 We’ll link to your book so everything that we didn’t get a chance to speak about is covered in your brilliant book. And if people want to get in touch or reach out to ask you something, what’s the best way to do that
Brenden Matthews 00:52:51 Email or, I mean if you Google my name with GitHub then you can find my GitHub account and, I don’t know, email is probably the best way, generally.
Gavin Henry 00:53:01 Brendan, thank you for coming on the show. It’s been a real pleasure. This is Gavin Henry for Software Engineering Radio. Thank you for listening.
[End of Audio]
