Tim McNamara, author of Rust in Action, an introduction Rust for programmers who have never used a systems programming language, discusses the top three benefits of Rust and why they make it a performant, reliable and productive programming language. Host Gavin Henry spoke with McNamara about its rich type system, ownership models, memory safety, thread safety, concurrency, UnSafe Rust, build tools, compilers, build times, running Rust with no OS, Rust 2021 edition, Rusts’ future, package managers, managing memory, memory leaks, runtimes, garbage collection, closures, version migrations, static analysis, what’s changed since our 2017 show on Rust, when to use Rust and how Rust compares to C, C++ and Go.
This episode sponsored by CACI.
Show Notes
- Episode 279: Florian Gilcher on Rust – https://www.se-radio.net/2017/01/se-radio-episode-279-florian-gilcher-on-rust/
- Securing UnSafe Rust Programs with XRust (Computer Society Digital Library)
- Rust 2021 Edition – https://doc.rust-lang.org/edition-guide/rust-2021/index.html
- Sahana Foundation
- Benchmarks Game – Which programming language is fastest? | Computer Language Benchmarks Game
- TechEmpower Benchmarks – TechEmpower Framework Benchmarks (7mil req/s)
- Episode sponsored by CACI: https://careers.caci.com/innovate
Transcript
Transcript brought to you by IEEE Software magazine.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Gavin Henry 00:00:56 Welcome to Software Engineering Radio. I’m your host Gavin Henry, and today my guest is Tim McNamara. Tim learned to program, so he could assist with humanitarian disasters around the world from New Zealand. His area of professional expertise is in natural language processing and machine learning. Tim also is the author of Rust In Action, an introduction to Rust for programmers who have never used a systems programming language before. He currently works at TCDI, an American legal services and cybersecurity company, as a Senior Software Architect. Tim, welcome to Software Engineering Radio. Is there anything I missed in your bio that you’d like to add?
Tim McNamara 00:01:34 No, that covers it, thanks very much.
Gavin Henry 00:01:35 So, the goals for today’s episode on Rust 2021 Edition, is to revisit why and when Rust was created, because we did a show — Episode 279 with Florian Gilcher — on Rust back in 2017. So, that was four years ago. So, we’re also going to learn to understand when to use Rust, its top three benefits to explore, what’s changed since our 2017 show, and to understand what an edition is and what the 2021 edition brings. So, first question is the Rust website headline is, “A language empowering everyone to build reliable and efficient software.” What does that mean?
Tim McNamara 00:02:16 That’s an excellent place to start. The Rust programming language is explicitly — or at least the community that surrounds the Rust programming language — is explicitly there to be able to support anyone achieve their goals with software by the safe and reliable aspect is probably the easiest one to mentally grasp. And that would be to say that, all things being equal, your application should use the least amount of memory and the least amount of compute resources in order to achieve its aims. So, effectively, Rust is attempting to eliminate waste in both in terms of compute resources, but also human resources. We shouldn’t necessarily need to have security vulnerabilities or random crashes. Our systems that we build the software systems should actually be stable and robust. And that is sort of the technical side of Rust. Now the empowering everything is kind of thereís value laid in the statement.
Tim McNamara 00:03:17 That’s intentionally there as a bit of a jibe, I think at other language communities or other open-source communities at saying, well, actually everyone is welcome. And you shouldn’t feel that if you’ve only done JavaScript or you’ve only done Python, or you’ve only programmed in Java, that you are not entitled to learn about low-level high-performance software, you definitely have a place in the room. We’re trying to attract a very wide spectrum of developers so that the whole ecosystem supports a better software industry. A software industry that’s robust, a software industry ‘s that respects its users, a software industry, that respects program is as a proficient.
Gavin Henry 00:04:04 Well, I think you’ve touched upon a few of my follow-up questions. So, that was good. Can you remind us why and when Rust was created for those that haven’t listened to the show in 2017?
Tim McNamara 00:04:13 Right. So, Rust was originally made public by Mozilla. So, the organization had this problem. One is an existential crisis, which is Chrome came about and they tried to rapidly enhance Firefox in response in particular, the performance of this JavaScript engine. Although the details are probably somewhere in between, but there turns out that C++ is extremely difficult to actually get working in a very large team. And there were concurrency bugs. They might be this memory management kind of problems about a classic example is called a double freight where you might say that memory has been freed and then some other part of the program actually goes and tries to free it again, which is illegal behavior. Another case that Rust completely eliminates by design is iterate and validation. So, if you change the array while you iterate through it, let’s say you’re adding or removing items.
Tim McNamara 00:05:15 This is a source of really bad bugs and Rust actually ensures that that illegal or, you know, we might have undefined behavior is completely impossible. We have a whole classes of bugs, which just don’t exist in Rust. And therefore why Rust was created was to create a bit of C++. It was there to be able to enable very large systems to be built with it. And that involved creating a productive and relatively ergonomic language that people who were not expert developers or would never see themselves as sort of expert system programmers, would still be able to produce reasonable code and possibly moreover, they wouldn’t be able to accidentally insert massive security vulnerabilities inside a very important piece of software.
Gavin Henry 00:06:11 To these checks and optimizations, well, not optimizations, but less, you know, about all these things when it’s running or before it gets to the point of being, I presume it creates a binary, does it?
Tim McNamara 00:06:24 Right, so Rust is a compiled language and therefore the compiler has an opportunity to check before any program has actually run and turns out that almost all of Rustís safety checks are actually checked at this compile stage. There’s actually no runtime cost at all to any of these safety features. What Rust does is kind of tag data with a lifetime just to measure, we didn’t have any conditionals in our program. We only had a very linear way through, and you would know that let’s say variable A, could last from say step 10 through to step 50. And then for whatever reason after step 50, accessing their variable A was illegal. Rust does a similar kind of analysis across the entire program, across every thread. And it does it all at compile time. There are one or two cases in which you will incur a runtime cost.
Tim McNamara 00:07:24 So, for example, if I try to access the ninth element of an array or so if I look up an array by, by index, there will be sort of what we call a bounce check to say that the ninth element actually exists. It is slightly possible to get around that with higher order programming and in particular, the functional paradigm, and you can mutate or perform transformations on some data structure with closures, which are anonymous functions. And they are typically very heavily optimized by the compiler and idiomatic Rust will often not require balance checking at all because it can guarantee that it will only access safe and valid memory,
Gavin Henry 00:08:11 So thereís certain ways to get round that, optimization and safety?
Tim McNamara 00:08:15 There are a couple of different styles of programming. Say an imperative style with follow-ups and accessing with an I. So, you can imagine that you’ve created your, your index variable I, and your sort of iterating from zero through 13, Rust, doesn’t require you to manually create an iterator like this index variable, that mode is available to you. And there’s sort of this mode of being very imperative and it works, but it’s sort of, it’s going against the grain slightly. And the sort of medic Rust approach will typically be faster. It’s certainly more memory efficient because it require the use of a temporary variables.
Gavin Henry 00:08:57 Weíve covered why and when Rust was created. Briefly, when should a programmer use Rust?
Tim McNamara 00:09:02 I don’t want to speak for every programmer, but I can talk about when I use Rust. So, before Rust, I basically used Python for about a decade, and that was my programming experience. And I used Rust, why I leaned towards Rust when things were too slow, is when I turned to Rust. But actually now that I am productive with it, it’s a surprisingly versatile tool. I tend to kind of use it for command line utilities and all the way through to kind of network services. It’s quite heavily used on the back end of the web site. So, for developing web APIs is quite a common use case, as well as this ability for Rust to compile to the front end via a technology called web assembly. Rust also has a special place to play inside end beated programming. It doesn’t require an operating system or an interpreter to run. And so it can actually be compiled to microcontrollers as well. It can fit in a very wide range of areas. And where I found it particularly useful is data processing, high volumes of terabytes and more of mostly text data, but it does it very quickly and it uses very little memory.
Gavin Henry 00:10:29 And what sort of companies, people, organizations are using it.
Tim McNamara 00:10:34 Rust has been adopted by all of the marquee tech companies. The Rust foundation, I think, is sponsored by at least Microsoft, Google, Facebook, Huawei of the Chinese tech company. Mozilla is heavily involved. The best milestone would be 2015 as when the 1.0 released there was quite a lot of work prior to that. So, I think the first public announcement at all was around 2008 and, or at least that may have been when the original developer Graydon Hoare actually started working on it. There was a long period where the language itself was effectively considered unstable where Rust wanted to be able to, if you think of its original use case, developing an alternative web browser, it was actually being developed in lock step with this big software project. And they were molding the language to make it more and more productive. So, there was a very, very long process of developing language features or roughing them out or inserting them pre-stabilization. The milestone that matters is 2015.
Gavin Henry 00:11:56 So what I was getting thought about, sort of three years after Go was coming around. I think.
Tim McNamara 00:12:02 When they were both announced, I think Rust may have been publicly announced around 2012, Rust and Go look very similar. So, they both had something that looked where it can go routines also known as core routines or lightweight threads. And the ability to kind of spawn work was baked into Rust, but there was actually pulled away, pulled out of the language as it stabilized. And it became a language that became much more aligned to its peer languages, predominantly C++. And it took on a mantra of zero cost abstractions. That is that any given abstraction, you should have little to no runtime cost at all and preferably no memory even. So, there are quite a few things that are available to you or they look like types inside the Rust source code. Then the compiler will actually completely remove inside the binary. They purely exist as the ability for you to mold your program as you’d like it, but they don’t actually impose any runtime cost and they don’t have any runtime memory
Gavin Henry 00:13:16 Who is using that? I was doing some GitHub action, workflow exploring last week. I noticed all of their containers that you can use in GitHub actions have Rust pre-installed. So, I thought that was a very good sign, but obviously GitHub is owned by Microsoft. So, I’d probably explain some of that. So, does Rust have a package manager?
Tim McNamara 00:13:35 Rust has an excellent package manager. Cargo is its name and Cargo is one of the most fantastic pieces of software that I’ve ever used. And I say that as someone who was always terrified whenever I was told that I needed to compile a native extension for some present package, because you would need a C compiler and then you would always encounter kind of weird linker issues and I had no idea what I was doing. Rust almost completely eliminates that whole problem. And Cargo just works. It’s ridiculous how good it is. It’s almost worth using Rust just for Cargo because it avoided you so many problems.
Gavin Henry 00:14:24 From the statement isnít it?
Tim McNamara 00:14:25 It’s surprisingly valid. I mean, coming from Python where there’s like 16 different ways, I exaggerate, but of the setup tools is just utilized. There’s a whole bunch of other new ones. Is it poetry? Is it PIP? Is it, you know, if you’re in scientific programming as a Condor and then all of that is eliminated in Rust and it feels fresh. I don’t want to evangelize too much, but also just, I keep looking for the disillusionment with the Rust, every single tech thing that I get excited about after, nine months say something will just hit me in a recipe, terrible and look for the next thing. But with Rust, Rust just keeps getting better and better and better. And yeah, I’m just wrapped at the ability to be part of that story.
Gavin Henry 00:15:13 Well, that’s really good to hear. So, now that we’ve had a good introduction to Rust, let’s start exploring some of the top three benefits. On the Rust website it says that Rust task performance. So, let’s talk about that in this section, how do we measure blazingly fast, which is one of the claims?
Tim McNamara 00:15:31 That is one of the claims. So, the claim is that Rust run blazingly fast. And that statement is kind of paired up with the, with the notion of a garbage collector and a garbage collector as a piece of software that runs alongside your application. If you’re running a Java app or anything on the CLR. So, if you’re in the .net ecosystem, or if you are a Python programmer or JavaScript programmer or a Go programmer, you’ll have a little utility that’s kind of bolted onto your application. That’s managing memory and requests from memory from the operating system on your behalf. It takes time to run. And in some cases, in fact, many cases, it will actually halt the execution of your program so that it could do its own work. And then very quickly swap that’ll turn itself off and give your program access to the CPU. Rust does not need a garbage collector. Every single allocation of memory is actually defined at compile time. But unlike say CNC++, you never have to manually request it. The compiler understands exactly when every single element of memory is required, that we will only access exactly what is being run.
Gavin Henry 00:16:57 We touched it initially in the introduction about most things being done at compile time, where all the safety checks and memory and tracking and looking at how things going to go. So, let’s just take an example where we’ve got a data structure that comes in over the network, but we don’t know what’s going to be in it the size wise. How do you deal with the memory and efficiency in predictive performance when you’ve got unknowns from users. How does Rust do that better?
Tim McNamara 00:17:27 Rust, will give you a couple of options. It provides you with the ability to say manually that I only want say four megabytes of RAM available to this incoming strength or this incoming stream of bytes. It provides you with the ability perhaps to say that you can just process them on the fly as they come in. Otherwise, it also gives you kind of tools to be able to say that you can use as much memory as needed. And so the language itself is going to try to provide you with the tools for you to kind of make the decision as the programmer is going to try to not impose its own will on you, in terms of that particular use case. Does that make it easy?
Gavin Henry 00:18:19 Yeah, I think so. The way Rust has been explained to make the best choices for all the stuff so you don’t have to worry about memory management and wait for the garbage collector. And that it always getting better every year, but it sounds as if you do know what you’re doing, you can help Rust make a decision, so it makes sense.
Tim McNamara 00:18:42 Right, I think it does. Let’s say that I wanted to accept an HTTP post request. So, if someone is submitting a form and we don’t know how long the format, but we want to guard the server that is hosting the application from being overrun with data, the thing that’s most likely to happen is that you will read in only the amount that is required. So, it might be one form element. And then that will be processed. The web framework itself will probably have a guard in the a, which is at once I’ve hit say two megabytes or whichever limit that you’ve provided that actually the client will be cut off. The framework were two in like the too large response in terms of, say, TCP or sort of raw TCP. The similar thing might happen.
Gavin Henry 00:19:34 Try to understand if there’s no runtime, no garbage collection in the build process, the complacent, how does it know to handle a variable memory requirement possible? I’m trying to drill down to I’m thinking the compilation style is like the stages — running scenarios and protections — but it’s not as, as looking at all the different flows through the program or?
Tim McNamara 00:20:01 That’s right. Yes. So, one of the things that kind of hits you in the face when you try to learn Rust is that suddenly you read something and jargon starts getting thrown at you. Some of the jargon that we talk about is the stack and the heap. Now the stack and the heap are actually not data structures in the classical sense. You never see them in your program, but the way that a systems programming language like Rust deals with an unknown number of bytes is through a process called dynamic memory allocation. Rust will request more memory from the operating system as it’s required. That’s why it’s called dynamic, but there’s a problem. And that is, it doesn’t know where to put it at compile time. So, instead Rust will allocate the size of a pointer, which has a memory address. So, it’s just the number, it’s another integer.
Tim McNamara 00:20:57 And that integer along with a length, which has another number. So, those are two things of fixed size will point to some other area of memory that will be allocated by the operating system at runtime. We encounter a couple of bits and pieces of jargon. And one of them is unfortunately named term of an unsized type in Rust. An unsized type is a type that doesn’t have a sort of a complete known length, like a string. We don’t actually know how a string from some unknown source. We don’t know how long it will be, but some parts of it are known. And normally the parts that are, will be a pointer, a length. And in the case of a string, we also were taking the capacity, which has how many bytes we’ve actually allocated with sort of, there’s a backing array as well that we point to. Now the cool thing I think about Rust is that you don’t actually need to know about pointers in order to make use of the language as your skills increase, you can kind of dig in and learn about them. It’s not forced upon you. That’s I think one of the reasons why the Rust community is kind of chosen to say, we are here to empower everyone.
Gavin Henry 00:22:18 It sounds like most of the effort has done a compile time to figure out what you’ve told Rust that you might need, but it also will figure out what it needs. So, does that mean bill time takes longer than it works with another language?
Tim McNamara 00:22:32 Rust compile times are relatively slower. So, it’s in some of its pure languages. The Rust compiler is doing more work. It is tracking the use of all of these variables, it turns out also that was some of the implementation details about how the compilation steps work mean that Rust’s compilation model is slower, but not for the reasons around dynamic, not for the memory allocation thing.
Gavin Henry 00:23:02 When we talk about blazingly fast, that obviously means runtime. Cause we’ve talked about the build time, where would you see blazingly fast set in with the rest of the language ecosystem, if rosters pitched as a C++ replacement and go as pitched as a C replacement, does Rust fit in somewhere between the performance starts of CNC++? Or how do we put some numbers against blazingly fast is what I’m trying to get to?
Tim McNamara 00:23:32 Yeah, sure. And that’s a perfectly valid question. The Rust community has put this outrageous claim out. So, it’s time for, for some evidence. So, I’m going to make a, another possibly outrageous claim. And that would be that I would find it easier to write faster code in Rust than say C or C++. And that is because a lot of the abstractions are really ergonomic in a very difficult to misuse. Whereas I think I’d be anxious about my own capability. If I were to try and reimplement something in a language where I would have much more manual control. Rust provides you with the equal level of manual control if you need it, but it’s there for you. So, it’s still available, whereas CNC++ in many instances, it’s required that you actually do that. So, where does Rusts sit? I’m thinking of two main benchmarks.
Tim McNamara 00:24:23 There is a site that used to at least been maintained by the DBN project called the Benchmarks Game. Now this is a set of a series of benchmarks, which consistently demonstrates that Rust is on par with the highest level of performance available. So, Rust is as fast as the fastest languages, but one of the reasons why it’s named Rust is that you can’t get closer to the middle. That at least that’s one of the slogans. The other thing in terms of benchmarks, there is a web benchmark. One of the Rust frameworks is able to serve, I think, concurrently 7 million requests per unit. And I can’t remember if the unit is second or minute, I believe it’s actually second, which strikes me as a ridiculous number of requests.
Gavin Henry 00:25:16 Yeah. I need a big pot of bone necks to serve that.
Tim McNamara 00:25:20 Yeah. You start saturating the neck. Yeah, so Rust is consistently good. One of the other… — this is much more anecdotal than those kinds of two benchmark sites, but someone who I respect a lot, his name’s Brian Cantrell, he has recently started with a couple of co-founders, the oxide computer company, and became notable as the creator of the D trace project for Solaris. And he has quite a fascinating blog post where he says that, like, “I’ve been a C programmer for decades, but my Rust is faster.” And that’s because it’s just much easier to write fast code in Rust.
Gavin Henry 00:26:02 That makes sense. Okay. So, one of the top things Rust lists as a major win is reliability. So, let’s take some time to explore that. What is a rich type system?
Tim McNamara 00:26:14 A rich type system is a type system that enables you to express your intent really well.
Gavin Henry 00:26:22 But it helps with reliability because it can optimize for that specific type or make it more predictable or…?
Tim McNamara 00:26:29 So, Rust does not have exceptions it’s place. It has this notion of a result type. A result is sort of a placeholder value that can either be a successful return value or an error case. So, we call this the okay or error. Now, if I return result from my function signature. So, if I declared my function as having result, then every coder that uses my function must take account of the fact that it could fail. It’s impossible to compile a Rust program that hasn’t included checking of error cases. It’s simply not possible to kind of skip over an error code.
Gavin Henry 00:27:16 That would apply to memory safety, threats safety, concurrency, anything?
Tim McNamara 00:27:21 Absolutely. The other time that I find particularly cunning really is the option. So, Rust in codes, missing values with this type that we’ll call option. And I think Pascal calls it optional. So Rust value could be present. We say, it’s, there’s some value or it could be missing in which we say, there’s none there. Now this looks like each of those two cases look as though they would take up memory, but it turns out that the none case is completely invisible in the final binary. It uses some really neat compile magic to kind of prove to itself that it actually never needs to represent none. And the actual binary itself because of the fact that if something is present, then it will never be known. It turns out that this is a genuine zero cost abstraction. We have complete type safety from the point of view that if something takes an argument option, it’s impossible to work with that function and not take account of the possibility that a value may not be present.
Gavin Henry 00:28:33 Does that touch on the ownership model guarantee?
Tim McNamara 00:28:36 Similar but different? Again, so ownership is another concept that gives you some, some of this reliability and robustness, but it works in a slightly different way. The optional case though, is really just there to say that it’s impossible to treat a function and sort of silently hope that a value will be present like a none or some sort of middle value, or null, depending on your preference, sort of turns up, you’re not going to suddenly crash. Or if I carry on with some kind of erroneous value, you raise a very good point though. There’s this kind of notion of an ownership or like an owner that we haven’t really discussed in this conversation.
Gavin Henry 00:29:19 If ownership is tracked, then you know who owns that bit of memory and you know who owns that data that gets shared between threads or things like that.
Tim McNamara 00:29:26 Absolutely. The first thing to say about ownership, I think it’s poorly named. So, ownership in a Western mindset implies that you are able to prevent trespassers from accessing things that you own. That is what ownership means. It’s got the ability to exclude others from access. And it turns out that in Rust, the owner does not have the ability to restrict other parts of the program. Like no one comes and checks for permission for the owner. That’s impossible for the owner to ever decline a request for accessing the data that has owned. The owner only has one responsibility in Rust, and that is to free the, or to delete the value that is owned at the end of its scope. If I own some integer at the end of my lifetime, I’m need to make sure that integer has already been deleted from memory. Turns out that this is all implemented with scopes, and there’s actually no requirement for you as a programmer to kind of call delete. It just happens to be the case that when my lifetime ends that everything that I own is also deleted.
Gavin Henry 00:31:17 We touched a little bit less on the introduction and something I want to ask just now is about the safety and ownership you’ve mentioned there, and the build time length. I’m just doing some C work at the moment. And I think part of what you’ve described in the build process, where it looks at the safety, if you were to take, C from what we’ve just spoken about scope, I’m doing something where it opens a socket and UDP socket, and then depending on whether that’s successful or not, and some other bits of memory you’ve returned from the function and the variables you’ve opened behind the scenes of that variable. So, memories opens go out of scope. So, it’s my job as a C programmer, to understand that that file descriptor take some stuff and leaves it open. And I’ve got to shut that down. Now, if I hadn’t thought about that, which I didnít, and I’ve got this program plugged into some static analysis tools like code QL on get hub climbing on the lives are on the cover T. One that checks for those different things and C, and it was only the cover T tool that picked the up and said, look here online, you know, 57, you’ve opened up some stuff, you’ve checked the return code, but the variables have gone out of scope and you’re leaking memory. So, how I understood what you’ve described just a minute ago and an introduction, it’s all of those third party, things are all scooped together and done at build time thrust.
Tim McNamara 00:32:48 You are correct. So, in terms of just Rust in terms of opening, closing sockets or some resources, Rust will kind of wind things down, but it’s actually dependent on the library itself — that you might have some UDP library that knows about sockets. One thing that Rust doesn’t guarantee is that your program will be free of memory leaks. So, it is potentially perfectly memory safe to just leak memory. If you know that nothing will ever access that region, then Rust will still compile it. If you were to kind of go under its back and kind of reimplement some of the functionality that’s provided in the standard library, for example, and you wanted to be able to rely on the fact that you’re writing a short-lived utility, then you expect that the operating system will just clear everything up when you’re done, because it’s only going to be alive for 10 seconds. Then you can leak memory deliberately. And there isn’t a check there.
Gavin Henry 00:33:50 Memory safety is more about accessing memory that isn’t yours.
Tim McNamara 00:33:55 Exactly. Rust will get very, very particular, very stroppy with you in as polite a way as possible, if you attempt to access, actually I’ll be a little bit more precise: What do I mean by access? Multiple readers is fine, but they can only ever be one place in your program that has write access to any given value and Rust implements that without actually any locking. So, it can guarantee in the most part that exactly when is safe to access values and to modify them without needing to add extra locking mechanisms. Now, if you do wish to have a shared variable that is writers and multiple threads, for example, you will actually be required to place some form of locking or some sort of synchronization mechanism in front of that variable. But in the single thread of case, if we had multiple references to some object, only one of them could ever have right exists.
Gavin Henry 00:35:01 Does it figure that all out, bill time, basically once you’ve got that binary, you know, you’re good.
Tim McNamara 00:35:07 Once you have your binary, you’re good. But wait for the but. Rust provides the ability for you to call what we call unsafe functions with an explicit unsafe block that is clearly very, very visible inside your source code. This is very common. This kind of pattern of, we talked about C just before, if I want to import or sort of link-in, a library that provides some useful functionality, say some compression library, and it requires a pointer and a link to the bytes that it wants to compress, Rust isn’t able to provide any guarantees at all about what the C library does. It doesn’t have any way to kind of introspect that libraries functionality. And so there are some sort of escape hatches to be able to create raw pointers, to do things that are dangerous. You can cause crashes in unsafe Rust, but you need to opt into that.
Gavin Henry 00:36:11 That makes sense. There’s an article I found on the IEEE Computer Society website, which is obviously what runs this podcast called ìSecuring UnSafe Rust Programs with X Rustî. And the quote here ìRust is a promising systems programming language that embraces both high-level memory safety and low-level resource manipulation. However, the dark side of Rust, unSafe Rust, leaves a large security hole as it bypasses the Rust type system in order to support low level operationsî. Recently, several reload many corruption, vulnerable theories have been discovered in Rustís shared libraries. I guess that’s just where people have flicked that switch to use the unSafe Rust and maybe not done it right.
Tim McNamara 00:36:56 Exceptionally difficult to do correctly.
Gavin Henry 00:36:59 And give someone a gun with bullets, you know, or not. You know what I mean?
Tim McNamara 00:37:03 Yeah, right. Going with bullets, here’s a loaded rifle. Please treat it with respect. The other side of Rust for me, just as someone who has written a book for people who have never done systems programming before, is that I often use the unsafe keyword examples with inside that book, because I want to demonstrate how things are built. And it’s quite fun actually to say, oh, look, here is a stream of bytes and they are all one after the other, but actually I want to now treat them as an array of integers. And so we need to actually take a series of four byte. We want to read in four bytes at a time and just treat that in memory as an integer. And actually that’s not magical. Your CPU does that all the time. And I really appreciate that Rust has these escape hatches available for people who do wish to kind of explore. And I’ve personally been really impressed with this idea that that Rust actually kind of supports people’s intellectual curiosity.
Gavin Henry 00:38:12 So, when you’re ready to do that and flick that switch, you can look behind the curtain and see what’s going on?
Tim McNamara 00:38:18 That’s right. You can also annotate your code with these kinds of forbid annotations, to say that I forbid the use of unsafe in this code base. You can actually decorate your source code with squeezy annotations, which actually attributes some of the attributes will refuse to compile your code on this, that complies with certain style guidelines, which is actually surprisingly useful with things relating to that same memory safety.
Gavin Henry 00:38:44 So, productivity for the developer or for the team? What’s the documentation like?
Tim McNamara 00:38:51 Pretty good. And on the whole, I think there are areas where there are spotty. So, that’s particularly in the sort of a third party ecosystem, but I think in the standard library, the documentation is excellent. The language itself as well documented the year is a free online book that you can download. That’s available to you. That details on that end.
Tim McNamara 00:39:19 Books are different, right? So, mine is intentionally different than other material. Mine is about examples and say, okay, so you want to know, let’s implement a CPU emulator, let’s implement a database, let’s implement an NTP client because I want to understand what NTP is and timekeeping is. And so my book is really around giving you the enough foundations of Rust so that you can actually manipulate that and extend it. Whereas the pre-work is you call the Rust programming language is really just all started integers and we’ll walk away through all the way through the language and all of its language features. And then at the end, we will get to an example,
Gavin Henry 00:40:04 What’s a friendly compiler?
Tim McNamara 00:40:06 A friendly compiler is one that provides you with specific errors, that point you at the right place. It gives you a concrete reason why something has broken and it gives you suggestions for fixing it. And most of the time, the suggestions are very, very good. So, Rust is a friendly compiler in the sense that every single error message includes a link to a webpage that has a very detailed explanation for what this error is and why it has caused and how to get out of it. For me, as someone who had never really used a compiled language before Rust, I am indebted, heavily indebted to the compiler team itself.
Gavin Henry 00:40:52 Is compiling the same thing as a build tool is that all in one?
Tim McNamara 00:40:56 Good point, there’s a small ecosystem of tools and it’s a trio of tools that comprise kind of the standard tool chain, sorry to throw more jargon into the conversation. So, we have the Rust compiler. So, this is known as Rust C. It is responsible for taking your source code and taking the source code of all the other dependencies and massaging them into an executable binary or a library. Alongside, there are two pieces of software. One of them we’ve talked about called Cargo. Cargo is responsible for downloading those dependencies, kind of rearranging everything so that it’s easy for Rust C to know where everything is. And it also actually invokes Rust see itself. So, your day-to-day Rust programmer will typically run Cargo build rather than interacting with Rust, so directly. The third member of this trio is called Rust up and Rust up is a tool that manages tool chains.
Tim McNamara 00:41:58 And you might say what’s a tool chain and the tool chain would be whether or not for example, you’ll running the stable Rust compiler, or you might actually opt into the nightly compiler, which has kind of got more features, but actually comes with the possibility of stability problems because things aren’t quite finished yet. It also provides you with the ability to install a Rust compiler for other compile targets. So, you might be running on Windows, but you want to compile for Linux or say your you’ve got Rust on your MacBook, but you actually want to compile for some other target or your little micro-controller, and Rust up manages all of that.
Gavin Henry 00:42:45 Oh, excellent. So, we did a show on Rust, like I said, in 2017, is there a quick way to say what’s changed since then?
Tim McNamara 00:42:52 Since your chat with Florian, there’ve been two Rust additions. A Rust addition as an opportunity for backwards incompatible changes to be included to Rust, notwithstanding that if you opt into a new edition, your old code will actually still continue to compile. And so Rusts actually a very stable language. There were several changes made in 2018.
Gavin Henry 00:43:18 So, would it just be a case of looking up, what changed and applying that to your code?
Tim McNamara 00:43:23 That’s right. The thing to search for, would be Rust 2018 edition. I think there is a, an addition guide as part of the official documentation and the 2018 changes were created out of playing with the language in anger for a couple of years, as an open source, more major open source project. And it’s simplified a lot of things which actually added to the ergonomics. The 2021 edition kind of just ends even just a tad extra polished. It provides us a couple of niceties. For example, the ability for you to have a conversion operation that may fail. So, instead of crashing on you, it will return a result, which we actually talked about a few minutes ago and just a couple of syntactic niceties that aren’t available to previous additions of Rust.
Gavin Henry 00:44:20 And what just a change to use a newer version of Rust? Is that the toolchain or Rest C?
Tim McNamara 00:44:27 To opt into the new version of Rust? Every Rust project managed by Cargo has a little metadata file. All you need is addition equals 2018. That’s sort of like a little mini file or a little conflict file.
Gavin Henry 00:44:43 If you are using quite a lot of libraries from crate to IO. Would it just be a case of doing the usual thing where, if one of those doesn’t work on the 2021, you open a ticket and that project can get up to speed, or should it always work?
Tim McNamara 00:44:57 You can still include third-party dependencies that themselves are using an older edition. So, you shouldn’t actually need to update any dependencies to your compiler version. Today’s compiler, can still compile code that was in the original 2015 edition. So, but if you want to contribute, yes, raise a ticket. And there shouldn’t be anything that could compile that can’t the changes are mostly additive rather than contradictory,
Gavin Henry 00:45:28 How are these changes proposed and discussed?
Tim McNamara 00:45:32 So, the rest community is quite special, and the fact that it has a very participatory RFC or kind of request for comments, process where a change might be suggested and a discussion forum, which has then iterated, you know, say, oh, that sounds like a sensible people have a chat and people might have a discussion and decide that it’s worth investigating. That will then be written up as a proposal. And then there are a couple of teams and depending on which team sort of sponsors the idea, then things can kind of move on from there as well as additions Rust has a six weekly update cycle in terms of its minor version updates. And so if something is not backwards incompatible, or you’re not trying to create something that the old compiler won’t understand, then quite often you can include substantive changes inside one of those six weeks cycles.
Gavin Henry 00:46:27 And if you are trying to keep up to date with that, would you just keep an eye on the announcements and rebuild your binary? Or when do you do that? Do you just leave it running where it’s deployed or do you just build that into your deployment type thing?
Tim McNamara 00:46:41 No, this I think is actually quite an important misconception to kind of squish, if we can. The language does have a minor update every six weeks, but the stability is still there. Your code will still continue to compile and no back was incompatible, changes are made in the newer versions. You kind of get new fancy things, you know, you can opt into that.
Gavin Henry 00:47:07 Okay. What does the future looked like for Rust since the Mozilla foundation creation?
Tim McNamara 00:47:12 My view is that Rustís future is extremely strong. Rust continues to grow exponentially in terms of its adoption. It’s receiving a lot of attention. Yeah. As we said, these marquee companies, but actually what I’m really intrigued by or really hopeful for is that smaller kind of maybe less than known companies might be taking it on for sort of side projects and for kind of exploratory work. And that to me is where I think a lot of the growth will be. People are excited that it has been the developers most loved programming language for the last six years in a row. But if you can actually work with it day to day, then it doesn’t mean a lot. And what I’m hopeful for is that the ecosystem will continue to, to really encourage creativity and really encourage people who are willing or interested in learning.
Gavin Henry 00:48:08 Excellent, thank you. Obviously, Rust is a very powerful language with a strong ecosystem, but if there was one thing a software engineer should remember from our show, what would you like that to be?
Tim McNamara 00:48:18 The one thing that I would like people listening to take away is that they can learn Rust. That they can share and its advantages don’t be put off by the fact that you bet you’ve heard that it’s difficult to learn. There are more resources, there are different resources available. You don’t need to follow the same path as everyone else.
Gavin Henry 00:49:19 Excellent. So, Tim, one thing I didn’t ask and the follow-up question was how does Rust run without an operating system? You mentioned that in the introduction,
Tim McNamara 00:49:28 This is called running code on bare metal, and it involves compiling your binary to something that can be kind of triggered by the bios, and this is something that’s actually supported by Rust sort of out of the box.
Gavin Henry 00:49:43 Where can people find out more?
Tim McNamara 00:49:45 If you’d like to learn more about Rust? I think a good place is at Rust-lang.org. Me, I’m Tim Clicks is what I normally go by. So, that’s T I M C L I C K S.
Gavin Henry 00:49:57 Perfect. And I’ve got a link for the Rust 2021 edition and the benchmarks you gave me. Thank you very much.
Tim McNamara 00:50:06 It’s my pleasure.
Gavin Henry 00:50:08 It’s been a real pleasure. This is Gavin Henry for Software Engineering Radio. Thank you for listening.
[End of Audio]
SE Radio theme: “Broken Reality” by Kevin MacLeod (incompetech.com — Licensed under Creative Commons: By Attribution 3.0)
