What's New in Rust 1.46 and 1.47Episode Page with Show Notes
Jon Gjengset: Hello, fellow humans. We are back yet again. Aren’t we, Ben?
Ben Striegel: Yes. Welcome back to Rustacean Station, for our new tradition of doing a podcast every two Rust releases.
Jon: You know, I like that a lot. I think it gives some nice sense of continuity, when you get to talk about how things have changed over a longer period of time. Builds anticipation.
Ben: It provides some perspective, certainly. We could talk about things that happened in the context of the previous release and the current release, at the same time.
Jon: It’s very exciting. I think it also helps that we’ve been very lucky and we’ve had a couple of guest submissions. You may have heard, if you’ve been listening to this podcast for a while. And keep in mind that Rustacean Station is a community podcast, is really the idea. So if you have some burning desire to make an episode about, really anything Rust-related you can think of, just reach out on the Rustacean Station Discord, and we’ll happily help you get an episode set up. The podcast becomes better when other people contribute, too. Ben and I are dull and boring, and we can’t really think of anything but these versions, you know?
Ben: Well, let’s begin.
Jon: All right. 1.46, Ben. What has changed?
Ben: Let’s look at the first thing here. So,
const fn improvements. And
so, in our previous episode, I think we talked for a while about how nightly was
starting to use
const fn, and how this was coming, and so we should still
probably give a bit of a recap. Because it’s been a few months, and I’m sure
there are some things that are worth going over again. So in this case now,
you’ve probably heard of
consts in Rust. They let you essentially inline an
expression at compile time and guarantee that it kind of gets copy-pasted into
any place that you use it at. So
const fn is kind of a way of saying, hey, so
I’m just having like these, like a raw, numeric literal that gets copy-pasted
around, you can actually write an entire function, and the output of that
function will get copy-pasted to wherever you use it. And so at the start this
was very limited. Kind of like, imagine, you know, a function that just returns
an integer literal. So it wasn’t much better than just using a raw integer. In
some cases, you could provide some encapsulation to enforce, you know, module
boundaries, privacy and whatnot, which was the original emphasis. In this case
though, we have now actually— I say we, as in the royal “we”, the Rust
developers— I have literally no involvement with this change at all— is to allow
you to start doing some actual programming. By actual I mean, like, branches and
Jon: Maybe we should refer to it as them. “They have. They have implemented.”
Ben: They have. We’re all a part of it, right? Everyone in the community is a part of Rust. Some have more involvement with the code than others. But we can all say that we love Rust equally.
Jon: Yeah, that’s true.
Ben: In any case. So they have implemented branching and looping, which are
kind of the basic fundamentals of programming, of a Turing complete language.
And certainly it’s a bit difficult, although possible to program without these.
But now we’ll be seeing in the discussion of the next release some things that
this has allowed. But to go over it now,
while let, and
loop are now all allowed within
const fn, so you can start
doing some actual cool things in these.
Jon: It’s interesting that
for is not supported yet. I’m guessing because
it relies on iterators.
Ben: Right. And so
for works on any type that implements the
trait. And I believe that implies some mechanisms that aren’t yet ready for
stabilization. Maybe not even implemented, I’m not sure. I’m not sure how far
along you know Miri is.
Jon: Yeah, I don’t know. It is pretty cool, though, that now you can do all
these additional things at compile time. Like now you can do, once you have
conditionals and loops, you can suddenly implement, like, logic in
which I assume, unlocks a lot of cool future things.
Ben: And we should go over again, that
const fn is powered by a Rust
interpreter, which is part of the compiler. And this interpreter is called Miri,
so named because it works on the MIR. The middle intermediate representation,
which is kind of a sub-language that Rust compiles down to, before it gets
passed on to LLVM, or cranelift, or any other backend you could imagine. And so,
this interpreter lives in the compiler, and it does certain things. It’s
actually a pretty cool thing. Like, so, Miri is used for conceptually more than
const fn. It’s a full Rust interpreter. And one of the cool things it can
do is evaluate your unsafe code for you, and tell you if you have undefined
behavior. It’s kind of like Valgrind in that way, or any of the other sanitizers
you might have for, like, a C++ code base, where it can kind of create this
virtual memory and then detect, hey, like, you touched this thing, and you
weren’t allowed to touch that thing, like you would have gotten undefined
Jon: Yeah, I really like Miri. It’s such a cool project. And its magic just continues to baffle me. Like, it recently got support for locks, which means that now you can start to actually test threaded programs with Miri, which is going to unlock a lot of cool unsafe testing as well.
Ben: Yeah. Speaking of unsafe testing, though, so I was curious. Kind of
like, so, one of the things with— the idea with Miri is that, like, with
const fn in Rust is that we want— “we” again; they want to—
Jon: The elders.
Ben: The elders. The council wants to prevent certain things from happening at compile time. And so, for example, a thing that you might— I don’t want to get, like, too into the weeds here. One thing that you can’t currently do, to my knowledge, at compile time, and may never be able to do, possibly, is floating point operations. And the reason for that is that different hardware is allowed to, by the spec, do different things for certain floating point operations.
Jon: Yeah, this is stuff like rounding for example, right?
Ben: Yeah, where, like, so— because Rust doesn’t want to tie itself to, like, any one particular kind of hardware. So the option is either you, like, re-implement all of these things in Rust and say, hey, here is like, the canonical way of doing it. But even if you do that, then you have run the risk of, for example, writing a function that has different output at compile time than at run time. Where if, say, at compile time, Miri tried to evaluate this float, using some math written to be a portable thing. Then at run time, maybe you’re using the hardware interface, which had a different result. You could have these two diverge, and that could lead to very bad things.
Jon: Yeah, interestingly, I had a conversation a while back with— I forget who it was. One of the Miri folks who was talking about how we might even be able to get what are essentially, like, compile time heap allocations where you can, like, build out a tree structure, for example, at compile time. And then the resulting memory and the pointers get, like, burned into the final binary. Which would be really neat. I don’t think this is something you can do yet. I think it was mostly hypothetical.
Ben: Kind of just becomes static memory.
Jon: Yeah, it’s really cool. It’s such a such a neat idea.
Ben: Is there an advantage there, to actually using, like, you know, I guess
if you wanted to use a
Vec and didn’t want— You could just use static memory
in that case. But if you wanted to maybe use a
Vec internally or have a
structure that just uses a
Vec, without having to reinvent all of the
collections on your own.
Jon: Well, I’m thinking if you want to do something like implement, like, a
B-tree or something, that is built at compile time, I mean, you could do it in
just a vector. But realistically, you sort of want separate allocations for
nodes and stuff, right? But this— it is true, this is getting very in the weeds.
But it’s a— I’m really excited to see where we get to with
const over the
Ben: One last thing though, is that with regard to things that you can’t do
const fn, is that
unsafe is a bit limited. I think intentionally, in
terms of not wanting to allow you to do
unsafe operations at compile time.
Possibly for risk of like, totally defeating type safety, are you aware of any
of the actual details there?
Jon: I don’t know exactly what’s going on, but I suspect you’re right that
it’s like, you don’t want
unsafe code that could undermine the compiler
unsafe code that runs at compile time can mess with the compiler.
I think there’s probably some hope there that Miri can contain the damage. But
that’s probably why they’re hesitant, would be my guess.
The next change is one that I’m pretty excited about, and I know it’s something
that is going to have some effects beyond just the immediate things it’s used
for. So back in March, with Rust 1.42, we got this really neat change where you
now get better error messages when you call
unwrap and methods like it.
Basically so that panics don’t include all this sort of extra output that
includes lots of internal Rust standard library calls, and instead gives you
just like, the
unwrap happened at this line in your code. And that was really
neat. And when it landed, I talked a little bit about
#[track_caller] and how
that was the underlying mechanism that enabled that. And in Rust 1.46
#[track_caller], the annotation, or attribute, has now been stabilized. Now
this attribute is really neat— and I highly recommend you read the RFC; it’s
pretty readable. But the basic idea with
#[track_caller] is that any function
that is annotated with the
#[track_caller] attribute, any code inside of that
function that tries to figure out what the— like, where the code was called
from, things like panics that access, sort of, the current location. They all
will use the, sort of, file and line number and column number of the caller to
that function instead. So if you have, like, the example they give in the
release notes is, if you want to define your own
unwrap function. Then if you
#[track_caller] and then
fn unwrap, then now, if you panic inside of
that function, the location that will appear for that panic will not be the
unwrap function, like the line inside the unwrap function where the panic is
called from, but rather the thing that called that unwrap function. And the
underlying mechanism here is pretty neat and actually surprisingly
straightforward, and the RFC goes into a lot of detail. Basically, it’s a
proc_macro that adds additional arguments to the function, sort of
transparently, and then modifies anything that calls that function to add those
arguments. And those arguments are basically the macros for the current file,
the current line, and the current column. So that way, the caller passes in the
location information, which can then be used internally by things like panic.
Now what’s really neat about
#[track_caller] is that you don’t just have to
use it for
unwrap. You could use it anywhere where you have a bunch of
internal library code and you want to point to an error that happened outside of
yourself. Usually, this will mean that you ended up calling panic somewhere. But
there are other use cases for this. I know that there’s been a bit of work on
using this in error reporting, for example. Because in error reporting, you
don’t really want to say that the cause of the error was somewhere deep inside
of the error handling library itself. You want to point back to where the
library that’s interacting with the error reporting tool or framework actually
said, like, an error occurred here. And so
#[track_caller] gets handy whenever
you want to sort of hide some of the internal complexity in giving messages to
the user about the code.
Ben: Those are the two big changes in 1.46. There’s still a few smaller
things I think are worth going over. First of all, we talked about how
const fn improvements landed, and they didn’t actually result in too many
const fns in this release. But there was
one, which is an important one:
std::mem::forget, which lets you intentionally
leak memory, is now a
const fn. I can’t actually think of a use myself for
that, Jon do you have any idea?
Jon: No, I’m not actually sure. I was reading up on the tracking issue for
it, and what was written there was, the usage would be in
dealing with generics, where you don’t want the generics to be dropped, which
I’m not quite sure how— I don’t even know if you can have generics in const
code— I guess you can. That makes sense. Yeah, I don’t know. I’m honestly not
mem::forget is useful in const code, but I think as with many of
the things we’ve talked about before, much of the use here is that if you don’t
know where it might be useful, it’s still worthwhile to make it
const so that
you’re enabling future innovations, that you might just not have thought of yet.
Ben: To be forward thinking in that way.
Jon: Exactly, yeah. I also dug through some of the detailed release notes,
as I always do. And there were some smaller ones that I thought were pretty
cool. In cargo, for example, cargo will now automatically pick up your
README.txt file without you having to explicitly set the like,
readme value in the
Cargo.toml. So this might mean that suddenly on
crates.io we’ll see a bunch more projects that finally now render a
because the authors didn’t know they had to explicitly list it. Another one that
is going to hit very few people, but when they hit people it matters, is that
env! macro, which lets you access an environment variable at
compile time. And this macro is really handy for things like, if you want to
compile in some constant into your code, and you want to modify it in the build
environment. However, in the past, if the value of that environment variable
that was compiled into the binary changed, cargo would not rerun the compilation
of that binary, so you would still have sort of, the compiled binary contained
the old value, and that’s now been fixed. So now, changes to environment
variables that are compiled into the binary will now trigger a re-compile. That
might fix some subtle bugs that have been painful.
There’s also one small change, that we talked a little bit about before we started recording, Ben, which is this ability to now index into nested tuples.
Ben: Right, so this is— you say the ability, it’s actually just the syntax
works now, so you’ve always had the ability to do this. And so let’s just go
over real quick. So if you have a tuple, you’re probably used to destructuring
it to actually get the things out of the tuple. But tuples are just structs,
conceptually. They’re structs, but you never gave them any names for their
fields, and so Rust, instead of destructuring, you can just use the normal dot
syntax to get the fields out of the struct. In this case, it’s a tuple with, you
know, an anonymous struct. And so the names it gives you are numbers starting
from zero. And so if you have a tuple
foo, you could say
foo.0 to get the
.1 to get the second field, and so on.
The thing is that before this release, if you had, like, a nested tuple, you
have, you know, several tuples within tuples. And you tried to get one of the
inner fields, you have, like,
foo.0.0, say, you get an error. Because what was
happening is that Rust was trying to parse
foo.0.0 as a float and saying, hey,
this is kind of a weird float you’ve got here. I have no idea what this is. So
now that’s just fixed.
Jon: It’s funny when you think about it, right? It’s basically, it’s parsing
foo. followed by a float, which means that it thinks that you’re
accessing, like, a fractional tuple element of
Ben: Yeah, so before you would, just to fix this, you would just use parentheses around it. And so it’s not a new ability, it’s always been there. But now you don’t need this workaround anymore for this kind of obscure feature, which I never ran into. But apparently Jon is happy about this.
Jon: Well, I think it’s because I use newtypes a decent amount internally in
the code to, like, try to give additional type safety when— just to disambiguate
types that are handled by the compiler. And then these new types, often you just
have, like, you just wrap a single type, so you just end up with a
sometimes you want to access what’s inside of it, inside of the newtype. Then
this comes up. And so this just saves me from having to name these fields, even
though, like the number of structs I have that have a single field called
inner is getting a little bit ridiculous. And this means that I no longer need
to have all those
Ben: One of the other minor release-note changes that caught my eye. And so
we mentioned before how
mem::forget was the only newly
const made function
in this release. But there’s kind of a weird exception, and so
also a very important function, highly unsafe. But now it is usable in both
const contexts, but not
const fns. And so it was not marked as
const fn. But now you can use it in
consts. And I’m not sure
exactly yet what that restriction is there for. I’m not sure. Jon, you didn’t
Jon: Yeah, no, I’m not sure. I could totally see why you might want it in
const, because, like if you have a pointer to something, sometimes you just
need to be able to cast it in a way that you can’t do with a primitive cast and
you need to use
transmute. But I don’t know the ways in which this may go
wrong, or why they’re now sure that it won’t. I’m not sure.
Ben: It might just be, like, an abundance of caution, where it’s like we
want this to be usable in
const context. But we’re not entirely sure. Unlike
transmute is an
unsafe function, and so they might just be
like, hey, let’s just make sure we know what we’re doing, getting into here,
before we, like, totally commit to using this in, like, all
Jon: Yeah, I think another thing that’s that’s worth mentioning about 1.46, before where we move on to 1.47, is that 1.46 was the first release, as far as I’m aware of, that had a pre-release. So on the Inside Rust Blog, the Rust Team, the Rust Release Team has started posting messages about, like, just in advance of the release of a new stable version, they post this little thing that explains that the 1.46 release is about to release, like, it’s going to be scheduled for this Thursday or something, and then it gives you instructions for how you can try out that release locally, just in order to, like, make sure that everything works ahead of the actual release. and I think they tried this with 1.45.1. That’s the first one I see at least. But I think 1.46 was the first major version that that had this pre-release testing phase. So that might be of interest to someone who’s using Rust in some big production setting, where you really need to be able to test things in advance.
Ben: I did have one more thing too, that I wanted to kind of just go over in
a more general sense. The change itself isn’t super exciting. It’s just that
now, if you have a tuple of a
String and a
u16, you can now convert that
into a socket address very easily. But I wanted to kind of go over this pattern
in the standard library of how, like, for example, so a socket address is going
to be a hostname, usually you’d see it
hostname:port#. Maybe you’ve seen this
in the web browser. Type in a URL to access, like, a special secret port, or a
server on a different port than the usual default of 80 or 443. And so in other
cases, though, so you say you have, like an IP address, and an IP address gets—
so let’s say version 4, it’s just like, it’s four
u8s, like four triplets of—
actually, octets. That’s it.
Jon: I think, octets, yeah.
Ben: That’s it. It’s just four octets. And so, but, like, before you
actually make it into an IP address, so an actual, like, a structure with all
the semantics of an IP address, maybe you have it as a
String. Maybe you have
it as a tuple of these four octets. Maybe you have it as like a bunch of
different things. And so Rust actually has a few constructors for this, and it’s
into pattern, which there definitely great blog posts out there
into. And the reason I want to go over this right now is because
people are coming from C++ or Java, one of the things is, “how do I do function
overloading in Rust?” Rust does not have traditional function overloading where
you would just, like, define a function and give it like a different input type.
And then you can call it with different parameters, and it would dispatch
automatically to these different functions. But it does have
Jon: Yeah, like, same name and different signature.
Ben: Right. And so it does have this
into thing instead. And so the idea
is that if you are writing, say, a library, or the standard library, where you
want someone to pass in an IP address, because maybe they’re making a network
connection of some kind. Instead of forcing them to beforehand, make an IP
address using one of, you know, some specific constructor, you can just say,
hey, like, we’re in this function. I’m going to as my primer type, I’m going to
Into<IpAddr>. And then, in that case, a user could then call this
function with, say, a
String or with their tuple, or with any of the other—
there’s a few different ways of making the IP address, and then we don’t need to
worry about this. And so effectively from the user side, the API consumer, it
looks like function overloading. From the side of the library implementer,
basically, what you can do is, you could have a bunch of these things, if you
want to like, say, like, dispatch to a single function after you have, like,
normalized, you’ve like, called the function. You have, hey, like we now, like,
we have this parameter. It’s just you know, so and so implements
we just called
.into() on this, and now we have, no matter what it used to be,
now we have our IP address. And so it’s a very, like, nice, like a bit more
principled, less flexible but more principled way of doing this function
overloading pattern from other languages, which new users might want to look
Jon: Yeah, and I think the other thing that’s nice, and this particular
change exemplifies that, right, is that over time you can keep adding new
into, and all of the functions that rely on
automatically start being able to do, like, accept arguments of that type is
Ben: Right. So that’s all that I had for 1.46. Shall we move on?
Jon: Yeah, Let’s do it. I’m excited. 1.47. Man, time flies, right?
Ben: We’re really going through it.
Jon: We’re at 1.46, now we’re at 1.47.
Ben: Rust moves fast.
Jon: Right. So this too, the first element here, on traits implemented on
larger arrays, is also something we’ve talked about before. About how Rust has
this feature that’s still baking, called const generics. And so, the idea is
that if you have an array, that is of some length
N, you might want to have
implementations that are generic over that number
,. And in theory, you could
have other types that are also generic over numbers, and not just over types.
Now, the primary way in which const generics is currently exposed to the world
is through implementations on arrays. So the idea is that if you have an array
of some length, you might want it to implement, say,
Debug, no matter how long
the array is. And it used to be that the Rust compiler would pull this trick of,
it just copy-pasted the implementation, like, 32 times and just implemented,
impl Debug for [T; 0]. And then it did the same for
And it just repeated this down.
And of course, internally, the compiler is allowed to use unstable features like
const generics. And so a little while back, most of the standard library was
updated to use const generics internally for this instead. So rather than
duplicating all this code, it just had a single const generic implementation for
N. In practice though, they, the elders, did not want to expose const
generics to users of Rust, because const generics are not stable yet. And so
they put this additional restriction on these generic implementations that said,
array::LengthAtMost32. So the idea here would be that they wouldn’t
expose any implementations of arrays— implementations of traits for arrays
longer than 32, because that’s what they had in the past with this manual
implementation, and that restriction has now been lifted.
Now things like
Debug are implemented for arrays of any length, through the
const generics internal to the compiler. And I think the decision for why they
chose to remove this sort of arbitrary restriction was that they realized, and
you’ll see in the PR that this argument is made as well, that there isn’t
really— you’re not really committing to const generics by exposing this feature.
All you’re committing to is that going forward, there will be implementations
for arrays of arbitrary length. And the mechanism just happens to currently be
const generics. But there’s nothing in the, in what has been stabilized that
requires that. So it might be that in the future, const generics just, like,
imagine const generics does not work out, they could still provide some other
built-in mechanism in the compiler that gives implementations for arrays of any
length, and that would be sufficient to still fulfill the contract that’s now
essentially been stabilized.
Ben: Stabilized in some sense, yeah. So again, actual stabilization of this
feature is still forthcoming with no timeline. But there has been movement here
in terms of the idea being, sort of trying to stabilize all of const generics
was a pretty big thing. Trying to stabilize just a minimally useful subset of
const generics, which now has its own feature flag at the compiler, the idea is
to get all current uses compiling on this, to kind of like shake out any bugs,
and I believe that’s coming along pretty well. I look at the— there’s a tracking
issue for the
min_const_generics feature flag, where they’re going over,
pretty recently, the remaining blocking issues and any bugs that are happening.
So, yeah, follow that if you want const generics in your own code.
Jon: Yeah, I mean— what I meant with stabilization here was more of, the stabilization of implementations for any length arrays. Like, that doesn’t bind them to const generics, even though it seems pretty likely that const generics, or at least some part of it, is going to come hopefully, maybe soon.
Ben: Yeah. Hopefully this year at least.
Jon: One interesting point that we don’t need to spend too much time on is
the fact that most traits are now implemented for arrays of any length. But
there’s an exception, which is the
Default trait. So they now have these
generic implementations for any
N for, like,
and whatnot. But
Default is not among them. And the reason for this is that in
the standard library today, like prior to 1.47, there’s an— the implementation
Default for arrays of length zero do not require that the type of the array
Default. So even if you have a type that doesn’t implement
Default, you can still get a zero-length array of that type, by using the
Ben: Which is totally reasonable, in a vacuum.
Jon: Yeah, because you don’t have any
Ts, so it doesn’t matter if
Default. Unfortunately, there isn’t, at least far as I’m aware, a
way to express this in today’s const generics, right? If you imagine that you
impl<T, const N> Default for [T; N]. You don’t have a way to say,
N is greater than one, or greater than or
equal to one, but don’t require it if
N is 0. You can’t currently write that.
And you can’t write it as two different impls, because they would be
overlapping. Now maybe specialization can combine with const generics in some
interesting way to deal with this in the future, but I think for the time being
they decided to keep the manual implementations for
Default this time around.
And then it’s something that might in the future be stabilized to also work for
any length array.
Ben: It’s one of those things where whenever you’re doing like, you know, an initial release of Rust 1.0, kind of, try to be forward thinking and think, hey, let’s not, like, you know, pin ourselves into a corner. But you can’t ever catch anything, so sometimes these things come back to bite you. Unfortunately, it’s not a super big deal in this case.
Jon: Yeah, but as you said too, I think the standard library, as it was,
makes a lot of sense, like you probably do want zero length arrays to implement
Default. And this might even be a thing that, like, macro authors could take
advantage of, I’m not sure. I’m sure there are various crates that take
advantage of zero-length arrays in some way. And so I think the standard library
originally did the right thing of saying, we’re going to implement
T if the length is zero. And now we just need to figure out how to
actually make that work with const generics.
Ben: The worst case scenario, I imagine you could do some kind of weird compiler hack. Obviously, it’s not the ideal solution, and you’d probably prefer not to do that if you can avoid it. But it’s not totally intractable, I think.
Jon: Yeah, I mean, I think the idea is to use specialization on constant
generics, right? So that you say you have a generic implementation that’s
generic over both
N, and then you have an implementation that’s
T but not
N, it gives a concrete value of
does not require
Default, and specialization in theory could could detect this
thing that’s specialized to
0 is more specific than the other one, and realize
that it needs to use that one.
Ben: Even then, though, I’m not sure because it seems like the— so the
second, this second specialized implementation where it’s like we want to
Default for this. It’s simultaneously both more and less specific
than the other one.
Jon: Yeah, it’s weird.
Ben: And so it’s like, could that even be done? And specialization is so up in the air right now, Who knows?
Jon: Yeah, I don’t know. But I guess this is why the elders have decided that it’s not time yet.
Ben: And we have fallen into the weeds once again, so— in our digressions.
Jon: Alright, let’s let’s backtrack back up, to shorter backtraces.
Ben: You’re talking about the
Jon: I’m sorry, Ben.
Ben: —in the previous release, and I get the impression that what’s
happening here is actually related to
#[track_caller], where there was a
regression a while back where the backtraces got a lot bigger. Where previously
Rust was trying to kind of compress backtraces to only be useful and then
#[track_caller], some changes there made them much longer, and now we’re kind
of getting back to the the good and proper, only having useful info in
backtraces. Do you have more information about this?
Jon: Yeah, I think what actually happened was just the— like, in certain
cases, the mechanism that Rust uses to shorten the backtraces didn’t work
correctly. In fact, looking at the PRs, I think what happened was that the sort
of marker thing that that Rust uses, which is this
__rust_begin_short_backtrace stack frame was being optimized away by the
compiler, using, like, tail call optimization. And so now, when walking the
stack, it couldn’t find the frame that told it to stop walking. Because it had
been optimized out. And so now, that frame has— there’s been some modifications
to ensure that frame does not get optimized out. Which means that now the
unwinding stops at the right place. And now you get the short backtraces again.
The other change that landed in 1.47, that I don’t really know what to say about, is that LLVM, which is sort of the backing compiler, or— is compiler the right word?
Ben: Code generator.
Jon: Code generator, yeah, that’s right. Was upgraded from the previous version to LLVM 11. Do you know more about what this means?
Ben: Not really. I mean that— the idea obviously is that LLVM is not, like, internally doesn’t have, like, a stable interface. And so you kind of have to keep, like, upgrading. Because it’s kind of like, you know, they provide an interface. But if you want all the new and shiny stuff in terms of all the improvements, you have to keep on upgrading. And so Rust is just on the normal update treadmill. And in this case, I think one of the— Rust doesn’t, you know, need to update as soon as LLVM has a release. And in this case, I think actually, it might even be on a pre-release of LLVM 11. But I think that a few releases back, there was some concern about some performance regressions in LLVM. And I think that from what we’ve seen, that that has kind of been clawed back in this release. And so obviously compiler performance is a big deal. And so I think people were just eager to upgrade.
Jon: Yeah, I managed to dig up the performance— like, there’s perf.rust- lang.org, which gives you, sort of, performance information about every release, including nightlies, of Rust. And that particular change that that integrated LLVM 11 seems to have just given mostly small but not insignificant compile improvements, for basically every benchmark that they have there. The fastest one is, like, sped up by almost 30% which is, it’s obviously pretty significant. But most of them hover around like 3%, 2% or so, which is still pretty good.
Ben: There are some new— actually, before we do that: Control Flow Guard on Windows. What is this?
Jon: Yeah, so this is— I mean, I don’t really use Windows much myself. But when I saw this in the release notes, I had to go digging a little to figure out what it was. And on the, like, Windows documentation page for Control Flow Guard, there’s a little diagram, and basically what it does is, it injects a little bit of code into your binary that does some runtime checks to make sure that if someone, like, attacks your program and manages to overwrite a buffer or something, they can’t easily take over control of the control flow of your program. So this would be something like, imagine that you take a function pointer F, and you call F through that function pointer. And then imagine that some attacker manages to overwrite F with an address of, like, some evil code they want to run instead. What CFG does, the Control Flow Guard, is basically inject this little check, just before the call to F, to check that F is still a valid target, that it doesn’t point into some, like, bad memory or something. And if it does, then it just terminates the process. There’s a little bit of sort of coordination that’s required here between the compiler and the Windows kernel at runtime that keeps track of, like, what are valid jump targets, like what are valid functions to call, and such. So it basically prevents some attacks that are bad when they happen. I don’t know how effective it is at preventing real attacks, like in the wild, but it certainly seems like something that that could be a pretty good idea, to just—
Ben: I’m sure it’s based on things that have been observed in the wild. I
know that, like, it might not be as useful for Rust in particular, in terms of,
like, if you’re writing
unsafe code in Rust, I’m sure it is, but it’s not
actually turned on by default in
rustc, because I think the idea is that
almost all Rust code actually doesn’t need this. Obviously, there are plenty of
things in Rust, that Rust does turn on by default. And so you have, like,
typical hardening things, like position independent code and all the other— we
have stack guards and stack canaries and things that kind of like, even though
normally you wouldn’t encounter these. Sometimes it’s just good to have for
additional security. But I think in this case, it was just decided that the
performance tradeoff to actual risk was not good enough to turn it on. But
obviously, you can turn it on if you so elect to, with this new flag to
Jon: Yeah, that sounds about right.
There were a bunch of smaller library changes in 1.47 as well. And I think
before we go to some of the specifics, do you want to talk a little bit about
const changes here?
Ben: Yeah. So as we mentioned before in the previous release,
has the capability of doing a bunch of
loops. And so there are
plenty of newly-const methods. In this case, although the scope is pretty
narrow, the biggest group are probably: for all integers, there are various
checked_mul-tiplication, and by “checked” we mean that in this case, if it
would overflow, it will provide an error. Also
saturating_sub, etc. In this case, if it would overflow, it will— or under
flow as well, I’m pretty sure, I guess in subtraction. I guess you can also add
a negative in the signed case.
Jon: Yeah, it’s—
Ben: But yeah, in either case, instead of overflowing, they will just kind
of clamp themselves to the highest value. And so, if you try to add two
ints, you just get
MAX int back. Or try to subtract zero or, you know, like a
MAX int from zero, you just get zero back. And so different people want
different behavior from integers. If they overflow, again, there are also, in
the standard library, overflowing versions of these operators, and then the
default, obviously, if you just use normal operators, is to panic in debug mode,
overflow in release mode, although that is implementation-defined and that can
change theoretically on, say, hardware that did this efficiently, they could
just say, hey, overflow or check.
Jon: Yeah, I think the standard library calls overflowing just wrapping, I think, because that’s usually what you observe in practice, right?
Ben: Yeah, So all of these are now
const fns, and so you can get this nice
overflow-free or overflow-checked behavior in your const math. Also a bunch of
operations on ASCII, and so you can check a string and say, hey, is this like an
ASCII string? Are all these upper case, or are all these, like, digits? There’s
a bunch of these. We’ll link to these in the release notes, but a lot of ASCII-
specific string functions are now implemented as
const. So you can do this at
Jon: Yeah, and I assume that many of these were really just waiting for branches—
Ben: Well, also loops, because you have to go over every single one.
Jon: Right. Yeah, that makes sense.
Ben: So yeah, it’s just a natural extension of, hey, we can do this now. Let’s do it.
Jon: Yeah, that makes sense. There are some other smaller changes that are
cool. I really like that we now have
Vec::leak is basically
the same as
Box::leak was already in the standard library; it
lets you take a heap allocation, like a
Box<T>, and then give you a static
mutable reference to
T. So the idea here is that if you allocate something on
the heap, and then you
Box, then because the
Box will never be
dropped, the memory is never freed. Therefore, that pointer lives forever. And
therefore the reference is valid for the
static lifetime. And now we have the
same thing for
Vec where you can leak a
Vec, so you leave the heap
allocation running forever, it never gets freed. And that gives you back a
mutable slice reference that can live for however long you want it to, up to
static because it will never be dropped or invalidated.
leak is a much more direct name than
forget, which is how you would
normally do this.
Jon: Yeah, because remember that in Rust, leaking memory is still safe, right? There’s no memory safety violation. It just causes you to run out of memory.
Ben: Well, so is
Jon: Yeah, exactly. I also thought that a
pointer::offset_from is kind of
cool. So this is something that comes up a lot if you end up doing some kind of
pointer arithmetic, especially if you deal with parsing or interacting with
certain low level C libraries.
pointer::offset_from is a pretty handy function
to have available to you.
Ben: I was curious to see some additions where, so now there are some new
constants. Numeric constants, not just, you know,
const in the Rust sense, but
like, actual numbers defined in the float modules,
f64, for a constant
called Tau. Are you aware of Tau, Jon?
Jon: Yeah, Tau is 2π, right?
Ben: It’s 2π. And so it’s kind of like saying, hey, like, why not, just, we already have— pi is already in there. It’s always been in there, and the idea being, like, why should we have tau in here? And there’s a whole— it’s kind of— I don’t know. It’s kind of a funny thing to me. I’m not a mathematician. I think some people feel very strongly about tau. I’ve read, there was a document called The Tau Manifesto, which argues about how pi is the wrong constant to use for circles. We should be using tau instead. And broadly, tau is defined as 2π. And because pi is defined as the ratio of the circumference of a circle to its diameter, tau is just circumference to radius. And so if you’re using— if you’re in application domains where radius makes more sense, I think a lot of trig functions, especially, care about this. And if you’re working in radians, tau is more natural than pi. Then you probably want to be using tau. Other than that, you could just be using 2π, honestly.
Jon: I mean, I think the summary of The Tau Manifesto is that we should be saying that pi is half tau.
Jon: That’s really what we should be saying, Rather than saying that tau is 2π.
Ben: We’ll link The Tau Manifesto in the release notes. It’s worth a read. It’s good. It’s just good fun math knowledge to have, in terms of like, illuminating why certain things are the way they are. It kind of just shows, too, like there’s path dependence in a lot of fields. Or maybe if people had been using tau for the past 2000 years, maybe we’d be like, we should add pi, now. Pi actually is the right constant to use, and so—
Jon: We would have The Pi Manifesto.
Ben: All fields have path dependence, and you know, change resistance, and Rust is not immune to these things. Rust has plenty of inheritance from C-style languages and other things before it, and things after it will have to decide, you know, languages that follow Rust, take up the mantle, decide what to take from Rust in the future.
Jon: It’s true. It’s true. In 1.47 as well, there were, like, some smaller
detailed changes that I’ll go over quickly. One of them is that build
dependencies, so these are often things like
other things that have to, like, parse code and spit out something else, like
things that depend on
syn very often. Those things used to be compiled in the
same mode that you’re compiling the main crate. The idea here was that, why
wouldn’t you? And the answer turns out that for these kind of build
dependencies, often they’re not doing that much processing. And so it doesn’t
really make sense to spend a lot of time in the compilation process to optimize
them, when they’re only going to run for a very short amount of time. And so,
with 1.47 the default is now to not optimize build dependencies. So that way you
can compile them quickly, and then if they run and only have to take a little
bit of input, you probably won’t even notice. So in theory, this should make
compilation of certain crates a decent amount faster. Of course, as the pull
request that made this change also argues, there are cases where this is not
true. Like if you have a
bindgen or something that has to process, like
megabytes of code, or gigabytes of code. Then at that point, you might actually
want your build script and
bindgen to be built in release mode. And there’s an
override you can use to do that.
There was also a really cool change to some of the Cargo documentation, that I
was very happy to see, which is basically a SemVer compatibility guide. So this
is a document that essentially goes through what are all of the things that
constitute breaking changes in Rust, and what things do not constitute breaking
changes. And also gives some examples of which are which, and why these matter.
So I highly recommend you give this a read. We’ll link that in the show notes as
well. And also, and this is very minor, but it’s going to matter on, like,
you’re going to notice this on the usability level, which is that
so if you type
cargo help, followed by some command on the command line, it
used to just give you this, like, short command line, brief help text, basically
what’s generated by
clap. And now it will actually display the man page. It
will give you a lot more detail about how each
cargo subcommand works, which
in theory should be a lot more helpful to people. So now, when you write, like,
cargo help publish or something, you’ll get more expansive text that might
include things like examples, rather than just a very terse explanation of
exactly which flags it accepts.
I notice one more thing, actually, that I’d like to talk about for 1.47. And
this is something that is not— you’re not really going to notice if you are
using Rust, as much as you’ll notice it if you’re helping contribute to Rust,
which is that
x.py, the tool that’s used to bootstrap the Rust compiler when
you build it from source, used to have all these weird settings, that every
guide on the compiler suggested you change the the settings for, or run it with
particular flags so that it wouldn’t take as long. And now all of its defaults
have been updated, so that normally you can just run
x.py build, or
x.py test, and it will just run the suggested things rather than build lots of
things you don’t need. This is a very minor change, but it’s a big quality of
life improvement, I think, especially for new contributors to Rust itself.
Ben: I think that’s all that I had for 1.47.
Jon: I think that’s all I had too. There’s a pretty exciting change that’s coming down the line. There’s a bit of a teaser, probably 1.48.
Ben: Jumping the gun?
Jon: Yeah. So this is— I’m only going to mention them briefly, because I’m
guessing that we’re going to talk about them a decent amount when it comes out.
But these are inter-doc links. The idea here is that if you have in your doc
comments in your code, let’s say that you have a library that has two methods.
It has two functions,
bar. And in the documentation for
want to link to
bar, and vice versa. Well, previously you had to write like
[`bar`](...), and then find, like, the filename that
cargo doc would
generate for the documentation for
bar and stick that in there. Basically, you
would have to write the URL, or at least the the relative URL directly into the
And now, with inter-doc links, you don’t need to do that anymore. The
documentation compiler basically is able to resolve symbols for you and then
automatically link them. So you would just write
[`bar`], and that’s all you
had to write, and the compiler would realize that you meant to link to
the same crate. Because you didn’t give a particular URL. And this works for all
sorts of things, like modules, or things in other crates. So you can do, like,
crate::some_type. It works for methods and types and consts, like whatever you
can think of, anything that can be— that you can, like, use in that file in the
Rust code, you can also just name directly in the documentation with the—
surrounded by ticks and square brackets, and the links will be generated
automatically. So this is just a really nice quality of life improvement for,
for writing good documentation.
Ben: Well, I think that’s it.
Jon: Yeah, I think that’s it. I think for people who are excited about, like, as we talked about, we the Rust community, right? There are two working groups that have recently been announced, or project groups, I guess they’re called, where if you’re interested in helping out the Rust community, I recommend you go give those a look. One is the error handling project group, which is going to look at how to improve the error handling story in Rust, both about libraries, but also how the language can help improve it. And the other is the Portable SIMD Project Group, which is working on how to get SIMD, the like, data parallel CPU operations, to work better in a portable way in Rust. And I think both of these groups are happy to get input and advice into their discussions. I think all their discussions are public. If you want to get involved and care about either of these use cases, then I highly recommend you do so. And we’ll link to those in the show notes as well.
Ben: Speaking of community, I know that— I think we mentioned last time, but also it’s worth reiterating, is that Jon has been expanding the role of our Rustacean Station Discord into a more broad direction beyond just podcasts. And now there are plenty of Rust streamers making their home there as well.
Jon: Yeah, that’s right. This came up because I noticed that, especially in the last maybe six months, maybe given the current world situation, it’s not that strange. A decent number of people have started to stream sort of technical content, whether it’s programming or just talking about Rust. And I figured that this is another great way to learn Rust beyond just, sort of, the written materials. And I figured that given that Rustacean Station is all about community content, that sort of teaching, learning content, exploring Rust, why limit ourselves only to the audible spectrum? Why not include the visual spectrum as well? And so now on the Rustacean Station Discord, you can find all these Rust streamers, you can chat to them. You can suggest ideas for streams you would like to see. And of course, you can also jump on there to just chat to Ben and I, and some of our great editors and other contributors.
Ben: If you wanted to help out, say—
Jon: For example…
Ben: If you wanted to edit this episode, which I mean, I guess it would be hard for you to edit it from hearing this specific request for help, considering that we’re currently making it and not—
Jon: A time traveler.
Ben: Who knows? Weirder things have happened in fiction.
Jon: That’s true. You are totally right.
I think that’s everything. I think we ran through 1.46 and 1.47. Good job, us!
Ben: We will see you again in 12 weeks. Three months.
Jon: Very, very exciting. And by then, hopefully in theory, I will have graduated, Ben.
Ben: Excellent. So you can spend full time on Rust at that point.
Jon: Yeah, very exciting.
Ben: No more of this silly waste of time on academics and personal achievement.
Jon: Yeah. I know, right?
Ben: Do you want to, even real quick— let’s say for you, let’s plug the thing you’ve been working on for your PhD, or whatever it is. It’s a project written in Rust.
Jon: It is. So over the past five years or so, I’ve been working on Noria, which is a database engine that’s written from scratch entirely in Rust. It’s like— I think it’s about 80,000 lines of Rust code, and the idea is basically that it builds a cache into the database. So rather than have to, like, write your application so that it talks to like MySQL, and then it also talks to Redis or Memcached or something. The database just does that for you, and makes all your queries fast, basically always. It is a research prototype, but it’s a really cool project. I’m very happy with it. I’m doing my defense in, like, a bit under two weeks. And I’m hoping to put the presentation of that work online. So maybe even by the time you listen to this, that video is online, you can see a bunch more about it there. But it’s a cool project. I encourage you to give it a look.
Ben: All right, so let’s say our farewells.
Jon: So long, Ben. See you in 12 weeks.
Ben: So long, farewell, auf wiedersehen goodbye—
Jon: Ah, beautiful. Bye, everyone.