Please see the disclaimer.

This post has been discussed on Hacker News, /r/rust, and lobste.rs.

I should not have posted this to Hacker News on a Saturday night right before going to bed, but in my defense, this post blew up in a way I didn’t expect.

Introduction

There was a recent dust-up on GitHub surrounding the decision by the Cryptography library (which I will call cryptography for convenience) to switch to Rust.

One of the distro maintainers of my distro of choice, Gentoo, filed a bug report with the crytography saying that the switch broke builds on several platforms that Gentoo still supports. The cryptography authors replied that those platforms are not really used anymore, and that they were going to stick with Rust because it has better memory safety than C. They also argued that it is better to force better programming languages on people because of better security.

At first glance, it appears that the better argument is on the side of the cryptography maintainers, but after thinking about it carefully, I think they are wrong.

Reasons

There are a few reasons why I believe the cryptography maintainers are at fault.

Due Diligence

First, their argument for Rust (and against C) because of memory safety implies that they have not done due diligence in finding and fixing such bugs.

I can almost hear the rage of my readers against that paragraph above and against the fact that I don’t have a commenting system on my blog. So let me answer the comments preemptively.

“They are volunteers, giving their time away for free!”

Yes, but they also intend for their code to be used widely. They managed to succeed in that, so they now have some obligation to their users.

“They don’t have any obligation!”

If a software project actively goes out and gets users, which just about any project with a serious number of users has done, then yes, they have an obligation to those users. The reason is that they sold users on the idea of using their software. In other words, they were marketing their software, which means making promises.

In fact, when people release libraries and try to get users for them, it’s because they want them to be used by downstream programmers. A programmer might write a program to scratch an itch and release it, but that reasoning applies much less to libraries, in my opinion.

“Okay, but they didn’t get anything in return, so there’s still no obligation.”

In return, the users gave them relevance.

One of the most vocal (in favor of Rust) developers of cryptography works for Red Hat Security Engineering (if I read his GitHub profile right). I don’t know if his work on cryptography helped him get that job, but it might have.

Another of the most vocal developers has a computer security company. I would bet money that his work on cryptography gives his company relevance.

So they did not get “nothing” from users. Quite the opposite, in fact.

By the way, this position comes from my own experience pushing my bc. I managed to convince the FreeBSD project to make it the default in FreeBSD 13.

Once I did that, I shouldered, willingly, the need to keep FreeBSD happy. And while I have made mistakes, I have done well so far.

And with my bc, I did my due diligence with memory safety. I fuzzed my bc and eliminated all of the bugs. I even run the generated fuzzer test cases through AddressSanitizer, and my entire test suite is run through Valgrind and AddressSanitizer. I also add failing fuzzer cases to my test suite, which means I run more and more test cases through both of those frightfully effective tools.

For the record, those tools are only effective with effective test suites, which I spent a lot of time building. But building such a test suite is part of due diligence itself.

So it follows that if the developers have not done their due diligence, their users should leave, either by forking the project or creating a new one. The relevance they gave to the cryptography authors should disappear.

Battle-Tested C Code

In fact, I have done enough due diligence with my bc that I would consider it a dereliction of duty to Rewrite It in Rust (RIIR).

RIIR has become a meme, a repo, and a list of offenders.

Why would it be a dereliction of duty? Because rewriting it in Rust would cause more bugs, not less. This is because of several reasons:

  1. I would need to redesign it to fit the language.
  2. I would need to reimplement it, and new implementations always have bugs.
  3. The C code is battle-tested, both by me (using fuzzing and other techniques) and by users.

That last point is the most crucial, especially in the case of cryptography.

If the developers of cryptography claim that they have, in fact, done their due diligence with regards to memory safety in their C code, then they are claiming that it’s battle-tested.

The saying that “a bird in the hand is worth two in the bush,” and in this case, if the cryptography developers are claiming that they have done their due diligence, they are throwing away a bird in the hand for a single one in the bush.

Edit (28 Feb 2021): This part of the post seems to be misunderstood widely, so I am going to attempt to clarify.

People are arguing that having safe C code requires a frozen, small codebase with a thorough test suite. And then they debate my position based on the belief that the codebase needs to evolve.

For the record, I agree with them that in order to have safe C code, the codebase must be small and frozen.

What I am arguing is that crypto code should be small and frozen, with a thorough test suite. I wrote about that here.

And if that’s the case, their users should leave and take cryptography's relevance with them.

Desktops and Smartphones Are Not the Only Computers

The users of cryptography were claiming in the bug report discussion that Rust is not portable to many platforms, and the authors said that they don’t have the time or resources to target those platforms. Fair enough.

But then they claim that the users should put in the effort to port Rust to their platforms. This is wrong.

The cryptography authors also claim that the platforms that Rust doesn’t support do not matter. As we will see below, this is false.

While I agree that the cryptography authors are not responsible for porting Rust to other platforms, the users of those platforms are not either.

That responsibility falls on the Rust developers.

They were the ones who sold Rust to those who have used it, so as above, they have the responsibility for supporting their users.

Granted, the Rust developers have made no claim about being portable to every platform. But they have claimed that it is appropriate for embedded software.

If it were true, this would be great. After all, IoT devices outnumber desktops and smartphones by at least one order of magnitude.

But there are a lot of them that LLVM, Rust’s backend, cannot generate code for. In fact, there are a lot of them that C++ cannot run on.

Edit (28 Feb 2021): Also, the Rust developers are the developers with the most experience reading ISA manuals knowing how to make a compiler generate code. So they are still the best placed to support those “esoteric” architectures.

And if they do not know how to read ISA manuals and generate code, it’s because they lean too heavily on LLVM.

Make no mistake; embedded software is still running the majority of devices in the world. And C is the king of embedded software.

I don’t know exact numbers, but I wouldn’t be surprised if the majority of programmable devices in the world cannot run Rust.

Thus, because Rust uses LLVM, it is not portable.

And in my opinion, Rust is not appropriate for the embedded space.

So in this case, I would consider that the cryptography developers were victims of the Rust developers.

gcc Is Not the Only Compiler

That isn’t the only problem.

There is a project to make gcc able to compile Rust. That’s commendable.

However, many people seem to believe that once it’s done, Rust will be portable. That is not the case.

Why? Simple: gcc is not the only compiler.

There is plenty of code out there that uses dead simple C compilers, like tcc, sdcc, and others. And often, they have good reason to do so.

Adding a gcc frontend, while it will improve the situation, will not make Rust as portable as C. Period.

Pushing for Progress Hinders It

The other thing that the cryptography authors claim is that their users who refuse to adopt Rust are hindering progress.

That may be true, but it is also true that forcing “progress” on others hinders true progress.

By forcing users to either adopt Rust or pin their dependency on cryptography to the most recent version without it, they are forcing those users to use stagnant code.

Isn’t that the very opposite of progress?

cryptography at Fault

Those reasons lay out why I think the cryptography authors should shoulder the blame for this situation, and I think I can explain where they went wrong.

  1. They let ideology come before pragmatic engineering.
  2. They sold cryptography to users.
  3. They knew that switching to Rust would break existing users and did it anyway.
  4. They either did not do their due diligence for memory safety, or they did and threw that battle-tested code out.

Portability

To be honest, I used to think that it was better to switch to a better language than C.

I had several talks with Linux distro maintainers, as well as a talk with my father-in-law, that convinced me that C’s portability is still important enough that it should be used.

Unless, of course, you explicitly target only certain platforms. But you had better be prepared to never target others.

In fact, after talking with the distro maintainers, who have had to build Rust, I am also convinced that it’s not just about being portable, it’s also about how easy it is to build software.

Rust’s bootstrap is complicated, and it is one of the worst things about it.

If building software in Rust means building Rust for a Tier 3 platform, you can bet that people will stick with C.

Zig

I want to take a moment to talk about Zig.

Zig might be one of the most promising up-and-coming languages of recent memory.

But it will ultimately fail to reach its goal.

You see, Zig is meant to replace C. But it, like Rust, also depends on LLVM.

Although its creator is trying to make it self-hosted.

But as long as Zig is written in C++, it will never replace C, simply because for those platforms where there is only a C compiler and no LLVM support, Zig cannot replace C. And as we saw above, that could be a lot of devices.

Now, there is an effort underway to write a compiler for Zig in C, which will help, but unless that compiler either spits out C code, to then be compiled by the platform’s C compiler, or it can generate code directly for all of those platforms, it will not replace C on those platforms.

Edit (28 Feb 2021): The creator posted this link and a link to Zig’s current C backend. He also posted a link explaining Zig’s relationship with LLVM.

This is all great, and it is why Zig is still one of the most promising up-and-coming languages.

Importance of Language Specs

And even if that Zig compiler in C does succeed (and I hope it does), there is another problem: it could create a schism.

Right now, as far as I know, there is no official language spec for Zig, like there is for Go. This means that this new compiler may not match what the official compiler does.

Edit (28 Feb 2021): The creator of Zig has made me aware of the effort to make a spec for Zig.

Who knows which one will win?

Rust has the same problem. The language seems to be defined as “whatever rustc does,” which means that the GCC frontend might never really be true Rust.

Edit (27 Feb 2021): After reading this blog post, I realized another good reason for having a solid language spec, and that is that if there is a language spec, it is more justifiable to make breaking changes to the compiler to fix bugs. In fact, having a language spec makes it easier to define what actually is a bug.

Embracing C

Because of all of this, it is my opinion that C cannot be replaced, only embraced.

This will be best done by making C the target language for the compiler and by writing the compiler for the language in C itself, either directly or indirectly.

But that will not be enough.

C is portable, yes, but that’s just because there are C compilers for every chip that has ever existed (pretty much).

However, that does not mean that C code is portable; because of undefined behavior, as well as unspecified behavior and implementation-defined behavior, almost no C code is portable.

So a language that wants to take C’s place needs to not only target C, it must generate C without any undefined behavior.

In other words, in order to take C’s place, a language needs to beat C at its own game of portability.

Generating Portable C

Now, generating perfectly portable C seems like a tall order, but it isn’t really.

The toughest undefined behavior to get rid of is concurrency problems like data races and race conditions. However, Rust has shown the way there, and I believe it is possible to do.

Beyond that, the biggest source undefined or implementation-defined behavior is arithmetic.

I have implemented fully-defined and portable arithmetic in C already, so that barrier is passed.

With time, I think that the other corner cases of C can be surpassed.

Announcement

With all of that said, I think it’s time for my announcement.

I am restarting work on Yao.

I need a fully portable language for my version control system, but I also need it to generate type information.

In other words, I need C, but I also need it to generate types.

Creating a dumb compiler that generates C is the perfect way to do that because it would be the same amount of work to write a program to generate type information from C code.

That reason, along with the fact that I am unsatisfied with the design of both Rust and Zig, is why I decided to go my own way.

Whether or not Yao gets used outside of my VCS, I don’t know. Probably not.

But we’ll see.

Conclusion

If there is one main takeaway from this post, it should be that software developers usually get stuck in bubbles. They don’t see the use of platforms that they don’t use themselves.

That creates friction when decisions are made to stop supporting those platforms, and those decisions will hopefully be reversed.

If they are not reversed, they will either kill the projects that make those decisions or hold back the industry from progressing.