Introduction

I was recently approached by a founder (who I will call TF for “The Founder”) of a startup (which I will call TS for “The Startup”) to audit their software in my spare time (for pay, of course). I was contacted on November 14 and told that the deadline was December 10.

The only problem: their codebase was almost 200k lines of code. They were going to give me less than a month of part-time work to audit all of that code.

It was an impossible deadline because auditing code, like writing code, is difficult.

TF has no way of understanding the amount of work that it takes to write, much less read or audit, a piece of software. And I would not expect them to because it’s not their domain of expertise.

When writing back to TF, I tried to communicate the reasons why I turned down the job, but because I am not the best communicator, I don’t think I was effective.

That is why, in this post, I would like to explain the difficulty of writing and auditing code in terms that non-programmers can understand.

Programmer Speed

There are two major things that programmers do directly with code: writing it and reading it. How fast can a programmer do them?

Writing Code

I am going to talk in terms of lines of code (LOC). It’s a bad metric, but it will do to illustrate my point.

Based on one of my projects, I know that I can write about 10,000 lines of nearly perfect code in a year. This seems to be true for other developers; for example, Colin Percival wrote about 30k LOC for Tarsnap in three years.

If we were to put that into perspective compared to what I was asked to audit, it would take me 20 years to write that much code to the quality that I expect of myself.

However, the time spent includes all of the time spent designing and testing the code, activities that are less important when auditing software. If I were to be generous and say that 90 percent of the time spent on writing code is designing and testing, that means I could audit code 10 times faster than I can write it.

This means that it would take me two years of full-time work to audit a codebase of 200k LOC. And I was given a month of part-time to do it.

Reading Code

But auditing code consists mostly of reading code, so maybe that is the more important measure of speed.

Let’s start with fiction. If this site is to be believed, the Harry Potter books have 1,084,170 words between all of them, and according to Martin Cutts in the Oxford Guide to Plain English, the average sentence length is 15-20 words. If we assume an average of 17.5 for Harry Potter, the expected number of sentences in the Harry Potter books is 61,953 sentences.

How long does it take someone to read the Harry Potter books? I haven’t read them recently, but according to this site, the average person reads about 250-300 words per minute (wpm).

Assuming an average of 275 wpm, this means that the average person can read all of the Harry Potter books in approximately 3942 minutes or about 65 hours and 42 minutes.

So, Harry Potter, with 61k sentences, can be read in 65 hours. That’s actually pretty great for us because that means we can assume that the average person can read 1000 sentences in about an hour.

Now, let’s assume that, on average, one line of code is equivalent to one English sentence.

How long would it take an average programmer to read 200k LOC?

About 200 hours.

How long would that take assuming regular part-time hours of 20 hours per week?

About 10 weeks.

And I was given 4 weeks.

Code Is Non-linear

But it gets worse.

Harry Potter can be read linearly; the sentences are written in a strict order and can be read from start to finish.

Unless the code has been written by someone with OCD worse than Monk’s, code is never so linear.

Code is (usually) written in chunks that programmers call “functions,” and functions can refer to (“call”) other functions. The start point of a program is usually a function with a special name (“main” in most mainstream languages), and the order functions are in is usually somewhat orderly but still random. And the location of the start function is not guaranteed, either!

Imagine reading a book where the pages are in a random order and labelled with a name. Each sentence on the page either:

1. Tells you the name of a page you must read before you can understand the page you are on, but it does not give you the page number of that page,
2. Refers you to other sentences on the current page,
3. Does both of the above, or
4. Does all of the above with multiple pages and/or multiple other sentences on the current page.

Then imagine that you have to find any relevant pages with a text search. And imagine that the start page is not the first page, but is placed randomly in the book.

That is what it’s like to read code.

Suffice to say, reading and comprehending a line of code does not happen as fast as reading a typical English sentence.

Computers Are Perfectly Autistic

But it gets worse.

An English sentence has to communicate enough to enable common understanding between people. Slight imperfections in a sentence will reduce the effectiveness of a sentence, but by and large, it will still work, even if someone, or many people, do not understand the full implications of what was said.

Code is a set of instructions that has to be understood by people and executed by machines.

When I have to explain to people why it’s hard to understand what computers do, I give them this analogy: A computer is like perfectly autistic three-year-old that will do exactly what you tell him to do. And if you make a mistake, he will either throw a loud tantrum (crash) or continue on blissfully ignorant of the fact that he just destroyed your life’s work.

What this means is that, just like a computer is perfectly exact in doing what it’s told to do, a programmer must be perfectly exact in telling it what to do.

This means, in turn, that code can’t just be understood on a superficial level; it must be understood on a deep level, including all of the implications, some of which are subtle. And the reason that the implications are subtle is because, as I mentioned above, they can depend on not just what the current line of code is, but what other lines of code are, some of which are not even in the same function!

And the number of connections between related entities can be exponential! ¹

In other words, reading code is not just slower than reading English, it’s exponentially slower.

Software Quality

Of course, all TS really wanted was my signature on a page saying that the code was, to the best of my knowledge, good code. I am sure they thought that it would not be hard for me to assure myself of how good that code was.

For example, one of the tasks I was given was to “perform input validation against all input.” They surely thought that task would not be terribly hard, but it is. Since the amount of possible inputs is basically infinite, that would be close to impossible without formal methods.

Professionalism

The last thing TF doesn’t know is that I try my best to be a professional, like doctors or engineers are expected to be. I won’t put my signature on something unless I have checked it for actual quality.

Why?

As a programmer, my product is my reputation, and by putting my signature on a paper saying that certain code is good, I am putting my reputation on the line for that code. I am not going to do that unless I am absolutely sure of the quality of that code.

On top of that, TS is in an industry where privacy of information is important and may be mandated by law (I didn’t check). This means that, on top of ensuring that the code works as intended, I would also need to ensure that it also protects the privacy of the data it stores. That is a much harder problem, one requiring a specialist in cryptography and cybersecurity. Maybe someday I will be one, but as of now, I am not. And I think it would be a breach of ethical standards to advertise myself as such.

I also don’t think TF understands what it takes to ensure quality in software. (Again, this is because it’s not their domain of expertise.) For example, one of the tasks I was given was to “perform input validation against all input.” And since the amount of possible inputs is basically infinite, that would be close to impossible without formal methods.

Conclusion

I hope that this attempt to communicate why I would not take the work and pay offered by TS succeeded, and I especially hope that non-programmers can now better understand why creating software, and auditing software, is so hard.