Introduction

This post will try not to be a mess, but it might be. It will be worth it, though; you will get some idea of how Yao looks in my head.

I will use all masculine pronouns for ease.

Process

To start, I want to talk about how I have designed Yao with regards to the process of software development, since it is the process that I started with.

When a programmer sits down at an IDE to start a new project, I envision him starting in a Python- or Ruby-like language. He may be intending to use that language for just prototyping, which means that he intends to rewrite the entire project later, probably for performance reasons.

Well, I want that language to be Yao; developers should be able to do fast prototyping with Yao.

But why must they rewrite the project in a different language later? Why is that even a thing?

(Probably because language designers have been too lazy to make their languages reach across the performance/prototyping spectrum, but what do I know?)

Prototyping

Here is the problem: a prototyping language must be flexible and fast to type in. It must be runnable without long compiles. Maybe a REPL is necessary too.

What makes Python fast to type in? Most dynamic (non-static) typing language fans will say that it is the lack of type annotations. Even as a static typing fan, I have to agree with them that it makes a significant difference. Maybe the significant whitespace does too, but that is a non-starter for reasons to be brought up later.

So…no type annotations. Sad.

And…Yao must be interpreted. You must be able to run it instantly, which means that the interpreter must also be wicked fast.

And maybe a REPL.

Actually, a little side quest for me is to make Yao the best choice for many types of tools mentioned in this post. The ones I am going after are:

• Generic build system (for Yao code).
• Scripting language (making it easy to prototype in will solve this).
• Shell language (yes, I am going to make a shell based on Yao).
• Systems language (more on this later).
• Workhorse language (by combining the four above into one and by having the best standard library of any programming language).
• Pocket calculator (Yao will have arbitrary-precision math and an extensive math library).

And in order to make a Yao a good shell language, I need a solid REPL. So there will be one.

Rewrite?

So, say Yao is made, and it is a great prototyping language. Our programmer has finished prototyping. Where does he go from here?

Well, if he is satisfied with the quality and performance, he can stop there. If he is not, he has a couple of choices:

1. He can attempt to get better quality and performance out of the prototyping language.
2. He can rewrite the prototype.

Sometimes, (1) works. When it does not, (2) kicks in.

In any other language (or most other languages), he must basically discard his work. Yao will not require that.

Towards Better Performance

How will I do that?

Well, remember how I said earlier that part of prototyping was to use an interpreter? To get performance, it is always better to compile (JIT still counts here) to native code.

So Yao will also have a compiler.

But surely, all that dynamic typing would defeat the optimizer and require a lot of expensive boxing. Yes, so it must be gotten rid of, with type annotations.

What?

Here’s the deal: dynamic typing is only good for speed of development, and only when a project is small. Static typing gives a lot of benefits, including better performance (because the compiler can make more assumptions) and better documentation of assumptions.

And it is that last one that makes development faster on large projects with static typing. Dynamic typing enthusiasts can’t really deny that. (And if they do, they are in denial.)

The good thing about the end of prototyping is that quick and dirty changes are not needed anymore; now, the code can start to be solidified into something more concrete. Yes, big changes might be a little harder (made easier by a good IDE), but at that point, big changes are not terribly necessary.

Yes, they will sometimes be necessary, but that is why good tools are important too.

And so, instead of rewriting his prototype in another language, the developer simply adds a simple build file (aided by tools) and starts adding type annotations. He does not finish right away, though, because Yao can still work with partially dynamically-typed code.

And from then on, he adds more type annotations where he would like. Also, whenever he has to add new features, he can still prototype them, since dynamic typing is still allowed.

That is the real reason for Yao.

Design

So let’s see how Yao’s design will accomplish this.

Any

First, Yao will have an Any type. This will basically be the boxing supertype. It will know the real type of the item that it is boxing, just like Python does.

Also, Yao will use type inference where possible, and it will automatically check types and unbox at runtime, where necessary.

Reflection

The type checking by the Any type will be made possible by support for complete reflection.

Heterogeneous Data

By default, heterogeneous data will be contained in an open data type.

However, there will be structs. In fact, Yao will go even further and allow users to create new ways of laying out heterogeneous data.

Numbers

Yao’s default number type will be an arbitrary-precision BigDecimal type, which will be implemented in the standard library. This is the same type that will be used for the calculator mode. This will be for reducing bugs related to precision, rounding, and overflow.

Yao will also include the standard integer and floating-point types, of course, and they will be selected by using type annotations.

Yao will also have its math operators (for integer and floating-point types) trap by default. However, like Swift, there will be wrapping operators available.

Logical Operators

To reduce bugs, the default logical (||, &&) will be non-short-circuiting, with short-circuiting ones available.

Compiler Library

For security reasons, Yao must be independent of C, so it will be implemented by a compiler that will be written from scratch. This library is called Yvm, and it will make many of the features of Yao possible, from the compiler to the interpreter to the build system.

Undefined Behavior

Also for security reasons, undefined behavior is unacceptable. Yao will have none.

Assembly

Yao’s unsafe will be assembly. The assembly will be in Yvm’s syntax, mostly.

This is to discourage its use (most programmers don’t like to touch assembly), except for when it actually is needed.

Memory Management

Memory management will be automatic by default. It will be automatic reference counting (ARC) with a cycle collector.

This will allow for automatic memory management across the board while also allowing fine-grained manual memory management.

How?

The cycle collector will use an algorithm that will use type information from reflection to figure out if a type can even be in a cycle. The compiler will be able to use that, too, so if a developer wanted to make sure that a particular type was not going to be managed by the cycle collector, he could tell the compiler to make sure that it could never be in a cycle.

This will go even further.

Like Rust, Yao will have “owned” and “borrowed” pointers, and it will have compiler passes to figure out if something illegal is done with them. However, they will not be default.

Also, unlike Rust, the standard library (and all Yao code, really) will be written such that functions don’t care what kind of pointers they get (except in cases where it should matter). Pointers will instead implement an interface that functions will expect instead.

Of course, the compiler will be smart about this.

Anyway, it will go even further. Users will be able to define their own pointer types, with their own automatic memory management, subject always to the reference counting (so that developers can’t just override it). And the standard library will recognize those, too.

User-defined pointer types are called type extensions.

Exceptions

Yao will not have exceptions. They are toxic.

Instead, Yao will have conditions and restarts, of Lisp fame.

Modules

Modules will basically be Clojure namespaces; they will be actual “objects” themselves that can be queried.

Mutable Data

Mutable data will be implemented as a type extension.

Symbols in Names

Yao will allow symbols (punctuation) in names, after the first letter (like Lisp). This means that postfix and binary operators will have to be separated from names by a space, but they usually are anyway, and it allows more descriptive names.

Operators

There will be user-defined operators, like Swift.

Syntax Shibboleths

Yao will use braces, not significant whitespace. The reason is that significant whitespace will always defeat an automatic code formatter.

But it will not require semicolons.

Conclusion

So, that was a brief dump of ideas I have for Yao’s design. With that done, I can finally get to work on it and Yvm.