In the software industry, three years is a long time, except in one area: paradigms. And I think the industry is about to experience a paradigm shift.
The new paradigm is called Structured Concurrency.
Why is it the next biggest paradigm? Well, let’s talk about that.
According to that Wikipedia article, it was first named and formulated in 2016 by Martin Sústrik (creator of ZeroMQ) in a blog post and an update and then refined and championed by Nathaniel J. Smith in the seminal post “Notes on structured concurrency, or: Go statement considered harmful”. Martin implemented it libdill, and Nathaniel implemented it in Trio.
It took until Nathaniel’s post for people to sit up and take notice, but notice they did. And that’s about when I ran into it.
I remember reading Nathaniel’s post and immediately knowing that he was right. And that it was the next biggest thing that would actually prove useful. I was determined to be a pioneer and jump on that ship before it had a helm, and I knew that I could make one.
Yao is that helm. Yao is going to be the first programming language where structured concurrency is the only concurrency paradigm. Yes, there will be asynchronous I/O and things like that, but if you create a thread in Yao, it will be done by structured concurrency, and only that.
The reason for this whole-hearted leap of faith is that I believe Nathaniel’s argument that structured concurrency will do the same thing for concurrency that structured programming did for programming in general. We all know how it took over all software, and I believe that structured concurrency will do the same, for the same reasons: it makes software easier to reason about.
Such a simple thing, but it makes all of the difference, especially when some software is beyond any one human’s comprehension.
But there is a problem: structured concurrency doesn’t have a good definition. That has made it hard for Martin, Nathaniel, and other people like me to talk to others about it. And I would like to change that.
Here is my definition:
- structured concurrency
- A programming paradigm wherein threads are encapsulated in, and do not leak out of, the last scopes they are contained in.
That’s it.
I think this definition covers all of the bases:
- It covers the usual case, where threads are created in a scope, like the
with trio.open_nursery() as nurseryconstruct in Trio. The threads are in the scope and they do not leave. - It covers the case where threads are contained (“encapsulated”) in an object that can be passed around because the object is the scope. When the object goes out of scope, the threads are joined. If that does not happen, then it is not structured concurrency.
- It covers the case where a thread itself is a first-class object and can be passed around, if it is joined when the program is exiting the last scope it is contained in.
If numbers 2 and 3 seem weird to you, they are the structured concurrency equivalent of creating a closure and passing it around. There is state that had a scope, and that scope is just being moved around. As soon as the closure itself goes out of scope, all of the scope inside is also cleaned up.
This all indicates two things: first, threads need to be first-class objects in a language with structured concurrency, and second, pure structured concurrency is only practical in a language with automatic resource management, just as closures are only practical in such a language.
I think that is about it, except that if this definition does not cover all bases, or is defective in some other way, please let me know.