Interaction Nets have been quite captivating to me lately, even though I don't understand much about them recently. I mean, their features seem quite interesting from my perspective, and I understand what I did wrong in Tarlox.
What was Tarlox? It was labor. A programming language I developed by locking myself in my room for two or three months. To be honest, I can't say it's a 100% original programming language. Think of when Bjarne Stroustrup (was this how this guy's name was spelled?) first released his "C with Classes." There's a development that's more like a few additions to C rather than a language. This is like that too, containing a few additions to the Lox programming language created in the first chapter of the book called "Crafting Interpreters."
So what are Tarlox's additions? Open the Github page and see. But what I like most is this: Parallel variable calculations.
I believe that when a variable is assigned, its assignment should not interrupt the flow of the code. For example, look at this:
var variable1 = at_least_one_minute_function();
var variable2 = dont_die_my_donkey_die_function();
Let's assume that calculating variable1 and variable2 takes two minutes each. If we're running this code in JavaScript, since variable1 and variable2 will be calculated sequentially, let's assume it will take a total of 4 minutes. Of course, no sensible programmer would settle for this and would probably run two threads to calculate these two problems. But having both of these threads process separately means including a truckload of synchronization tools in the code. In Tarlox, both variables are calculated simultaneously and neither blocks the main flow of the program until they need to be accessed. If you want to avoid blocking, you can query whether variables are ready using the "is_ready" operator.
Actually, I think this is quite a useful tool for network operations rather than just parallel computation. After all, an additional thread doesn't necessarily mean "parallelism." A function assigned to a variable might well be waiting for a response coming from the network.
In such cases, the preference is usually "async/await." Not a bad idea, certainly. I guess there's something that people who designed and enjoy using Async/Await know. Still, I honestly don't like Async/Await. Even the abomination called Async Rust could be reason enough to dislike Async/Await, but I won't take such an easy way out. Async/Await, like "exceptions," is something that complicates the flow of code, makes it incomprehensible, and makes it difficult to follow the deterministic idea of "This should be done at this time, this should happen at that time." The flow of code should be predictable.
I can't pass without appreciating Erlang on this matter. With its own "preemption" mechanism, let's describe it as "the mechanism that decides which thread should process when," it allows waiting threads or threads continuing to work in parallel to work without sabotaging the main flow. Moreover, it can do this without communicating with the operating system.
My goal in Tarlox was to make concurrency/parallelism the default behavior by loading parallelism into something as fundamental to the language as variables, and to control it through basic tools. But of course, since we have work to do and I didn't want to deal with how a preemption mechanism works, I used the operating system's threads and actually did something a bit unperformant due to "context switch" costs (I'm talking about the cost between the operating system and the program's execution). Actually, we can consider this a choice too; the preemption mechanism is a structure that complicates runtime. Maybe we should leave the operating system's job to the operating system.
But the problem is that ultimately the solution set I have is built only on forcibly creating something. For example, there are currently indefinite deadlocks in Tarlox. The reason stems from the concurrent HashMap library I used. For instance, this was a problem that quite confused me:
var x = x + 1
Looks easy, doesn't it? Tarlox was deadlocking exactly at this point because it needed to access x to assign x's value. The HashMap library I used either expected me to constantly query "is it available?" in such situations, or if I continued without querying (we don't have the luxury of throwing a spinlock... Okay, maybe I should have been more open to using condvar), it preferred to wait directly until the conflict in between was resolved, which... was impossible under this condition.
I got out from under this, but I can't say the solution I found is very good, and Tarlox will definitely deadlock somehow in the relationships between variables under complex scenarios.
As I said, there will be things I can work on and deal with here, but we're already heading toward a point where everything consists of ugly solutions that save the day. Of course, I could build my own HashMap library to store variables, make Tarlox transform into a structure that forces and overwhelms the user with rules for accessing the environment where variables are stored, while trying to do good. (Why should I throw shade at Rust? We love Rust. Arc<Mutex<...)
Of course, I researched programming languages that try to do what Tarlox was trying to do and found Chapel. Chapel is an amazing language in terms of parallelism, its syntax seemed quite readable to me, but I personally concluded that it's quite difficult to learn. So I thought this was inherently complex due to the nature of this work and gave up without fundamentally turning back.
I didn't think about this topic for a while because I was bored and gave my attention to different areas for a while. I had also convinced myself that this issue was unsolvable, meaning that the best thing that could be done wasn't actually more guaranteed or better than classical solutions. However, for the parallelism language I designed in my mind, which was more proper than Tarlox, I had established these rules:
- There should definitely be no mutability in the language. New values should be produced, then old values should be cleaned from memory, or while the programmer produces new values, old values should simply be updated in the background. (Interior mutability)
- Information about the current state should not be somewhere. Everything should be created as new values and functions should also send new state information to each other.
- Functions should not create side effects no matter what.
- The solution to repetitions should be recursion, not loops.
Of course, it's no coincidence that these conditions remind me of Haskell. Since I don't particularly like pure functional logic, I have an immutable object-oriented programming language in mind. This is one of my long-term goals.
Later I saw HVM being mentioned in one of the Rust groups. When I first examined it, there was no Bend programming language on HVM, it was described as an alternative VM to Haskell. (Haskell... Its nature is quite suitable for parallel programming) Its features have things that would quite entertain a parallel computing amateur like me: It allows parallel computation by its own nature while doing calculations. For example, this expression is calculated in parallel:
(f(x) * g(x)) + (f(y) * g(y))
Here's how parallel computation works: both sides of the addition operator can be calculated in parallel first, separately and in parallel. Each side of the multiplication operation within the operands of the addition is also calculated in parallel. Thus, a total of four different threads get up without us saying anything.
Here's the thing: while Tarlox works on variables, it actually locks up during parallel calculations of data to be saved to memory and during access to parallel-calculated memory areas, and even if it doesn't lock up, it produces unpredictable results. In contrast, HVM eliminates this problem much earlier by dividing expressions into parts that can be processed in parallel.
Besides this, HVM doesn't need memory control. This means it doesn't require a garbage collector like in Java, memory management like in C, or an ownership model like in Rust.
Understanding this is very difficult at first because HVM focuses on the very, very foundation of the problem. The problem is this: The widely used computational model is essentially the Turing machine from the 1940s. RiscV processors that can perform very simple operations to Ryzen 9 processors essentially model this in their core. Today we use RAM memory instead of punched paper, and transistor technology with silicon allows us to model this machine at the nanometer level. However, the Turing machine model has some gaps:
- Since it's based on unlimited papers (read as memory), it doesn't make predictions about how these papers will be used effectively. Poorly configured instructions can cause paper waste because it essentially brings no limitations.
- The Turing machine has a single head that will read and scan the papers. Multiple Turing machines can work in parallel with each other and process compatible instructions, but the synchronization of this remains an additional burden on the person providing the instructions, namely the programmer.
The problem in programming and generally in all engineering is not that something cannot be done, but that when it's done, it's not understandable and simple at a glance. Every solution that a smart person finds in that complexity corresponds to extra time for that smart person to remember what was what when they return later. The solutions we obtain should be as close to mathematics as possible so that anyone who knows mathematics can understand. Of course, the person talking about this is a Psychology graduate who last saw mathematics in high school, please excuse that.
Speaking of Psychology, let me open another topic. By human nature, in Psychology we cannot represent people with a mathematical model. If we can, the society we live in has turned into an entropic society like in Yevgeny Zamyatin's novel "We." (Asimov's Psychohistory in his books is both a sociological topic and doesn't seem realistic to me.) Therefore, each person needs to be dealt with as a special case. This means an endless workload in itself. We need to show that software development doesn't resemble this.
The Turing machine models how a human thinks. Therefore, despite being mathematical, like human thinking, it doesn't require the instructions coming as input to have a mathematical nature, and creates the danger that each "instruction list" (i.e., program) becomes a world to be discovered from scratch, just like humans themselves. Of course, this is a thesis and I might be exaggerating a bit.
Tarlox's problem lies here. There's no meaningful mathematical model at the core of parallelism logic. If we could somehow concretize the computational model, we would see Turing machines reading the same part of the same paper sequentially. For these Turing machines not to stumble over each other in the simplest operation, the person preparing the instructions would need to show special sensitivity. So we can reduce the possibility of them stumbling around with a bit more optimization, but still, the person who will provide the instructions would need to make extra effort.
HVM, like every VM (whether JVM, Erlang VM, this or that), models a processor, but this time it represents a completely different computational model, not a Turing machine. That model is Interaction Nets. This is a computational model that emerged in the 90s, and instead of taking a paper and modeling operations, it represents each value with a function and shows its new value through the target function by simplifying it with a certain method. Here you can think of "Lambda Calculus." In Lambda Calculus, each number or value has a functional equivalent. The encoding that Alan Church, who gifted us Lambda Calculus, used for the number 2 is something like this:
(function_that_gives_one_value, function_that_gives_zero_value) => function_that_gives_one_value(function_that_gives_one_value(function_that_gives_zero_value()))
If I write it with Lambda notation, which is more elegant than JavaScript notation, it becomes this handsome thing:
λx.λy.x x y
By the way, friends without functional programming background might think that every function must have a result. What we want here is not to reach that value but to have a function representing that value in our hands.
Interaction Nets also operate with this logic. Of course, beyond this, they operate with "agent" things. I honestly haven't been able to understand it. I need to focus my attention and understand its content more deeply.
So what excites me? I may have said that I don't particularly like "pure functional programming," but having "pure functional programming" be the processor itself is something completely different. I also see the future of processors and computers in this computational model. A new processor can be built on this logic, and probably a processor with thousands of cores like GPUs but capable of doing very different things than GPUs can be built. Embedded systems can be designed with an assembly resembling a simple version of Haskell, and even real-time systems requiring no memory control can be designed.
This of course means a computing revolution, but it will be difficult to predict who will carry out this computing revolution and how. What will computers look like? What will we do, what will we program if the Von Neumann style disappears? If C means what it means for today's computers, will Haskell have that meaning in the future? It's a bit difficult to foresee this.