Mechanicalism in the real world
How does one exhaustively scope software development, so that it can be broken down into steps?
Although I haven’t made much fanfare about it, recently I have become probationally employed by a virtual reality startup called Unai. I want to thank Lewis first of all for opening the door to that opportunity for me, as well as Maxim and the rest of the team for actually bothering to read my work and take me seriously. To count how many times I have been skimmed and brushed aside by prospective employers makes this kind of thing all the more of a blessing to me, and blessings I very much count.
During the course of my onboarding, I had spent some time with part of the team just discussing whatever was on our minds or what compelled us. Naturally, I had to attempt explaining a lot of my most seminal theoretical research by analogy and example, and for one reason or another, it did not go as well as I had expected. In this post, I will complete what I had told Maxim I needed to do back when I had first spoken to him about this outcome several weeks ago: grounding my theoretical work in the real world by painting a day-to-day picture of what this sort of thing can look like, how that builds up over the long-term, and how it differs from an admittedly more functionalist conventional approach to software development.
This work of mine has been the culmination of several years of stubbornness and deep thought, with a lot of firsthand and secondhand experiences brought together to attempt to (a) coherently explain why software development as we know it is broken, and (b) find a sustainable way of approaching it. Relatedly, this has led me to create something I call Project Anodyne, which is in cryo for the time being as I attempt to acclimate into building things at Unai instead.
One of the most important experiences I had for grounding mechanicalist thinking was with a friend of mine and fellow security guard named Hayden. (If you’re reading this, hi! I hope you’re gainfully in game dev by now!) I met him in community college at Wake Tech, as I was in a default sort of Associate’s transfer track, and he was in the school’s game dev programme, SGD. While he was being taught Unity and all the conventional tools everyone uses in the industry, he came across my independent work that I was doing in C at the time, and found a strong interest and appreciation. He asked me a lot of specific questions about the system’s fundamentals and I answered in kind. This led to him having a positive appreciation for how approachable my code was compared to Unity. Even though he had never studied C in his life and was barely proficient in C♯ at the time, he could comprehend everything significant about what I wrote, including the internals, as I had documented things so thoroughly and backed them up with further reading materials. While he may not have known it at the time, he was looking at an early product of the new frame of thinking I was developing and would later come to call mechanicalism. My lesson from all the conversations I had with him was that if I did a good and thorough job and thought through the things I make, anyone who wanted to could wrap their heads around it and do the same things I did. At the time, Hayden was taking community college classes while still in high school, and had little more to his name than a good attitude and a personal passion for game dev. He was such a model figure for consuming this work I was doing that I could never forget it.
Once upon a time, software development followed a very different cadence. In a time when software was shipped on diskettes and even optical media, making sure that code was well-written was absolutely imperative, because it couldn’t be patched after the fact. Indeed, as late as the 1990s, we see the perilous consequences of shipping bad code in the form of litigation against Microsoft for their DoubleSpace software, what with its infamous propensity to delete important data. Decades prior, IBM was pioneering the enterprise software market with their System/360, a flagship selling point of which was their unparalleled warranty regime. IBM acknowledged that writing software was expensive, and wanted to own the game by selling a product that would run what you wrote for it until kingdom come.
Or well, that’s kind of the truth. See, when I said that IBM found writing software to be expensive, I kind of lied. More exactly, authorship is expensive, while writing is cheap (copy that floppy and see just how cheap!). Even more exactly, this is one of the key conjectures of mechanicalism! In functionalist thought, the contrapositive is true: authorship is cheap, while writing is expensive. This is why we see an obsession with “not reinventing the wheel” manifesting in an algal bloom of third party code, surreptitiously copied around and maintained by foundations and corporations. In functionalism, the better call is importing LLVM; in mechanicalism, the better call is writing a simple weekend parser, like Ken Thompson did in making UNIX way back when. This is because functionalist programmers are expendable compared to their code – what was discovered? Never mind who discovered it, that’s for the footnotes and the accolades, if there are any. By contrast, mechanicalist code is expendable compared to its authors – how well was this made? That is a product of craftsmanship and care, not novel discovery.
Why does this distinction exist? As I explained in my introductory article for Functionalism and Mechanicalism, each broad branch has different strengths and applications:
Functionalism is the sole cornerstone of experimental research in computing. It is the method for approaching unsolved problems, both imaginary and real world, thanks to its unparalleled ability to model in the abstract.
…and likewise:
Mechanicalism provides the sole basis for creating well-behaved and fault-tolerant systems. This makes it the only paradigm suitable for mission critical computing, such as that involving finance or military use. The concept is also indispensable for approaching bare metal silicon in order to maximise performance – this was demonstrated in the weird architecture of Parallela’s Epiphany-V processor. Often times functionalist systems are built dependent on rigid assumptions about the underlying real machine that makes their runtimes unable to cope with such architectures as the Epiphany-V. This is an obvious loss of capability on their part.
The reason I find these terms so relevant is because the overwhelming majority of code being written in the world is uncritically functionalist, and most of it benefits none from being written this way because it is not written in a context of coming to know the unknown.
The simple truth is, we have trillions of dollars sloshing around the world to maintain tools that are in very well-understood domains, such as web pages, source control, all kinds of basic algorithms, rendering, networking, and more. We know all of the important parts of how to make something like GitHub or GitLab; why in the hell are we continuously employing thousands of people to maintain this as if they are exploring anything new? It has more to do with a mandate from venture capitalists to “move fast and break stuff” than it does with the core business of such sites, which is providing a public service.
Functionalism is inherently unstable and is only worth approaching in a context of discovery and analysis. We need to confine the school of thought to tasks such as machine learning, language analysis, data science, exploratory creative work, and the like. It’s just not the way to build a web-based software service or an operating system. The continuous operating costs of electricity and real estate are burden enough – our problem with big tech is also a problem of the fact that no one else can afford to compete. When social media sites turn politically sour and flex their total lack of public accountability, courting government regulation out of sheer austerity, remember that it’s only possible because of how everyone writes code.
Of course, breaking out of it is a lot easier said than done. The main hold-up is that software is extremely horizontally integrated to a point where it is to a large extent pointless to switch away from because other layers that the software depends on will not also change in kind. You can write your web server code in an exhaustively researched mechanicalist way, but if Linux is going to keep changing under your feet – and all portents point to it very much doing that – then it doesn’t help you much unless you stay on a fixed version of the kernel, with all of the potential for security vulnerabilities that you don’t have the resources to address. See how this explosive complexity bites everyone in the ass? You can’t get away from it. You are forced by the greater environment into maintenance of your code until the end of time. You probably can’t afford that.
This is a moral catastrophe for two reasons: (a) it stratifies the power of software development, concentrating it into the vanishingly few hands who have the money to hire infinitely large teams of developers to build & maintain what they want, and permanently shutting out individuals and upstarts from meaningfully changing the face of software at all, and (b) even if you have the resources to overcome this, you are still increasingly limited into the arbitrarily-chosen limits of existing precedent, which shrinks the possibility space of all computing irrevocably for no purpose or benefit to anyone. Even billionaires will be forced to accept calling into LLVM as a simple fact of life, because they are forced to do things in the functionalist way by inertia, leasing all of the guts of the things they create from opaque foundations that get funded by incumbents for the sake of their own monopolistic entrenchment. Charles recognises this as a local minima, but this is also pretty dystopian, isn’t it? I discussed this angle in depth here:
Worse yet, you don’t even get the best kind of functionalism: because everyone is depending on LLVM in this way as a poor crutch for the mechanicalist work nobody is doing, it is vanishingly unlikely that you will ever see something that meaningfully differs from it in architecture serving a similar purpose. Was there some other kind of compilation you wanted to experiment with? Maybe you wanted to study data visualisation in programs like my friend Charles does. That’s too bad, because nobody is funding a toolchain that will be good for strange stuff like that. You’ll just have to fit that square peg into the round LLVM hole like everyone else, even though you aren’t even directly trying to do anything mechanicalist here. LLVM has to be good for All Use Cases™ through genericism, making it a jack of all trades and a master of (almost) none. This is why a Bronze Age style collapse is one serious implication of the continued ignorance of these broad branches:
This is what I have come to call a dependence on convention, which not coincidentally is the most major (and probably fatal) flaw with the increasingly notorious software stack called Urbit:
Okay, so there is much to negatively anticipate if this approach is not taken at least sometimes. Maybe you are sceptical of that kind of angle. What is there to positively appreciate about it, then?
I have already pointed to one of the things, which is Anodyne. Charles’ work on parallel processing is poised to be multiple orders of magnitude faster than conventional GPGPUs, putting it in the same ballpark as AI accelerator ASICs in terms of operations-per-second and energy efficiency. The difference is his chip is capable of general-purpose computations while the ASICs are solely built for doing massive matrix multiplication. He has already predicted that this will beckon a paradigm shift of algorithm design, opening up developers to many algorithms in exponential time complexities that are normally avoided due to the poor architecture of conventional processors.
If the Epiphany-V is any clue, these kinds of chips will not be running Linux any time soon, if ever. They are comparatively strange, and require a lot of special care and vertical integration of software to flatten out a lot of complexities that serve no benefit on machines like this. You might take file systems for granted in all of your innumerable experience with software development, but you would be shooting yourself in the foot to expect that kind of thing to be available here. It just doesn’t make sense.
Finally, something like Anodyne is necessary in its completeness, simply because we need to walk this kind of work all the way to the product, so that the general public can actually monetise and complete the cashflow cycle that makes any company profitable. It is simply not realistic to sell a portion of the device to any outstanding effect because of the qualitative regression to the mean I discussed before.
But maybe you don’t know about Anodyne, or for whatever reasons of brevity or interest you just cannot grok enough to believe that in its fullness. This brings me to the other major consequence of mechanicalist programming: the end of maintenance.
If approached properly, code written in a mechanicalist bent will never need to be maintained, any more than a book needs to be maintained once it hits the printing presses. You might have a minor blip here or there, but overwhelmingly the software is done, it does its purpose well, and it is totally open and documented for the perusal of computer scientists for a thousand years to come. You are free to engage in the wholly positive and additive work of revising new editions at your leisure, instead of the drudge of maintaining something just to keep it in the state it was already in. It is not an archaeology assignment in the making, not for people generations from now, and not for the author themselves six months down the line when they have forgotten about it and come back to be as clueless as anyone else would be upon encountering it.
The implications of the end of maintenance are as liberating as they are immense. The painful agile software development methodology is rendered totally redundant, as sprints turn into a motif that do more to kneecap thoroughness than they do to effect well-written code. Stand up becomes a pointless formality as the whole orientation of development reverts to a process more resembling waterfall, rigorously sequential and thorough with little to no backtracking on a purely technical level. Requirements get laid out well in advance through thorough modelling by management, to the point that one could predict all kinds of tangential trivia about the engineers’ workdays down to the calendar day.
Most crucially, the need for ever-more developers to develop ambitious projects simply disappears. In the blindly functionalist way, a company’s developer head count is kept at a close parity with how much complexity they are dealing with in the course of their product development. Instead of this ridiculously capital-intensive approach, highly gifted computer scientists are free to soak up all of the relevant complexity into their brains before they write a single line of code. They can then pen deep and exhaustive theses about all of the parameters of the product development job at hand, including listing all of the priorities and shortcuts and making decisions together with management on what to approach when, and how to choose features that effect worthwhile prototypes for company goals. This is not much different from how prototyping is done in materials science or other kinds of engineering – do we make the device in balsa wood? How about 3D printing? What do we need to be in metal? How much do these things all cost? How are we handling clearances and tolerances and how does this all feel? What can I live without for this prototype and what is really, truly needed?
In the conventional, cargo-culted functionalist approach, this may seem to be what happens as well, but in the details it is not the case. If we approached materials science the way we do software systems programming, it would be akin to copying the latest titanium nanomaterials from Boeing, just because copying is free in this analogy and there’s no reason not to because “it’s the best”. Even if you could just borrow exotic materials for free in unlimited quantity, you can probably appreciate a lot of the other inevitably strange and undesirable consequences of working with such things. It may be hard to work with, or require special shipping constraints. It might just bend more easily in your hands than aluminium foil wrap, which even if you have some magic Swiss Army knife for holding the thing, is pretty darn silly compared to just using something normal like aluminium, isn’t it? Does it really need to be state-of-the-art for a prototype? Can’t we all kind of get the important parts about your thing, which have everything to do with the geometries you authored and little to nothing to do with the Boeing nanomaterials you had the luxury of creating it with? There are basic realities about the state-of-the-art that just aren’t supposed to apply to the laptop case you might be making, or the toy robot you always dreamed of, and that’s okay.
I often enjoy likening this kind of thing to a clip from The Simpsons where Homer is seen to be house sitting for Mr. Burns. When Marge is introduced to the house, Homer shows her his sleeping chambers, where with the pull of a lever, the bed is dropped through a trap door into an incinerator, and a new bed is fed in through the wall. Homer proudly exclaims that “the bed never needs to be made”:
In a virtual context, this is something we do every day with tools like Docker and Kubernetes. It’s “easier” to just incinerate the old and remake a new one automagically, except this is kind of insane.
You might say that this isn’t a fair comparison because unlike Mr. Burns’ bed, software is free to duplicate and destroy. The problem with this is that one guy a long time ago was tasked with “making the bed” once and for all, as he put in the master blueprints for the bed generating machine. You are busy exclaiming about how great it is not to need to make the bed, while slowly the very knowledge of bed making itself is eroded and eventually lost. Like Joe Rogan once said in his 2006 standup special on the topic of “what if all the smart people disappeared”, you just assume that someone, somewhere simply must know how to fix it. This is true, until it isn’t. If those people retire and die and you’re none the wiser because you took it for granted, the knowledge will be lost, forever. That audio clip is highly succinct, by the way – I strongly recommend listening to the whole four minutes! It’s at least some food for thought in this vein.
A secondary consequence of never needing to “make the bed” is that no one is compelled to think about how better these things might be made. Admittedly I am stretching the analogy at this point, but suffice to say that being able to explore and innovate is a hugely important task in computer science, and this is a situation where such work has been kneecapped in service of lazy and grossly excessive automation of the mundane. For example, the thinking you might want to do about compilers is made artificially harder by the entire field’s insistence on forcing you into the box called LLVM and libclang – and if you want to go it alone because there’s just no other way, you will be very, very lonely and lost. It’s a travesty that our existing corpus of code makes it so much more difficult for fellow scientists to collaborate freely, because it is constantly forcing them into a mode of work that does more to profit Mark Zuckerberg and shareholders of Google than it does to move computers forward for the entire human race.
So yeah, not having to maintain code is a ridiculously immense load off of all of our backs. We can actually afford to do new things with all of this time we gained, because whatever isn’t new has already been done in such a studious and thorough way that we don’t need to worry about maintaining permission from the people who wrote it, else forking it off in some hopeless way that we could never hope to maintain. There is no political angle to using other people’s existing works. You don’t have to be within the guidelines set by a foundation, and you don’t need to involve them if you make some kind of major changes, or if you want to use their work in a way that they never intended it to be used. Liberty is fully restored to the task of software development, it doesn’t cost you any extra time out of your day, and so everybody wins.
Lastly, I must briefly explain my personal angle about all of this. I had endeavoured for years to discover and term these things because I knew that something was wrong with how we were all accustomed to developing software. It was brittle, and buggy, and nobody talked about this seriously, so I simply had to figure out why. And because this is, to the best of my knowledge, the right way to approach software development, by virtue of it being sustainable and useful to people without my involvement or consent even far, far into the future, and because it allows me to create code and move on from it with 100% of my attention, it is a moral imperative for me to develop things in this way. If I don’t, I won’t even succeed in getting very far because I will be trapped in a never-ending maintenance cycle of my own creation. This is why I have an unshakable personal mandate to create something like Anodyne, along with all of the tools that build up towards it like Hinterlib and Alabaster – it is simply required before any of the free form, “real fun” can begin where I create things like video games and SaaS apps without worry for the engineering that underpins them. I have attempted to forsake this in the past, in the interest of expediency foisted onto me at profit by others, and it did not work out for me.
As crazy as it might seem, for better or worse, I must continue this kind of work in some shape or another. I personally found Anodyne in partnership with Charles, but there is no reason that this work cannot manifest in other forms, such as in my work at Unai, for instance. I am a terrifically convicted person, and while I am plenty mature enough to set aside my own visceral or personal feelings about things (and I indeed did this a lot in exploring my research), I simply cannot bring myself to do things that I know are wrong. Perhaps this is a curse, but it is one that at this point I must bear, whatever the price. I have to keep going with these things because I know that they are immensely, game-changingly powerful, and because I know I can get through the thick of it all soon enough. Most of the bedrock of my research is done; now I am just finding the path by which it will light up the world. It’s ridiculously hard, especially when people don’t understand or don’t know me by name, but I am doing it anyway. I don’t blame anyone for that, and it being my burden is why I was so compelled to write this article in the first place. I will write more and more until everyone knows all of the details and we can all start doing this together.
Thanks for reading! This article was a long one, but I made it a freebie because I noticed a lot of you have subscribed recently for free and I want to give you a proper demonstration of how my Substack goes, beyond a retroactive perusal of the archives. If you would consider subscribing it would help me out a lot; I ask a nominal $5.55/month, or an annually billed $55.55/year (one month for free), as this helps me stay independent and focussed on my work at hand. No matter how much anyone sends, nary a week goes by that I’m not thinking through all of this stuff and spurring discussions with colleagues anyway. All of your support is greatly appreciated.