The Future of Programming Systems - four thoughts
Last week I gave a 20 minute mini talk at M&I Partners in Zeist about the Future of Programming Systems, and how the work of architects and engineering teams will grow closer together in the future. I'm completely obsessed by this topic so I'm excited! Here is that talk expressed in a blog post, slightly modified based on some feedback and new thoughts.
Ok! Thank you for inviting me and sorry for not keeping to the original format as requested. I proposed this idea instead and it was accepted, so in part Matthijs (the organizer) is to blame and I hope it is OK with all of you as well ;)
When cycling here, I realized I needed to make a last minute change and added "Systems" to the title. I believe that limiting ourselves to the activities of the programmer is too narrow, and we should have solutions that involve everyone who is in the lifecycle of software. So it's about the System and that includes the designers, architects, users, database administrators, operators, and yes, also developers.
Your request was to talk about one of the classic texts of the software engineering / software architecture field. Now I didn't pick a classic, and I didn't pick a book. Instead I picked this semi-random blog post I found on Hacker News a couple of years ago.
Why this blog post? It's not so revolutionary or great that it is going to be the next classic work. But, it had some imagery that I just could not get out of my head. Open the link and read the first parts until the cave man animation. Here are some stills if you don't want to open the link for some reason.
So why did I pick it? Well, after a 7 year stint at Mendix, a Low-Code pioneer where I left in 2018 (see footnote 1), I am obsessed with better ways of programming. It was already better in many ways, but we still only scratched the surface with that product. It made me see the light, at least in my mind. This blog author seems to have some of the same ideas. Over the years I've collected bookmarks of all interesting hacker news posts about this field, and this is one of the top ones in the list. And it had nice visuals that made for a good story.
So let's go to the four thoughts.
Today we are going to explore four ideas that lead to the way I view the future of programming systems. The middle box will remain a question mark, because it is currently a quest that I am on and I do not have the answers yet. The ideas around it will give some kind of direction for the center box though. I think you'll get it at the end.
The first idea is that the way we program now is stuck in a local maximum due to the way that software development evolved.
The human body has an appendix. It is a useless vestigial* organ that served our predecessors and now sometimes kills us. That was the story at least, now science seems to say it is useful. But for today's story let's ignore that new science ;) The point is that software is riddled with historic vestigial artifacts. If we would start over from scratch, we could build something much better.
*Vestigial: [....] having become functionless in the course of evolution. - Oxford Languages definition
[frame 1 & 2] In the fifties we programmed by connecting wires between vacuum tubes to make our circuits. Then computers became more flexible with punch cards, and then punch cards could be written with high level languages like Assembly and FORTRAN. Yes, in those days even Assembly was high level.
[frame 3] Fast forward 40 years and we are still manipulating source code files with instructions. Or maybe not instructions if you use nice functional languages, but it's still very similar.
[frame 4] But programs today are much more complex! Here is some code that connects to the internet and lets another program do something by sending it some data. And this is just one example of things that became more complex. In your daily lives you see how complex it can get, from self-driving cars to ChatGPT to the software that controls our international electricity grid.
[frame 5] Nevertheless, a typical project is still structured like this. Folders with source code files, config files, makefiles, documentation etc. They looked like this 30 years ago and still do so today. The fact that "you can throw anything here and call it a project" is in my view a vestigial aspect of coding. It screams to me that no one is running the asylum. If I stick a random image in a random directory, very few IDEs or CI/CD systems will complain about that. Isn't that weird?
[frame 6] Another vestigial aspect of code is the fact that we still have to bother with tabs vs spaces!
(Audience member) - but.... that's now taken care of right? Everyone has linters and autoformatters?
No! The fact that a linter has to fix something that shouldn't be there in the first place is horrible!
It is like we have built a whole bundle of tools that we've connected to each other and compensate for each others leaky abstractions to get the situation under control. To programs we've added databases, logging, monitoring, error tracing, compilers, linters, formatters, ORMs, test frameworks, mocks, LSPs, and all of these things as separate tools. And now if we want to change the way we program substantially, we have to change all of these things! It's very optimized and very powerful, but it feels like everything has been glued on.
The point is: we've evolved and are now at a sub-optimal point. Let's go to thought 2.
The next point is that we can understand things much faster with the right medium.
Understanding is crucial for being productive, and it goes for both machines and humans. If an IDE recognizes what some code or config actually does, it can apply special rules to it. For example, the Laravel framework in PHP is highly structured in what Events, Database Models, and Routes the application has. The framework treats these as High Level concepts. It's really great! But, because it's just coded as Class-based PHP, virtually every IDE still treats the code defining e.g. Events as regular PHP code. The IDE just sees the characters, and maybe some of the PHP, but knows nothing about Laravel!
Understanding is crucial for humans too.
Here is a text based description of what a combustion engine does. Thanks ChatGPT!
Of course I much prefer a visual animation. Check this gorgeous website for this and more examples. I grasp what it does much faster and much more accurately with a proper medium. That medium took way longer to generate than some text, but it pays off!
Or when I'm asking for directions in a city. It's super confusing.
I much prefer a map!
When architects or developers are trying to make sense of a software landscape, the current mediums for that seem to be:
- reading powerpoint presentations by architects that are probably not at all grounded in reality
- walking to a whiteboard with some seniors
- reading documentation which is very likely out of date
So the second thought is: other mediums can often explain things much, and we should try to find the right ones.
Let's continue to the next point.
Programming systems were originally built for the silicon. Humans were cheap and computers were expensive. Of course this has changed significantly as now the situation is completely reversed. We have powerful IDEs and there are some programming systems that really tried to do better (Mendix, etc), but nothing mainstream goes far enough.
As a piece of silicon I just need to know how I can execute a piece of code through my circuits.
As a programmer I have all the questions on the right. Please take a look, I'm not going to repeat them here. I think virtually no IDE helps you with these questions. Yes, some have dead code detection, but there is a lot of false negatives and if you stop using one of the entry points of the application (an API route for example) and you forget to delete it in the code, no IDE will warn you about that. Your monitoring tool knows it's not used. Why doesn't your IDE? I suspect that the fancy programming systems in Meta or Google will help you with a lot of these things, but those are not accessible to 98% of the software engineers. VS Code also highlights who wrote a piece of code and when, but if a function got moved once over the last years, good luck! Again, a sign of a bolted-on technology, which fundamentally does not capture meaning but code characters stored on disk.
So, the conclusion here is that programmers have a lot of questions while programming that A) need a lot of discipline to do right and B) lead to a lot of manual work. They have unmet needs.
The last thought is one of hope.
I think software is truly truly special. It's in a category of its own if you compare it to any other industry out there. (Please note that I have not tried them all ;) )
These are some of the reasons that make software different and that make people fall in love with software. I fell a long time ago, and I still am in love with it. The last point is made incredibly well by Fred Brooks. I managed to sneak in a reference to a proper classic text of the software engineering field after all!
Because software is so inherently different we have the power to re-invent our programming systems radically.
So please, help me out by 1) not accepting the status quo, 2) keep an eye out for the innovators and try their solutions, 3) always look for better abstractions in your code and architecture.
The four thoughts once again. These ideas are central to my belief that the future of software systems will be very different from where we are now. In the future, programmers, users, designers, enterprise architects, etc, will be supported by much better tooling the caters to all of their needs. And maybe most importantly we will finally get rid of tabs vs spaces ;)
Like I said, over the years I have curated a long list of articles that resonated with the way I was thinking. I'm now considering starting a newsletter or series of blog posts with the best links of the month with my commentary or something. Then again, maybe that's just a distraction and I should just start building my own company that actually builds the future of programming systems.
Thanks again for having me and letting me talk about my passion. What questions do you have?
[ discuss on HN ]
Footnotes:
1. Mendix made a lot of things better through offering the right high-level concepts. I think criticasters would say that we were artificially limiting the developers. I would say that we put concepts like Forms, Database entities, Scheduled events, API endpoints etc as first class citizens in the development environment, so that it could help you. And help you in ways that make more sense than stupid-simple auto-complete. I love AI coding tools like Cursor or CoPilot, but the fact that they use text character based completion makes me mad as hell! It should be so much better!
Comments
Post a Comment