The software that I love
Pride. When interviewing candidates I always ask "tell me something that you are proud of." Sometimes you get a blank stare. That person is down and dealing with some shit. Luckily most of the time the eyes light up and the person starts talking about them at their best. I love those moments. As in Peter Thiel's "From 0 to 1", I want to see the best in people and then see if we can deal with the bad. In the end, I ask "so this is all great, but what will we have to learn to live with if we hire you?". On the one hand I'd like to prepare myself, but the question tests for self-reflection too. It works well.
But back to proud. I've been dealing with some shit too, and there have been a bit more downs than ups lately. This blog post/diary is a way to make my eyes sparkle and remind myself of the things I have built and still love. These are short stories in chronological order. There's a lot more to life than software, but this focused format is simple and it works. I might write from other perspectives in other posts. I've written this post for myself, but if you enjoy it, that's all the better.
These stories are how I remember them. My memory isn't amazing and in most of them I only had a small part with a limited perspective. If you have additions, questions or other feedback, send me a message at firstname.lastname@example.org.
Note 1: This is a work in progress and will be updated when I have time.
Note 2: Wow, this post got long but also popular. It had over 10k views on day 1. To make it easier here are some links:
- Part 1 - Fun but somewhat forced programming ('96-'00)
- [added Oct 2.] 1996 - JTJ Rekenen
- [added Oct 2.] 1997 - Plotter
- [added Oct 2.] 1999 - Reverse engineering the binary format of HP ChemStation
- Part 2 - High School & Uni ('00-'11)
- [added Oct 2.] 2004 - The HP 49g
- [added Oct 3.] 2005 - Porting an Atari tax program to Delphi
- [added Oct 2.] 2009 - SnelTrein
- [added Oct 30.] 2010 - Snapp
- Part 3 - The Mendix Years ('11-'18)
- [added Oct 8.] 2011 - newnode.py
- [added Oct 2.] 2011 - Social Gravity
- [added Oct 2.] 2012 - autopullpep8 and the facebook adventure
- [added Oct 2.] 2012 - JUDO
- [added Oct 30.] 2012 - Decrypt.py
- [added Oct 2.] 2012 - Mendix WebModeler v0.1
- [added Oct 5.] 2012 - mxplient
- [added Oct 6.] 2012 - Access2Mendix
- [added Oct 9.] 2013 - Stacky Bird
- [added Oct 3.] 2014 - Sandboxes
- [added Oct 2.] 2014 - cf-mendix-buildpack
- [added Oct 5.] 2015 - certinator
- [added Nov 28.] 2015 - buildpacks for Docker
- [added Oct 3.] 2015 - InstaDeploy
- [added Oct 5.] 2015 - Mendix2Java
- [added Jul 25] 2016 - Mendix Cloud v4 - Part 1 - How it got started
- [added Oct 9.] 2016 - Packing all mxruntimes in git
- [added Oct 5.] 2016 - The too clever scheduling service
- [added Oct 2.] 2017 - The Reaper
- Part 4 - CTO @ easee ('18-now)
- [added Oct 2.] 2018 - The easee Wizard Engine
- [added Oct 10.] 2019 - The easee Diff Viewer
- [added Oct 11.] 2019 - easee RemoteControl
- [added Oct 4.] 2020 - easee TeleVisus
- [added Oct 22.] 2020 - QMS Integrity Checker
- [added Oct 30.] 2021 - FDA Crawler
- [added Oct 11.] 2022 - Cowboy e-Bike Remote Control
Part 1 - Fun but somewhat forced programming ('96-'00)
1996 - JTJ Rekenen
Niels, my dad, had a 13-year career at HP (later Agilent). He started servicing mass spectrometers and ended up as a programmer. He always loved computers. He was pushing me to get into software from about age 8 or 9. We started developing a small program together that taught kids how to do maths. My dad did most of the work, and we created this together with my friend Joscha. We called it JTJ Rekenen. JTJ stood for "Jouke-Thiemo Joscha", Rekenen is Arithmatic. We contemplated calling it JJT but JTJ sounded cooler. The program did simple addition, multiplication etc. In hindsight, I believe the logic for division was full of problems.
The UI let you practice +, -, * and /, and turned the screen green if the answer was good and red if bad. I sold this little program to my school for about 50 guilders. I was pretty proud, but in reality my dad made most of the software while my friend and I watched and did some simple things and nodded a lot. JTJ Rekenen was built in Delphi or Visual Basic. I forgot which, but we used both at this time.
1997 - Plotter
As my dad worked at HP he saw all kinds of equipment in the office. At some point he brought home a couple of broken inkjet printers. He took out a lot of parts, ordered a microcontroller from the US and created a simple plotter for A4 paper with the stepper motors. Back then microcontrollers were expensive and difficult to program, there was no Arduino IDE or anything like it. If you weren't careful you could blow out your parallel port.
Dad built all the hardware, connected the microcontroller, and hooked up a driver and a small library so we could control it from Delphi. My job was to program the code to move the head around and up and down. Up means the marker is not touching the paper, down means you're drawing. This was great, as you could program something and see the results in the real world right away! I still enjoy try to get instant feedback loops whenever I'm working on a system.
I started with creating the logic for the letters. If you draw an A, you had to create 3 lines, 2 diagonal and 1 horizontal. On and on for the letters B to Z. It was quite a lot of work but after some time I got good at it and start optimizing patterns. Dad showed me how to make procedures to re-use logic. I figured out how to abstract things so that letters could be drawn from a relative starting position, how to do line wrapping, etc. Lots of fun.
1999 - Reverse engineering the binary format of HP ChemStation
At this time of my life I did not program for fun, I did it because my dad asked me to. I did a lot of other things with computers though, mostly games and drawing (CorelDraw). But in the summer of '99, my dad was struggling with the software that Agilent used for their mass spectrometers, ChemStation. It was a pretty big software suite and development was done in California. He could not influence it. He was often frustrated with how it worked, and like a true hacker, wanted to read and manipulate the data files with his custom software. He still does that to this day, now mostly with ChemStation's successor MassHunter. The ChemStation data format was super optimized for storage space and thus hard to understand. It stored the raw data from the spectrometers and some data that was annotated by the analysts in the lab.
On Friday nights dad, me and my two younger brothers used to go swimming in Amstelveen next to the HP office. One night in the car on the way back, he said that he could use my help for a couple of weeks during the summer break and could pay me about 100 guilders for it. He wanted me to figure out the data format that ChemStation used, so that he could write a parser for it. He showed me how to manipulate the data in ChemStation, and then inspect the files with a hex editor. If I made a copy of the file and changed the data I could compare the before & after versions and see how things were stored. Wow.
I learned so much in these couple of weeks. I think this was the first time I actually created something useful for someone. He could have done it 10x quicker himself, but what a fantastic learning opportunity for me. I learned about little/big-endianness, hex/binary/string encoding, and variable length fields. The result was a pretty good description of the data format. When my dad implemented the library he would sometimes call me over to explain some details that he didn't understand. I felt awesome and was only 11 years old!
In 2010 I worked for him for a year, and one of my projects was to create a C# library for the same old ChemStation data format. I think he still uses it to this day.
Part 2 - High School and Uni ('00-'11)
2004 - The HP 49g
In secondary school, everyone that did "advanced" maths had to get a TI 83+ calculator. Being an "HP family", my dad bought me an HP 49g+ calculator with RPN. I loved that thing. Going against the grain and doing things differently is very important to me. Here I had to get special permission from school to use this device because it was not the approved crappy TI calculator. Luckily I had a really cool maths teacher (hello Mr. Pieke!) and he made it happen.
I loved RPN and stack-based programming. I created all kinds of small programs in User RPL. On holidays I would be programming stuff in my tent. On hpcalc.org I downloaded all kinds of knowledge bases for e.g. the periodic table and loved reading things from the calculator. The calculator broke down at some point, then I bought a 49g+, but that was stolen from my student room later. I still miss it.
2005 - Porting an Atari tax program to Delphi
2009 - SnelTrein
While I was off studying Computer Science the smartphone revolution kicked off. Being an Apple hater, I got the first Android phone I could get my hands on: the T-mobile G1. Loved it and still think killing physical foldable keyboards was a shame. I had all kinds of ideas for apps, for some reason most of them revolved around trains. Like a GPS alarm clock that would wake you up when you got close to your destination.
I traveled by train a lot, and planning a trip or checking train times was a pain. I decided to build a train times app, but the name Trein was already taken, so I named mine SnelTrein. The Dutch name for high-speed trains. It was a simple wrapper around the mobile website of the NS (Dutch Railways). My app was super fast and super simple in the UI, as it remembered your trips. Your most often used trips were always on top. Once the app opened, it was literally one click to see the departure times of any of your 5 most used trips. I loved it, users loved it, and it grew to 3000 installs.
After some time, the NS released their own app, and I figured they are probably not scraping their own mobile site. After MITM'ing the traffic, it turned out they indeed had an XML based API which was password protected but they did not use HTTPS. I created a blog post post about this, submitted it to Dutch tech website tweakers.net. It was posted to the front page and I got my 15 minutes of Dutch tech fame. https://tweakers.net/nieuws/73527/ontwikkelaar-verkrijgt-toegang-tot-ns-api-voor-treintijden.html
I stopped maintaining SnelTrein because the NS app caught up a bit and is good enough. Early in 2022 I also switched to an iPhone SE from 2016 because Android phones are so big.
Stephanie's uncle Rob had a small design / marketing agency in Amersfoort, together with Bart. They contacted me to create a web-based "company personality test". A bit like 16 personalities / MBTI but then for companies. I liked the idea and could use the money, so I went ahead and said yes. The basic concept was to get companies back in touch with their core to brand themselves more effectively. E.g. you might discover that your company is a true "pioneer" or "peace keeper". If you're in touch with your personality, it should give you a sort of compass for your actions and for how to communicate more effectively. It was about 12 questions and you had to drag and drop your 3 favorite answers per question. Then you got 1 of 12 different personalities at the end. After the test they could help you create a brand strategy and implement it.
About a year or two later Rob & Bart needed a similar website with a quiz, and they asked if I could create the same thing again. It was using crappy jQuery technology so I wasn't too excited about it. The Dont-Repeat-Yourself mantra kicked in, I had a bit of a feeling that there was a cool tech solution and decided to create an engine and editor for creating quizzes. This turned out to be a really cool engineering challenge. I went a bit wild on my own time and within months you could:
- create quizzes in diagrams with a drag & drop editor interface
- embed it on any website
- see live traffic as people were taking the quiz
- have persistent data stored in mongodb
- have non-linear presentations with paths dependent on previous answers
- have a pub-quiz-like interface for many users
Slowly but surely it was becoming a full-fledged programming language. I experimented with different names, and went through Snap, Snapp, SnapLogic, WebFlow, SurveyMaker. Most of these were taken by similar companies though. Rob & Bart were very impressed and really loved the direction this product was going in. We started talking about creating a company together. I would do tech, and they would find customers and do implementations. However, by the time that it became really serious it was mid 2013. I was very busy with my day job, getting a kid and interviewing at facebook. I couldn't afford to go all-in as we had no funding and I was pretty much living paycheck to paycheck. Besides, the tech at Mendix was a lot more exciting than this.
In conclusion Snapp was just too low on my list of priorities and it slowly died after this. In hindsight there was a lot of potential. I had a great time with my career choices and don't have any regrets, but still I sometimes wonder "what if" about this venture. We'll never know :)
Part 3 - The Mendix Years ('11-'18)
2011 - newnode.py
- Hosting Reverse Proxies, JVM based apps, Databases
- Networking protocols: DNS, TCP/IP, HTTP, SSL/TLS, XMPP, Load Balancers, NTP
- Servers: RAM, CPU, hard drives
- Services: Backups, Monitoring, Alerting
- Distributed Systems and operating them 24/7
- Scripting in Bash, Python etc.
2011 - Social Gravity
I got onto twitter in 2010 and at some point realized the immense knowledge stored in the network. If you know who someone follows, and who follows that user, it tells you a lot.
Magically little groups of your different spheres of influence or interests appeared. If you hovered over an area, it highlighted frequent keywords in the bios of the twitter accounts. You can try to market yourself in any way you like, but your network doesn't lie.
As twitter restricted their APIs a lot, and I was very busy with my cool new day job at Mendix, this project died down. I created a video with how it worked here: https://www.youtube.com/watch?v=pyVUMTVarpQ . It's a bit embarrassing because it tells you so much about my interests back then. Anyway, most social network APIs don't expose this data anymore, but they use this data for all kinds of advertisement targeting. Pretty scary.
2012 - autopullpep8 and the facebook adventure
At Mendix, Achiel, my first team lead showed me Python and it became my go-to language. I got frustrated with the different coding styles, whitespace errors, and learned about pep8, flake, etc. Nowadays we use `black` but back then there was only `autopep8`. A tool that reformatted python code to the pep8 standard.
Learning about GitHub, its API and autopep8 I got a brilliant idea. What if I created a script to clone the top 100 most starred python projects, applied autopep8 and submitted a Pull Request. All completely automated. This was simple. I was nice enough to check the diffs before sending the PR. Autopep8 got a lot wrong and made some things really ugly. I spent 20 hours cleaning up code and then submitting PRs. I got my (small) contributions into a handful of top python projects. Pretty cool! Now more people figured out this trick and it is frowned upon in the open source community, but I think I did more good than harm.
I will never know for sure, but I suspect getting patches into top python libraries triggered some recruitment algorithm in silicon valley. In mid 2013 I got a message from a recruiter at Facebook for a position in their Production Engineering team. I actually hated Facebook and had deleted my account a few months prior. On the other hand the job was super cool and I loved California. I thought I had 0 chance of making the interview but it might be a great learning exercise. So, the process started. Over a period of 2 months I had 5 late night calls with the US West Coast that I had to keep secret from my bosses. I didn't prepare much and had the feeling that I passed every interview by the skin of my teeth.
My brain works a bit strangely. I can't grasp theoretical knowledge well, so while studying Computer Science a lot of concepts did not stick. But in these interviews, I finally understood what the engineers were struggling with and how we could create solutions. The theory finally had a useful application! I learned so much during and after these calls, and I'm super grateful for the experience.
At this time I was offered an "incentive package" at Mendix because of all the projects I did in my free time. I felt obliged to come clean and told Johan about my "affair" with Facebook, and that I appreciated the effort but first had some choices to make. I think the news spread quickly and most people considered me a lost cause. Facebook flew me out to Palo Alto for two days of on-site interviews. I did a bit of a road trip with my dad who by pure chance was freelancing in LA. He came over to the Bay Area for 2 days, we rented a fancy car and had fun.
The Valley was cool. While at 1 Hacker Way I saw Sheryl Sandberg doing a walking meeting across the FB campus with 5 of her reports. When I checked into the Sheraton there was a guy in an Instagram shirt standing next to me, he looked very happy. Instagram was bought for $1B a couple of months prior and the guy was now worth north of $50M but he still looked pretty normal. This was a different world. I also saw my first Teslas on the road and was stupid enough to put my actual current salary on some HR form.
Back home I got a good offer, negotiated a bit and got a better offer. IIRC it was 128k + 2x yearly 20k signing + stock, and for a 25-year-old engineer from Europe back in 2013 that was amazing. I still don't make close to this amount now, even though I probably could ;) In the Valley it was probably "meh" and that's why they recruited from overseas. But remember, I never expected to get an offer because in my head I wasn't good enough for the Champions League of the tech industry. So now I had to think. While thinking I discovered that:
- I don't care about money beyond basic needs.
- I'm very loyal and liked our mission at Mendix a lot.
- I care about the environment. Flying back to Europe 2 times a year with Stephanie my wife and a brand-new kid did not sit well with me. We now have 3 kids and it still doesn't.
- I don't like joining a winning team, I like a challenge and have a giant weak spot for the underdog.
- I'm actually a pretty great engineer.
- The H1B visa would not allow Stephanie to do any paid work. The H1B spouse club is real and that's a pity.
- I want to work on something good and meaningful and still hate Facebook.
I declined the offer. I felt bad about the super smart engineers and the recruiter for going through all this trouble (sorry Phil). I had made it clear from the start that declining was a very real opportunity. Thank you Zuck for sponsoring the free trip and the great learning experience. At Mendix, Roald told me he was very relieved I decided to stay. That was nice.
2012 - JUDO
At Mendix I worked on the Cloud team, we hosted Mendix apps for customers. Hosting should be as easy as building. The Cloud/Ops/DevOps bunch was a tightly-knit team, as we were in the firing line together. With that I mean 24/7/365 uptime guarantees and you need to rely completely on your colleagues at all times of the day. It's something else. When I joined we were 4, when I left it was 12. Now it's 100. I'm in touch with a lot of these people to this day.
Every night the system created backups of the apps which admins could download over HTTP. The download consisted of the postgres dump, but also a lot of files from object storage. We could store ZIPs for every backup every night, but then the storage requirements would explode. Instead, we did incremental backups with rsync and hardlinks. Achiel wrote his BSc. thesis on this project for the sole reason of graduating within 10 years of starting college. If not, he would have to pay back his government study subsidies. That was fun. Hans was actually the mastermind behind the backup system. He's an incredible engineer and always built rock-solid stuff. Now this system was pretty cool, but the download needed to be a single zip file and that was difficult.
We built JUDO, the Java Upload Download Organizer. When getting a download request over HTTP, it assembled the zip file on-the-fly while walking over the files storage directories. This was a pretty complicated project and Xiwen, Frank and I needed a lot of help from the senior engineers (Hans and Achiel). Over the years we found obscure bugs and fixed most.
The streaming approach was very elegant. There was no temporary storage and no asynchronous processes. The fundamental problem was that the size of the file was not known at the start. The size and Content-Length header could not be predicted because of the compression. When a connection got interrupted, browsers were not sure whether the download was complete. This led to some problems. This is why most products use a "we are preparing your download" approach. Less efficient, but when dealing with backups it's important to know you have downloaded it completely.
With my new python skills I created a funny little program called decrypt.py. I had done some hard disk recovery for a friend, and found it a bit ironic that the end result was so bland. It just said "19783 files recovered". In movies it's always way more dramatic. My creation brings the drama back to boring things. It takes input, shows garbled text on the screen and slowly reveals the original content. You can view a demo on the github page.
It only took a couple of hours to build, but the main thing about decrypt.py is that it got me really hooked on Hacker News. I had discovered it a couple of months before, but had never posted anything. Now with this post I reached the top spot for a couple of hours (I think?) and saw the comments and upvotes coming in. It was very addictive. I've reached the top place a couple of times now and it's a rush every time. There have been very few days that I have not visited the orange website since 2012, and I did learn a lot from it.
2012 - Mendix WebModeler v0.1
2012 - mxplient
2012 - Access2mendix
- the data model
- the data
- the logic
2013 - Stackybird
- teaching my kids programming
- creating a programming language from scratch
- the "meme-games" Flappy Bird, 2048 and SpaceChem that were popular in 2013/2014.
2014 - Sandboxes
- Cloud infrastructure that could support dynamic re-assignment of apps to virtual machines.
- A "last request" counter in the runtime.
- A deploy button in the Desktop Modeler.
- A "your app is resuming" page.
- A runtime license for the free apps.
- An SSO service for inviting your colleagues.
- MxBuild, a cloud based compiler for the apps (not sure why it was needed, but we did build it around this time).
2014 - cf-mendix-buildpack
2015 - InstaDeploy
2015 - mprtools
2015 - Mendix2java
2015 - Certinator
- Certinator needed to autocomplete intermediate certificates.
- For that I wanted to find all common intermediate certificates in the world.
- For that I needed to connect to millions of popular websites and look at their certificate chain.
- For that I needed to find a list of millions of websites.
- For that I could use the CommonCrawl dataset and extract URLs.
- For that I set up a huge EC2 instance in the same region as the CommonCrawl data. I optimized and processed 10TB of data on it within a couple of hours.
- Then I connected to all domains and collected the certificate chain with a tiny `golang` program.
2015 - buildpacks for Docker
2016 - Mendix Cloud v4 - Part 1 - How it got started
2016 - Packing all mxruntimes in git
2016 - The too clever scheduling service
2017 - The Reaper
Part 4 - CTO @ easee ('18-now)
2018 - The easee Wizard Engine
- Scalability. V1 of the eye test would crash with about 10 concurrent users. There was a spike of users after we got on RTL Nieuws and the database did not look pretty after that.
- The performance of our cylinder test. A clinical study was underway in the UMC Utrecht, and things looked good, except for the cylinder test. We had a new one but needed to implement it.