Software etc.

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2018 - Mendix Cloud v4 - Part 2 - How to launch a complicated product Note : This was a multi-year mega project (at least for my standards) and ran from about 2015 till 2018 when I left Mendix. It's still one of my proudest professional achievements. If you haven't yet, read part 1 about Cloud v4 . During 2017/2018 our team was doing a lot of things at the same time: Handle the operational fires in Mendix Cloud v1/v2/v3. Add new hardware with 50%-100% YoY growth, as well as handle all kinds of customer requests, like resizing VMs or adding more storage. These were all manual operations by our engineers. Migrate v1/v2 apps to v3 to ensure standardization. Build v4 which would make everything standard, scalable and self-service. It would be the solution to solve the chaos from v1/v2/v3. Fires? What fires? Running a 24/7 hosting operation for custome...

Years ago I was a Product Manager of a product with lots of teething troubles. The Director of Support and I sat down from time to time to discuss the most urgent customer issues. One day he suddenly said, visibly annoyed, "I see that you're agreeing with me and writing down things in your notebook, but are you actually going to do something with it this time?" . Needless to say, that hit home. I've seen many LinkedIn posts that go like "here are ten things that take 0 talent: following up on commitments, being on time, ... " etc. . But I think that's wrong. Making sure you "follow up" is hard work. I'm a total pleaser by nature, I want to be agreeable and I dread confrontation. Telling someone "I'm not going to do anything with this" is very difficult. So my natural tendency is to say "Hm yes that's a good point, it would be great if we did something with it, perhaps we can do ..." and then go off and brainstorm...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2016 - Mendix Cloud v4 - Part 1 - How it got started We had been building a big new cloud for a year or two, but we did not have a name. I was the Product Manager for "it". Should we call it the Real Mendix Cloud? The Scalable Mendix Cloud? The Next-Gen Mendix Cloud? Or a brand name like e.g. Heroku, HP Helion, IBM BlueMix were doing. After brainstorming with Roald it dawned on me that this was actually our 4th generation cloud offering. Why not simply call it v4? Customers had always had lots of confusion on where their apps were hosted. They called it "the Mendix Cloud", the "Achiel Cloud" or the "Hans Cloud". If we now said "you are on v2" or v3, and you need to go to v4 because of X, that'd be a pretty simple explanation. I really liked the simplicity of the v4 name and pushed for it. No one had a ...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2015 - Buildpacks for Docker As I got more familiar with buildpacks and docker, I figured out that we can use buildpacks to create runnable docker images. Not just droplets that you can only run in Cloud Foundry, but plain old docker images. It only took about 100 lines of bash! I still love the elegance of it: https://github.com/jtwaleson/buildpacks-for-docker next :  2015 - Mendix2Java previous :  2015 - certinator

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2016 - Packing all mxruntimes in git One of the things that made the buildpack slow, was that it had to download and unpack the whole Mendix Client and Mendix Runtime when building the container image. Over time these got dependencies got bigger and bigger. The images were bloated and needed time to transfer over the network. We needed a way to add a "cache" to our Cloud Foundry clusters, so that the buildpack could just copy the files from disk. We would not even have to include the dependencies in the final image. That would be a lot faster because the apps themselves were usually pretty small. Storing all the hundreds of runtime versions in the cache of every server was going to take up a lot of disk space though, we needed something more clever. I realized that most versions were minor updates and that a lot of files were shared between mult...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2011 - newnode.py At Utrecht University I had done a dual BSc. degree in Computer Science and Cognitive Artificial Intelligence. Most if it was technical but I had done plenty of courses in the humanities as well. Some psychology, history of philosophy and Arabic. I was also lucky enough to have joined a book club where we read the classics. I made great friends there. One of them invited me to an honours program in Rhetoric in Amsterdam. That class had about 30 students, and Alexander Klöpping was one of them. I remember that he flew to NYC to get the first iPad and then talked about it on national television in "De Wereld Draait Door". He was in a very different world from all the other students and it was really cool to see.  When I started working life I knew I was going to miss the humanities. I registered for an evening program for a BA in...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2012 - Access2mendix I'm not sure how this project got handed to me. Maybe we were a bit desperate to find new revenue, and in that search, someone had realized that a lot of enterprises used mini-apps built on Microsoft Access. Microsoft Access was installed on most systems because it was bundled with Office, and it allowed mildly technical users to build apps. Going through proper IT processes involved layers of red-tape so lots of people had built lots of solutions with Access. As I hated red-tape too I understood those people very well. They were what was called "shadow-IT". These apps were stored on some shared folder which only a couple of people knew about, and no one knew how to maintain it. Lots of processes were going through these "unofficial" apps and this was seen as a problem, because IT was not in control. With Mendi...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2016 - The too clever scheduling service In Mendix Cloud v4 we only wanted to use fault-tolerant services. So no single VMs, but "services" that were "web-scale", "clustered" and "horizontally scalable". Most services we created were request based. So a request would come in, and the service would respond. Simple. We built everything according to the 12-factor app architecture and ran apps on Cloud Foundry. On the AWS side we used lambda and SQS. This was our toolbox for the new architecture. One service that was not request-based was the backup service. Every night we had to create backups. Users didn't trigger this backup creation, we had to trigger it ourselves. If you use a traditional VM, you'd set up a cron job and you're done. In our new world this was somehow considered bad, but we had no tools i...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2015 - Mendix2java Roald called me up one day and said we had a problem. We were about to close a major deal with a large company that I can't name here, but the sales process was stuck at the last stage. Roald was co-founder and pretty important and he never called me, so that was interesting. He's an interesting character: driven, business focused, and very smart. He's unbeatable at the interplay of tech and sales. He's very confrontational, so I had to get used to him in the first couple of years, but after that I deeply respected him. He's confrontational because he cares. I think the respect was mutual because he called me in this crisis, and we worked together quite a bit when I was a PM. This almost-customer was stuck on vendor lock-in. Building things on any platform is risky because it's expensive to migrate off of it. The...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2015 - Certinator My brain is different from that of most people. This is a bit exaggerated, but I either understand something completely or I'm very confused. A coworker recently said that I somehow "know what I don't know". This is a blessing and a curse. A blessing because when I get it I am able to see bugs or solutions in the blink of an eye. A curse because I'm kind of useless and doubt everything until I understand a topic 100%. Both `git` and SSL/TLS were topics that I didn't get for a long time. When I learned the git commands I was useless, but once I learned the data structures I became a git expert overnight. It's a beautiful idea brilliantly executed. SSL/TLS was the same. In the Mendix Cloud, customers needed to add custom domains for their apps. As we were enterprisey, this had to be HTTPS but no low-code devel...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2012 - mxplient The Mendix core had three parts: - The MxModeler (IDE) - The MxClient (Powering the app in the browser) - The MxRuntime (Powering the back-end and managing the database) I was in the Cloud team and used a Mendix app for the Cloud Portal. As we needed to automate a lot with external scripts, I needed an API in the app. Mendix had no REST functionality yet, and being Python fanatics we didn't want to touch XML WebServices. Then I realized the app already had an API. In the browser UI, I was doing all these actions I needed to automate, and those actions went to the Runtime as HTTP requests. Using the browser debugger I discovered the undocumented API that the MxClient and MxRuntime used. I proposed just making this API public, but the Mendix core teams were very against this. Who did this junior engineer think he was? Michiel Kalkman was...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2015 - InstaDeploy When we launched "Free Apps" at Mendix I was pretty proud. It was basically "Sandboxes V2 + WebModeler". Under the hood we had this whole new architecture with all kinds of clever "cloud-scale" solutions. It turned out that users were not so happy. Every time you changed something in your app you had to deploy it. We wanted people to use Free Apps so we hid the old "local deployment" option. The Free Apps deployment service was buggy and slow. Local deployment used to take 10 seconds. The old sandboxes took 1 minute, the Free Apps took 4. On bad days it took 8. The cloud team had launched the deployment service without any SLAs or basic explanation of the limitations to the other teams. A bit immature. The WebModeler and Modeler teams just had to use it and see how it worked. In testing it was alr...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2014 - Sandboxes Derek had a dream for the Mendix trial and he kept talking about it. Sandboxes. You could already try Mendix for free, create an app and run it on your own PC. But you couldn't share your apps to your colleagues. We needed a trial environment in the cloud. Derek was an excellent CEO because he had a clear vision which he kept repeating. He said what users needed and didn't micromanage the solutions. For all I know the higher-ups might have experienced a different side of him, but being at the bottom of the food chain this was my perspective. I have tremendous respect for him. We could not have thousands of free apps running on our infrastructure, costs would go through the roof. Serverless didn't exist and the Mendix runtime was quite heavy. The only remaining solution was what Heroku did: stop the app when not active and resu...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2017 - The Reaper We launched Mendix Cloud v4 by starting the new Free Apps tier on it. These apps without SLA allowed us to take more risks and learn to operate Cloud Foundry. The Mendix Free Apps cluster had enough RAM for about 100 concurrent apps. It could scale up and down, but we wanted to keep the costs reasonable. An app would run as long as there was HTTP traffic to it. If an app did not have any traffic for 30 minutes, we killed it. If an HTTP request would come in, we'd serve a "loading" page while the app was spinning back up. We had a small app (The Resumer) that would catch all traffic for apps that were not running & serve the loading page. I believe Xiwen wrote most of it, but my memory is failing me. It would fire a request to the CF API to find and start the app.  The Reaper was its counterpart. It would talk to the CF ...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2014 - cf-mendix-buildpack The 12-factor app manifesto made an enormous impact on the world of cloud. If you didn't do 12-factor apps, you were stuck in the stone age. At Mendix we were stuck in the stone age, and we wanted out. I believe it was in 2013 that Johan, the CTO, spent a weekend on getting Mendix to run on Heroku. He was still close enough to the Java code that he could compile his custom runtime for this purpose. I think this is the last serious programming I saw him do. He's a fantastic CTO and unlike me learned to let go of coding. We never pursued Heroku again, but a year later Mark Rogers, who was leading Business Development, discovered Pivotal and their Cloud Foundry offering. This was an enterprise platform that targeted the same market as us. Cloud Foundry was Heroku for the Enterprise, and it was awesome. It would take us out ...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2012 - Mendix WebModeler v0.1 Mendix was (and is) a fantastic platform. Most Mendix developers use the Desktop Modeler / Mendix Studio Pro to create apps. Back in 2013 this was the only option. I had a different vision for the product. Maybe because I had no love for Windows apps, maybe because my stack was the browser, maybe because of the actual user needs. In any case, I wanted a web-based app creator. No installers, no friction. It would need a lot of difficult front-end engineering and a complete revamp of the codebase.  Think of how much effort Microsoft had to put in Visual Studio Code. Add another re-architecture for client-server and you get the idea.  No one within Mendix was crazy enough to try it. Being young and unhindered by wisdom or experience, I spent a couple of nights on creating v0.1. It was a PHP + jQuery based application that could ...

What are you proud of?  When interviewing candidates I always ask "tell me something that you are proud of." Sometimes you get a blank stare. That person is probably down and dealing with some crap. Luckily most of the time the eyes of the candidate light up and the person starts talking about themselves   at their best . I love those moments. As in Peter Thiel's "From 0 to 1", I want to see the best in people and then see if we can deal with the bad. In the end, I ask "so this is all great, but what will we have to learn to live with if we hire you?". On the one hand I'd like to prepare myself, but the question tests for self-reflection too. It works well. But back to proud. In the end of 2022 I was not in the best place either, and there were a bit more downs than ups. This blog post/diary was a way to make my eyes sparkle and remind myself of the things I had built and still loved. These are short stories about pieces of software, and what happe...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2012 - JUDO At Mendix I worked on the Cloud team, we hosted Mendix apps for customers. Hosting software should be as easy as building software. The Cloud/Ops/DevOps bunch was a tightly-knit team, as we were in the firing line together. With that I mean 24/7/365 uptime guarantees and you need to rely completely on your colleagues at all times of the day. It's something else. When I joined we were 4, when I left it was 12. Now it's 100. I'm in touch with a lot of these people to this day. Every night the system created backups of the apps which admins could download over HTTP. The download consisted of the postgres dump, but also a lot of files from object storage. We could store ZIPs for every backup every night, but then the storage requirements would explode. Instead, we did incremental backups with rsync and hardlinks. Achiel wrote his BSc. ...

[ This is a part of " The Software That I Love ", a series of posts about Software that I created or had a small part in ] 2012 - Autopullpep8 and the Facebook adventure Achiel, my team lead at Mendix, showed me Python and it quickly became my go-to language. I got frustrated with the different coding styles, whitespace errors, and learned about pep8, flake, etc. Nowadays we use `black` but back then there was only `autopep8`. A tool that reformatted python code to the pep8 standard. Learning about GitHub, its API and autopep8 I got a brilliant idea. What if I created a script to clone the top 100 most starred python projects, applied autopep8 and submitted a Pull Request. All completely automated. This was simple. I was nice enough to check the diffs before sending the PR. Autopep8 got a lot wrong and made some things really ugly. I spent 20 hours cleaning up code and then submitting PRs. I got my (small) contributions into a handful of top python projects. Pretty cool! Now...