[TSTIL] Packing all mxruntimes in git

[ This is a part of "The Software That I Love", a series of posts about Software that I created or had a small part in ]

2016 - Packing all mxruntimes in git

One of the things that made the buildpack slow, was that it had to download and unpack the whole Mendix Client and Mendix Runtime when building the container image. Over time these got dependencies got bigger and bigger. The images were bloated and needed time to transfer over the network. We needed a way to add a "cache" to our Cloud Foundry clusters, so that the buildpack could just copy the files from disk. We would not even have to include the dependencies in the final image. That would be a lot faster because the apps themselves were usually pretty small.

Storing all the hundreds of runtime versions in the cache of every server was going to take up a lot of disk space though, we needed something more clever.

I realized that most versions were minor updates and that a lot of files were shared between multiple versions of the runtime. I went ahead and created a git repository that stored all of the runtime versions as a tag. You can see the archive here: https://github.com/mendix/runtimes . This repository would be about 1.5 GB and stored 210 runtime versions of +/- 500MB each. Excellent compression!

We would save this repo in our own fork of the cflinux2 base image in Cloud Foundry, so it was already present on all machines in the cluster. Using the copy-on-write mechanisms in Cloud Foundry, we could even do "git fetch" if the app used a brand new Mendix version that was no stored in the cache yet. This was pretty clever. The downside was that it depended on github.com for building apps. Not ideal, but solvable. Another downside was that it would give smart people early notice on when we were releasing a new Mendix version, as they could see the new runtime being pushed.

For some reason we stopped using this approach in early 2018. I don't remember what the team replaced it with.


Popular posts from this blog

AI programming tools should be added to the Joel Test

The unreasonable effectiveness of i3, or: ten years of a boring desktop environment

The long long tail of AI applications