[TSTIL] FDA Crawler

[ This is a part of "The Software That I Love", a series of posts about Software that I created or had a small part in ]

2021 - FDA Crawler

Some industries see a lot of competition even though there's not a lot of commercial traction yet. There the battles are fought not over customers and revenue, but over patents and certifications. With easee we were operating in a similar environment. It was very important for me as CTO / product person to figure out what our competitors were doing. One of the tools that I developed for this is the FDA crawler. It scrapes the FDA API for product and company registrations and serializes it in a git repository. Every week the script runs and with the git history you can see exactly if there are any created, updated or deleted listings. Any new company or product category is instantly visible. Well, sort of, because the database is only updated manually once every couple of weeks. We typically see news via a google alert weeks before it is visible in the official channels. Nevertheless, this serialization to git is a nice pattern that I hope to use for more purposes. It helps us keep track of any new competitors in the USA. It's a tiny script but really effective and sure beats checking the database manually.


Popular posts from this blog

AI programming tools should be added to the Joel Test

The unreasonable effectiveness of i3, or: ten years of a boring desktop environment

The long long tail of AI applications