2025-06-11 - Day 11 - 100 Days of Code

Hello, Friends -

Today I spent more time working on the Eensy ETL. I’m excited to launch into this project and spent far too long today shopping for an open API with streaming data. So far, the Github events API is looking good for this. It is live data but it is not exactly streaming (not like a Kafka stream). It can use pagination, but it also offers the use of a ETag value and “If-None-Match” header. This should help prevent retrieving duplicate records.

The Github event record contains a few different types: PushEvent, CreateEvent, ForkEvent (these should be interesting), WatchEvent,and PullRequestEvent.

My code time was spent setting up the bones of Terraform code for a serverless application and event grid trigger. My goal is to write the function in Java but if it becomes too complicated or takes too long to run then I’ll change it to Python. I haven’t decided how much of the event data to keep in each record because these are definitely document-shaped instead of tabular so some information will have to be stripped. It would be cool if the event record came with some kind of location data because I would love to visualize this data as a heatmap. I’ll have to think of some other visualization to represent this data. Stay tuned…