Writing
Thoughts on Data, AI, and building products.
A few months ago, I was building a data investigation agent. The architecture felt straightforward: build an internal application, connect it to an LLM, and give it access to Databricks through a service princi…
An LLM rarely disappoints because it cannot think. It disappoints because of what we forgot to place in front of it. The model is only ever as good as the world we hand it, and when the output arrives malformed…
Every few decades, the world convinces itself that a new technology will change everything. And the curious part is that the world is usually right. The railways changed everything. The internet changed everyth…
When an agent fails, we blame its intelligence. The real constraint is quieter: not whether it can reason, but whether it has the room to.
I built Support Agent on a thousand support tickets without a vector database. At this scale, brute-force cosine similarity in numpy is faster, simpler, and exact. Here is why the absence is a feature.
Place an order, then publish the event. Two lines of code, one silent data-corruption bug. How the transactional outbox pattern closes the gap between what your database commits and what the rest of the system…
You launch with three columns. Six months later, the schema has been altered a dozen times by three different engineers across four environments, and nobody can describe with confidence what the production data…
Why the built-in Datadog integration falls short on serverless, and how custom DLT hooks plus Terraform-managed monitor fill the gap.
I built a smart grid meter pipeline on Databricks Free Edition with medallion architecture, DQX validation, SCD modeling, and Terraform IaC. Here is what worked, what broke, and what I learned.
Think the LLM is the final product? Think again. Raw models are powerful but unpredictable. Like a horse without a harness. Learn why a well-engineered AI harness is the secret to turning non-deterministic mode…