Topic
AI Engineering
11 essays
Designing Human-in-the-Loop Approvals for Agents That Act
Human-in-the-loop only works if people actually read the ask. Here is how I decide what needs approval, show the real action, and treat a no as final.
Designing LLM Features Around Structured Outputs You Can Trust
Treat the schema as the product contract. Here is how I design typed LLM outputs, validate and repair them, version the shape, and eval field by field.
Designing Tools LLM Agents Can Actually Use
Tool schemas are prompts. Here is how I name tools, shape parameters, return compact results, write recoverable errors, and eval whether agents can use them.
Context Windows Are Not Memory
A long context window feels like memory, but it is just a working set. Real memory needs selection, persistence, and a plan for what to forget.
RAG Evaluations That Actually Catch Regressions
A practical framework for building retrieval and answer-quality evaluations that catch real regressions instead of producing vanity metrics.
Prompt Caching Is a Product Feature, Not Just an Infra Trick
Why prompt caching changes latency, cost, and product design decisions — and why teams should treat it as a user-facing capability.
What Agent Observability Should Actually Show You
A practical view of agent observability: what to log, what to surface, and what teams need to understand before agents become trustworthy in production.
How I Think About Model Routing in Production
A practical way to think about model routing across cheap, fast, and strong models without turning your stack into a chaotic science project.
MCP Feels Like the Missing UI Layer for Agents
Why the Model Context Protocol matters: it gives agents a cleaner way to discover tools, carry context, and interact with software systems.
When Not to Use RAG
RAG is powerful, but it is not a default answer to every AI product problem. Here are the cases where I would avoid it or simplify it.
Synthetic Data for LLM Evals: Useful, but Easy to Abuse
Synthetic data can speed up AI evaluation pipelines, but only if it is used with strong guardrails and real-world calibration.