Skip to content
Said Yaka

Topic

Evaluation

4 essays

  1. Designing LLM Features Around Structured Outputs You Can Trust

    Treat the schema as the product contract. Here is how I design typed LLM outputs, validate and repair them, version the shape, and eval field by field.

  2. Designing Tools LLM Agents Can Actually Use

    Tool schemas are prompts. Here is how I name tools, shape parameters, return compact results, write recoverable errors, and eval whether agents can use them.

  3. RAG Evaluations That Actually Catch Regressions

    A practical framework for building retrieval and answer-quality evaluations that catch real regressions instead of producing vanity metrics.

  4. Synthetic Data for LLM Evals: Useful, but Easy to Abuse

    Synthetic data can speed up AI evaluation pipelines, but only if it is used with strong guardrails and real-world calibration.