2 posts tagged with "rag"

++

Our GPT-5.6 token-economics analysis ended with a challenge: benchmark promises are a hypothesis, not a reason — run the eval on your workload before you believe a "fewer tokens per task" claim. This tutorial is us taking our own advice.

We take the RAG docs assistant from part 5, wrap it in a DeepEval regression suite — containerized, judged by gemma4 on the WEC Inference API, zero OpenAI dependency — and then use that suite to answer a genuinely open question: should we swap the service's generation model? Qwen2.5-3B-Instruct (current) vs gemma4 (candidate), pass rate and tokens per task, measured head to head.

Spoiler: the eval saved us from a pointless migration. And along the way we hit four real production failures — a disk-full death spiral, a startup race condition, a Docker Compose variable trap that silently evaluated the wrong model, and the fix that makes that class of bug impossible. Every command, number, and error below is from a real run.

++

Everything in this series was building to this. You can prove a model works (part 1), make its output machine-reliable (part 2), generate real test data (part 3), and observe production (part 4). Now we spend all four skills at once on the pattern behind almost every serious LLM product: RAG — retrieval-augmented generation.

We'll build a docs assistant: a containerized HTTP service that answers customer questions from WEC's own documentation. Not a notebook — a service, Docker-first, the shape you'd actually deploy. And because this series doesn't do happy-path demos: along the way our RAG hallucinates a GPU price, we root-cause it to our own scraper, fix it, and pin the fix with a regression test. Every command, number, and error below is from a real run.