Catch what your tests miss: observe and score your WEC app in production with Langfuse
+- 1Prove a model works
- 2Trustworthy JSON
- 3Real test data at scale
- 4Observe & score production
- 5RAG, end to end
- 🏆Catch regressions in CI
A customer says your support bot promised them a refund policy that doesn't exist. Your feature made two LLM calls — classify, then reply. Which one invented it? If you can't answer that, your app is a black box — and this guide fixes exactly that.
You can now prove a model works (part 1), make its output machine-reliable (part 2), and generate a real test set to check it against (part 3). But all of that runs offline, in CI, on inputs you chose. Production doesn't play along.
This guide closes the gap. We'll self-host Langfuse — the open-source, self-hostable alternative to LangSmith — trace every real call, auto-score live traffic with an LLM judge, drill into the exact step that breaks, and feed failures back so your part-3 dataset gets stronger. Offline eval tells you it worked on your test set; this tells you it works in the wild.