What metrics should you track for a RAG system?
Track retrieval and generation separately. On the retriever, measure recall at k and precision at k against a labelled set. On the generator, measure groundedness — whether each claim is supported by the retrieved context — and answer quality against a rubric. A single end-to-end score tells you something broke but not which half.
Groundedness is the metric that moves hallucination rates. Scoring each sentence of the answer against the retrieved chunks surfaces the specific claims the model invented, which is actionable in a way that an aggregate quality score is not.
Watch recall before precision. Most RAG systems that hallucinate are not retrieving the right chunk at all — the model then fills the gap. Fixing chunking and query rewriting usually beats fixing the prompt.
Across engagements, splitting the metrics this way and then attacking retrieval first has taken RAG hallucination rates down by around 68% from baseline to live.
Written by Binary AI Labs · Reviewed