RAG evaluation framework: what to measure first

Most teams test RAG systems with a few happy-path prompts and a subjective “looks good.” That is not enough for production. A robust RAG evaluation framework should isolate the pipeline into measurable layers: query understanding, retrieval quality, answer quality, and operational reliability. If you only score final answers, you cannot diagnose why quality drops. In my RAG Equity Research Agent, this layered approach was the difference between a good demo and a stable system under real market questions.

The 4-layer metric model for RAG evaluation

Layer	Core metrics	Why it matters	Failure signal
Query handling	Query rewrite success, intent classification accuracy	Ensures retrieval starts from the right intent	Correct docs exist but are never retrieved
Retrieval	Recall@k, MRR, nDCG, context overlap	Measures whether useful evidence is fetched	Hallucination risk rises despite strong model
Generation	Groundedness, citation precision, answer completeness	Validates answer quality against retrieved evidence	Fluent but unsupported responses
Operations	P95 latency, token cost per answer, timeout rate	Protects UX and margins	Quality unstable under load or budget pressure

This is the key principle: retrieval metrics predict answer quality earlier than user complaints do.

Build a gold dataset from real user intent

Use real support tickets, analyst prompts, and search logs. Synthetic-only sets usually miss ambiguity, domain jargon, and messy phrasing. For each test sample, store:

• User query (raw, not cleaned)
• Expected answer points
• Required source documents
• Disallowed claims
• Difficulty label (easy/ambiguous/multi-hop)

A useful starting target is 150–300 high-quality examples per critical workflow.

Release gates that prevent silent regressions

Before every release, run an offline evaluation suite with hard thresholds:

• Retrieval Recall@10 >= 0.85 on critical intents
• Citation precision >= 0.90
• Groundedness score >= 0.88
• No critical safety policy violations

If one gate fails, block deployment. This keeps velocity high while avoiding expensive rollback cycles.

Online RAG evaluation in production

Offline tests are necessary, but they do not capture drift. You still need online controls:

• Sample and score live traces daily
• Track retrieval miss clusters by intent category
• Compare citation usage trends week-over-week
• Trigger alerts on latency/cost spikes

The observability layer implemented for DAISI follows this model: trace quality, trace cost, and policy outcomes are monitored together.

Failure taxonomy for faster triage

When quality drops, classify failures before changing prompts or models:

• Index failures: stale or missing documents
• Ranking failures: relevant context exists but ranks too low
• Prompt failures: context retrieved, but instructions under-specify evidence usage
• Policy failures: answer should have been blocked or escalated

This taxonomy helps teams fix the right subsystem first.

Recommended implementation sequence

• Define RAG evaluation metrics and gates
• Build the gold dataset from real requests
• Automate offline evaluation in CI
• Add production trace sampling and drift dashboards
• Review top failure clusters weekly

If you do these five steps well, your RAG system becomes explainable, measurable, and much easier to improve over time.