Prompt Engineering
Advanced
Retrieval Quality and Ranking
Evaluate whether your RAG system retrieved the right evidence before blaming the prompt.
35 min
3 sections
retrieval
evaluation
ranking
1
2
3
01. Separate retrieval failures from generation failures
Section 1 of 3
If the answer is wrong, first inspect retrieved context. The prompt cannot cite a runbook that retrieval never returned. Track whether failures come from missing documents, poor chunking, weak ranking, stale sources, or generation mistakes.
text
RAG failure analysis:
- No relevant chunk retrieved: improve indexing or query rewriting.
- Relevant chunk ranked too low: improve reranking.
- Relevant chunk stale: fix document lifecycle.
- Relevant chunk present but ignored: improve prompt and source citation rules.
- Answer conflicts with source: add groundedness checks.Back to Course