A weekend RAG prototype feels like magic. Then the knowledge base grows, the documents go stale, and the model starts confidently citing things that are no longer true. Production RAG is a data pipeline problem wearing an AI costume.
Retrieval quality is the ceiling
The model can only be as good as what you hand it. Chunking strategy, embedding choice, re-ranking and freshness matter more than which LLM you picked. We spend most of our RAG effort on retrieval evaluation — measuring whether the right context actually surfaces.
Cite, or do not claim
Grounded answers must link back to their sources so users — and your own team — can verify them. If the system cannot cite it, it should not assert it. That single rule kills most hallucination complaints.
Treat your knowledge base like code: versioned, tested, and continuously updated. RAG is not set-and-forget; it is a living system.