Retrieval-augmented generation lives or dies on the retrieval half. A great model with poor context still hallucinates.

Chunk with meaning, not by fixed length — split on headings and semantic boundaries so each chunk answers a question on its own. Store rich metadata (source, section, date) so you can filter before you search and cite after.

One-shot vector search is rarely enough. Retrieve broadly, then rerank with a cross-encoder to push the truly relevant passages to the top. Gate on confidence: if the best passages are weak, say so and hand off rather than inventing an answer.

Most importantly, evaluate. Build a small set of real questions with known answers and score retrieval and generation on every change. Grounded, cited, measured — that's the difference between a demo and a system people trust.