RAG means retrieval-augmented generation. The application searches authorised documents and gives relevant passages to a model as context for an answer. It does not need to incorporate every document into the model’s parameters.
This can make internal knowledge available, but it does not guarantee correctness. An outdated instruction, a missing page or a citation that does not support a claim can lead to a convincing but wrong answer.
The project is more than uploading PDFs: it needs curated sources, access controls and tests of whether the system can answer, cite and admit that information is missing.
Two paths: prepare knowledge and answer a question
Indexing runs when documents change; answering runs when somebody asks a question. Keeping the paths separate makes updates manageable and helps locate failures in extraction, retrieval or generation.
Indexing prepares passages; a query retrieves only authorised ones and carries their provenance into the answer.
- Documents
Identified sources, versions and permissions.
- Extraction and chunking
Text, tables and coherent units.
- Embeddings and metadata
Search representations and verifiable provenance.
- Search index
Vector and text search, not the source of truth.
- Authorised retrieval
Apply identity and filters before fetching context.
- Model context
Relevant passages with references.
- Answer and sources
A checkable explanation or an explicit limit.
Vector search is one option and can be combined with text search and filters. The core principle is to retrieve before answering.
The research paper that introduced RAG combines retrieval and generation for knowledge tasks. An enterprise implementation must also solve permissions, versions and operations.
Inventory sources before generating embeddings
Identify which document is authoritative for each topic. If two manuals disagree, the system may retrieve both and blend their instructions. A knowledge owner must resolve current versions and historical archives.
- Stable identifier and link to the source.
- Version, effective date and owner.
- Document type, language and scope.
- Permissions and use restrictions.
- A procedure for adding, updating and retiring material.
Check reading order and tables in PDFs. OCR adds another step for scans and must be evaluated. The document automation guide covers those checks.
Split documents without losing meaning
Chunking divides documents into retrievable units. A fixed size can separate a rule from its exception. For example, a travel policy needs its spending limit, approval condition and exceptions together.
An embedding represents content numerically for semantic matching. Similarity is not proof of truth or relevance. Exact product codes may work better with text search, while differently worded questions may benefit from semantic search. Test a combined approach on real abbreviations, short queries and vague requests.
Permissions follow the document into the answer
A person’s identity determines which documents may enter retrieval. Filtering only the final screen is too late: a restricted passage in model context may leak into the response. Suggestions and shared caches must observe the same boundary.
Azure AI Search documents a security-filter pattern based on identities or groups. The application must bind the authenticated identity to that filter; a user must not be able to substitute an arbitrary group.
| Situation | Expected result |
|---|---|
| Two roles ask about one topic | Each retrieves only permitted sources |
| Folder access is revoked | The change reaches the index and caches |
| A document instructs the model to reveal another | The text cannot alter permissions |
| A cited link needs access | Opening it checks identity again |
External embedding or generation services may receive content. Decide that data flow before indexing. RAG does not automatically mean private AI.
Citations should enable verification
Each important claim needs a source that actually supports it, ideally linked to a page or section. A link to an entire hundred-page manual shifts most verification back to the reader.
Distinguish supported answers, contradictory sources and missing information. “I could not find this rule in the available documents” is a useful result with an escalation route. An application can check that cited identifiers were retrieved; evaluation must also check whether the passage supports the claim.
Three evaluations and an update procedure
| Layer | Evaluation question | If it fails |
|---|---|---|
| Extraction | Are text and tables complete? | Correct extraction or the source |
| Retrieval | Was the needed authorised passage found? | Review chunks, metadata and search |
| Answer | Is the explanation faithful and cited? | Review context and answer checks |
Prepare real questions with expected sources and questions without answers. Hold out some for testing changes. Include withdrawn documents, contradictions and unauthorised access attempts.
When replacing a document, withdraw old chunks or mark them inactive. Deletion policy should cover the index, storage and caches. Record index version and cited references while minimising personal data in logs.
When RAG is the wrong tool
Current stock or order status belongs in a transactional system, not a document index that may lag. Use deterministic tools for calculations. Fine-tuning can change model behaviour, but does not replace source updates or access control.
For a few well-organised files, conventional search may be enough. With many sources, recurring questions and an owner who can maintain the knowledge, a private knowledge assistant may merit a pilot.
From knowledge to a working process
Make your documents useful and verifiable.
We can assess sources, permissions and a representative question set before choosing the search architecture.
Discuss a knowledge assistant ↗
