Enterprise search and RAG that respects document permissions
Cohash builds enterprise search and retrieval-augmented generation that only returns what each user is allowed to open in the source system. We capture permissions with every chunk at ingestion, filter by the user's groups before ranking, keep permission changes and deletions in sync, and run leak tests with accounts from different roles. Where a platform already enforces permissions natively, we use it instead of building our own.
- 01Sources and permission models
- 02Ingestion with ACLs
- 03Filter before ranking
- 04Keep permissions in sync
- 05Evals and leak tests
- 06Citations and logging
The story
Told without the client's name or anything that could identify them.
A professional services firm's AI team had a RAG demo that answered from one shared vector index, and in testing it quoted a document the person asking couldn't open in SharePoint. Nobody could say how a permission change in Confluence, or a leaver's private files, would reach the index. We started by listing each source and how it expresses access, then rebuilt ingestion so every chunk carries its allowed users and groups. Queries now filter by the user's groups before ranking, permission changes sync on a freshness target security agreed to, and leak tests with accounts from different roles run in CI on every change.
why it was urgentOWASP's LLM08:2025 Vector and Embedding Weaknesses names unauthorized access and data leakage through embeddings, and recommends permission-aware vector stores. Microsoft's Copilot connectors attach an access control list to every indexed item. A first RAG demo on one shared index rarely does either.
- openIn the demo, the answer came from a SharePoint document the asker wasn't allowed to open.
- recurringEvery document lands in one vector index with no record of who may read it.
- blockedSomeone left the company and their private files still show up in answers.
- openSecurity asked how a permission change in Confluence reaches the index, and nobody knew.
The estate, before and after
Before: a RAG demo on one shared index
A bulk job copied SharePoint, Confluence and file share content into one vector index and lost the permissions along the way. Everyone's questions hit that same index, so an answer could quote a document the person asking wasn't allowed to open.
Read the diagram as text
- SharePoint → Bulk ingestion job: documents
- Confluence → Bulk ingestion job: pages
- File shares → Bulk ingestion job: files
- Bulk ingestion job → Shared vector index: chunks · embeddings
- Any employee → RAG demo app: question
- RAG demo app → Shared vector index: similarity search
After: retrieval filtered by each user's groups
Every chunk carries its allowed users and groups from ingestion onward, and a sync job tracks permission changes and deletions. At query time the user's groups come from the identity provider, filtering happens before ranking, and answers are cited and logged.
Read the diagram as text
- SharePoint → ACL-aware ingestion: documents
- Confluence → ACL-aware ingestion: pages
- File shares → ACL-aware ingestion: files
- SharePoint → Permission sync: permission changes
- ACL-aware ingestion → Permission-aware index: chunks with ACLs
- Permission sync → Permission-aware index: ACL updates · deletes
- Identity provider → Group-filtered search: user groups
- Group-filtered search → Permission-aware index: filtered query
- Group-filtered search → LLM with citations: permitted chunks
- LLM with citations → Query audit log: query log
- Leak tests in CI → Group-filtered search: leak tests
How we run it, phase by phase.
6 phases take you from A RAG demo on one shared index to Permission-aware retrieval in production. Each one ends with something you can check before the next starts. As you scroll, the blocks show the estate moving across.
- phase 01
Sources and permission models
Each source handles access its own way. SharePoint has groups and sharing links, Confluence has space and page restrictions, file shares have NTFS rights and Salesforce has sharing rules. We write those down, then decide per source between indexing and querying live.
- phase 02
Ingestion with ACLs
We tag every chunk with the users and groups allowed to read it, by stable identity provider ID, and resolve nested groups. Documents with unclear permissions don't make it into the index.
- phase 03
Filter before ranking
At query time we look up the user's groups and filter both vector and keyword search down to the chunks they can read. Ranking and generation never see anything else.
- phase 04
Keep permissions in sync
Where a source can push permission changes and deletions, we subscribe. Where it can't, we re-crawl. Security signs off on a freshness target for each source.
- phase 05
Evals and leak tests
Test retrieval quality on real questions, and run leak tests with accounts from different roles that ask for content they shouldn't see. Both run in CI on every change.
- phase 06
Citations and logging
Answers link back to their sources. For each query we log the user, the documents retrieved and the model used. The log is kept under the same retention rules as the sources.
What the client got
What changed for the client once the work was done. Described, not scored: we don't publish client figures.
- result 01
Answers only use text the user may open
We filter vector and keyword search to the user's groups before ranking. If the user couldn't open a chunk at the source, the model never sees it.
- result 02
Every chunk carries its source permissions
We store allowed users and groups with stable identity provider IDs at ingestion, resolve nested groups, and hold back documents whose permissions are unclear.
- result 03
Permission changes and deletions reach the index
Where a source sends change events we listen for them. Where it doesn't, we re-crawl on a schedule. Each source has a freshness target that security signed.
- result 04
Leaks are tested on every change
Accounts from different roles ask for content they shouldn't see, alongside retrieval quality evals on real questions, in CI.
- result 05
Every answer is cited and logged
Answers link back to their sources. We log who asked, which documents came back and which model answered, and keep those logs exactly as long as the sources themselves.
Risks we plan for
-
Stale permissions
When ACLs refresh only nightly, someone whose access was revoked can keep retrieving content for hours. How long that window is allowed to be is a decision the sync design has to make.
-
Aggregation leaks
Permitted fragments can still add up. Take a few drafts, each readable on its own, and together they might reveal a reorg plan. Filtering won't catch it. Sensitivity labels and source exclusions do.
-
Instructions hidden in documents
Some retrieved documents carry text written to steer the model. We treat retrieved content as data and limit which tools the model can call. Prompt injection cases go into the test set.
-
Embeddings as a copy of the data
Your vectors and chunks are one more copy of confidential content. Give them the same encryption and residency rules as the originals, and delete the chunks when an original is deleted.
Where AI helps, and where it needs guardrails
In a lot of cases the platform already enforces permissions. Copilot connectors filter by source ACLs, and federated connectors fetch live through MCP under the user's own OAuth sign-in. For custom RAG, we expose retrieval as an MCP tool that only accepts audience-bound, user-delegated tokens, as the MCP authorization spec expects. An agent can't search with more rights than the person who asked.
forward deployed engineers →What you're left with
We write these for your setup, and they're working documents. Your team should be able to keep things running from them long after we've gone.
- inventory01
Source inventory with each permission model and an index-or-federate decision
yours to keep - code02
Ingestion pipeline that stores ACLs with every chunk
yours to keep - document03
Query-time filtering tied to identity provider groups
yours to keep - tests04
Retrieval eval set and leak tests running in CI
yours to keep - evidence05
Query and retrieval audit logging with retention rules
yours to keep
Platforms and services on this move
Learn the background
Terms from this page in plain English. If you want the why, the field note behind it is right there.
- 01Prompt injection
- Instructions tucked into content an AI reads, like a document or a ticket comment, meant to make it do something its user never asked for.in context →
- 02RAG
- Retrieval-augmented generation, or RAG. A search first pulls relevant passages from your documents, and a language model writes the answer from those passages.in context →
Certified engineers, wherever you need them.
Some clients want a project team, some want a couple of engineers on theirs. Either works. Onsite, nearshore or offshore, with handovers across time zones that keep things moving.
Onsite
Engineers at your sites in the US
Nearshore
Overlapping hours with US teams
Offshore
Overnight progress on your backlog
All time zones
Coverage handed over across time zones
01Should we build our own RAG or use Copilot or a platform search?
If your content lives mostly in Microsoft 365 or one data platform, its built-in search usually enforces permissions with less work. Custom RAG makes sense when sources span systems those tools don't cover, or when answers must follow your own rules.
02Which vector database should we use?
One that filters on metadata at query time and meets your hosting rules. Most mainstream vector stores can. Spend the energy on the permission design instead.
03How do you prove nothing leaks?
Nobody can prove a negative. You can test hard, though. We run leak tests on every change, using accounts from different roles, and we log what each query retrieved so any leak can be traced back.
04Do you also clean up the permissions underneath?
Yes. Most leaks we find trace back to broad sharing in the source system. Cleaning that up with the identity and content owners is often the first useful thing we deliver.
Often on the same roadmap
On-premises Active Directory to cloud-first Microsoft Entra ID
We shrink on-premises Active Directory one application at a time, moving groups, users and app authentication to Microsoft Entra ID as each app is ready.
read the case studyAD FS federation to Entra ID authentication
Converting federated domains from AD FS to Entra ID cloud authentication, moving relying parties app by app and retiring the AD FS farm.
read the case studyAI pilot to production with forward deployed engineers
Taking an AI pilot into production with forward deployed engineers who work with the business team, build evals, connect real systems and hand over cleanly.
read the case study7d631a81 - 01OWASP: LLM08:2025 Vector and Embedding Weaknessesgenai.owasp.org ↗
- 02Microsoft Learn: Copilot connectors overviewlearn.microsoft.com ↗
- 03Model Context Protocol: Authorization (2026-07-28)modelcontextprotocol.io ↗
- 04OWASP Top 10 for LLM Applications 2025genai.owasp.org ↗
Planning this move?
A 30-minute working session with a senior engineer. No slides.
- The platform and the version you're on
- Where you need to get to, and by when
- What's custom: rules, plugins, integrations
- Any audit finding or deadline driving it