The knowledge layer: connecting AI to company docs

Part 1 gave every employee a governed assistant. The next question arrived within days, from the assistant’s own users: why can it not answer questions about our documents?

The business case for closing that gap is old and quantified. Gartner’s digital-worker research finds 47% of workers struggling to find the information they need, and its digital-friction work puts the loss at 5.5 hours per employee per week. McKinsey’s much-cited figure, 19% of the knowledge worker’s week spent searching and gathering, dates to 2012, before AI raised expectations of search.¹ For a 200-person company, single-digit-percentage recovery of that time pays for any architecture in this article many times over.

The risk side is newer and sharper, and it is why this article spends a third of its length on permissions:

• Concentric’s data-risk analysis across half a billion records found 16% of business-critical data overshared, about 802,000 files at risk per organization.
• Microsoft’s Copilot rollouts made a category of incident famous: the AI does not break permissions, it makes bad permissions discoverable at conversational speed. A share set to “everyone” on an HR folder five years ago becomes a summarizable answer for any employee who asks.
• EchoLeak (CVE-2025-32711) demonstrated the ceiling: a zero-click prompt injection against a production assistant that could exfiltrate anything in its retrieval scope. What the assistant can retrieve sets the size of the breach.

So connecting AI to company knowledge is three decisions stacked: which platform, on what licence terms, with what permission model. Published comparisons address the first and skip the other two, and both of the skipped ones overturned our platform choice.

Benchmark on your corpus → Exercise the licence surface → Test the permission model → Adopt

↺ the next candidate runs the same three tests against the same gold set

The buy-or-build boundary

Before self-hosting anything, price the buy option, because for some companies it wins:

Option	Reported price	Where it fits
Glean	~$45-75/user/mo, ~100-seat minimum (reported; not published)	Large orgs, no infra appetite; the reference for permission-mirrored search
Microsoft 365 Copilot	$30/user/mo	All-Microsoft estates, if the tenant’s sharing hygiene survives it
ChatGPT Enterprise “company knowledge”	quote-only	Light knowledge needs inside an existing OpenAI relationship
Self-hosted (this article)	infra + your time	Data residency, model choice, cost control, regulated data

At 200 people, Glean’s reported floor is roughly $110K a year. The self-hosted path costs a fraction of that in infrastructure and more in attention, and it is the only path that keeps documents inside your own boundary, which for us was the deciding constraint, not the price.

The landscape: five products, three categories

The self-hosted field gets compared as one flat list, and the flat list is the first trap. As of August 2026:

	Category	Version	Stars	Per-document permissions	Licence
Onyx	Org-knowledge product	v4.5.0	~31.6k	Yes, paid edition (ACL sync from sources)	MIT core + enterprise directories
PipesHub	Org-knowledge product	v0.6.0	~3.2k	Yes, in the OSS build	Apache 2.0
RAGFlow	RAG engine	v0.26.4	~88.5k	No; dataset-level only me/team	Apache 2.0
Dify	App builder	1.16.1	~152.5k	No; app and workspace scoped	Apache 2.0 with added conditions
AnythingLLM	Team doc chat	v1.15.0	~64.7k	No; workspace scoped	MIT

flowchart LR
  subgraph K["Org-knowledge products"]
    O["Onyx"] 
    P["PipesHub"]
  end
  subgraph B["Builders"]
    R["RAGFlow: RAG engine"]
    D["Dify: app platform"]
  end
  subgraph C["Team chat"]
    A["AnythingLLM"]
  end
  K -->|"connect sources, sync, permissions handled"| G["Company assistant"]
  B -->|"you assemble the assistant"| G
  C -->|"upload docs, chat"| G

An org-knowledge product is the finished thing: connectors, continuous sync, a bot surface, and a permissions story. A builder gives you an excellent engine (RAGFlow’s document parsing is genuinely strong) and leaves the assistant, and the permissions, to you. Feature-grid comparisons flatter the builders on flexibility and hide that the permissions row is empty, and the permissions row decides enterprise deployments.

Test one: a benchmark with a control arm

No published comparison of these platforms contains a measured accuracy number on a shared corpus; the comparisons that exist are feature reviews. So we measured, and the method transfers to any corpus:

• A gold set from your own documents. 72 questions, each written by opening the cited file and asking something with a concrete, checkable answer (a default, a threshold, a path). The authoring rule is written down: never write a question from memory of the file.
• A no-retrieval control arm. Same answer model, no context: it passed 5.6%. That number is what makes the others meaningful, because it proves the corpus is internal enough that gains are retrieval’s contribution.
• Two metrics, kept apart. Hit@5 (did a gold source appear in the top five retrieved) and pass rate (did an LLM judge accept the answer, with confident wrong answers and “I do not know” both failing). They fail independently: in an earlier round, fixing one platform’s index coverage lifted its hit rate 12 points and its pass rate only 5. Retrieval bounds what is possible; synthesis sets what users get.
• Confidence intervals on every difference. Paired bootstrap, 2,000 resamples. On small question sets, a few points is noise, and the interval says so.

The result, all platforms on identical local footing:

Backend	Hit@5	Pass rate	Latency p50	Files indexed
No-retrieval baseline	n/a	5.6%	1.6s	n/a
Onyx (MIT build)	97.2%	77.8%	8.1s	534/534
RAGFlow	90.3%	66.7%	2.0s	518/534
PipesHub	63.9%	16.7%	3.1s	479/534

Three readings, with the caveats the record itself carries:²

1. Retrieval against no retrieval is enormous (+72 points for the leader), the one result robust to every caveat.
2. The Onyx-RAGFlow gap could be noise. +11.1 points, but the 95% CI runs from 0.0 to +22.2, so the honest reading is a tie. With accuracy tied, operations decide: RAGFlow retrieved 4x faster, in half the containers, which for a chat assistant is the difference between an answer and a spinner. Our recorded recommendation at this stage was RAGFlow.
3. Every platform silently dropped files. Onyx’s upload path rejects source-code extensions, RAGFlow cannot index YAML, and PipesHub’s document model cannot represent a file path, which lost 93 files without an error. Index coverage is a benchmark metric, and the only way to see it is counting your own files on the other side.

Test two: the licence boundary

The benchmark cannot see licensing, and licensing removed our first winner before this table existed: an earlier Onyx deployment returned HTTP 402 on API-key minting in the MIT build, on exactly the surface (API keys, MCP) that a bot integration authenticates through. Since v4.0.0 Onyx also ships tiered licensing with enforced validation on by default, and its permission auto-sync from sources (Confluence, Drive, Slack, SharePoint) is an enterprise feature; the MIT boundary is visible in the repo as whole /ee directory trees. Dify’s licence is Apache with conditions (multi-tenant and branding restrictions). RAGFlow and PipesHub are clean Apache 2.0.

The transferable rule: exercise the licence boundary the way you exercise accuracy, on the exact API surfaces your architecture depends on, in the build you intend to run. A feature grid tells you what exists somewhere in the product; only a deployment tells you what is gated in your tier.

Test three: permissions, where the ranking got overturned

Then the scope firmed up: the knowledge layer would serve around 200 people across mixed-sensitivity sources (wikis, ticketing, file shares, chat). At that point “employees only see answers derived from documents they can open” stops being a feature preference. It is the requirement the whole deployment stands on, and the Copilot oversharing record above is what it looks like violated.

RAGFlow cannot meet it, structurally. Its dataset permission field is binary (only me, or team). Its document model carries no ACL. And its metadata filters are constructed by an LLM at query time, which can never carry a security guarantee, because generation’s failure mode is precisely what a permission check exists to prevent. None of this is a missing toggle, and the maintainers’ issue tracker confirms finer controls are enterprise roadmap, not OSS.

So the decision overturned the benchmark’s ranking: Onyx’s MIT build, with permission enforcement built in-house at the query layer. Both documents, the measurement that said RAGFlow and the dated decision that says Onyx, sit side by side in the repo. When a non-negotiable requirement outweighs the measured metrics, write that down; re-weighting scores until the numbers agree with the choice is how benchmarks stop being trusted.

Permission-aware retrieval, fail-closed

The published prior art on RAG permissions is mostly cloud-vendor patterns: Azure AI Search does security trimming via group filters (and, since 2025, native Entra-based document ACLs); AWS Bedrock does metadata filtering you construct yourself; Glean mirrors source ACLs into its index and evaluates the signed-in user at query time. All document the mechanism. Almost none of it states the failure behavior, and the failure behavior is the design. Ours:

flowchart TD
  U["User asks in Slack"] --> Q["Query + user identity"]
  Q --> F["ACL filter inside knn.filter
  (during search, never after)"]
  F --> CH["Chunks, each pointing to a
  permission container"]
  CH --> CN["Container membership
  synced from sources"]
  CN -->|"user in container"| Y["Chunk eligible"]
  CN -->|"not in container"| N["Invisible"]
  CN -->|"sync state UNKNOWN"| N

The five choices, each answering a leak path documented in the wild:

1. ACLs at the chunk level. Retrieval returns chunks. A document-level check on a chunked index is how the confidential annex of a partly-public document leaks through a neighboring chunk; the pricing-annex example is the canonical one in the literature.
2. Chunks point to permission containers, never to principals. Re-permissioning a 200,000-document site updates one container membership and rewrites zero chunk payloads. Stamping users directly onto chunks turns every HR change into a reindex, which guarantees the sync falls behind.
3. The ACL filter runs inside the vector search’s knn.filter. An outer filter applied after top-k selection silently returns fewer than k results to restricted users, so the people with the narrowest view get the worst recall, and nothing reports it.
4. Unknown is excluded, unconditionally. A document whose permissions have not synced is invisible to everyone. This is the fail-closed default, and it inverts the default we found in the wild, where a failed ACL fetch resolved to inherit-and-visible. Sync problems then degrade recall, and only recall. As far as I can find, this fail-open-vs-fail-closed choice has no dedicated published treatment; every sync design forces it, silently or explicitly.
5. A two-account negative test per connector, in CI. Account A retrieves the document, account B must not, or the connector does not ship. Permission code without negative tests is a policy document with extra steps.

Deployment follows the same posture: isolated cells per trust boundary rather than one shared index, the most restricted corpus first, embeddings routed through the part-1 gateway on their own key so knowledge spend lands in the same ledger, and retrieval answering 403 until the permission checks pass verification. The first production state is fail-closed and empty, and the verification order follows from it: confirm the permission boundary holds before any document becomes an answer.

Choosing for your constraints

Your situation	Direction
Everyone may see everything (small team, open culture)	RAGFlow or AnythingLLM; permissions machinery buys nothing
Mixed-sensitivity sources, must mirror source ACLs	Onyx paid, PipesHub OSS, or Onyx MIT + your own enforcement
No infra appetite, budget available	Glean-class SaaS; audit the sharing hygiene first (the Copilot lesson)
Building LLM apps, knowledge is one feature	Dify; read its licence conditions first
Regulated data, EU residency, model choice	Self-hosted, behind your own gateway

And in every column: benchmark on your corpus with a control arm, exercise the licence surface before adopting, and state your permission default out loud. If a vendor or a design cannot tell you what happens to a document with unsynced permissions, the answer is fail-open, and fail-open plus conversational search is how 802,000 at-risk files become answers.

Key takeaways

• Connecting AI to company knowledge is three stacked decisions: platform, licence boundary, permission model. Feature comparisons cover only the first; the other two overturned our choice twice.
• The candidates span three categories (org-knowledge products, builders, team chat), and per-document permissions exist only in Onyx (paid) and PipesHub. RAGFlow’s absence is structural, including LLM-generated filters that cannot carry security.
• Benchmark with a no-retrieval control arm and confidence intervals: our leader’s 11-point win had an interval touching zero, and operations (4x latency, half the containers) were the real differentiator at that point.
• Count your files on the far side of indexing. All three platforms silently dropped files, each for a different reason.
• Exercise the licence on the API surfaces you depend on, in the build you will run; ours failed with an HTTP 402 on key minting, after the benchmark said yes.
• Enforce permissions at the chunk level, through container indirection, inside the search filter, with unknown-permission documents invisible to everyone and a two-account negative test per connector in CI.
• Start fail-closed and empty. The assistant’s retrieval scope sets the size of any breach, and the oversharing record (16% of business-critical data, EchoLeak) shows what the alternative looks like at conversational speed.

Footnotes

1. The 19%/1.8-hours figure is McKinsey Global Institute, “The Social Economy,” 2012, and its age is part of the point: the cost of not finding information was quantified before assistants existed. Gartner’s 47% (2023 Digital Worker Survey) and 5.5 hours/week digital-friction figures are the fresher anchors; Glean pricing figures are reported by competitors, not published. ↩

2. The recorded caveats: the LLM judge was unvalidated against human labels, and in this round judged its own answer model’s outputs, so pass rates read as directional and orderings as the reliable part. RAGFlow’s bundled default embedder failed to start in our environment and was substituted with bge-small-en-v1.5, so its measured quality belongs to that combination. The gold set (72 questions) is large enough to separate retrieval from no retrieval decisively and too small to separate the top two platforms; both facts are visible in the intervals. ↩