Two products can offer you a "knowledge base" and mean entirely different things. One means a capability your organization owns, with a steward who decides what belongs in it. The other means a retrieval configuration a developer set up one afternoon. Both descriptions are honest. Only one survives a compliance review.
The same split runs through the retrieval itself, and that is the half worth understanding first, because most RAG running in production today is the shallow version and its failures are quiet.
Naive RAG, and why it disappoints
The default recipe is four steps. Split each document into fixed-size chunks. Embed each chunk. At query time, embed the question and take the top k by cosine similarity. Paste the winners into the prompt.
It is easy to build, which is why it is everywhere, and it fails in ways that are hard to see from outside:
- Chunks are cut by character count, not meaning. A definition lands in one chunk and its qualifying clause in the next. Retrieve either alone and the answer is confidently wrong.
- A chunk has no idea what it belongs to. No section, no document, no neighbours. The model receives a paragraph with no way to ask what came before it.
- Tables and figures become noise. Plain text extraction flattens a table into a stream of numbers with no rows or columns, which then embeds into something that matches nothing useful.
- Top-k is one guess with no recourse. If the right passage ranks eleventh and k is ten, there is no second attempt.
- Everything is one granularity. A question that wants a document's gist and a question that wants one specific clause both get the same fixed-size fragments.
Notice that none of these produce an error. The pipeline reports success, the endpoint returns text, and the model answers fluently from bad context. That is what makes naive RAG expensive: it fails silently, and the first person to notice is usually a user who trusted the answer.
Semantic RAG: the document becomes a graph
The alternative — sometimes called graph RAG, though semantic RAG describes the intent better — changes what ingestion produces. Instead of a flat list of chunks, a document becomes an entity subgraph: documents, sections, pages, chunks, tables and figures, wired together with typed edges.
That structural difference pays off at every later stage:
- Chunking follows structure. Semantic, sliding-window, recursive or hybrid strategies, with layout-aware extraction so tables and figures survive as themselves rather than as flattened text.
- Retrieval navigates, rather than only ranking. Having found a relevant chunk, the query agent can walk to its section, pull a neighbouring passage, or exclude material that turned out to be irrelevant.
- Multiple granularities per entity. Each carries
full_text,compressedandsummaryforms, and the agent picks a level per source to fit the token budget you asked for — summaries for breadth, full text where precision matters. - Retrieval is iterative. A first pass that comes back thin triggers another, up to a cap you set. Top-k-once becomes a search loop.
- Hybrid by default. Vector, keyword, or both with tunable weighting, behind a classifier that routes between structured lookup and vector retrieval per question.
And because a knowledge base is a partition of the entity graph rather than a parallel store, ingested content sits alongside the rest of your application's data — inspectable, joinable and extensible through the same APIs. That is also what makes it governable, instead of an opaque index someone has to take on trust.
What that looks like in practice
Ingestion is one call that returns immediately with a workflow handle, because building a subgraph with embeddings and summaries at several granularities is a multi-minute job and nobody should hold an HTTP request open for it:
curl -X POST "https://<gateway-host>/api/kb-ingest" \
-H "Content-Type: application/json" \
-d '{
"kbId": "kb-partition-uuid",
"documentId": "doc-uuid",
"blobPath": "s3://corpus/2024-annual-report.pdf",
"config": {
"extractionMode": "layout",
"chunkingStrategy": "semantic",
"maxChunkTokens": 512,
"summarizationLevels": ["compressed", "summary"]
}
}'Querying asks for a token budget rather than a chunk count. That is the tell that you are talking to a graph rather than a list — the agent decides how many sources, and at which detail level, to spend that budget on:
curl -X POST "https://<gateway-host>/api/rag-query" \
-H "Content-Type: application/json" \
-d '{
"query": "What are the key findings about error rates in production systems?",
"kbIds": ["kb-partition-uuid"],
"maxTokens": 4096,
"strategy": {
"maxIterations": 5,
"detailLevel": "auto",
"searchMode": "hybrid",
"includeMetadataFilters": { "published_date": { "gte": "2024-01-01" } }
}
}'And the response carries its own provenance, chunk by chunk, with the level each source was read at:
{
"context": "The key findings indicate that error rates in production systems...",
"sources": [
{
"documentId": "doc-uuid",
"chunkId": "chunk-uuid",
"title": "Production Error Rate Analysis",
"relevance": 0.92,
"detailLevel": "full_text"
}
],
"iterations": 3,
"tokenCount": 3847
}
"iterations": 3 is worth pausing on. That query searched, considered what it found, and
searched again twice before it was satisfied. Naive RAG has no equivalent field, because it only ever gets
one attempt.
A failure mode that survives both
Better architecture does not exempt you from checking that ingestion worked. A hard-won warning: a pipeline that reports success has told you documents were processed. It has not told you the resulting chunks are useful, or in some cases that they contain anything at all.
Empty and near-empty chunks embed perfectly happily. They index. They are simply never usefully retrieved. The pipeline is green and the knowledge base is hollow, and nobody finds out until someone asks a question the corpus should have answered and gets a confident non-answer.
So validate chunk quality inside the pipeline rather than in a post-incident review: assert that embedded count equals chunk count per document, and probe the vector store for a non-zero hit before declaring the ingest complete. The lesson travels well beyond RAG — a component reporting a negative should say which negative it means. "I checked and found nothing" and "I could not check" look identical in a log line and mean opposite things.
The governance half
Architecture is only part of what separates a knowledge base from an index. The rest is ownership, and it needs four things that have nothing to do with vectors:
- An owner — usually a subject-matter expert rather than the developer who wrote the ingestion code, who can say which of three overlapping policy documents is authoritative.
- An approval step — a path by which a source becomes production knowledge deliberately. Without one, a knowledge base grows by accretion.
- A lifecycle — documents expire, policies are superseded, retention applies. One that can only grow is a liability with a search interface.
- An audit trail — which documents were retrieved, from which base, under whose access, at what version. A different question from "was retrieval accurate", and the one regulators ask.
In FireFoundry a knowledge base is a managed asset: created and administered in the Console, fed by the managed ingestion pipeline above, and governed by access controls that name applications, agents, and real end users and groups †.
† In beta or active development.
The questions to ask
Of any platform's knowledge capability, ask two things. What does ingestion actually produce — a list or a graph? And who owns this knowledge, and who approved it? If the answers are "a list" and "whoever has write access to the storage account", you have a retrieval index. That may be all you need. Just do not find out during an audit.
More on where that second question comes from in What Each Enterprise AI Product Is Actually Selling, or explore the platform.