Chapter 12: Retrieval & Knowledge Layer (RAG + MCP)

Matryoshka representation (OpenAI, Cohere, and others) is the single most useful storage lever: you can truncate a 3072-d vector to 1024 or 512 dimensions with graceful quality loss. A 1024-d float32 vector is 4 KB; at 10M chunks that is 40 GB before quantization. Halving dimensions halves that bill and roughly halves query latency.

Practical defaults for a CMS: start with text-embedding-3-small (or BGE-M3 if you must self-host for privacy). Move to Voyage 4 or embed-v4.0 only if a measured eval shows retrieval misses, or if you are heavily multilingual. Always store the model name + dimension + chunking version alongside each vector so a mixed corpus during a migration is detectable and queryable. Never compare cosine distances across two different embedding models — they live in different spaces.

3. Vector store choice

The store must do filtered approximate-nearest-neighbour (ANN) over your content vectors and respect CMS metadata (locale, status=published, tenant, content type, access role) at query time. The 2026 contenders:

For most CMS projects, start with pgvector. The decisive argument is architectural, not benchmark-driven: your content, its permissions, its publish state, and its embeddings live in one transactional database, so a single SQL query can do WHERE status='published' AND locale='en' AND tenant_id=$1 ORDER BY embedding <=> $query LIMIT 20. No sync pipeline, no dual-write consistency bug, no "the vector store says published but the CMS unpublished it an hour ago" incident. Supabase and Tiger Data (Timescale) benchmarks in 2026 show pgvector with HNSW matching or beating dedicated stores at ~1M scale and equivalent recall.

Reach for a dedicated store when you cross roughly 50–100M vectors, need sub-5 ms filtered latency at high QPS, or want to decouple the retrieval system's scaling from your primary OLTP database. Qdrant is the cleanest "fastest filtered search" pick; Turbopuffer wins on cost for very large, read-skewed corpora because it keeps data on S3 and only caches hot vectors.

pgvector indexing, briefly

• HNSW (graph index): ~95%+ recall out of the box, absorbs inserts without rebuilds, 2–5× more memory than IVFFlat. Default choice for active-write CMS content under ~10M vectors.
• IVFFlat (k-means cells): faster build, less memory, but recall drifts as data changes and needs periodic reindex. Only for large mostly-static corpora.
• pgvectorscale StreamingDiskANN (Timescale): disk-based ANN; Timescale reports 28× lower p95 latency than Pinecone's storage-optimized index at 50M vectors and ~75% lower cost — the way to push pgvector well past its in-memory ceiling.
• halfvec (16-bit): ~half the bytes, minimal recall loss, raises the indexable dimension cap from 2,000 to 4,000. Binary quantization / SBQ shrinks storage further for huge corpora at a recall cost you must measure.

pgvector 0.8.x (current in early 2026) supports HNSW, IVFFlat, halfvec, sparsevec, and binary quantization — enough to cover most CMS needs without leaving Postgres.

4. Chunking your own content

Chunking is where CMS retrieval quietly wins or loses, and it is easier in a CMS than in a pile of PDFs because your content is already structured. Exploit that.

Chunk along the content model, not blindly by character count

A naive 1,000-character splitter shreds a structured entry mid-table and mid-sentence. Instead:

• Field-aware chunking: treat a rich-text body, a FAQ answer, a product description, and a spec table as distinct chunk types, each with its own size and metadata. A short FAQ answer is one chunk; a long article is split by heading.
• Structure-aware splitting: split Markdown/HTML on headings (H2/H3) so chunks align with sections. Keep the heading path (Article > Section > Subsection) in metadata.
• Recursive token splitting as the default body strategy: target ~512 tokens with ~10–15% overlap. A February 2026 Vecta benchmark across 50 papers found recursive 512-token splitting first at 69% accuracy — ahead of naive semantic chunking, which fragmented to ~43-token shards and dropped to 54%. The lesson: simple, size-bounded, structure-aware chunks beat clever-but-tiny ones.

Three upgrades that consistently help

1. Contextual Retrieval (Anthropic, 2024). Before embedding each chunk, prepend a short LLM-generated sentence situating it in its document ("This chunk is from the Refunds section of the EU Returns Policy, covering the 14-day window"). Anthropic reported ~49% fewer retrieval failures (up to ~67% when combined with reranking). In a CMS you often get this for free from metadata — the entry title, taxonomy, and heading path — so generate context only where structure is thin.
2. Hierarchical / parent-child chunking. Index small chunks for precise matching, but return the larger parent section (or whole entry) to the LLM for context. This resolves the precision-vs-context tension and is the most widely adopted production pattern in 2025–2026.
3. Late chunking (Jina). Embed the whole document first with a long-context model, then pool token embeddings into chunk vectors so each chunk's vector carries document-level context. Gains grow with document length — useful for long-form editorial content.

Always store rich chunk metadata

Every chunk row should carry: entry_id, content_type, locale, status, tenant_id, access_roles, heading_path, chunk_index, source_url, updated_at, embedding_model, chunk_version. This metadata is what powers permission-aware filtering, freshness, deduplication, and citation back to the canonical entry.

5. The retrieval pipeline: hybrid search + reranking

Pure vector search misses exact terms (SKUs, names, code, acronyms); pure keyword search misses paraphrase. Hybrid search runs both and fuses results.

Reciprocal Rank Fusion (RRF) is the workhorse fusion method: it combines the ranks (not the incompatible raw scores) from BM25/full-text and vector search, sidestepping the score-normalization bug that breaks naively weighted blends. In Postgres you can do this in one query using tsvector/ts_rank (or pg_search/ParadeDB BM25) alongside the <=> vector operator and fuse with a small RRF expression. Benchmarks consistently show BM25 + dense + RRF beating either method alone across datasets.

Reranking is the second-biggest quality lever after good chunking. Retrieve a generous candidate set (say top 20–50 via hybrid search), then re-score with a cross-encoder reranker — Cohere rerank-v4, Voyage rerank-2.5, or an open model like BGE-reranker — that reads the query and each candidate together, and keep the top 5–8 for the LLM. Cost is bounded because the reranker only sees pre-filtered candidates, not the whole corpus. Budget +100–300 ms for this stage.

A solid default CMS retrieval pipeline:

query → (LLM) query rewrite/expand
      → hybrid search: BM25 + vector, RRF fuse, filter by locale/status/role → top 30
      → cross-encoder rerank → top 6
      → assemble prompt with parent-chunk context + citations
      → generate

6. Traditional vs agentic retrieval

• Traditional (single-shot) RAG: retrieve once, then generate. Predictable latency (~50–200 ms retrieval) and cost. Right for FAQ-style answers and editor "explain this field" helpers.
• Advanced RAG: add query rewriting + reranking. ~500 ms–2 s latency depending on reranker; better precision but still struggles with multi-hop questions.
• Agentic RAG: retrieval becomes a control loop — retrieve, evaluate, decide to answer or retrieve differently, possibly across multiple tools/sources. Best for multi-hop and "compile an answer across many entries" tasks, but each loop adds a model round-trip in latency and tokens.

The production rule in 2026 is adaptive routing: classify the query's complexity first and send simple questions down the cheap single-shot path while reserving the agentic loop for queries that genuinely need it. Do not pay agentic latency on every "what's our refund window?" query.

7. An MCP server over your CMS

The Model Context Protocol (current spec 2025-11-25, with a 2026 release candidate in progress) is the open, JSON-RPC-2.0 standard for letting LLM applications connect to external data and tools. Shipping an MCP server is how your CMS becomes a connectable knowledge source for Claude Desktop/Code, Cursor, ChatGPT, internal agents, and any future MCP host — without each of them needing a bespoke integration.

MCP defines three server-side primitives, and your CMS server should expose all three:

The killer tool is search_content, which simply wraps the §5 hybrid+rerank pipeline and returns reranked, citation-bearing chunks plus their canonical entry URLs. With it, an editor in Cursor or an agent in Claude Code can ask natural-language questions about the entire content base and get grounded, linkable answers.

Transport & auth. Use stdio for a local server an editor runs alongside their IDE, and Streamable HTTP for a remote, multi-user server. Remote servers must implement OAuth 2.1 so the MCP layer enforces the same permissions as the CMS — never expose a tool that bypasses your access control. The 2026 RC moves toward a more stateless core (so a remote server can sit behind a plain round-robin load balancer and clients can cache tools/list), which matters for scaling a multi-tenant CMS MCP endpoint.

Security is non-negotiable. The spec itself stresses that tool descriptions and annotations are untrusted, tools are arbitrary code execution, and hosts must get explicit user consent before invoking them. For a CMS specifically:

• Run read tools (search_content, get_entry) and write tools (create_draft) with different scopes; never let an unauthenticated or low-trust client publish.
• Apply the same row-level/locale/tenant filters in the MCP tool as in your app retrieval path. The MCP server is a new front door to the same data — it must not be a permissions bypass.
• Treat prompt-injection in content itself as a real risk: a malicious entry body could try to steer an agent. Keep retrieved content clearly delimited as data, not instructions, and constrain write tools behind human approval.

MCP server vs. plain REST/GraphQL API — when to ship which

You almost certainly already have a content API. MCP does not replace it; it adds an agent-native facade. Ship REST/GraphQL for your apps and front end; ship MCP when you want third-party AI hosts and internal agents to discover and use your content with zero custom glue. Many teams implement the MCP tools as thin wrappers over their existing API + retrieval service.

8. The internal knowledge base angle

The same machinery doubles as an internal knowledge base for your own organization: index not just published content but editorial guidelines, style guides, past campaigns, component docs, and runbooks (with stricter access_roles). An MCP server then lets writers ask "have we covered this topic before?", "what's our tone for product launches?", or "show me the three most similar existing entries" directly from their authoring tool — turning the CMS into institutional memory rather than a write-only store.

9. Pitfalls to avoid

• Treating the vector store as source of truth. It is a rebuildable index. Keep CMS as canonical and re-embedding as a background job.
• Forgetting freshness. Unpublished/edited content must invalidate or re-embed its chunks. Drive embedding updates off the CMS's content-change events/webhooks, not a nightly full re-crawl, or stale answers will leak.
• Mixing embedding models in one index without recording the model/version — distances become meaningless across spaces.
• Permission leaks via retrieval. If filters aren't applied inside the ANN query (and inside the MCP tool), you will return draft, embargoed, or other-tenant content. Filter at the index, not after.
• Over-chunking into tiny fragments. Semantic splitters that emit ~40-token shards underperform plain 512-token recursive chunks.
• Skipping reranking. It is the cheapest large quality win after sane chunking.
• Agentic everything. Multi-hop loops on trivial queries waste latency and tokens; route by complexity.
• No eval set. Without a small golden set of question→expected-citation pairs, you cannot tell whether a new embedding model, chunk size, or reranker actually helped. Build it before you tune.

Key Takeaways

• The knowledge layer grounds generation; design it as a derived, rebuildable index over your canonical CMS data, never a second source of truth.
• Default to text-embedding-3-small (or BGE-M3 for self-host); upgrade to Voyage 4 / Cohere embed-v4 only when an eval shows misses or you're heavily multilingual. Use Matryoshka truncation to control storage.
• Start with pgvector (HNSW; add pgvectorscale StreamingDiskANN past ~50M vectors) so vectors live transactionally beside content and permissions; move to Qdrant/Turbopuffer at large scale or for sub-5 ms filtered latency.
• Chunk with your content model: structure-aware ~512-token recursive chunks with overlap, hierarchical parent-child returns, and contextual prefixes — much of which CMS metadata gives you free.
• Retrieval quality = hybrid search (BM25 + vector via RRF) + cross-encoder reranking + permission-aware filtering applied inside the query.
• Use adaptive routing: single-shot RAG for simple queries, agentic loops only for multi-hop tasks, to control latency and cost.
• Ship an MCP server (Resources + Tools + Prompts; search_content is the killer tool) with OAuth 2.1 and the same access control as your app — it's a new front door, not a permissions bypass.
• Always store per-chunk metadata (entry_id, locale, status, roles, model/version) and drive re-embedding off content-change events to keep answers fresh and citable.

Key References

• Anthropic. Introducing Contextual Retrieval. 2024. https://www.anthropic.com/news/contextual-retrieval — Source for the contextual-chunk technique and the ~49%/67% retrieval-failure reductions.
• Model Context Protocol. Specification 2025-11-25. 2026. https://modelcontextprotocol.io/specification/2025-11-25 — Authoritative spec: Resources/Tools/Prompts, JSON-RPC, transports, security principles.
• Model Context Protocol Blog. The 2026-07-28 MCP Specification Release Candidate. 2026. https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/ — Stateless core, Extensions, Tasks; load-balancer-friendly remote servers.
• Tiger Data (Timescale). pgvector Is Now Faster than Pinecone at 75% Less Cost. 2026. https://www.tigerdata.com/blog/pgvector-is-now-as-fast-as-pinecone-at-75-less-cost — pgvectorscale StreamingDiskANN benchmark (28× lower p95 at 50M vectors).
• dbi-services. pgvector, a guide for DBA — Part 2: Indexes (update March 2026). 2026. https://www.dbi-services.com/blog/pgvector-a-guide-for-dba-part-2-indexes-update-march-2026/ — Current pgvector 0.8.x index options (HNSW, IVFFlat, halfvec, SBQ).
• Firecrawl. Best Vector Databases in 2026: A Complete Comparison Guide. 2026. https://www.firecrawl.dev/blog/best-vector-databases — Cross-store comparison incl. Turbopuffer S3-first cost model.
• Digital Applied. Hybrid Search: BM25, Vector & Reranking Reference 2026. 2026. https://www.digitalapplied.com/blog/hybrid-search-bm25-vector-reranking-reference-2026 — RRF fusion and cross-encoder reranking practice.
• Digital Applied. RAG Chunking Strategies: A 2026 Retrieval Playbook. 2026. https://www.digitalapplied.com/blog/rag-chunking-strategies-2026-retrieval-quality-playbook — Contextual/late/hierarchical chunking and the Vecta 512-token vs semantic finding.
• Voyage AI / MongoDB. Voyage 4 embedding family (MoE) launch. 2026. https://blog.voyageai.com — RTEB benchmark claims, shared vector space across model sizes.
• OpenAI. New embedding models and API updates (text-embedding-3 + Matryoshka). 2024–2026. https://platform.openai.com/docs/guides/embeddings — Dimensions, truncation, pricing for the 3-small/3-large models.
• Cohere. Embed v4 + Rerank v4 documentation. 2026. https://docs.cohere.com/docs/embeddings — Multilingual embeddings and cross-encoder reranking pairing.
• Towards Data Science. Agentic RAG vs Classic RAG: From a Pipeline to a Control Loop. 2026. https://towardsdatascience.com/agentic-rag-vs-classic-rag-from-a-pipeline-to-a-control-loop/ — Control-loop framing, latency/cost trade-offs, adaptive routing.
• CrafterCMS. What Is the Model Context Protocol? A CMS Developer's Guide to AI Integration. 2026. https://craftercms.com/blog/technical/what-is-the-model-context-protocol — Concrete CMS-as-MCP-resources/tools mapping.
• arXiv:2504.19754. Reconstructing Context: Evaluating Advanced Chunking Strategies for RAG. 2025. https://arxiv.org/abs/2504.19754 — Empirical evaluation of chunking strategies.