Chapter 15: Analytics & Content-Intelligence Build

2. Choosing the privacy-first analytics layer

These four products are not interchangeable. GA4 and Plausible/Umami answer "what's happening on my site"; PostHog answers "what are users doing in my product." Pick by job-to-be-done, not by feature count.

Decisions for a CMS specifically:

• Content site / publisher: start with Umami or Plausible. They give the page-level metrics that matter for content (pageviews, sources, entry/exit, time, goals) without a consent banner — which removes the single biggest source of analytics data loss (banner-blocked GA4 can lose 20-50% of sessions). Umami's free self-host with no event cap and built-in funnels/retention is the most generous starting point; Plausible wins where EU-hosted infrastructure is a procurement requirement.
• Product / app with content (docs portal, SaaS with a marketing site): PostHog is the pragmatic single platform — its 1M-event + 5k-replay free tier covers most early-stage CMS projects, and session replay plus feature flags are directly useful for testing AI-authored content variants.
• GA4 earns its place only when Google Ads attribution or Search Console-linked acquisition reporting is a hard requirement. Even then, run it alongside a cookieless tool so you have a banner-independent source of truth. GA4's free BigQuery export is its one genuinely useful feature for a build like this — it puts raw events in a warehouse you control.

A common 2026 pattern: Plausible/Umami for the public-facing privacy-clean numbers, plus the warehouse export (PostHog → ClickHouse, or GA4 → BigQuery) for the deep modeling and AI narratives. Privacy-first does not mean data-poor; it means consent-free collection of non-PII event data you can still warehouse and reason over.

3. The event pipeline in practice

Event schema first. Before choosing storage, define a flat, append-only event table. A workable CMS event looks like:

event_id, timestamp, event_name, content_id, content_slug, content_type,
url, referrer, referrer_class (search|ai|social|direct|internal),
session_id (rotating daily salt, no PII), country, device, utm_*,
scroll_depth, time_on_page, is_bot, bot_name, props (JSON)

Two CMS-specific fields earn their keep: referrer_class with an explicit ai bucket (see §7), and bot_name so you can separate AI-crawler hits from human reads. Keep PII out by design — hash or rotate session IDs daily so you never store a stable cross-day identifier, which is what keeps you out of consent-banner territory.

Transport. For a CMS you rarely need Kafka on day one. Three tiers:

1. Small: the analytics tool ingests directly (PostHog/Umami API). No transport layer.
2. Medium: an open-source CDP — RudderStack, Jitsu, or Segment — fans one event out to your warehouse and your analytics tool, so you collect once and route many.
3. Large/streaming: Kafka/Redpanda + a stream processor (Flink) for windowed real-time alerts, with the warehouse as the serving layer. Most CMSes never need this.

Storage — the 2026 OLAP picture. The "OLAP renaissance" gives you genuinely good open options:

A clean, low-cost default for most CMS builds: collect via Umami/PostHog, export to a warehouse, and use DuckDB as the analysis engine — heavy ClickHouse for the billions-of-rows tier, DuckDB for the interactive thousands-to-millions the AI narratives actually read. The "ClickHouse for heavy lifting, DuckDB for light interactive" split is a recognized pattern.

4. The semantic layer: stop metric drift before it starts

The single biggest mistake in an analytics build is letting every dashboard re-define "active reader" or "engaged session" in its own SQL. Define metrics once in a semantic layer so dashboards, alerts, and the LLM narrative all compute the same number.

• dbt is the de-facto modeling layer: write staging → marts models, define metrics in YAML, and ship tests so a broken metric fails CI rather than silently lying.
• Lightdash reads the dbt semantic layer directly — metrics and dimensions defined in YAML become the foundation for all exploration, which "eliminates the metric-drift problem where BI calculations diverge from dbt definitions over time." Lightdash repositioned in 2026 as "agentic BI," exposing the semantic layer to LLM agents.
• Cube is the stack-agnostic alternative if you are not on dbt — a headless semantic layer that any tool (or an LLM) queries via SQL/REST/GraphQL.

The payoff for AI-native: when your LLM generates SQL or narratives, it queries the semantic layer, not raw tables. That is what makes text-to-SQL trustworthy — the model picks from governed metrics instead of hallucinating joins.

5. Dashboards: pick by who reads them

For a CMS, Metabase for editors + Evidence.dev for the engineering content report is a strong, cheap combination. Evidence in particular fits the AI-native story: a content report is itself content — Markdown with embedded SQL and charts, regenerated on a schedule, and a natural place to embed LLM-written narrative.

6. AI-generated insight narratives over your own data

This is the pillar realized. A raw dashboard requires a human to interpret it; an AI-native CMS writes the interpretation. Per Gartner (2025), 59% of enterprises now use some form of AI analytics, up from 33% in 2022 — narratives are the mainstream surface for it.

The narrative job, decomposed:

1. Detect — a scheduled job runs governed queries (week-over-week deltas, anomalies, top movers) against the semantic layer.
2. Summarize — an LLM converts the numeric result + metric metadata into prose: "Organic clicks to /guides/* fell 27% over the last 8 weeks, concentrated in three posts citing 2023 data; AI-referral sessions to the same cluster doubled but convert at one-third the rate."
3. Recommend — the model proposes actions tied to your content model: refresh post X, consolidate Y and Z, add an FAQ block to capture AI Overview citations.
4. Deliver — push the narrative into the editor dashboard, a weekly email, or an Evidence report.

Architecture rules that keep narratives honest:

• Deterministic math, generative prose. The LLM never computes the numbers — SQL/dbt does. The model only narrates a structured result set. This is the same discipline as text-to-SQL with a semantic layer: let the model pick governed metrics, then describe them. (Mixing the two is how you get confidently wrong analytics.)
• Ground every claim in a row. Pass the model the exact aggregated table and require it to reference values, ideally with a source_query it summarized. No free-floating percentages.
• Model choice (2026). General data-analysis benchmarks in 2026 put GPT-5.2 at the top with Claude 4.x and Qwen 3 as mature alternatives for long-context, multilingual reasoning. For a narrative-over-aggregates job the bottleneck is rarely the frontier model — a mid-tier model is fine because the hard part (the math) is already done. Use a cheap model for routine weekly narratives and reserve a frontier model for the quarterly deep-dive.
• Natural-language exploration as a second surface. Beyond scheduled narratives, expose ad-hoc text-to-SQL ("weekly active readers by content type, last 90 days") through the semantic layer so editors self-serve. The 2026 caveat from practitioners: this "collapses the analyst-to-chart loop from hours to minutes — assuming the semantic layer is clean." Garbage semantic layer, garbage answers.

7. Tracking AI traffic — the new, mandatory dimension

A 2026 CMS analytics build that ignores AI traffic is missing the most important trend in content distribution. Two distinct flows:

(a) AI crawlers (bots reading your content). Cloudflare data (May 2026) shows training is the single largest purpose of AI crawling at ~52.5% of requests, and only ~8.4% of AI bot traffic is search — the only category that can send a referral. Crawl-to-referral ratios are brutal: ClaudeBot ~13,528 pages crawled per human visit, OpenAI ~1,252:1, Perplexity ~95:1, vs Googlebot ~5:1. Implication for your pipeline: classify bot_name (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) and report crawl volume separately from human reads, so AI crawling doesn't inflate your engagement numbers.

(b) AI referrals (humans arriving from an AI answer). Add a referrer_class = ai bucket keyed on referrers like chatgpt.com, perplexity.ai, gemini.google.com, copilot.microsoft.com. These convert poorly: a SparkToro study (Jan 2026) found only 12-18% of Perplexity citations produce click-through, so 82-88% of citations send nobody. Track citations and visibility, not just clicks. Specialized monitors (Otterly.AI, Scrunch AI, SEOcrawl AI) watch whether your content is cited in AI Overviews / ChatGPT / Perplexity even when no click results — a leading indicator your decay model should incorporate, because organic CTR for informational queries with AI Overviews fell ~61% from June 2024 to Sept 2025.

The llms.txt standard (covered in this report's distribution chapters) is the supply side; this chapter is the measurement side — instrument both the crawlers reading your llms.txt and the referrals coming back.

8. Content-decay and refresh signals

This is the most CMS-specific analytics output and the cleanest place to close the loop.

Define decay as a multi-signal pattern, not a single drop. Industry guidance: a 20-40% drop in organic clicks over 8-12 weeks is confirmed decay needing intervention. But because AI Overviews now cannibalize clicks even at stable rankings, single-metric monitoring under-counts. Track signals moving together:

Build a decay score. Materialize a content_health table (per content_id) that joins GSC + analytics + CMS metadata, computes an 8-12-week trend per signal, and emits a weighted score plus a recommended action: refresh / consolidate / delete / leave. AI tooling here flags pages early — when impressions, clicks, CTR and position start moving together but before traffic fully collapses. Run it weekly; the cost of a DuckDB query over a few hundred thousand rows is negligible.

The action taxonomy (from refresh frameworks): Update (replace outdated stats, add sections, refresh metadata) is the default; Consolidate when two posts compete; Delete/redirect thin or duplicate pages; Leave seasonal or evergreen pages whose decline is expected.

9. Closing the loop: feedback into authoring & the AI pipeline

The analytics build is only "intelligence" if its output changes what gets written. Three loop integrations, in increasing autonomy:

1. Editor-facing (human-in-the-loop, default). The decay score and the LLM narrative surface inside the CMS as a "Content Health" panel on each entry: score, the specific decaying signals, the AI-suggested action, and a "Draft refresh" button. The editor decides. This is the safe, recommended baseline.

2. Draft generation (agent-assisted). When a page is flagged "refresh," a job assembles a refresh brief — the stale statistics, the queries now winning impressions, the AI-answer gaps where you're cited-but-not-clicked — and hands it to the authoring AI (see the report's authoring-pipeline chapters) to produce a draft update against your content model. The draft enters the normal review workflow; nothing publishes unattended.

3. Topic discovery (strategic). Aggregate rising queries, AI-referral landing patterns, and zero-click-but-cited topics into a backlog of new content the AI pipeline can draft. The same warehouse that measures decay now drives the editorial calendar.

Guardrails for the loop: keep a human approval gate before publish (matching the report's broader human-in-the-loop posture); log every AI-suggested action and its outcome so the decay model can learn which refreshes actually recovered traffic; and never let the AI both measure success and decide to publish without an independent check — separation of "judge" and "author" prevents the system from optimizing for its own metric. Feed recovered-vs-not outcomes back as labels to tune the decay weights over time.

Key Takeaways

• Treat analytics as a six-stage pipeline (capture → transport → store → model → serve → act); decouple stages so any component is swappable. The "act" stage is what makes it AI-native.
• GA4, Plausible/Umami, and PostHog answer different questions. For a content CMS, start with Umami or Plausible (cookieless, no banner, no data loss); use PostHog when you also want replay/flags/A-B; reserve GA4 for Google Ads attribution (and exploit its free BigQuery export).
• Define your event schema first, including a referrer_class=ai bucket and bot_name, and keep PII out by rotating session IDs daily — that's what keeps you consent-banner-free.
• Use a semantic layer (dbt + Lightdash, or Cube) so dashboards, alerts, and the LLM all compute the same metric — this is the prerequisite for trustworthy text-to-SQL and narratives.
• For storage, DuckDB/MotherDuck is the cheap interactive default; ClickHouse for billions of rows; BigQuery if you already have GA4 export.
• AI narratives must follow "deterministic math, generative prose" — SQL/dbt computes every number, the LLM only narrates a grounded result set; a mid-tier model suffices because the hard part is already solved.
• Instrument AI traffic on both sides: crawlers (report separately — ClaudeBot crawls ~13,500 pages per human visit) and referrals (12-18% Perplexity citation→click; track citations, not just clicks).
• Build a multi-signal content-decay score (20-40% click drop over 8-12 weeks is confirmed decay, but watch signals moving together because AI Overviews cannibalize clicks at stable rankings) that emits refresh/consolidate/delete/leave.
• Close the loop with a human-in-the-loop Content Health panel and AI-drafted refreshes that enter normal review; log outcomes to tune the decay model, and never let the same AI both measure and publish.

Key References

• PostHog. PostHog pricing — usage-based, free tier. 2026. https://posthog.com/pricing — Free 1M events + 5k recordings/mo; per-event and per-recording step-down pricing; self-host community edition (ClickHouse) with no limits.
• PostHog. The 9 best GDPR-compliant analytics tools. 2026. https://posthog.com/blog/best-gdpr-compliant-analytics-tools — Comparison framing GA4 vs Plausible/Umami vs PostHog by job-to-be-done and consent posture.
• OpenPanel. Self-Hosted Web Analytics 2026 — Plausible vs Matomo vs Umami vs OpenPanel. 2026. https://openpanel.dev/articles/self-hosted-web-analytics — Self-host licensing (Plausible CE/AGPL, Umami MIT), feature parity, cookieless compliance.
• Plausible. Plausible Analytics (cookieless, EU-hosted). 2026. https://plausible.io — Cloud from ~$9/mo, EU (Germany) hosting, Community Edition self-host requiring Postgres + ClickHouse.
• ClickHouse. What's new in ClickHouse — 2025 roundup. 2025. https://clickhouse.com/blog/clickhouse-2025-roundup — Fivetran connector, BigQuery→ClickHouse Dataflow template; event-stream analytics positioning.
• MotherDuck. Best database for real-time analytics in 2026. 2026. https://motherduck.com/learn/best-cloud-data-warehouses-real-time-analytics-2026/ — DuckDB growth (~1M weekly downloads), DuckDB+ClickHouse split pattern, storage/compute decoupling.
• Lightdash. Lightdash semantic layer + agentic BI. 2026. https://docs.lightdash.com/guides/lightdash-semantic-layer — dbt-native semantic layer eliminating metric drift; LLM-agent access to governed metrics.
• FutureAGI. AI for Creating Dashboards in 2026: Tools and Workflow. 2026. https://futureagi.com/blog/ai-for-creating-dashboards/ — Narrative generation, NL-to-chart loop, "clean semantic layer" caveat.
• LLM Intel. AI Analytics: The Complete Guide for 2026. 2026. https://llmintel.pro/blog/ai-analytics-complete-guide-2026 — Gartner 59% enterprise AI-analytics adoption; narrative summaries and anomaly alerts as mainstream surfaces.
• Foundry CRO. Track AI Search Referrals: ChatGPT & Perplexity (2026). 2026. https://foundrycro.com/blog/tracking-ai-search-referrals/ — Building an ai referral bucket; crawler vs referral distinction.
• TechnologyChecker. ChatGPT Statistics 2026 — Cloudflare Crawl Data. 2026-05. https://technologychecker.io/blog/chatgpt-statistics — Crawl-to-referral ratios (ClaudeBot ~13,528:1, OpenAI ~1,252:1, Perplexity ~95:1); training = 52.5% of AI crawling.
• Wellows. How to Detect SEO Content Decay Early Using AI. 2025. https://wellows.com/blog/detect-seo-content-decay/ — Multi-signal decay detection, early flagging before full traffic loss, AI Overview CTR -61% (Jun 2024–Sep 2025).
• ALM Corp. Content Decay: What It Is, Why It Happens, How to Fix It. 2026. https://almcorp.com/blog/content-decay/ — 20-40% click drop over 8-12 weeks threshold; outdated-statistics signal; update/consolidate/delete taxonomy.
• SEOTesting. Brilliant Tools for Monitoring Content Decay in 2025. 2025. https://seotesting.com/blog/content-decay-tools/ — GSC + Ahrefs + dedicated decay reports as the monitoring stack.
• Basedash. Best open source BI tools compared 2026. 2026. https://www.basedash.com/blog/best-open-source-bi-tools-compared-2026 — Metabase (60k+ orgs), Evidence.dev code-first reporting, Lightdash semantic-layer BI.