Tier 2 Metabase vs Rill verdict

PRD #75 splits the analytical contract into three surface tiers. Tier 2 is the local BI tier: a running, self-hostable server for analysts who need follow-up questions, filters, pivots, and dashboard editing over the same DuckDB warehouse used by the rest of the tutorial. Issue #70 made both candidates runnable. This lesson turns that prototype evidence into the M13-S6 verdict.

Verdict: Metabase is the Tier 2 winner. It fits the analyst-facing BI job more directly because it offers a broader question-building loop, clearer schema discovery, deeper chart controls, and a better path to a narrated dashboard walkthrough without turning the production lesson into a metrics-modeling project. Rill stays in the repo because it is useful evidence: it shows how far a BI-as-code surface can go when the analytical contract is already clean.

Rill loses because its strongest shape is a governed metrics application for known dimensions and measures, while this tier needs a general local BI surface where an analyst can start from the warehouse, inspect the schema, ask a question, reshape the answer, and explain the result in one sitting.

The decision is captured in ADR-0026.

What Tier 2 has to prove

Tier 1 is for durable authored pages. Tier 3 is for conversational, editor-like analysis. Tier 2 is the middle shape: not just a static report, and not an AI copilot. It should feel like a local PowerBI-style surface that a platform candidate can run during an interview or screencast, point at the DuckDB warehouse, and use to answer second-order questions without writing a bespoke frontend.

The comparison therefore uses the same three reference views from the prototype: the Pareto tradeoff between NDCG@10 and intra-list diversity, the constraint history table, and the per-publisher metric breakdown. Those views exercise the important parts of the analytical contract. They join the model evaluation table to constraint weights, preserve editor-facing audit fields, and expose publisher differences across EB-NeRD, Adressa, and MIND.

The winner is not the tool with the cleanest demo page. It is the tool that best supports an analyst who sees a suspicious metric, drills into the publisher or configuration that caused it, adjusts the view, and explains what changed without leaving the local-first story from ADR-0014.

Query-builder ergonomics

Metabase wins the query-builder ergonomics round. The prototype currently seeds native SQL questions because DuckDB support enters through the DuckDB driver and because the tutorial needs exact reference views. Even with that seed path, Metabase gives the analyst a familiar loop: saved questions, dashboard cards, filters, visualisation controls, and a route from question to dashboard without turning every change into repository work. Its interface is built around the unit the analyst understands, which is the question.

Rill is strongest when the modeling work has already happened. Its project files define models, metrics views, and explores. That is attractive for a governed metrics layer because the dimensions and measures are versioned in YAML. It is less attractive for this Tier 2 job because a follow-up question often begins before the final semantic shape is known. The analyst wants to ask “what if I split this by publisher and guardrail state?” before committing that shape as a reusable model.

Clicks-per-insight therefore favors Metabase for exploratory BI. Rill can make known metrics fast once they are modeled, but Metabase makes the transition from a seeded reference question to an analyst-owned variant shorter.

Auth flow

Auth flow is the cleanest Rill win. The local Rill compose service starts a preview server on 127.0.0.1:9009 with telemetry disabled and no account setup inside the tutorial. That is excellent for a local lesson: one command, one port, and the analyst is in the surface.

Metabase has more ceremony. The compose file starts a warehouse seed service, then a Java Metabase service, then a Node seed script that waits for the setup token, creates a local admin user, registers the DuckDB database, and seeds the saved questions and dashboard. That ceremony is automated, but it still means the first interaction is shaped by login and application state. The seeded email and password make the lesson reproducible; they do not make the auth flow invisible.

This does not overturn the verdict because auth is one part of the Tier 2 workflow, not the whole workflow. Rill has the smoother first minute. Metabase has the stronger second hour.

Schema discovery

Schema discovery is a Metabase strength. Tier 2 needs to handle a wide impressions table without forcing the analyst to read project YAML first. The candidate table, stg_unified_impressions, is intentionally plain: publisher, user, article, timestamp, click, recency, category, sentiment, and platform context columns need to remain visible enough for an analyst to reason about publisher-specific degradation. In a larger run this is the kind of table that reaches thirty columns quickly.

Metabase treats database browsing as a first-class BI activity. The analyst can register the DuckDB database, inspect tables, open saved questions, duplicate them, and learn the warehouse shape from the UI. The native SQL seed is not a dead end; it is an entry point into the database.

Rill discovers schema through its project structure. Models expose a curated subset, metrics views define the usable dimensions and measures, and explores present those selected fields. That is clean once the semantic contract is settled, but it is not as strong when the task is “find which publisher column or fixture volume explains this metric.” For this tutorial’s BI tier, schema discovery should start broad and narrow through the question. Metabase starts broader.

Chart customisation

Metabase also wins chart customisation depth. The seeded cards use a scatter plot, a table, and a bar chart because those are the minimum shapes needed for the comparison. The important part is what happens next: an analyst can change display type, labels, axes, aggregation, table columns, dashboard layout, and card placement inside the same surface.

Rill’s dashboards are crisp when the dashboard should be an Explore over a metrics view. The defaults in the prototype are understandable: compare by publisher, show average NDCG@10, diversity, coverage, and seeded volume measures. That is a strong governed-metrics experience. The tradeoff is that customisation flows through model and metrics files. That is excellent for review and repeatability, but less ergonomic when the analyst is shaping a story live.

For a public lesson, this matters. The screencast should show a candidate moving from a dashboard to a refined explanation, not editing YAML for every change in the narrative.

Per-publisher graceful degradation

Both candidates can represent per-publisher graceful degradation because both read the same DuckDB warehouse and both keep the publisher dimension explicit. The prototype forces the important cases into the reference views: EB-NeRD, Adressa, and MIND all appear in the Pareto and metric-breakdown views, and the constraint history keeps the guardrail settings visible beside metric movement.

Metabase has the better analyst loop for explaining degradation. A saved native question can carry the exact SQL that maps fixture names to reader-facing publisher labels, then the dashboard can expose filters and chart changes without hiding the query. When Adressa has weaker category coverage or MIND has different sensitivity behavior, the analyst can duplicate the question, add a publisher predicate, and keep the result as another saved card.

Rill has a cleaner reusable metric layer. Its publisher metric model carries impression, user, and article counts alongside metric averages, which is useful when a publisher degrades because the fixture is thinner rather than because the ranker behaved differently. That is a real advantage. The verdict still leans Metabase because Tier 2 is not only a metrics layer; it is an investigation and communication surface.

Operational shape

Operational shape favors Rill. The Rill Dockerfile installs the Rill binary into a slim Debian image, mounts the project read-only, mounts the warehouse read-only, disables telemetry, and starts one preview process. Its state is mostly project files plus a runtime volume.

Metabase is heavier. It uses a Java runtime image, downloads metabase.jar, adds the DuckDB driver as a plugin, persists an application database, and needs the seed process to create the local account, collection, database, saved questions, and dashboard. Startup has more moving parts and idle memory will be higher than the Rill preview process. The upside is that this heavier service is doing more of the generic BI product job: user state, saved questions, dashboard composition, and interactive editing.

ADR-0014 does not require the lightest possible container. It requires the surface to be local-first and inspectable. Both candidates pass that bar. Rill passes it more elegantly; Metabase spends more resources to deliver the right Tier 2 behavior.

Public-vs-private question

Neither candidate becomes the public docs surface in this issue. Tier 2 is a local BI service, so the deliverable should be a screencast-as-deliverable and a runnable local command rather than a public hosted dashboard. That keeps the module honest: the static public artefacts belong to Tier 1 and the docs site; Tier 2 proves the interactive analyst loop.

Metabase is better suited to that private-to-public story. A screencast can open the seeded dashboard, show the three reference questions, duplicate one, adjust the chart, explain a publisher-specific tradeoff, and end with a saved dashboard state. The viewer sees a BI workflow rather than a repository change.

Rill can produce a strong screencast too, especially for a platform engineer showing how metrics are modeled as code. That is not the primary Tier 2 buyer. The primary Tier 2 buyer is the analyst who needs a local BI surface for follow-up questions. Metabase tells that story with fewer translation steps.

Decision

Metabase becomes the Tier 2 winner. Rill remains runnable as the losing candidate and as a useful contrast for future work. The next productionisation slice should keep the existing Docker-only local shape, harden the Metabase seed path, and produce the screencast around the three reference views.

The decision is intentionally reversible at the module boundary. If a future lesson decides the production Tier 2 surface should be metrics-as-code first, Rill is still present, still mounted against the same warehouse, and still usable as evidence. For PRD #75, though, the BI tier should optimize for the analyst’s ad hoc question loop. That makes Metabase the clearer winner.