Start here

What this is (read this first)

This site looks like a tutorial about news recommenders. It is — but that is not why it exists. It exists so that one person — the owner of this lab — actually learns this stuff and makes it stick. The recommender, the data platform, the editor interface: those are not the deliverable. They are the material to practise against. The real product is durable understanding in the owner’s head, and the proof that the understanding is real is being able to rebuild the work without help. This decision is recorded in ADR-0033, and everything below follows from it.

That sounds like a small distinction. It changes everything. A normal tutorial is “done” when the code works. This one is never “done” by that test, because code working once teaches almost nothing — you learn a craft by doing it many times, not once. So the goal is not finish the tutorial. The goal is re-run the tutorial until the concepts are second nature. You can think of the whole repository as a gym, and each lesson as an exercise you come back to and repeat.

If that framing is new to you, the one-paragraph version is: build fluency by repetition. Read a lesson, understand the concept, then rebuild it by hand. Come back next week and rebuild it again — this time with less help. The day you can do a lesson cold, from an empty directory, with the docs closed, is the day that concept has stuck. That day is the only definition of success here.

What you are actually learning

Two things, drilled until automatic:

The technical stack — the muscle memory of standing up a real data and application system: DuckDB and Parquet for storage, dlt for ingestion, dbt for transformation, Dagster for orchestration, FastAPI for the service layer, sentence embeddings for the model — and the TypeScript editor interface (Express + HTMX) on top. The editor counts as much as the Python: building a genuinely good interface that lets a non-technical person drive a model is a skill in its own right, and it is the part with the most transferable value.
The domain reasoning — how to think about an accountable recommender: why editorial judgment belongs in the platform and not buried inside a model score, what a “candidate set” is versus a “ranking”, and how editorial constraints reshape what a reader sees. This is the part you can reason about, defend, and explain — not just type.

Things you are not here to master: the agent workflow that builds the lessons (that is how the gym gets stocked, not what you are drilling), and any claim of being a production recommender researcher (the model is deliberately modest, on purpose — see below).

How to practise this

There are three ways the work happens, and they are not in competition — they are different phases of the same loop:

Stocking the gym (agents, often AFK). Background agents build and deepen the lessons ahead of you: clean setup, seed data, teardown, a single clear concept, checkpoint questions, and docs that read like a textbook. This is preparation, not progress. Twenty correct lessons you never practise is not success — it is just a well-stocked gym nobody trained in.
Assisted reps. You do a lesson by hand while an assistant explains, quizzes you with the checkpoint questions, and catches misunderstandings. Training with a spotter.
Unassisted reps. You do the lesson cold, no help. This is the only thing that proves a concept stuck.

A word on “focused”, because it is easy to misread: focused does not mean short or minimal. It means full, verbose, and deep enough to get under your skin — the opposite of shallow and quickly forgotten. A good lesson here over- explains rather than under-explains. The rule “go deeper, not wider” only means: don’t bolt on a shiny new module to look impressive — earn real depth on what is already here first.

The documentation (this site) is your textbook. You return to it to refresh a concept, then close it and do the rep. The lessons themselves carry the self-test (the checkpoint questions); the docs just explain, clearly and at length.

The question the material is built around

The practice material happens to be a news recommender, and it is built around one genuinely interesting question:

If a news recommender shapes what readers see, how does the newsroom keep editorial responsibility visible?

The short version: the recommender model is deliberately modest — it just finds plausible articles for a reader. The platform around the model is the important part. It exposes the editorial choices, the application contract, the analyst contract, and the evidence needed to review what changed. If you only have five minutes for the domain side, read the thesis essay: it explains the core claim in plain English — editorial accountability belongs in the platform, not inside a hidden model score. For the story of how the lab builds itself with AI agents, read how this was built.

What you are looking at

The tutorial builds a small but inspectable recommendation platform. Each piece below is also a thing you will rebuild by hand:

A data platform ingests public news-recommendation data, shapes it into DuckDB/Parquet tables, and computes recommendation candidates.
A ranker applies editorial constraints to those candidates: topical mix, freshness, sentiment balance, editorial promotion, and sensitive-topic limits.
An editor interface lets a newsroom editor adjust those constraints and preview the recommendation list before saving a configuration.
An app contract exposes stable HTTP/OpenAPI endpoints for editor-facing tools.
An analytical contract exposes queryable tables so analysts can inspect the same underlying platform with SQL.
Evaluation and dashboard surfaces show whether a configuration is buying clicks at the cost of coverage, diversity, recency, tone, or sensitivity.

The tutorial is not trying to prove that this toy recommender beats production systems. It is trying to make the surrounding platform legible enough that an editor, analyst, or manager — or you, on your tenth rep — can ask better questions about the recommender.

The mental model

flowchart LR
    request["Reader request"]
    platform["Data platform<br/>DuckDB + Parquet + dbt"]
    model["Simple model<br/>finds candidate articles"]
    ranker["Ranker<br/>applies editorial policy"]
    editor["Editor interface<br/>tune and preview"]
    appContract["App contract<br/>OpenAPI over HTTP"]
    analyst["Analytical contract<br/>DuckDB tables"]
    evidence["Evaluation + dashboards<br/>show the tradeoffs"]

    platform --> model
    request --> model --> ranker
    editor --> appContract --> ranker
    platform --> appContract
    platform --> analyst
    ranker --> evidence
    analyst --> evidence

The model answers, “which articles might fit this reader?” The platform answers the harder editorial questions: “which constraints did we apply, who changed them, what did the list look like before and after, and what tradeoff did we accept?” Being able to draw this diagram from memory — and explain every arrow — is one of the concepts that should stick.

Where to go next

Thesis essay: the domain argument you are learning to reason about.
How this was built: the AI-native workflow that stocks the gym.
Tour: the same architecture in pictures.
Hub: deployed artefacts and operator links.
Editor module: the editor-facing surface and its app contract.
Data reference: generated dbt model docs and lineage.
API reference: generated OpenAPI reference for the app contract.

A click-optimised recommender can look successful while quietly narrowing a newsroom’s judgment. The platform is where that tradeoff becomes visible, tunable, testable, and reviewable — and learning to build that platform, by hand and repeatedly, is the point of this lab.