docs: write the README and map the build workflow

.dockerignore

@@ -0,0 +1,10 @@
+.git
+**/.venv
+**/__pycache__
+**/.mypy_cache
+**/.pytest_cache
+**/.ruff_cache
+frontend/node_modules
+eval/fixtures/raw
+dist
+**/dist

.env.example

@@ -1,5 +1,17 @@
-# Copy to .env and adjust. Without a .env the app starts in demo mode.
+# Runtime mode; set to demo for no-key fixtures or live for Spotify and Anthropic.
-APP_MODE=demo
+APP_MODE=
 
-# The redirect URI must exactly match the URI registered in the Spotify dashboard.
+# Spotify app client ID; required in live mode and unused in demo mode.
 SPOTIFY_CLIENT_ID=
+
+# Spotify OAuth callback URI; set in live mode if the local default is not registered.
+SPOTIFY_REDIRECT_URI=
+
+# Anthropic API key; required in live mode and unused in demo mode.
+ANTHROPIC_API_KEY=
+
+# Pre-authorized Spotify token; only for hosted instances without interactive login.
+SPOTIFY_SEED_REFRESH_TOKEN=
+
+# Secure session cookie flag; set to true on every HTTPS deployment.
+SESSION_COOKIE_SECURE=

.github/workflows/ci.yml

@@ -15,9 +15,9 @@ jobs:
       - uses: actions/checkout@v4
       - uses: astral-sh/setup-uv@v5
       - run: uv sync --frozen
-      - run: uv run ruff check .
+      - run: uv run ruff check . ../eval
-      - run: uv run ruff format --check .
+      - run: uv run ruff format --check . ../eval
-      - run: uv run mypy app
+      - run: uv run mypy app tests ../eval
       - run: uv run pytest
 
   frontend:

.gitignore

@@ -18,3 +18,8 @@ frontend/dist/
 
 # eval: raw (unredacted) recordings never enter the repo
 eval/fixtures/raw/
+eval/reports/
+
+# eval outputs (reports, baseline snapshots) are generated, never shipped
+eval/reports/
+eval/snapshots/

AGENTS.md

@@ -26,10 +26,9 @@ api -> pipeline -> ports <- adapters        domain imports nothing app-level
 - Spotify JSON never escapes `adapters/spotify/mapping.py`.
 - Pipeline tunables (counts, thresholds, budgets, model id, effort, TTLs) live
   in `app/config.py` (pydantic-settings). No magic numbers in code.
-- LLM prompts are versioned template files under `app/prompts/`, not inline
+- LLM prompts live as constants in `app/prompts.py`, not inline in adapters.
-  strings.
+- `eval/scenarios.yaml` defines golden eval queries and fixture-recording keys.
-- `eval/scenarios.yaml` is one source of truth for three consumers: golden
+  UI suggestion chips live in `frontend/src/lib/suggestions.ts`.
-  eval queries, demo fixture keys, and UI suggestion chips.
 
 ## Naming
 

README.md

@@ -1,35 +1,168 @@
 # discovery-by-llm
 
-This is a demo that serves as a proof of concept for LLM utilization for music discovery, specifically using the Spotify API. The form-factor is an LLM-chat like experience, with direct Spotify integration.
+Music discovery in a chat form: describe the moment, get verified Spotify
+tracks with a reason per track, save the result as a playlist.
 
-> Work in progress. This file is filled in during the build.
+Spotify removed its recommendation and audio-intelligence endpoints for new
-> Chronological build log (Dutch): [docs/logboek.md](docs/logboek.md).
+apps, so in this PoC the LLM fills that role. It interprets the request and
+proposes candidates; Spotify verifies them and supplies the taste profile,
+album art and playback. The eval suite compares the output against bare
+Spotify search on the same questions.
 
-## Demo
+> Chronological build log, including what was dropped and why (Dutch):
+> [docs/logboek.md](docs/logboek.md).
 
-*(to follow: hosted instance + local `docker compose up --build`, with and
+## Running it
-without API keys)*
+
+**Hosted instance** (credentials in the submission mail): a live deployment
+bound to my own Spotify account, with real personalisation and playlist
+writes. Spotify dev mode caps an app at 5 allowlisted users, so a reviewer
+cannot log in with their own account; I can allowlist a reviewer account on
+request.
+
+**Without any keys:**
+
+```
+docker compose up --build
+```
+
+Demo mode replays recorded API cassettes through the same pipeline and
+streaming path as live mode. The suggestion chips map to the recorded
+scenarios; any other input replays the nearest scenario, with a banner
+naming it. Playlist writes are simulated and labeled.
+
+**Live with your own keys:** copy `.env.example` to `.env`, set
+`APP_MODE=live`, a Spotify client id (redirect URI
+`http://127.0.0.1:8888/callback`) and an Anthropic API key, then
+`docker compose up --build` and log in via Spotify.
 
 ## How it works
 
-*(to follow: pipeline diagram and module map)*
+```
+query ──► call 1: intent + candidates ──► grounding ──► call 2: rerank ──► cards
+            (LLM, structured output)      (Spotify        (LLM, streamed
+                    ▲                       search          per track)
+            taste profile                   fan-out)            │
+            (cached, /me/top + saved)          │           playlist write
+                                          verified pool
+```
+
+- **Call 1** interprets the request (mood, activity, era, language,
+  familiarity) and proposes 35 candidate tracks, informed by a compressed
+  taste profile fetched once per session.
+- **Grounding** resolves every candidate against Spotify search in a
+  bounded concurrent fan-out with early stop, a request deadline and a
+  name-to-id cache. Unverified candidates are dropped. Misses and
+  mismatches are logged as separate rates.
+- **Call 2** reranks the verified pool and streams one justification per
+  track as NDJSON events; cards render as they arrive.
+- Only track ids from the verified pool can reach the user. In live
+  testing at familiarity `new`, the model fabricated 12 of 32 candidates;
+  the matcher dropped all 12 and the user still received 15 verified
+  tracks.
+- An honest limit on discovery depth: "new to you" means "not in your top
+  or saved tracks", and the intent prompt favors well-known tracks from
+  adjacent scenes because those verify reliably (chasing obscure work is
+  where the model starts inventing titles). A listener deep in those
+  scenes will recognize a fair share of the results. Going deeper needs a
+  repair loop and a better novelty signal; see the last section.
+- Refinement turns rerank the existing pool; turn 2 makes zero Spotify
+  calls.
 
 ## Choices
 
-*(to follow)*
+- **Two LLM calls around a deterministic grounding step**, no agent loop:
+  predictable latency and every stage testable on its own.
+- **Hand-rolled Spotify client** (~200 lines over httpx). The endpoint
+  surface is small and the retry, rate-limit and caching behavior is where
+  the engineering lives. Retry policy is per endpoint: bounded Retry-After
+  honor on reads, one token refresh, and no retries on playlist writes
+  (side effects would be ambiguous).
+- **Ports and adapters.** Demo mode is a second implementation of the same
+  ports. The domain model never sees Spotify JSON.
+- **NDJSON over a streamed POST.** The request has a body and EventSource
+  cannot POST. Typed events (metadata / track / warning / error / done)
+  with a strict client-side phase machine.
+- **Degradation is explicit.** Below the grounding floor the app returns
+  what resolved plus a warning; quota exhaustion (QUOTA_EXCEEDED) is
+  recognized and never retried; live mode never falls back to fixtures.
+- **Hosted seed session.** The public instance installs a session from an
+  escrowed refresh token at startup, so every visitor shares my account,
+  including playlist writes. HTTP basic auth at the edge is what makes
+  that acceptable. Playlists carry a fixed name prefix for bulk cleanup.
+- **No embeddings.** Search is the only entry point into the catalog; an
+  index does not pay for itself inside a PoC.
 
-## Performance and optimisations
+## Performance
 
-*(to follow)*
+Time to first card is 20 to 30 seconds, measured through the UI and in the
+eval suite; the spread is LLM latency. Streaming keeps the wait honest:
+metadata arrives early, cards render one by one, progress states name what
+is happening.
+
+Under the hood:
+
+- Search `limit` is capped at 10 since Feb 2026, so resolving 35 candidates
+  requires a fan-out: bounded concurrency, early stop once the rerank pool
+  is full, request deadline, partial results on quota exhaustion.
+- Name-to-id resolution cache and a session-scoped taste-profile cache.
+- Anthropic prompt caching between calls.
+- Per-request counters (Spotify calls, cache hits, LLM tokens) in
+  structured logs, surfaced in the dev panel.
+
+Bare search matches words; it has no notion of mood, era, energy or who is
+asking. The pipeline answers from the listener's actual scenes. Across all
+8 eval scenarios the pipeline delivered 11 to 15 verified tracks from 8 to
+20 distinct artists; on the discovery scenario, all 15 were outside the
+listener's top and saved tracks on Spotify. A sample (full table via
+`eval/run_eval.py --baseline`; the eval checks are plain code assertions,
+no LLM judging):
+
+| "Energy for the gym" | Pipeline                                 | Bare search                                                    |
+| -------------------- | ---------------------------------------- | -------------------------------------------------------------- |
+| 1                    | Baddadan by Chase & Status, Bou, Flowdan | Redbone (with GloRilla) by Lil Baby                            |
+| 2                    | Tough Talk by Chase & Status, Kwengface  | Rock That Body by Black Eyed Peas                              |
+| 3                    | Solar System by Sub Focus                | Instigator by Inpatient, Ren, Chris Webby                      |
+| 8                    | DJ Turn It Up by Dimension               | Gym Power Beat by Ultimate Fitness Playlist Power Workout Trax |
 
 ## Where to look
 
-*(to follow)*
+| Criterion                  | Where                                                                                                  |
+| -------------------------- | ------------------------------------------------------------------------------------------------------ |
+| Code quality and structure | `backend/app/` (domain / ports / adapters / pipeline / api), `AGENTS.md`                               |
+| AI application             | `backend/app/prompts.py`, `backend/app/adapters/anthropic/llm.py`, `backend/app/pipeline/grounding.py` |
+| Spotify integration        | `backend/app/adapters/spotify/` (PKCE, retry policy, mapping)                                          |
+| Performance                | `backend/app/pipeline/grounding.py` (fan-out, cache, early stop), `backend/app/observability/`         |
+| Streaming contract         | `backend/app/api/recommendations.py`, `frontend/src/lib/recommendationStream.ts`                       |
+| Evaluation                 | `eval/` (runner, scenarios, baseline arm, recorded fixtures)                                           |
+| Pragmatic cuts over time   | `docs/logboek.md`                                                                                      |
 
 ## Method
 
-*(to follow)*
+About one hour of preparation (API research, a design spike) and about
+eight hours of hands-on build time across three days. The build was
+agent-assisted: coding agents implemented scoped tracks from briefs I
+wrote, while I owned the architecture, the API contracts, every code
+review, every merge and all live verification against the real APIs. Work
+landed in reviewed batches, which is why commits arrive in bursts. The
+logboek documents the process as it happened; the setup behind it (server,
+Forgejo/GitHub mirror, deployment, how the agents are used) is mapped in
+[docs/workflow.md](docs/workflow.md).
 
-## What I cut / what I would do next
+## What I cut, and what would come next
 
-*(to follow)*
+- A faster model on the intent call, which would cut most of the time to
+  first card; the grounding stage already measures the trade-off to watch
+  (fabrication rate), so the experiment is cheap to run safely.
+- Deeper discovery. A repair loop that re-prompts the model for candidates
+  that failed verification would let the prompt chase less obvious work
+  without losing the pool, and a novelty signal beyond exact-id exclusion
+  (artist-level familiarity, recently played) would push results past
+  "famous in an adjacent scene".
+- Refinement turns that re-ground; today turn 2 reranks the existing pool.
+- Chat history persistence. Conversations are client-side only and the
+  server keeps no conversation state; sessions and caches are in-process,
+  Redis is the named production step.
+- Playback queue control, an i18n framework (copy is already centralized),
+  per-stage byte snapshots in the eval, and a semantic index over grounded
+  candidates.

backend/Dockerfile

@@ -6,11 +6,15 @@ RUN npm ci
 COPY frontend/ ./
 RUN npm run build
 
-FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim
+FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim AS live
-WORKDIR /srv
+WORKDIR /srv/backend
 COPY backend/pyproject.toml backend/uv.lock ./
 RUN uv sync --frozen --no-dev --no-install-project
 COPY backend/app ./app
+COPY eval/scenarios.yaml ../eval/scenarios.yaml
 COPY --from=frontend /build/dist ./app/static
 EXPOSE 8000
 CMD ["uv", "run", "--no-sync", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
+
+FROM live AS demo
+COPY eval/fixtures ../eval/fixtures

backend/app/adapters/anthropic/llm.py

@@ -112,7 +112,7 @@ class AnthropicRecommender:
         if parsed is None:
             raise RecommenderOutputError("Intent response contained no structured output")
         validated = IntentOutput.model_validate(parsed.model_dump())
-        return _to_intent(validated)
+        return to_intent(validated)
 
     async def stream_rerank(
         self,
@@ -125,7 +125,7 @@ class AnthropicRecommender:
     ) -> AsyncGenerator[RerankSelection]:
         """Yield each complete valid selection while the JSON is streaming."""
         schema = transform_schema(RerankOutput.model_json_schema())
-        parser = _RecommendationObjectParser()
+        parser = RecommendationObjectParser()
         async with self.client.messages.stream(
             model=self.settings.llm_model,
             max_tokens=self.settings.rerank_max_tokens,
@@ -166,8 +166,11 @@ class AnthropicRecommender:
             raise RecommenderOutputError("Rerank response returned too many selections")
 
 
-class _RecommendationObjectParser:
+class RecommendationObjectParser:
+    """Incrementally extract complete recommendation objects from JSON text."""
+
     def __init__(self) -> None:
+        """Create an empty parser for one rerank response."""
         self.complete_text = ""
         self._scan_index = 0
         self._object_start: int | None = None
@@ -177,6 +180,7 @@ class _RecommendationObjectParser:
         self._has_found_array = False
 
     def feed(self, text_delta: str) -> list[RerankSelectionOutput]:
+        """Consume a text delta and return newly completed selections."""
         self.complete_text += text_delta
         if not self._has_found_array:
             match = _RECOMMENDATION_ARRAY.search(self.complete_text)
@@ -221,7 +225,8 @@ class _RecommendationObjectParser:
         return None
 
     def _finish_object(self) -> RerankSelectionOutput:
-        assert self._object_start is not None
+        if self._object_start is None:
+            raise RecommenderOutputError("Rerank parser lost the object start position")
         object_text = self.complete_text[self._object_start : self._scan_index + 1]
         self._object_start = None
         try:
@@ -230,7 +235,8 @@ class _RecommendationObjectParser:
             raise RecommenderOutputError("Rerank item failed validation") from error
 
 
-def _to_intent(output: IntentOutput) -> Intent:
+def to_intent(output: IntentOutput) -> Intent:
+    """Map validated intent output to the application domain."""
     return Intent(
         mood=tuple(output.mood),
         activity=output.activity,

backend/app/adapters/demo/cassette.py

@@ -0,0 +1,128 @@
+"""Decode recorded Spotify and Anthropic responses for demo replay."""
+
+import base64
+import json
+from dataclasses import dataclass
+from functools import cache
+from pathlib import Path
+
+from app.adapters.demo.scenario import FIXTURE_ROOT
+
+
+@dataclass(frozen=True)
+class RecordedResponse:
+    """One decoded response retained from a recorded HTTP interaction."""
+
+    method: str
+    url: str
+    status: int
+    response_body: bytes
+    response_chunks: tuple[bytes, ...]
+
+    def json_body(self) -> object:
+        """Parse the decoded response body as JSON."""
+        return json.loads(self.response_body)
+
+
+@dataclass(frozen=True)
+class DemoCassette:
+    """The recorded service responses needed by one demo scenario."""
+
+    key: str
+    spotify_responses: tuple[RecordedResponse, ...]
+    intent_response_bodies: tuple[bytes, ...]
+    rerank_response_chunks: tuple[bytes, ...]
+
+    @property
+    def intent_response_body(self) -> bytes:
+        """Return the final intent response recorded for this scenario."""
+        return self.intent_response_bodies[-1]
+
+    @property
+    def spotify_search_responses(self) -> tuple[RecordedResponse, ...]:
+        """Return recorded Spotify search responses in capture order."""
+        return tuple(response for response in self.spotify_responses if "/search?" in response.url)
+
+    @property
+    def spotify_taste_responses(self) -> tuple[RecordedResponse, ...]:
+        """Return recorded Spotify taste responses in capture order."""
+        return tuple(
+            response
+            for response in self.spotify_responses
+            if any(
+                endpoint in response.url
+                for endpoint in ("/me/top/artists", "/me/top/tracks", "/me/tracks?")
+            )
+        )
+
+
+@cache
+def load_cassette(scenario_key: str) -> DemoCassette:
+    """Load and decode both service cassettes for one scenario."""
+    scenario_root = FIXTURE_ROOT / scenario_key
+    spotify = _load_responses(scenario_root / "spotify.json")
+    anthropic = _load_responses(scenario_root / "anthropic.json")
+    intent_responses = tuple(
+        response for response in anthropic if _is_anthropic_message(response.response_body)
+    )
+    rerank_responses = tuple(
+        response for response in anthropic if not _is_anthropic_message(response.response_body)
+    )
+    if not spotify or not intent_responses or not rerank_responses:
+        raise ValueError(f"Scenario cassette is incomplete: {scenario_key}")
+    return DemoCassette(
+        key=scenario_key,
+        spotify_responses=spotify,
+        intent_response_bodies=tuple(response.response_body for response in intent_responses),
+        rerank_response_chunks=rerank_responses[-1].response_chunks,
+    )
+
+
+def _load_responses(path: Path) -> tuple[RecordedResponse, ...]:
+    payload = json.loads(path.read_text(encoding="ascii"))
+    if not isinstance(payload, list):
+        raise ValueError(f"Cassette must contain a response list: {path}")
+    return tuple(_decode_response(entry) for entry in payload)
+
+
+def _decode_response(payload: object) -> RecordedResponse:
+    if not isinstance(payload, dict):
+        raise ValueError("Cassette response must be a mapping")
+    try:
+        chunks_value = payload["response_chunks_base64"]
+        if not isinstance(chunks_value, list):
+            raise TypeError
+        chunks = tuple(_decode_base64(value) for value in chunks_value)
+        return RecordedResponse(
+            method=_string(payload["method"]),
+            url=_string(payload["url"]),
+            status=_integer(payload["status"]),
+            response_body=_decode_base64(payload["response_body_base64"]),
+            response_chunks=chunks,
+        )
+    except (KeyError, TypeError, ValueError) as error:
+        raise ValueError("Cassette response is invalid") from error
+
+
+def _decode_base64(value: object) -> bytes:
+    return base64.b64decode(_string(value), validate=True)
+
+
+def _string(value: object) -> str:
+    if not isinstance(value, str):
+        raise TypeError
+    return value
+
+
+def _integer(value: object) -> int:
+    if not isinstance(value, int):
+        raise TypeError
+    return value
+
+
+def _is_anthropic_message(body: bytes) -> bool:
+    try:
+        payload = json.loads(body)
+    except (json.JSONDecodeError, UnicodeDecodeError):
+        return False
+    return isinstance(payload, dict) and payload.get("type") == "message"

backend/app/adapters/demo/catalog.py

@@ -0,0 +1,89 @@
+"""Spotify catalog adapter backed by one recorded demo cassette."""
+
+from collections import defaultdict
+from urllib.parse import parse_qs, urlparse
+
+from app.adapters.demo.cassette import DemoCassette, RecordedResponse
+from app.adapters.demo.scenario import normalize_text
+from app.adapters.spotify.mapping import (
+    parse_saved_track_page,
+    parse_search_tracks,
+    parse_top_artists,
+    parse_track_page,
+)
+from app.domain.models import Track
+from app.ports.protocols import TimeRange
+
+
+class DemoCatalog:
+    """Replay recorded Spotify search and taste responses without HTTP."""
+
+    def __init__(self, cassette: DemoCassette) -> None:
+        """Index one scenario cassette for deterministic request replay."""
+        self.cassette = cassette
+        self._search_pages = _index_search_pages(cassette.spotify_search_responses)
+        self._search_cursors: dict[str, int] = defaultdict(int)
+
+    async def search_tracks(self, query: str, limit: int = 10) -> list[Track]:
+        """Return the next recorded search page for a normalized query."""
+        normalized_query = normalize_text(query)
+        pages = self._search_pages.get(normalized_query, ())
+        cursor = self._search_cursors[normalized_query]
+        if cursor >= len(pages):
+            return []
+        self._search_cursors[normalized_query] += 1
+        return parse_search_tracks(pages[cursor].json_body())[:limit]
+
+    async def fetch_top_artists(self, time_range: TimeRange, limit: int) -> list[str]:
+        """Return the recorded synthetic top artists for a time range."""
+        response = self._taste_response("/me/top/artists", time_range)
+        return parse_top_artists(response.json_body())[:limit]
+
+    async def fetch_top_tracks(self, time_range: TimeRange, limit: int) -> list[Track]:
+        """Return the recorded synthetic top tracks for a time range."""
+        response = self._taste_response("/me/top/tracks", time_range)
+        return parse_track_page(response.json_body())[:limit]
+
+    async def fetch_saved_tracks(self, limit: int) -> list[Track]:
+        """Return recorded synthetic saved-track pages in offset order."""
+        responses = sorted(
+            (
+                response
+                for response in self.cassette.spotify_taste_responses
+                if urlparse(response.url).path.endswith("/me/tracks")
+            ),
+            key=_saved_track_offset,
+        )
+        tracks: list[Track] = []
+        for response in responses:
+            tracks.extend(parse_saved_track_page(response.json_body()))
+            if len(tracks) >= limit:
+                break
+        return tracks[:limit]
+
+    def _taste_response(self, endpoint: str, time_range: TimeRange) -> RecordedResponse:
+        for response in self.cassette.spotify_taste_responses:
+            parsed_url = urlparse(response.url)
+            query = parse_qs(parsed_url.query)
+            if parsed_url.path.endswith(endpoint) and query.get("time_range") == [time_range]:
+                return response
+        raise ValueError(f"Cassette lacks {endpoint} for {time_range}")
+
+
+def _index_search_pages(
+    responses: tuple[RecordedResponse, ...],
+) -> dict[str, tuple[RecordedResponse, ...]]:
+    pages: dict[str, list[RecordedResponse]] = defaultdict(list)
+    for response in responses:
+        query = parse_qs(urlparse(response.url).query).get("q")
+        if query:
+            pages[normalize_text(query[0])].append(response)
+    return {key: tuple(value) for key, value in pages.items()}
+
+
+def _saved_track_offset(response: RecordedResponse) -> int:
+    value = parse_qs(urlparse(response.url).query).get("offset", ["0"])[0]
+    try:
+        return int(value)
+    except ValueError:
+        return 0

backend/app/adapters/demo/pipeline.py

@@ -0,0 +1,78 @@
+"""Demo-only pipeline decorator for honest fuzzy replay disclosure."""
+
+import time
+from collections.abc import AsyncGenerator
+from contextlib import aclosing
+
+from app.adapters.demo.catalog import DemoCatalog
+from app.adapters.demo.recommender import DemoRecommender, parse_recorded_intent
+from app.adapters.demo.scenario import select_replay_scenario
+from app.config import Settings
+from app.domain.models import ConversationTurn, PreviousRecommendation, Track
+from app.pipeline.event import PipelineEvent, PipelineMetadataEvent, PipelineWarningEvent
+from app.pipeline.grounding import Grounder
+from app.pipeline.orchestrator import RecommendationPipeline
+from app.ports.protocols import MusicCatalog
+
+DEMO_REPLAY_CODE = "demo_replay"
+
+
+class DemoReplayPipeline:
+    """Decorate the real pipeline with a fuzzy-replay warning event."""
+
+    def __init__(self, settings: Settings) -> None:
+        """Create one cache-preserving pipeline with a replay recommender."""
+        self.settings = settings
+        self.pipeline = RecommendationPipeline(DemoRecommender(settings), settings)
+
+    async def stream(
+        self,
+        session_id: str,
+        request_id: str,
+        catalog: MusicCatalog,
+        query: str,
+        history: tuple[ConversationTurn, ...],
+        previous_recommendations: tuple[PreviousRecommendation, ...],
+    ) -> AsyncGenerator[PipelineEvent]:
+        """Stream the selected fixture and disclose non-exact selection."""
+        match = select_replay_scenario(query, bool(previous_recommendations))
+        seeded_pool = await self._prepare_refinement_pool(catalog)
+        event_stream = self.pipeline.stream(
+            session_id,
+            request_id,
+            catalog,
+            query,
+            history,
+            previous_recommendations,
+            seeded_pool=seeded_pool,
+        )
+        async with aclosing(event_stream) as events:
+            async for event in events:
+                yield event
+                if isinstance(event, PipelineMetadataEvent) and not match.is_exact:
+                    yield PipelineWarningEvent(
+                        code=DEMO_REPLAY_CODE,
+                        message=f'Demo replay is showing the recorded "{match.chip}" scenario.',
+                    )
+
+    async def _prepare_refinement_pool(
+        self,
+        catalog: MusicCatalog,
+    ) -> tuple[Track, ...] | None:
+        if not isinstance(catalog, DemoCatalog):
+            return None
+        intent_bodies = catalog.cassette.intent_response_bodies
+        if len(intent_bodies) < 2:
+            return None
+        # Refinement cassettes record the parent intent first and refinement intent last.
+        parent_intent = parse_recorded_intent(intent_bodies[0])
+        result = await Grounder(self.settings).ground(
+            catalog,
+            parent_intent.candidates,
+            # The recorded parent pool is listener-neutral, so replay has no known-track exclusions.
+            frozenset(),
+            parent_intent.familiarity,
+            self.settings.rerank_count + self.settings.rerank_pool_buffer,
+            time.monotonic() + self.settings.request_deadline_seconds,
+        )
+        return result.tracks

backend/app/adapters/demo/playlist.py

@@ -0,0 +1,28 @@
+"""Playlist writer that makes demo saves explicit and network-free."""
+
+import hashlib
+
+import structlog
+
+from app.domain.models import CreatedPlaylist
+
+
+class DemoPlaylistWriter:
+    """Simulate playlist writes with stable Spotify-shaped URLs."""
+
+    async def create_playlist(self, name: str, description: str) -> CreatedPlaylist:
+        """Return a deterministic fake playlist without an external write."""
+        digest = hashlib.sha256(f"{name}\n{description}".encode()).hexdigest()[:12]
+        playlist_id = f"demo-{digest}"
+        return CreatedPlaylist(
+            id=playlist_id,
+            url=f"https://open.spotify.com/playlist/{playlist_id}",
+        )
+
+    async def add_tracks_to_playlist(self, playlist_id: str, track_uris: list[str]) -> None:
+        """Log the simulated track addition without writing to Spotify."""
+        structlog.get_logger().info(
+            "demo_playlist_simulated",
+            playlist_id=playlist_id,
+            track_count=len(track_uris),
+        )

backend/app/adapters/demo/recommender.py

@@ -0,0 +1,128 @@
+"""Anthropic recommender adapter backed by recorded response chunks."""
+
+import asyncio
+import json
+from collections.abc import AsyncGenerator
+from contextvars import ContextVar
+
+from app.adapters.anthropic.llm import (
+    IntentOutput,
+    RecommendationObjectParser,
+    RerankOutput,
+    to_intent,
+)
+from app.adapters.demo.cassette import DemoCassette, load_cassette
+from app.adapters.demo.scenario import select_replay_scenario
+from app.config import Settings
+from app.domain.models import (
+    ConversationTurn,
+    Intent,
+    PreviousRecommendation,
+    RerankSelection,
+    Track,
+)
+from app.ports.protocols import RecommenderOutputError
+
+
+class DemoRecommender:
+    """Replay recorded intent and rerank output through live validation."""
+
+    def __init__(self, settings: Settings) -> None:
+        """Bind replay pacing and task-local scenario state."""
+        self.settings = settings
+        self._cassette: ContextVar[DemoCassette | None] = ContextVar(
+            "demo_recommender_cassette",
+            default=None,
+        )
+
+    async def create_intent(
+        self,
+        query: str,
+        history: tuple[ConversationTurn, ...],
+        previous_recommendations: tuple[PreviousRecommendation, ...],
+        taste_summary: str,
+        candidate_count: int,
+    ) -> Intent:
+        """Parse the selected scenario's recorded structured intent."""
+        match = select_replay_scenario(query, bool(previous_recommendations))
+        cassette = load_cassette(match.key)
+        self._cassette.set(cassette)
+        return parse_recorded_intent(cassette.intent_response_body)
+
+    async def stream_rerank(
+        self,
+        intent: Intent,
+        grounded_tracks: tuple[Track, ...],
+        taste_summary: str,
+        history: tuple[ConversationTurn, ...],
+        selection_count: int,
+        correction: str | None = None,
+    ) -> AsyncGenerator[RerankSelection]:
+        """Replay recorded SSE chunks through the live incremental parser."""
+        cassette = self._cassette.get()
+        if cassette is None:
+            raise RecommenderOutputError("Demo rerank has no selected scenario")
+        parser = RecommendationObjectParser()
+        async for text_delta in _text_deltas(
+            cassette.rerank_response_chunks,
+            self.settings.demo_chunk_delay_seconds,
+        ):
+            for selection in parser.feed(text_delta):
+                yield RerankSelection(selection.track_id, selection.justification)
+        try:
+            validated = RerankOutput.model_validate_json(parser.complete_text)
+        except ValueError as error:
+            raise RecommenderOutputError("Recorded rerank response is invalid") from error
+        if len(validated.recommendations) > selection_count:
+            raise RecommenderOutputError("Recorded rerank returned too many selections")
+
+
+def parse_recorded_intent(response_body: bytes) -> Intent:
+    """Parse an Anthropic message body through the live intent output model."""
+    try:
+        response = json.loads(response_body)
+        content = response["content"]
+        text = content[0]["text"]
+        if not isinstance(text, str):
+            raise TypeError
+        return to_intent(IntentOutput.model_validate_json(text))
+    except (IndexError, KeyError, TypeError, ValueError) as error:
+        raise RecommenderOutputError("Recorded intent response is invalid") from error
+
+
+async def _text_deltas(
+    chunks: tuple[bytes, ...],
+    delay_seconds: float,
+) -> AsyncGenerator[str]:
+    buffer = b""
+    for chunk_index, chunk in enumerate(chunks):
+        if chunk_index and delay_seconds > 0:
+            await asyncio.sleep(delay_seconds)
+        buffer += chunk
+        buffer = buffer.replace(b"\r\n", b"\n")
+        while b"\n\n" in buffer:
+            event, buffer = buffer.split(b"\n\n", 1)
+            text_delta = _event_text_delta(event)
+            if text_delta is not None:
+                yield text_delta
+    if buffer:
+        text_delta = _event_text_delta(buffer)
+        if text_delta is not None:
+            yield text_delta
+
+
+def _event_text_delta(event: bytes) -> str | None:
+    data_lines = [line[5:].strip() for line in event.splitlines() if line.startswith(b"data:")]
+    if not data_lines:
+        return None
+    try:
+        payload = json.loads(b"\n".join(data_lines))
+    except (json.JSONDecodeError, UnicodeDecodeError):
+        return None
+    if not isinstance(payload, dict) or payload.get("type") != "content_block_delta":
+        return None
+    delta = payload.get("delta")
+    if not isinstance(delta, dict) or delta.get("type") != "text_delta":
+        return None
+    text = delta.get("text")
+    return text if isinstance(text, str) else None

backend/app/adapters/demo/scenario.py

@@ -0,0 +1,151 @@
+"""Select replayable demo scenarios from the shared scenario catalog."""
+
+import re
+import unicodedata
+from dataclasses import dataclass
+from difflib import SequenceMatcher
+from functools import lru_cache
+from pathlib import Path
+
+import yaml
+
+_REPOSITORY_ROOT = Path(__file__).parents[4]
+SCENARIO_PATH = _REPOSITORY_ROOT / "eval" / "scenarios.yaml"
+FIXTURE_ROOT = _REPOSITORY_ROOT / "eval" / "fixtures"
+_WORD_PATTERN = re.compile(r"[a-z0-9]+")
+
+
+@dataclass(frozen=True)
+class Scenario:
+    """One shared suggestion and its optional parent scenario."""
+
+    key: str
+    chip: str
+    query: str
+    after: str | None = None
+
+    @property
+    def is_refinement(self) -> bool:
+        """Return whether this scenario refines an earlier result."""
+        return self.after is not None
+
+
+@dataclass(frozen=True)
+class ScenarioMatch:
+    """Describe which recorded scenario will answer a demo request."""
+
+    key: str
+    is_exact: bool
+    scenario_query: str
+    chip: str
+
+
+@lru_cache(maxsize=1)
+def load_scenarios() -> tuple[Scenario, ...]:
+    """Load and validate the shared scenario catalog once."""
+    payload = yaml.safe_load(SCENARIO_PATH.read_text(encoding="ascii"))
+    if not isinstance(payload, dict) or payload.get("version") != 1:
+        raise ValueError("Scenario catalog must have version 1")
+    entries = payload.get("scenarios")
+    if not isinstance(entries, list) or not entries:
+        raise ValueError("Scenario catalog must contain scenarios")
+    scenarios = tuple(_parse_scenario(entry) for entry in entries)
+    keys = {scenario.key for scenario in scenarios}
+    if len(keys) != len(scenarios):
+        raise ValueError("Scenario keys must be unique")
+    if any(scenario.after not in keys for scenario in scenarios if scenario.after is not None):
+        raise ValueError("Scenario catalog contains an unknown parent")
+    return scenarios
+
+
+def select_scenario(query: str) -> ScenarioMatch:
+    """Select an exact scenario or the nearest available recorded fixture."""
+    scenarios = _replayable_scenarios()
+    normalized_query = normalize_text(query)
+    for scenario in scenarios:
+        if normalized_query in {normalize_text(scenario.query), normalize_text(scenario.chip)}:
+            return _to_match(scenario, is_exact=True)
+
+    selected = max(
+        scenarios,
+        key=lambda scenario: max(
+            SequenceMatcher(None, normalized_query, normalize_text(scenario.query)).ratio(),
+            SequenceMatcher(None, normalized_query, normalize_text(scenario.chip)).ratio(),
+        ),
+    )
+    return _to_match(selected, is_exact=False)
+
+
+def select_replay_scenario(
+    query: str,
+    has_previous_recommendations: bool,
+) -> ScenarioMatch:
+    """Map a request with prior results to its recorded refinement when available."""
+    selected = select_scenario(query)
+    scenarios = _replayable_scenarios()
+    selected_scenario = next(scenario for scenario in scenarios if scenario.key == selected.key)
+    if selected_scenario.is_refinement or not has_previous_recommendations:
+        return selected
+    refinement = next(
+        (scenario for scenario in scenarios if scenario.after == selected_scenario.key),
+        None,
+    )
+    return _to_match(refinement, is_exact=False) if refinement is not None else selected
+
+
+def suggestion_items() -> list[dict[str, str]]:
+    """Return chip and query pairs for non-refinement scenarios."""
+    return [
+        {"chip": scenario.chip, "query": scenario.query}
+        for scenario in load_scenarios()
+        if not scenario.is_refinement
+    ]
+
+
+def normalize_text(value: str) -> str:
+    """Normalize text for exact and similarity matching."""
+    decomposed = unicodedata.normalize("NFKD", value.casefold())
+    ascii_text = decomposed.encode("ascii", errors="ignore").decode("ascii")
+    return " ".join(_WORD_PATTERN.findall(ascii_text))
+
+
+@lru_cache(maxsize=1)
+def _replayable_scenarios() -> tuple[Scenario, ...]:
+    fixture_keys = {path.name for path in FIXTURE_ROOT.iterdir() if path.is_dir()}
+    scenarios = tuple(scenario for scenario in load_scenarios() if scenario.key in fixture_keys)
+    if not scenarios:
+        raise ValueError("Demo mode requires at least one scenario fixture")
+    return scenarios
+
+
+def _parse_scenario(payload: object) -> Scenario:
+    if not isinstance(payload, dict):
+        raise ValueError("Each scenario must be a mapping")
+    required = {"key", "chip", "query", "facets"}
+    if not required <= set(payload):
+        raise ValueError("Scenario is missing a required field")
+    after = payload.get("after")
+    if after is not None and not isinstance(after, str):
+        raise ValueError("Scenario parent must be a key")
+    return Scenario(
+        key=_required_string(payload, "key"),
+        chip=_required_string(payload, "chip"),
+        query=_required_string(payload, "query"),
+        after=after,
+    )
+
+
+def _required_string(payload: dict[object, object], key: str) -> str:
+    value = payload.get(key)
+    if not isinstance(value, str) or not value:
+        raise ValueError(f"Scenario {key} must be a non-empty string")
+    return value
+
+
+def _to_match(scenario: Scenario, is_exact: bool) -> ScenarioMatch:
+    return ScenarioMatch(
+        key=scenario.key,
+        is_exact=is_exact,
+        scenario_query=scenario.query,
+        chip=scenario.chip,
+    )

backend/app/adapters/spotify/auth.py

@@ -13,11 +13,7 @@ from app.adapters.spotify.errors import SpotifyAuthenticationError
 
 AUTHORIZE_URL = "https://accounts.spotify.com/authorize"
 TOKEN_URL = "https://accounts.spotify.com/api/token"
-SCOPES = (
+SCOPES = "user-top-read user-library-read playlist-modify-public playlist-modify-private"
-    "user-top-read user-library-read user-read-recently-played "
-    "playlist-modify-public playlist-modify-private "
-    "user-read-playback-state user-modify-playback-state"
-)
 TOKEN_EXPIRY_SKEW_SECONDS = 60.0
 
 

backend/app/adapters/spotify/client.py

@@ -7,6 +7,7 @@ import httpx2
 from app.adapters.spotify.auth import refresh_access_token
 from app.adapters.spotify.errors import (
     SpotifyAuthenticationError,
+    SpotifyQuotaExhaustedError,
     SpotifyRateLimitedError,
     SpotifyRequestError,
     SpotifyUnavailableError,
@@ -119,7 +120,7 @@ class SpotifyClient:
         params: dict[str, str | int] | None = None,
         json: dict[str, object] | None = None,
     ) -> httpx2.Response:
-        access_token = await self._access_token()
+        access_token, refresh_generation = await self._access_token()
         response = await self._send(
             method,
             path,
@@ -127,18 +128,27 @@ class SpotifyClient:
             params=params,
             json=json,
         )
-        response = await self._retry_once_if_unauthorized(
+        response, refresh_generation = await self._retry_once_if_unauthorized(
             response,
             method,
             path,
-            access_token,
+            refresh_generation,
             params=params,
             json=json,
         )
-        response = await self._retry_once_if_rate_limited(
+        response, refresh_generation = await self._retry_once_if_rate_limited(
             response,
             method,
             path,
+            refresh_generation,
+            params=params,
+            json=json,
+        )
+        response, _ = await self._retry_once_if_unauthorized(
+            response,
+            method,
+            path,
+            refresh_generation,
             params=params,
             json=json,
         )
@@ -154,6 +164,7 @@ class SpotifyClient:
         json: dict[str, object] | None,
     ) -> httpx2.Response:
         increment_spotify_calls()
+        try:
             return await self.http.request(
                 method,
                 f"{self.settings.spotify_api_base_url.rstrip('/')}{path}",
@@ -161,27 +172,33 @@ class SpotifyClient:
                 json=json,
                 headers={"Authorization": f"Bearer {access_token}"},
             )
+        except httpx2.HTTPError as error:
+            raise SpotifyUnavailableError(504, "Spotify request failed") from error
 
     async def _retry_once_if_unauthorized(
         self,
         response: httpx2.Response,
         method: str,
         path: str,
-        access_token: str,
+        refresh_generation: int,
         *,
         params: dict[str, str | int] | None,
         json: dict[str, object] | None,
-    ) -> httpx2.Response:
+    ) -> tuple[httpx2.Response, int]:
         if response.status_code != 401:
-            return response
+            return response, refresh_generation
 
-        await self._refresh_if_current(access_token)
+        await self._refresh_if_current(refresh_generation)
-        return await self._send(
+        access_token, retry_generation = self._token_snapshot()
+        return (
+            await self._send(
                 method,
                 path,
-            self.session.tokens.access_token,
+                access_token,
                 params=params,
                 json=json,
+            ),
+            retry_generation,
         )
 
     async def _retry_once_if_rate_limited(
@@ -189,12 +206,13 @@ class SpotifyClient:
         response: httpx2.Response,
         method: str,
         path: str,
+        refresh_generation: int,
         *,
         params: dict[str, str | int] | None,
         json: dict[str, object] | None,
-    ) -> httpx2.Response:
+    ) -> tuple[httpx2.Response, int]:
         if response.status_code != 429:
-            return response
+            return response, refresh_generation
 
         retry_after_seconds = _parse_retry_after(response)
         _, reason = _parse_error_details(response)
@@ -204,15 +222,19 @@ class SpotifyClient:
             or retry_after_seconds is None
             or retry_after_seconds > self.settings.spotify_retry_after_cap_seconds
         ):
-            return response
+            return response, refresh_generation
 
         await asyncio.sleep(retry_after_seconds)
-        return await self._send(
+        access_token, retry_generation = self._token_snapshot()
+        return (
+            await self._send(
                 method,
                 path,
-            self.session.tokens.access_token,
+                access_token,
                 params=params,
                 json=json,
+            ),
+            retry_generation,
         )
 
     def _raise_for_error(self, response: httpx2.Response) -> httpx2.Response:
@@ -223,26 +245,33 @@ class SpotifyClient:
         if response.status_code == 401:
             raise SpotifyAuthenticationError("Spotify rejected refreshed authentication")
         if response.status_code == 429:
+            if reason == "QUOTA_EXCEEDED":
+                raise SpotifyQuotaExhaustedError(_parse_retry_after(response), reason)
             raise SpotifyRateLimitedError(_parse_retry_after(response), reason)
         if response.status_code >= 500:
             raise SpotifyUnavailableError(response.status_code, message)
         raise SpotifyRequestError(response.status_code, message)
 
-    async def _access_token(self) -> str:
+    async def _access_token(self) -> tuple[str, int]:
-        access_token = self.session.tokens.access_token
+        tokens = self.session.tokens
-        if self.session.tokens.is_expired:
+        refresh_generation = self.session.refresh_generation
-            await self._refresh_if_current(access_token)
+        if tokens.is_expired:
-        return self.session.tokens.access_token
+            await self._refresh_if_current(refresh_generation)
+        return self._token_snapshot()
 
-    async def _refresh_if_current(self, access_token: str) -> None:
+    async def _refresh_if_current(self, refresh_generation: int) -> None:
         async with self.session.refresh_lock:
-            if self.session.tokens.access_token != access_token:
+            if self.session.refresh_generation != refresh_generation:
                 return
             self.session.tokens = await refresh_access_token(
                 self.http,
                 client_id=self.settings.spotify_client_id,
                 tokens=self.session.tokens,
             )
+            self.session.refresh_generation += 1
+
+    def _token_snapshot(self) -> tuple[str, int]:
+        return self.session.tokens.access_token, self.session.refresh_generation
 
 
 def _parse_retry_after(response: httpx2.Response) -> float | None:

backend/app/adapters/spotify/errors.py

@@ -11,7 +11,7 @@ class SpotifyAuthenticationError(SpotifyError):
     """Spotify rejected authentication or token refresh."""
 
 
-class SpotifyRateLimitedError(SpotifyError, CatalogQuotaExhaustedError):
+class SpotifyRateLimitedError(SpotifyError):
     """Spotify rate limited a request that could not be retried."""
 
     def __init__(self, retry_after_seconds: float | None, reason: str | None = None) -> None:
@@ -21,6 +21,10 @@ class SpotifyRateLimitedError(SpotifyError, CatalogQuotaExhaustedError):
         self.reason = reason
 
 
+class SpotifyQuotaExhaustedError(SpotifyRateLimitedError, CatalogQuotaExhaustedError):
+    """Spotify rejected a request because the application quota is exhausted."""
+
+
 class SpotifyUnavailableError(SpotifyError):
     """Spotify returned a server-side failure."""
 

backend/app/adapters/spotify/session.py

@@ -17,6 +17,7 @@ class SpotifySession:
     tokens: TokenSet
     account_id: str
     display_name: str
+    refresh_generation: int = 0
     refresh_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
 
 
@@ -51,7 +52,15 @@ class PendingLogins:
 
     def add(self, state: str, code_verifier: str) -> None:
         """Store a PKCE verifier for a newly issued OAuth state."""
-        self._entries[state] = (code_verifier, time.monotonic())
+        now = time.monotonic()
+        expired_states = (
+            pending_state
+            for pending_state, (_, created_at) in self._entries.items()
+            if now - created_at >= PENDING_LOGIN_LIFETIME_SECONDS
+        )
+        for expired_state in tuple(expired_states):
+            del self._entries[expired_state]
+        self._entries[state] = (code_verifier, now)
 
     def pop(self, state: str) -> str | None:
         """Consume a verifier unless its OAuth state is unknown or expired."""

backend/app/api/recommendations.py

@@ -1,4 +1,4 @@
-"""Authenticated recommendation streaming and playlist creation routes."""
+"""Recommendation streaming and playlist creation routes."""
 
 import asyncio
 from collections.abc import AsyncIterator, Callable
@@ -8,7 +8,7 @@ import structlog
 from fastapi import APIRouter, HTTPException, Request
 from fastapi.responses import StreamingResponse
 
-from app.adapters.spotify.errors import SpotifyError
+from app.adapters.spotify.errors import SpotifyAuthenticationError, SpotifyError
 from app.adapters.spotify.session import SpotifySession
 from app.api.routes import resolve_session
 from app.api.schemas import (
@@ -38,6 +38,10 @@ from app.pipeline.orchestrator import RecommendationPipeline
 from app.ports.protocols import MusicCatalog, PlaylistWriter
 
 SpotifyClientFactory = Callable[[SpotifySession], MusicCatalog | PlaylistWriter]
+MusicCatalogFactory = Callable[
+    [SpotifySession, str, tuple[PreviousRecommendation, ...]],
+    MusicCatalog,
+]
 
 router = APIRouter()
 
@@ -47,14 +51,12 @@ async def recommendations(
     request: Request,
     payload: RecommendationRequest,
 ) -> StreamingResponse:
-    """Stream one authenticated discovery response as NDJSON."""
+    """Stream one session-resolved discovery response as NDJSON."""
     resolved = resolve_session(request)
     if resolved is None:
         raise HTTPException(status_code=401, detail="Not authenticated")
 
     pipeline = cast(RecommendationPipeline, request.app.state.recommendation_pipeline)
-    factory = cast(SpotifyClientFactory, request.app.state.spotify_client_factory)
-    catalog = cast(MusicCatalog, factory(resolved.session))
     request_id = cast(str, request.state.request_id)
     history = tuple(ConversationTurn(turn.role, turn.content) for turn in payload.history)
     previous = tuple(
@@ -66,10 +68,11 @@ async def recommendations(
         )
         for item in payload.prior_recommendations
     )
+    factory = cast(MusicCatalogFactory, request.app.state.music_catalog_factory)
+    catalog = factory(resolved.session, payload.query, previous)
 
     return StreamingResponse(
         _stream_lines(
-            request,
             pipeline,
             resolved.session_id,
             request_id,
@@ -87,7 +90,7 @@ async def create_playlist(
     request: Request,
     payload: PlaylistCreateRequest,
 ) -> PlaylistCreateResponse:
-    """Create and fill one authenticated Spotify playlist."""
+    """Create and fill one live or simulated playlist."""
     resolved = resolve_session(request)
     if resolved is None:
         raise HTTPException(status_code=401, detail="Not authenticated")
@@ -101,6 +104,9 @@ async def create_playlist(
             "Music discovery selected by the listener.",
         )
         await writer.add_tracks_to_playlist(playlist.id, payload.track_uris)
+    except SpotifyAuthenticationError as error:
+        structlog.get_logger().warning("playlist_write_failed", error_type=type(error).__name__)
+        raise HTTPException(status_code=401, detail="Spotify authentication expired") from error
     except (SpotifyError, ValueError) as error:
         structlog.get_logger().warning("playlist_write_failed", error_type=type(error).__name__)
         raise HTTPException(status_code=502, detail="Spotify playlist creation failed") from error
@@ -108,7 +114,6 @@ async def create_playlist(
 
 
 async def _stream_lines(
-    request: Request,
     pipeline: RecommendationPipeline,
     session_id: str,
     request_id: str,
@@ -127,8 +132,6 @@ async def _stream_lines(
     )
     try:
         async for event in event_stream:
-            if await request.is_disconnected():
-                return
             yield f"{_to_wire_event(event).model_dump_json()}\n"
     except asyncio.CancelledError:
         raise
@@ -137,7 +140,6 @@ async def _stream_lines(
             "recommendation_stream_failed",
             error_type=type(error).__name__,
         )
-        if not await request.is_disconnected():
         failure = ErrorEvent(
             code="recommendation_failed",
             message="Recommendation could not be completed.",

backend/app/api/routes.py

@@ -1,4 +1,4 @@
-"""HTTP routes for Spotify login and session management."""
+"""HTTP routes for login and session management."""
 
 from dataclasses import dataclass
 from typing import cast
@@ -9,7 +9,7 @@ from fastapi.responses import JSONResponse, RedirectResponse, Response
 
 from app.adapters.spotify.login import begin_login, complete_login
 from app.adapters.spotify.session import PendingLogins, SessionStore, SpotifySession
-from app.config import Settings
+from app.config import AppMode, Settings
 
 SESSION_COOKIE_NAME = "discovery_session"
 
@@ -26,8 +26,10 @@ class ResolvedSession:
 
 @router.get("/api/auth/login")
 def login(request: Request) -> RedirectResponse:
-    """Start Spotify Authorization Code with PKCE login."""
+    """Start the configured live or demo login flow."""
     application_settings = cast(Settings, request.app.state.settings)
+    if application_settings.app_mode is AppMode.DEMO:
+        return RedirectResponse("/?login=demo", status_code=307)
     pending_logins = cast(PendingLogins, request.app.state.pending_logins)
     authorize_url = begin_login(application_settings, pending_logins)
     return RedirectResponse(authorize_url, status_code=307)
@@ -40,11 +42,13 @@ async def callback(
     state: str | None = None,
     error: str | None = None,
 ) -> RedirectResponse:
-    """Complete Spotify login and establish an opaque cookie session."""
+    """Complete live login and establish an opaque cookie session."""
+    application_settings = cast(Settings, request.app.state.settings)
+    if application_settings.app_mode is AppMode.DEMO:
+        return RedirectResponse("/?login=demo", status_code=307)
     if error is not None or code is None or state is None:
         return _login_error_redirect()
 
-    application_settings = cast(Settings, request.app.state.settings)
     http = cast(httpx2.AsyncClient, request.app.state.http)
     pending_logins = cast(PendingLogins, request.app.state.pending_logins)
     session_store = cast(SessionStore, request.app.state.session_store)
@@ -80,6 +84,12 @@ def current_session(request: Request) -> JSONResponse:
     return JSONResponse({"display_name": resolved.session.display_name})
 
 
+@router.get("/api/suggestions")
+def suggestions(request: Request) -> list[dict[str, str]]:
+    """Return shared non-refinement suggestion chips and queries."""
+    return cast(list[dict[str, str]], request.app.state.suggestions)
+
+
 @router.post("/api/auth/logout", status_code=204)
 def logout(request: Request) -> Response:
     """Remove the current application session and clear its cookie."""
@@ -105,7 +115,11 @@ def _login_error_redirect() -> RedirectResponse:
 
 
 def resolve_session(request: Request) -> ResolvedSession | None:
-    """Resolve the cookie session or the installed live seed session."""
+    """Resolve the stable demo identity, cookie, or live seed session."""
+    application_settings = cast(Settings, request.app.state.settings)
+    if application_settings.app_mode is AppMode.DEMO:
+        demo_session = cast(SpotifySession, request.app.state.demo_session)
+        return ResolvedSession("demo", demo_session)
     session_store = cast(SessionStore, request.app.state.session_store)
     cookie_session_id = request.cookies.get(SESSION_COOKIE_NAME)
     if cookie_session_id is not None:

backend/app/config.py

@@ -41,6 +41,8 @@ class Settings(BaseSettings):
     llm_model: str = "claude-sonnet-5"
     intent_effort: str = "low"
     rerank_effort: str = "medium"
+    # Bound provider calls independently from the grounding deadline.
+    llm_timeout_seconds: float = 120.0
     # Ceilings include adaptive thinking tokens, which is why they sit far
     # above the size of the structured output itself.
     intent_max_tokens: int = 16384
@@ -71,6 +73,9 @@ class Settings(BaseSettings):
     resolution_cache_max_entries: int = 2048
     taste_profile_ttl_seconds: float = 900.0
 
+    # Demo replay paces recorded stream chunks so cards appear as they did live.
+    demo_chunk_delay_seconds: float = 0.05
+
     # Taste profile fetch bounds: enough signal to describe a listener
     # without paging through an entire library on session start.
     top_items_limit: int = 50

backend/app/main.py

@@ -4,18 +4,27 @@ from collections.abc import AsyncIterator
 from contextlib import asynccontextmanager
 from pathlib import Path
 
+import httpx
 import httpx2
+import structlog
 from anthropic import AsyncAnthropic
 from fastapi import FastAPI
 from fastapi.staticfiles import StaticFiles
 
 from app.adapters.anthropic.llm import AnthropicRecommender
+from app.adapters.demo.cassette import load_cassette
+from app.adapters.demo.catalog import DemoCatalog
+from app.adapters.demo.pipeline import DemoReplayPipeline
+from app.adapters.demo.playlist import DemoPlaylistWriter
+from app.adapters.demo.scenario import select_replay_scenario, suggestion_items
 from app.adapters.spotify.auth import TokenSet, refresh_access_token
 from app.adapters.spotify.client import SpotifyClient
+from app.adapters.spotify.errors import SpotifyError
 from app.adapters.spotify.session import PendingLogins, SessionStore, SpotifySession
 from app.api.recommendations import router as recommendations_router
 from app.api.routes import router
 from app.config import AppMode, Settings, settings
+from app.domain.models import PreviousRecommendation
 from app.observability.logging import configure_logging
 from app.observability.timing import RequestTimingMiddleware
 from app.pipeline.orchestrator import RecommendationPipeline
@@ -26,6 +35,7 @@ FRONTEND_DIST = Path(__file__).parent / "static"
 def create_app(
     application_settings: Settings | None = None,
     http_transport: httpx2.AsyncBaseTransport | None = None,
+    anthropic_http_client: httpx.AsyncClient | None = None,
 ) -> FastAPI:
     """Build the FastAPI app: API routes plus the built SPA on one port."""
     active_settings = application_settings or settings
@@ -34,17 +44,37 @@ def create_app(
     @asynccontextmanager
     async def lifespan(application: FastAPI) -> AsyncIterator[None]:
         _validate_live_settings(active_settings)
+        application.state.session_store = SessionStore()
+        application.state.pending_logins = PendingLogins()
+        application.state.settings = active_settings
+        application.state.seed_session_id = None
+        application.state.suggestions = suggestion_items()
+        if active_settings.app_mode is AppMode.DEMO:
+            application.state.demo_session = _demo_session()
+            application.state.recommendation_pipeline = DemoReplayPipeline(active_settings)
+
+            def demo_catalog_factory(
+                session: SpotifySession,
+                query: str,
+                previous_recommendations: tuple[PreviousRecommendation, ...],
+            ) -> DemoCatalog:
+                match = select_replay_scenario(query, bool(previous_recommendations))
+                return DemoCatalog(load_cassette(match.key))
+
+            application.state.music_catalog_factory = demo_catalog_factory
+            application.state.spotify_client_factory = lambda session: DemoPlaylistWriter()
+            yield
+            return
+
         async with httpx2.AsyncClient(
             timeout=active_settings.spotify_timeout_seconds,
             transport=http_transport,
         ) as http:
             application.state.http = http
-            application.state.session_store = SessionStore()
-            application.state.pending_logins = PendingLogins()
-            application.state.settings = active_settings
-            application.state.seed_session_id = None
             anthropic_client = AsyncAnthropic(
-                api_key=active_settings.anthropic_api_key or "unused-demo-key"
+                api_key=active_settings.anthropic_api_key,
+                timeout=active_settings.llm_timeout_seconds,
+                http_client=anthropic_http_client,
             )
             application.state.anthropic = anthropic_client
             application.state.recommendation_pipeline = RecommendationPipeline(
@@ -55,16 +85,30 @@ def create_app(
             def spotify_client_factory(session: SpotifySession) -> SpotifyClient:
                 return SpotifyClient(http, session, active_settings)
 
+            def music_catalog_factory(
+                session: SpotifySession,
+                query: str,
+                previous_recommendations: tuple[PreviousRecommendation, ...],
+            ) -> SpotifyClient:
+                return spotify_client_factory(session)
+
             application.state.spotify_client_factory = spotify_client_factory
+            application.state.music_catalog_factory = music_catalog_factory
             if (
                 active_settings.app_mode is AppMode.LIVE
                 and active_settings.spotify_seed_refresh_token
             ):
+                try:
                     application.state.seed_session_id = await _install_seed_session(
                         http,
                         application.state.session_store,
                         active_settings,
                     )
+                except (SpotifyError, ValueError) as error:
+                    structlog.get_logger().warning(
+                        "seed_session_install_failed",
+                        error_type=type(error).__name__,
+                    )
             try:
                 yield
             finally:
@@ -129,3 +173,11 @@ async def _install_seed_session(
             display_name=current_user.display_name,
         )
     )
+
+
+def _demo_session() -> SpotifySession:
+    return SpotifySession(
+        tokens=TokenSet(access_token="demo", refresh_token="demo", expires_at=float("inf")),
+        account_id="demo-listener",
+        display_name="Demo Listener",
+    )

backend/app/pipeline/grounding.py

@@ -7,8 +7,10 @@ from collections.abc import Callable
 from dataclasses import dataclass
 from enum import StrEnum
 
+import httpx2
 import structlog
 
+from app.adapters.spotify.errors import SpotifyError
 from app.config import Settings
 from app.domain.matching import candidate_key, judge_candidate_match, track_key
 from app.domain.models import Familiarity, Track, TrackCandidate
@@ -22,6 +24,7 @@ class ResolutionStatus(StrEnum):
     RESOLVED = "resolved"
     MISS = "miss"
     MISMATCH = "mismatch"
+    FAILED = "failed"
     QUOTA = "quota"
 
 
@@ -32,6 +35,7 @@ class GroundingMetrics:
     attempted_count: int
     miss_count: int
     mismatch_guard_count: int
+    failed_count: int
     cache_hit_count: int
     did_reach_deadline: bool
     did_exhaust_quota: bool
@@ -111,6 +115,7 @@ class Grounder:
             settings.resolution_cache_ttl_seconds,
             settings.resolution_cache_max_entries,
         )
+        self._semaphore = asyncio.Semaphore(settings.grounding_concurrency)
 
     async def ground(
         self,
@@ -119,6 +124,7 @@ class Grounder:
         known_track_ids: frozenset[str],
         familiarity: Familiarity,
         pool_target: int,
+        deadline_at: float,
     ) -> GroundingResult:
         """Resolve candidates until the target, deadline, or quota boundary."""
         accepted: dict[int, Track] = {}
@@ -127,18 +133,18 @@ class Grounder:
         metrics = _MutableMetrics()
         pending: dict[asyncio.Task[_ResolutionAttempt], int] = {}
         next_index = 0
-        deadline_at = time.monotonic() + self.settings.request_deadline_seconds
-        semaphore = asyncio.Semaphore(self.settings.grounding_concurrency)
         try:
             while next_index < len(candidates) or pending:
                 if len(accepted) >= pool_target:
                     break
+                if time.monotonic() >= deadline_at:
+                    metrics.did_reach_deadline = True
+                    break
                 next_index = self._launch_tasks(
                     catalog,
                     candidates,
                     pending,
                     next_index,
-                    semaphore,
                 )
                 if not pending:
                     break
@@ -183,12 +189,9 @@ class Grounder:
         candidates: tuple[TrackCandidate, ...],
         pending: dict[asyncio.Task[_ResolutionAttempt], int],
         next_index: int,
-        semaphore: asyncio.Semaphore,
     ) -> int:
         while next_index < len(candidates) and len(pending) < self.settings.grounding_concurrency:
-            task = asyncio.create_task(
+            task = asyncio.create_task(self._resolve(catalog, next_index, candidates[next_index]))
-                self._resolve(catalog, next_index, candidates[next_index], semaphore)
-            )
             pending[task] = next_index
             next_index += 1
         return next_index
@@ -228,9 +231,8 @@ class Grounder:
         catalog: MusicCatalog,
         index: int,
         candidate: TrackCandidate,
-        semaphore: asyncio.Semaphore,
     ) -> _ResolutionAttempt:
-        async with semaphore:
+        async with self._semaphore:
             return await self._resolve_with_slot(catalog, index, candidate)
 
     async def _resolve_with_slot(
@@ -265,6 +267,15 @@ class Grounder:
                 bare_results = []
         except CatalogQuotaExhaustedError:
             return _ResolutionAttempt(index=index, status=ResolutionStatus.QUOTA)
+        except (SpotifyError, httpx2.HTTPError) as error:
+            structlog.get_logger().info(
+                "candidate_unresolved",
+                title=candidate.title,
+                artist=candidate.artist,
+                status=ResolutionStatus.FAILED,
+                error_type=type(error).__name__,
+            )
+            return _ResolutionAttempt(index=index, status=ResolutionStatus.FAILED)
 
         if matched_track is not None:
             self.cache.put(key, matched_track)
@@ -289,6 +300,7 @@ class _MutableMetrics:
     attempted_count: int = 0
     miss_count: int = 0
     mismatch_guard_count: int = 0
+    failed_count: int = 0
     cache_hit_count: int = 0
     did_reach_deadline: bool = False
     did_exhaust_quota: bool = False
@@ -297,6 +309,7 @@ class _MutableMetrics:
         self.attempted_count += 1
         self.miss_count += attempt.status is ResolutionStatus.MISS
         self.mismatch_guard_count += attempt.status is ResolutionStatus.MISMATCH
+        self.failed_count += attempt.status is ResolutionStatus.FAILED
         self.cache_hit_count += attempt.is_cache_hit
         self.did_exhaust_quota = self.did_exhaust_quota or attempt.status is ResolutionStatus.QUOTA
 
@@ -305,6 +318,7 @@ class _MutableMetrics:
             attempted_count=self.attempted_count,
             miss_count=self.miss_count,
             mismatch_guard_count=self.mismatch_guard_count,
+            failed_count=self.failed_count,
             cache_hit_count=self.cache_hit_count,
             did_reach_deadline=self.did_reach_deadline,
             did_exhaust_quota=self.did_exhaust_quota,
@@ -340,6 +354,7 @@ def _log_grounding(result: GroundingResult) -> None:
         attempted_count=metrics.attempted_count,
         miss_rate=metrics.miss_rate,
         mismatch_guard_rate=metrics.mismatch_guard_rate,
+        failed_count=metrics.failed_count,
         cache_hits=metrics.cache_hit_count,
         deadline_reached=metrics.did_reach_deadline,
         quota_exhausted=metrics.did_exhaust_quota,

backend/app/pipeline/orchestrator.py

@@ -7,6 +7,7 @@ from contextlib import aclosing
 
 import structlog
 
+from app.adapters.spotify.errors import SpotifyError
 from app.config import Settings
 from app.domain.models import (
     CompressedTasteProfile,
@@ -27,7 +28,12 @@ from app.pipeline.event import (
     PipelineWarningEvent,
 )
 from app.pipeline.grounding import Grounder
-from app.ports.protocols import MusicCatalog, Recommender, RecommenderOutputError
+from app.ports.protocols import (
+    CatalogQuotaExhaustedError,
+    MusicCatalog,
+    Recommender,
+    RecommenderOutputError,
+)
 
 RERANK_FALLBACK_CODE = "rerank_fallback"
 RERANK_FALLBACK_MESSAGE = "Ranking output was invalid, so grounded results are shown instead."
@@ -112,9 +118,12 @@ class RecommendationPipeline:
         query: str,
         history: tuple[ConversationTurn, ...],
         previous_recommendations: tuple[PreviousRecommendation, ...],
+        seeded_pool: tuple[Track, ...] | None = None,
     ) -> AsyncGenerator[PipelineEvent]:
         """Yield ordered events for one recommendation request."""
         started_at = time.monotonic()
+        deadline_at = started_at + self.settings.request_deadline_seconds
+        try:
             taste = await self.taste_cache.get(session_id, catalog)
             intent = await self.recommender.create_intent(
                 query,
@@ -123,13 +132,35 @@ class RecommendationPipeline:
                 taste.text,
                 self.settings.candidate_count,
             )
+        except RecommenderOutputError:
+            yield PipelineErrorEvent(
+                code="intent_failed",
+                message="Recommendation intent could not be generated from the model response.",
+            )
+            return
+        except CatalogQuotaExhaustedError:
+            yield PipelineErrorEvent(
+                code="quota_exhausted",
+                message=(
+                    "Spotify request quota was exhausted before recommendations could be prepared."
+                ),
+            )
+            return
+        except SpotifyError:
+            yield PipelineErrorEvent(
+                code="spotify_unavailable",
+                message="Spotify was unavailable while preparing recommendations.",
+            )
+            return
         yield PipelineMetadataEvent(
             request_id=request_id,
             intent_summary=intent.intent_summary,
             candidate_count=len(intent.candidates),
         )
 
-        pool = await self._grounded_pool(session_id, catalog, intent, taste)
+        pool = await self._grounded_pool(
+            session_id, catalog, intent, taste, deadline_at, seeded_pool
+        )
         if not pool:
             yield PipelineErrorEvent(
                 code="no_grounded_results",
@@ -160,10 +191,14 @@ class RecommendationPipeline:
         catalog: MusicCatalog,
         intent: Intent,
         taste: CompressedTasteProfile,
+        deadline_at: float,
+        seeded_pool: tuple[Track, ...] | None,
     ) -> tuple[Track, ...]:
-        """Reuse the session's pool on refinement, otherwise ground anew."""
+        """Reuse the seeded or session pool on refinement, otherwise ground anew."""
         if intent.is_refinement:
-            cached_pool = self.last_pools.get(session_id)
+            cached_pool = (
+                seeded_pool if seeded_pool is not None else self.last_pools.get(session_id)
+            )
             if cached_pool:
                 return cached_pool
         result = await self.grounder.ground(
@@ -172,6 +207,7 @@ class RecommendationPipeline:
             taste.known_track_ids,
             intent.familiarity,
             self.settings.rerank_count + self.settings.rerank_pool_buffer,
+            deadline_at,
         )
         if result.tracks:
             self.last_pools[session_id] = result.tracks
@@ -290,19 +326,31 @@ class _TasteProfileCache:
         return profile
 
     async def _fetch(self, catalog: MusicCatalog) -> CompressedTasteProfile:
-        short_artists, long_artists, short_tracks, long_tracks, saved_tracks = await asyncio.gather(
+        try:
-            catalog.fetch_top_artists("short_term", self.settings.top_items_limit),
+            async with asyncio.TaskGroup() as task_group:
-            catalog.fetch_top_artists("long_term", self.settings.top_items_limit),
+                short_artists_task = task_group.create_task(
-            catalog.fetch_top_tracks("short_term", self.settings.top_items_limit),
+                    catalog.fetch_top_artists("short_term", self.settings.top_items_limit)
-            catalog.fetch_top_tracks("long_term", self.settings.top_items_limit),
-            catalog.fetch_saved_tracks(self.settings.saved_tracks_limit),
                 )
+                long_artists_task = task_group.create_task(
+                    catalog.fetch_top_artists("long_term", self.settings.top_items_limit)
+                )
+                short_tracks_task = task_group.create_task(
+                    catalog.fetch_top_tracks("short_term", self.settings.top_items_limit)
+                )
+                long_tracks_task = task_group.create_task(
+                    catalog.fetch_top_tracks("long_term", self.settings.top_items_limit)
+                )
+                saved_tracks_task = task_group.create_task(
+                    catalog.fetch_saved_tracks(self.settings.saved_tracks_limit)
+                )
+        except ExceptionGroup as errors:
+            raise errors.exceptions[0] from None
         return compress_taste_profile(
             TasteProfile(
-                short_term_artists=tuple(short_artists),
+                short_term_artists=tuple(short_artists_task.result()),
-                long_term_artists=tuple(long_artists),
+                long_term_artists=tuple(long_artists_task.result()),
-                short_term_tracks=tuple(short_tracks),
+                short_term_tracks=tuple(short_tracks_task.result()),
-                long_term_tracks=tuple(long_tracks),
+                long_term_tracks=tuple(long_tracks_task.result()),
-                saved_tracks=tuple(saved_tracks),
+                saved_tracks=tuple(saved_tracks_task.result()),
             )
         )

backend/app/prompts.py

@@ -49,15 +49,23 @@ appears prominently in the taste profile. Avoid duplicate titles by the same art
 avoid multiple editions of the same recording. Use a range of strong fits so a later
 ranking step has meaningful choices rather than thirty near-identical songs.
 
-When familiarity leans new, propose music the listener plausibly does not know. Move beyond
+When familiarity leans new, novelty means plausibly new to this listener, not obscure.
-the named top artists and tracks while retaining understandable bridges through genre,
+Known tracks are excluded after grounding, so novelty relative to the taste profile is
-scene, production style, instrumentation, energy, era, or songwriting. Do not simply select
+enforced downstream; do not chase rarity. Name each artist's signature track, biggest
-deep cuts from every familiar artist. Prefer adjacent artists, overlooked catalogs,
+single, or a similarly famous recording from their catalog: a title you have seen written
-regional scenes, and credible cross-genre connections. The profile is not exhaustive, so
+down many times, spelled exactly as released. Never guess a title that merely sounds like
-never claim that a candidate is definitely unknown. When familiarity is familiar,
+the artist; if no specific famous title comes to mind for an artist, pick a different
-candidates may include supplied top or saved tracks, but remain responsive to the current
+artist whose hit you can name with certainty. Famous artists adjacent to the taste profile
-request. When familiarity is mix, combine recognizable anchors with adjacent discoveries
+are welcome; their best-known work is still new to a listener who does not play them.
-rather than splitting into unrelated halves.
+Prefer the anthems of the profile's own scenes and neighboring scenes over global chart
+hits: the certainty bar stays the same, but a scene's signature tracks fit the taste far
+better than whatever topped the general charts.
+Retain understandable bridges through genre, scene, production style, instrumentation,
+energy, era, or songwriting. The profile is not exhaustive, so never claim that a
+candidate is definitely unknown. When familiarity is
+familiar, candidates may include supplied top or saved tracks, but remain responsive to the
+current request. When familiarity is mix, combine recognizable anchors with real,
+confidently identifiable adjacent discoveries rather than splitting into unrelated halves.
 
 Use musical knowledge conservatively. Base selection on durable, commonly knowable
 attributes of recordings. Do not invent listening statistics, personal memories, release

backend/pyproject.toml

@@ -6,8 +6,10 @@ requires-python = "==3.13.*"
 dependencies = [
     "anthropic==0.121.0",
     "fastapi>=0.116",
+    "httpx>=0.28",
     "httpx2>=2.10",
     "pydantic-settings>=2.10",
+    "pyyaml>=6.0",
     "structlog>=25.4",
     "uvicorn[standard]>=0.35",
 ]
@@ -17,6 +19,7 @@ dev = [
     "mypy>=1.17",
     "pytest>=8.4",
     "ruff>=0.12",
+    "types-pyyaml>=6.0",
 ]
 
 [tool.ruff]
@@ -36,4 +39,5 @@ module = ["app.domain.*", "app.ports.*", "app.pipeline.*"]
 strict = true
 
 [tool.pytest.ini_options]
-testpaths = ["tests"]
+pythonpath = [".", "../eval"]
+testpaths = ["tests", "../eval/tests"]

backend/tests/test_anthropic_llm.py

@@ -0,0 +1,13 @@
+"""Focused tests for Anthropic response parsing."""
+
+import pytest
+
+from app.adapters.anthropic.llm import RecommendationObjectParser
+from app.ports.protocols import RecommenderOutputError
+
+
+def test_parser_missing_object_start_raises_typed_output_error() -> None:
+    parser = RecommendationObjectParser()
+
+    with pytest.raises(RecommenderOutputError, match="object start position"):
+        parser._finish_object()

backend/tests/test_auth_routes.py

@@ -113,6 +113,29 @@ def test_seed_session_authenticates_requests_without_a_cookie() -> None:
     assert response.json() == {"display_name": "Seed Listener"}
 
 
+def test_seed_session_failure_keeps_application_serving() -> None:
+    async def spotify_handler(request: httpx2.Request) -> httpx2.Response:
+        return httpx2.Response(400)
+
+    app = create_app(
+        application_settings=Settings(
+            app_mode=AppMode.LIVE,
+            spotify_client_id="client-id",
+            anthropic_api_key="test-key",
+            spotify_seed_refresh_token="seed-refresh",
+        ),
+        http_transport=httpx2.MockTransport(spotify_handler),
+    )
+    with TestClient(app, follow_redirects=False) as client:
+        health_response = client.get("/api/health")
+        login_response = client.get("/api/auth/login")
+
+        assert app.state.seed_session_id is None
+
+    assert health_response.status_code == 200
+    assert login_response.status_code == 307
+
+
 def _live_settings() -> Settings:
     return Settings(
         app_mode=AppMode.LIVE,

backend/tests/test_demo_adapters.py

@@ -0,0 +1,68 @@
+"""Offline replay tests for demo service adapters."""
+
+import asyncio
+
+from app.adapters.demo.cassette import load_cassette
+from app.adapters.demo.catalog import DemoCatalog
+from app.adapters.demo.recommender import DemoRecommender
+from app.config import Settings
+
+
+def test_cassette_decodes_bodies_and_preserves_rerank_chunks() -> None:
+    cassette = load_cassette("focus-coding")
+
+    assert cassette.intent_response_body.startswith(b'{"model"')
+    assert len(cassette.spotify_search_responses) > 20
+    assert len(cassette.spotify_taste_responses) == 6
+    assert len(cassette.rerank_response_chunks) > 1
+    assert b"event: content_block_delta" in b"".join(cassette.rerank_response_chunks)
+
+
+def test_demo_catalog_replays_search_and_synthetic_taste_pages() -> None:
+    async def run() -> None:
+        catalog = DemoCatalog(load_cassette("focus-coding"))
+
+        tracks = await catalog.search_tracks('track:"Stay" artist:"Hybrid Minds"')
+        artists = await catalog.fetch_top_artists("short_term", 50)
+        top_tracks = await catalog.fetch_top_tracks("long_term", 50)
+        saved_tracks = await catalog.fetch_saved_tracks(100)
+
+        assert tracks[0].title == "Stay"
+        assert artists[0] == "Synthetic Focus Artist"
+        assert {track.title for track in top_tracks} >= {
+            "Synthetic Focus Track",
+            "Synthetic Jazz Track",
+        }
+        assert saved_tracks[0].title == "Synthetic Saved Track"
+
+    asyncio.run(run())
+
+
+def test_demo_recommender_parses_intent_and_streams_recorded_rerank() -> None:
+    async def run() -> None:
+        recommender = DemoRecommender(Settings(demo_chunk_delay_seconds=0))
+        intent = await recommender.create_intent(
+            "Focus while coding",
+            (),
+            (),
+            "Synthetic taste",
+            35,
+        )
+        selections = [
+            selection
+            async for selection in recommender.stream_rerank(
+                intent,
+                (),
+                "Synthetic taste",
+                (),
+                15,
+            )
+        ]
+
+        assert intent.activity == "programming"
+        assert len(intent.candidates) == 35
+        assert len(selections) == 15
+        assert selections[0].track_id == "2nIixNuuV5eHCJydG5aYIB"
+        assert all(selection.justification for selection in selections)
+
+    asyncio.run(run())

backend/tests/test_demo_app.py

@@ -0,0 +1,208 @@
+"""Full HTTP contract tests for key-free demo mode."""
+
+import asyncio
+
+import httpx
+from fastapi.testclient import TestClient
+from pydantic import TypeAdapter
+from starlette.types import ASGIApp, Message, Receive, Scope, Send
+
+from app.adapters.demo.scenario import load_scenarios
+from app.api.schemas import StreamEvent
+from app.config import Settings
+from app.main import create_app
+
+
+class _MetadataBarrier:
+    """Pause one marked ASGI response immediately after its metadata event."""
+
+    def __init__(self, app: ASGIApp) -> None:
+        self.app = app
+        self.metadata_sent = asyncio.Event()
+        self.resume_response = asyncio.Event()
+
+    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
+        is_refinement = dict(scope.get("headers", ())).get(b"x-demo-request") == b"refinement"
+
+        async def send_with_barrier(message: Message) -> None:
+            await send(message)
+            if message["type"] == "http.response.body" and b'"type":"metadata"' in message.get(
+                "body", b""
+            ):
+                self.metadata_sent.set()
+                await self.resume_response.wait()
+
+        await self.app(scope, receive, send_with_barrier if is_refinement else send)
+
+
+def test_demo_request_streams_valid_events_without_a_session() -> None:
+    app = create_app(Settings(demo_chunk_delay_seconds=0))
+    with TestClient(app) as client:
+        response = client.post(
+            "/api/recommendations",
+            json={"schema_version": 1, "query": "Focus while coding"},
+        )
+
+    adapter: TypeAdapter[StreamEvent] = TypeAdapter(StreamEvent)
+    events = [adapter.validate_json(line) for line in response.text.splitlines()]
+    assert response.status_code == 200
+    assert events[0].type == "metadata"
+    assert events[-1].type == "done"
+    assert sum(event.type == "track" for event in events) == 15
+
+
+def test_unknown_demo_query_discloses_the_replayed_scenario_before_tracks() -> None:
+    app = create_app(Settings(demo_chunk_delay_seconds=0))
+    with TestClient(app) as client:
+        response = client.post(
+            "/api/recommendations",
+            json={"schema_version": 1, "query": "Focus while codign"},
+        )
+
+    adapter: TypeAdapter[StreamEvent] = TypeAdapter(StreamEvent)
+    events = [adapter.validate_json(line) for line in response.text.splitlines()]
+    warning_index = next(
+        index
+        for index, event in enumerate(events)
+        if event.type == "warning" and event.code == "demo_replay"
+    )
+    first_track_index = next(index for index, event in enumerate(events) if event.type == "track")
+    warning = events[warning_index]
+    assert warning_index < first_track_index
+    assert warning.type == "warning"
+    assert "Focus while coding" in warning.message
+
+
+def test_demo_refinement_reconstructs_the_pool_recorded_with_its_cassette() -> None:
+    app = create_app(Settings(demo_chunk_delay_seconds=0))
+    with TestClient(app) as client:
+        initial_response = client.post(
+            "/api/recommendations",
+            json={"schema_version": 1, "query": "Focus while coding"},
+        )
+        initial_adapter: TypeAdapter[StreamEvent] = TypeAdapter(StreamEvent)
+        initial_events: list[StreamEvent] = [
+            initial_adapter.validate_json(line) for line in initial_response.text.splitlines()
+        ]
+        prior_recommendations = [
+            {
+                "rank": event.rank,
+                "track_id": event.track.id,
+                "title": event.track.title,
+                "artists": event.track.artists,
+            }
+            for event in initial_events
+            if event.type == "track"
+        ]
+        response = client.post(
+            "/api/recommendations",
+            json={
+                "schema_version": 1,
+                "query": "More electronic",
+                "prior_recommendations": prior_recommendations,
+            },
+        )
+
+    adapter: TypeAdapter[StreamEvent] = TypeAdapter(StreamEvent)
+    events = [adapter.validate_json(line) for line in response.text.splitlines()]
+    warning_codes = [event.code for event in events if event.type == "warning"]
+    assert "rerank_fallback" not in warning_codes
+    assert sum(event.type == "track" for event in events) == 15
+
+
+def test_concurrent_demo_refinement_uses_its_request_local_recorded_pool() -> None:
+    async def run() -> None:
+        settings = Settings(demo_chunk_delay_seconds=0)
+        previous = [
+            {
+                "rank": 1,
+                "track_id": "previous",
+                "title": "Previous track",
+                "artists": ["Previous artist"],
+            }
+        ]
+        payload = {
+            "schema_version": 1,
+            "query": "More electronic",
+            "prior_recommendations": previous,
+        }
+
+        baseline_app = create_app(settings)
+        baseline_transport = httpx.ASGITransport(app=baseline_app)
+        async with (
+            baseline_app.router.lifespan_context(baseline_app),
+            httpx.AsyncClient(
+                transport=baseline_transport,
+                base_url="http://test",
+            ) as client,
+        ):
+            baseline_response = await client.post("/api/recommendations", json=payload)
+        baseline_events = _stream_events(baseline_response)
+        expected_track_ids = [event.track.id for event in baseline_events if event.type == "track"]
+
+        concurrent_app = create_app(settings)
+        barrier = _MetadataBarrier(concurrent_app)
+        transport = httpx.ASGITransport(app=barrier)
+        async with (
+            concurrent_app.router.lifespan_context(concurrent_app),
+            httpx.AsyncClient(transport=transport, base_url="http://test") as client,
+        ):
+            refinement_task = asyncio.create_task(
+                client.post(
+                    "/api/recommendations",
+                    json=payload,
+                    headers={"x-demo-request": "refinement"},
+                )
+            )
+            try:
+                await asyncio.wait_for(barrier.metadata_sent.wait(), timeout=2)
+                unrelated_response = await client.post(
+                    "/api/recommendations",
+                    json={"schema_version": 1, "query": "Energy for the gym"},
+                )
+            finally:
+                barrier.resume_response.set()
+            refinement_response = await refinement_task
+
+        events = _stream_events(refinement_response)
+        warning_codes = [event.code for event in events if event.type == "warning"]
+        track_ids = [event.track.id for event in events if event.type == "track"]
+        assert unrelated_response.status_code == 200
+        assert refinement_response.status_code == 200
+        assert "rerank_fallback" not in warning_codes
+        assert track_ids == expected_track_ids
+
+    asyncio.run(run())
+
+
+def test_demo_auth_playlist_and_suggestions_are_explicitly_simulated() -> None:
+    app = create_app(Settings(demo_chunk_delay_seconds=0))
+    with TestClient(app, follow_redirects=False) as client:
+        current_user = client.get("/api/auth/me")
+        login = client.get("/api/auth/login")
+        playlist = client.post(
+            "/api/playlists",
+            json={
+                "schema_version": 1,
+                "name": "Night drive",
+                "track_uris": ["spotify:track:demo"],
+            },
+        )
+        suggestions = client.get("/api/suggestions")
+
+    expected_suggestions = [
+        {"chip": scenario.chip, "query": scenario.query}
+        for scenario in load_scenarios()
+        if not scenario.is_refinement
+    ]
+    assert current_user.json() == {"display_name": "Demo Listener"}
+    assert login.headers["location"] == "/?login=demo"
+    assert playlist.json()["url"].startswith("https://open.spotify.com/playlist/demo-")
+    assert suggestions.json() == expected_suggestions
+    assert not hasattr(app.state, "http")
+    assert not hasattr(app.state, "anthropic")
+
+
+def _stream_events(response: httpx.Response) -> list[StreamEvent]:
+    adapter: TypeAdapter[StreamEvent] = TypeAdapter(StreamEvent)
+    return [adapter.validate_json(line) for line in response.text.splitlines()]

backend/tests/test_demo_scenario.py

@@ -0,0 +1,32 @@
+"""Tests for deterministic demo scenario selection."""
+
+from app.adapters.demo.scenario import select_replay_scenario, select_scenario
+
+
+def test_scenario_selection_prefers_normalized_exact_query_or_chip() -> None:
+    query_match = select_scenario("something calm for while I am programming, but not boring")
+    chip_match = select_scenario("  FOCUS, while CODING!  ")
+
+    assert query_match.key == "focus-coding"
+    assert query_match.is_exact
+    assert chip_match.key == "focus-coding"
+    assert chip_match.is_exact
+
+
+def test_scenario_selection_falls_back_to_nearest_fixture() -> None:
+    match = select_scenario("Focus while codign")
+
+    assert match.key == "focus-coding"
+    assert not match.is_exact
+
+
+def test_prior_results_select_a_matching_refinement_or_same_scenario() -> None:
+    refinement = select_replay_scenario("Focus while coding", True)
+    direct_refinement = select_replay_scenario("More electronic", False)
+    same_scenario = select_replay_scenario("Rainy Sunday", True)
+
+    assert refinement.key == "focus-coding-refine"
+    assert not refinement.is_exact
+    assert direct_refinement.key == "focus-coding-refine"
+    assert direct_refinement.is_exact
+    assert same_scenario.key == "rainy-sunday"

backend/tests/test_grounding.py

@@ -1,8 +1,14 @@
 """Deterministic tests for bounded Spotify grounding."""
 
 import asyncio
+import time
 from collections.abc import Awaitable, Callable
 
+from app.adapters.spotify.errors import (
+    SpotifyQuotaExhaustedError,
+    SpotifyRateLimitedError,
+    SpotifyUnavailableError,
+)
 from app.config import Settings
 from app.domain.models import Familiarity, Track, TrackCandidate
 from app.pipeline.grounding import Grounder
@@ -47,7 +53,14 @@ def test_early_stop_honors_pool_target() -> None:
         grounder = Grounder(_settings(grounding_concurrency=2))
         candidates = tuple(TrackCandidate(f"track-{index}", "Artist") for index in range(6))
 
-        result = await grounder.ground(catalog, candidates, frozenset(), Familiarity.MIX, 2)
+        result = await grounder.ground(
+            catalog,
+            candidates,
+            frozenset(),
+            Familiarity.MIX,
+            2,
+            _deadline(),
+        )
 
         assert len(result.tracks) == 2
         assert len(catalog.search_queries) == 2
@@ -74,6 +87,7 @@ def test_miss_and_mismatch_are_counted_separately() -> None:
             frozenset(),
             Familiarity.MIX,
             2,
+            _deadline(),
         )
 
         assert result.metrics.miss_count == 1
@@ -92,9 +106,23 @@ def test_resolution_cache_hit_skips_catalog() -> None:
         grounder = Grounder(_settings())
         candidates = (TrackCandidate("Cached Song", "Artist"),)
 
-        await grounder.ground(catalog, candidates, frozenset(), Familiarity.MIX, 1)
+        await grounder.ground(
+            catalog,
+            candidates,
+            frozenset(),
+            Familiarity.MIX,
+            1,
+            _deadline(),
+        )
         first_call_count = len(catalog.search_queries)
-        second = await grounder.ground(catalog, candidates, frozenset(), Familiarity.MIX, 1)
+        second = await grounder.ground(
+            catalog,
+            candidates,
+            frozenset(),
+            Familiarity.MIX,
+            1,
+            _deadline(),
+        )
 
         assert len(catalog.search_queries) == first_call_count
         assert second.metrics.cache_hit_count == 1
@@ -124,6 +152,7 @@ def test_deadline_returns_resolved_partial_pool() -> None:
             frozenset(),
             Familiarity.MIX,
             2,
+            time.monotonic() + settings.request_deadline_seconds,
         )
 
         assert [track.id for track in result.tracks] == ["fast"]
@@ -132,6 +161,127 @@ def test_deadline_returns_resolved_partial_pool() -> None:
     asyncio.run(run())
 
 
+def test_spotify_failure_is_counted_and_remaining_candidates_continue() -> None:
+    async def run() -> None:
+        async def search(query: str) -> list[Track]:
+            if "Failing Song" in query:
+                raise SpotifyUnavailableError(503)
+            title = query.split('track:"', 1)[1].split('"', 1)[0]
+            return [_track(title.lower().replace(" ", "-"), title, "Artist")]
+
+        catalog = FakeCatalog(search)
+        candidates = (
+            TrackCandidate("Failing Song", "Artist"),
+            TrackCandidate("First Good Song", "Artist"),
+            TrackCandidate("Second Good Song", "Artist"),
+        )
+
+        result = await Grounder(_settings()).ground(
+            catalog,
+            candidates,
+            frozenset(),
+            Familiarity.MIX,
+            2,
+            _deadline(),
+        )
+
+        assert [track.title for track in result.tracks] == [
+            "First Good Song",
+            "Second Good Song",
+        ]
+        assert result.metrics.failed_count == 1
+        assert result.metrics.attempted_count == 3
+        assert not result.metrics.did_exhaust_quota
+
+    asyncio.run(run())
+
+
+def test_plain_rate_limit_continues_but_quota_exhaustion_stops_fanout() -> None:
+    async def run() -> None:
+        async def plain_rate_limited_search(query: str) -> list[Track]:
+            if "Rate Limited" in query:
+                raise SpotifyRateLimitedError(6.0)
+            return [_track("found", "Found Song", "Artist")]
+
+        plain_catalog = FakeCatalog(plain_rate_limited_search)
+        candidates = (
+            TrackCandidate("Rate Limited", "Artist"),
+            TrackCandidate("Found Song", "Artist"),
+        )
+        plain_result = await Grounder(_settings()).ground(
+            plain_catalog,
+            candidates,
+            frozenset(),
+            Familiarity.MIX,
+            1,
+            _deadline(),
+        )
+
+        async def quota_search(query: str) -> list[Track]:
+            raise SpotifyQuotaExhaustedError(0.0, "QUOTA_EXCEEDED")
+
+        quota_catalog = FakeCatalog(quota_search)
+        quota_result = await Grounder(_settings()).ground(
+            quota_catalog,
+            candidates,
+            frozenset(),
+            Familiarity.MIX,
+            1,
+            _deadline(),
+        )
+
+        assert [track.id for track in plain_result.tracks] == ["found"]
+        assert plain_result.metrics.failed_count == 1
+        assert not plain_result.metrics.did_exhaust_quota
+        assert quota_result.tracks == ()
+        assert quota_result.metrics.did_exhaust_quota
+        assert len(quota_catalog.search_queries) == 1
+
+    asyncio.run(run())
+
+
+def test_grounding_concurrency_is_shared_across_requests() -> None:
+    async def run() -> None:
+        active_searches = 0
+        maximum_active_searches = 0
+
+        async def search(query: str) -> list[Track]:
+            nonlocal active_searches, maximum_active_searches
+            active_searches += 1
+            maximum_active_searches = max(maximum_active_searches, active_searches)
+            await asyncio.sleep(0.01)
+            active_searches -= 1
+            title = query.split('track:"', 1)[1].split('"', 1)[0]
+            return [_track(title, title, "Artist")]
+
+        catalog = FakeCatalog(search)
+        grounder = Grounder(_settings(grounding_concurrency=2))
+        candidates = tuple(TrackCandidate(f"Song {index}", "Artist") for index in range(2))
+
+        await asyncio.gather(
+            grounder.ground(
+                catalog,
+                candidates,
+                frozenset(),
+                Familiarity.MIX,
+                2,
+                _deadline(),
+            ),
+            grounder.ground(
+                catalog,
+                candidates,
+                frozenset(),
+                Familiarity.MIX,
+                2,
+                _deadline(),
+            ),
+        )
+
+        assert maximum_active_searches == 2
+
+    asyncio.run(run())
+
+
 def _settings(**overrides: object) -> Settings:
     values: dict[str, object] = {
         "grounding_concurrency": 1,
@@ -141,6 +291,10 @@ def _settings(**overrides: object) -> Settings:
     return Settings.model_validate(values)
 
 
+def _deadline() -> float:
+    return time.monotonic() + 1.0
+
+
 def _track(track_id: str, title: str, artist: str) -> Track:
     return Track(
         id=track_id,

backend/tests/test_health.py

@@ -1,7 +1,11 @@
 """Smoke test for the application factory."""
 
+from unittest.mock import AsyncMock, Mock
+
+import pytest
 from fastapi.testclient import TestClient
 
+from app.config import Settings
 from app.main import create_app
 
 
@@ -10,3 +14,22 @@ def test_health_reports_mode() -> None:
     response = client.get("/api/health")
     assert response.status_code == 200
     assert response.json()["mode"] in ("live", "demo")
+
+
+def test_anthropic_client_uses_configured_timeout(monkeypatch: pytest.MonkeyPatch) -> None:
+    anthropic_client = Mock()
+    anthropic_client.close = AsyncMock()
+    constructor = Mock(return_value=anthropic_client)
+    monkeypatch.setattr("app.main.AsyncAnthropic", constructor)
+
+    live_settings = Settings(
+        app_mode="live",
+        spotify_client_id="client-id",
+        anthropic_api_key="api-key",
+        llm_timeout_seconds=42.0,
+    )
+    with TestClient(create_app(live_settings)) as client:
+        response = client.get("/api/health")
+
+    assert response.status_code == 200
+    constructor.assert_called_once_with(api_key="api-key", timeout=42.0, http_client=None)

backend/tests/test_orchestrator.py

@@ -3,6 +3,7 @@
 import asyncio
 from collections.abc import AsyncGenerator
 
+from app.adapters.spotify.errors import SpotifyQuotaExhaustedError, SpotifyUnavailableError
 from app.config import Settings
 from app.domain.models import (
     ConversationTurn,
@@ -13,7 +14,7 @@ from app.domain.models import (
     Track,
     TrackCandidate,
 )
-from app.pipeline.event import PipelineEvent, PipelineTrackEvent
+from app.pipeline.event import PipelineErrorEvent, PipelineEvent, PipelineTrackEvent
 from app.pipeline.orchestrator import RecommendationPipeline
 from app.ports.protocols import RecommenderOutputError, TimeRange
 
@@ -61,11 +62,15 @@ class FakeRecommender:
         intents: list[Intent],
         selection_ids: tuple[str, ...] = (),
         failure_count: int = 0,
+        intent_error: Exception | None = None,
+        intent_delay_seconds: float = 0.0,
     ) -> None:
         """Store deterministic outputs for successive calls."""
         self.intents = intents
         self.selection_ids = selection_ids
         self.failure_count = failure_count
+        self.intent_error = intent_error
+        self.intent_delay_seconds = intent_delay_seconds
         self.rerank_call_count = 0
 
     async def create_intent(
@@ -77,6 +82,9 @@ class FakeRecommender:
         candidate_count: int,
     ) -> Intent:
         """Return the next fixed intent."""
+        await asyncio.sleep(self.intent_delay_seconds)
+        if self.intent_error is not None:
+            raise self.intent_error
         return self.intents.pop(0)
 
     async def stream_rerank(
@@ -115,6 +123,108 @@ def test_event_order_and_rerank_ids_stay_inside_grounded_pool() -> None:
     asyncio.run(run())
 
 
+def test_intent_stage_failures_yield_one_typed_error_event() -> None:
+    async def run() -> None:
+        cases = (
+            (RecommenderOutputError("invalid intent"), "intent_failed"),
+            (
+                SpotifyQuotaExhaustedError(0.0, "QUOTA_EXCEEDED"),
+                "quota_exhausted",
+            ),
+            (SpotifyUnavailableError(503), "spotify_unavailable"),
+        )
+        for error, expected_code in cases:
+            catalog = FakeCatalog(())
+            recommender = FakeRecommender([], intent_error=error)
+
+            events = await _run_pipeline(catalog, recommender)
+
+            assert len(events) == 1
+            assert events[0].type == "error"
+            assert events[0].code == expected_code
+
+    asyncio.run(run())
+
+
+def test_taste_failure_cancels_sibling_fetches() -> None:
+    class FailingTasteCatalog(FakeCatalog):
+        """Fail one taste request after all sibling requests have started."""
+
+        def __init__(self) -> None:
+            super().__init__(())
+            self.started_count = 0
+            self.cancelled_count = 0
+            self.all_started = asyncio.Event()
+            self.never_finishes = asyncio.Event()
+
+        async def fetch_top_artists(self, time_range: TimeRange, limit: int) -> list[str]:
+            await self._wait_for_all_fetches()
+            await self._wait_until_cancelled()
+            return []
+
+        async def fetch_top_tracks(self, time_range: TimeRange, limit: int) -> list[Track]:
+            await self._wait_for_all_fetches()
+            await self._wait_until_cancelled()
+            return []
+
+        async def fetch_saved_tracks(self, limit: int) -> list[Track]:
+            await self._wait_for_all_fetches()
+            raise SpotifyUnavailableError(503)
+
+        async def _wait_for_all_fetches(self) -> None:
+            self.started_count += 1
+            if self.started_count == 5:
+                self.all_started.set()
+            await self.all_started.wait()
+
+        async def _wait_until_cancelled(self) -> None:
+            try:
+                await self.never_finishes.wait()
+            except asyncio.CancelledError:
+                self.cancelled_count += 1
+                raise
+
+    async def run() -> None:
+        catalog = FailingTasteCatalog()
+        events = await _run_pipeline(catalog, FakeRecommender([]))
+
+        assert len(events) == 1
+        assert events[0].type == "error"
+        assert events[0].code == "spotify_unavailable"
+        assert catalog.cancelled_count == 4
+
+    asyncio.run(run())
+
+
+def test_grounding_deadline_starts_before_intent_generation() -> None:
+    async def run() -> None:
+        track = _track("found", "Found Song")
+        catalog = FakeCatalog((track,))
+        recommender = FakeRecommender(
+            [_intent(track)],
+            intent_delay_seconds=0.02,
+        )
+        pipeline = RecommendationPipeline(
+            recommender,
+            Settings(
+                rerank_count=1,
+                rerank_pool_buffer=0,
+                grounding_floor=1,
+                grounding_concurrency=1,
+                request_deadline_seconds=0.01,
+            ),
+        )
+
+        events = await _collect(pipeline, catalog, "query")
+
+        assert [event.type for event in events] == ["metadata", "error"]
+        assert isinstance(events[-1], PipelineErrorEvent)
+        assert events[-1].code == "no_grounded_results"
+        assert catalog.search_call_count == 0
+
+    asyncio.run(run())
+
+
 def test_rerank_fallback_warns_then_streams_grounded_order() -> None:
     async def run() -> None:
         first = _track("first", "First Song")

backend/tests/test_recommendation_api.py

@@ -3,13 +3,16 @@
 import time
 from collections.abc import AsyncGenerator
 
+import pytest
 from fastapi.testclient import TestClient
 from pydantic import TypeAdapter
 
 from app.adapters.spotify.auth import TokenSet
+from app.adapters.spotify.errors import SpotifyAuthenticationError
 from app.adapters.spotify.session import SessionStore, SpotifySession
 from app.api.routes import SESSION_COOKIE_NAME
 from app.api.schemas import StreamEvent
+from app.config import AppMode, Settings
 from app.domain.models import ConversationTurn, CreatedPlaylist, PreviousRecommendation, Track
 from app.main import create_app
 from app.pipeline.event import PipelineDoneEvent, PipelineMetadataEvent, PipelineTrackEvent
@@ -37,13 +40,16 @@ class FakePipeline:
 class FakePlaylistWriter:
     """Capture playlist writes without external calls."""
 
-    def __init__(self) -> None:
+    def __init__(self, error: Exception | None = None) -> None:
         """Create an empty write trace."""
+        self.error = error
         self.name: str | None = None
         self.track_uris: list[str] = []
 
     async def create_playlist(self, name: str, description: str) -> CreatedPlaylist:
         """Record the prefixed name and return a stable playlist."""
+        if self.error is not None:
+            raise self.error
         self.name = name
         return CreatedPlaylist("playlist", "https://open.spotify.com/playlist/playlist")
 
@@ -52,7 +58,13 @@ class FakePlaylistWriter:
         self.track_uris = track_uris
 
 
-def test_recommendations_stream_lines_validate_against_frozen_schemas() -> None:
+def test_recommendations_stream_lines_validate_against_frozen_schemas(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    async def fail_if_polled(request: object) -> bool:
+        raise AssertionError("Request disconnect state must not be polled")
+
+    monkeypatch.setattr("starlette.requests.Request.is_disconnected", fail_if_polled)
     app = create_app()
     with TestClient(app) as client:
         _authenticate(client, session_store=app.state.session_store)
@@ -70,8 +82,13 @@ def test_recommendations_stream_lines_validate_against_frozen_schemas() -> None:
     assert [event.type for event in events] == ["metadata", "track", "done"]
 
 
-def test_recommendations_require_a_valid_session_without_seed() -> None:
+def test_live_recommendations_require_a_valid_session_without_seed() -> None:
-    with TestClient(create_app()) as client:
+    application_settings = Settings(
+        app_mode=AppMode.LIVE,
+        spotify_client_id="client-id",
+        anthropic_api_key="test-key",
+    )
+    with TestClient(create_app(application_settings)) as client:
         response = client.post(
             "/api/recommendations",
             json={"schema_version": 1, "query": "focused electronic music"},
@@ -103,6 +120,26 @@ def test_playlist_endpoint_prefixes_name_and_adds_tracks() -> None:
     assert writer.track_uris == ["spotify:track:track"]
 
 
+def test_playlist_authentication_failure_signals_relogin() -> None:
+    app = create_app()
+    writer = FakePlaylistWriter(SpotifyAuthenticationError("expired"))
+    with TestClient(app) as client:
+        _authenticate(client, session_store=app.state.session_store)
+        app.state.spotify_client_factory = lambda session: writer
+
+        response = client.post(
+            "/api/playlists",
+            json={
+                "schema_version": 1,
+                "name": "Night drive",
+                "track_uris": ["spotify:track:track"],
+            },
+        )
+
+    assert response.status_code == 401
+    assert response.json() == {"detail": "Spotify authentication expired"}
+
+
 def _authenticate(client: TestClient, session_store: SessionStore) -> None:
     session_id = session_store.create(
         SpotifySession(

backend/tests/test_spotify_auth.py

@@ -6,12 +6,14 @@ import hashlib
 import time
 
 import httpx2
+import pytest
 
 from app.adapters.spotify.auth import (
     TokenSet,
     derive_code_challenge,
     refresh_access_token,
 )
+from app.adapters.spotify.session import PendingLogins
 
 
 def test_code_challenge_is_unpadded_base64url_sha256() -> None:
@@ -43,3 +45,18 @@ def test_refresh_keeps_existing_refresh_token_when_omitted() -> None:
 def test_token_expiry_uses_sixty_second_skew() -> None:
     assert TokenSet("access", "refresh", time.monotonic() + 59).is_expired
     assert not TokenSet("access", "refresh", time.monotonic() + 61).is_expired
+
+
+def test_pending_login_add_sweeps_expired_entries(monkeypatch: pytest.MonkeyPatch) -> None:
+    current_time = 0.0
+    monkeypatch.setattr(
+        "app.adapters.spotify.session.time.monotonic",
+        lambda: current_time,
+    )
+    pending_logins = PendingLogins()
+    pending_logins.add("expired", "old-verifier")
+
+    current_time = 601.0
+    pending_logins.add("current", "new-verifier")
+
+    assert set(pending_logins._entries) == {"current"}

backend/tests/test_spotify_client.py

@@ -9,10 +9,16 @@ import pytest
 
 from app.adapters.spotify.auth import TokenSet
 from app.adapters.spotify.client import SpotifyClient
-from app.adapters.spotify.errors import SpotifyRateLimitedError, SpotifyRequestError
+from app.adapters.spotify.errors import (
+    SpotifyQuotaExhaustedError,
+    SpotifyRateLimitedError,
+    SpotifyRequestError,
+    SpotifyUnavailableError,
+)
 from app.adapters.spotify.session import SpotifySession
 from app.config import Settings
 from app.domain.models import Track
+from app.ports.protocols import CatalogQuotaExhaustedError
 
 TransportHandler = Callable[[httpx2.Request], Coroutine[None, None, httpx2.Response]]
 
@@ -44,19 +50,19 @@ def test_unauthorized_response_refreshes_once_and_returns_result() -> None:
 def test_concurrent_unauthorized_responses_share_one_refresh() -> None:
     async def run() -> None:
         token_calls = 0
-        old_api_calls = 0
+        api_calls = 0
-        both_old_requests_arrived = asyncio.Event()
+        both_initial_requests_arrived = asyncio.Event()
 
         async def handler(request: httpx2.Request) -> httpx2.Response:
-            nonlocal old_api_calls, token_calls
+            nonlocal api_calls, token_calls
             if request.url.host == "accounts.spotify.com":
                 token_calls += 1
-                return _token_response()
+                return _token_response("old-access")
-            if request.headers["Authorization"] == "Bearer old-access":
+            api_calls += 1
-                old_api_calls += 1
+            if api_calls <= 2:
-                if old_api_calls == 2:
+                if api_calls == 2:
-                    both_old_requests_arrived.set()
+                    both_initial_requests_arrived.set()
-                await asyncio.wait_for(both_old_requests_arrived.wait(), timeout=1)
+                await asyncio.wait_for(both_initial_requests_arrived.wait(), timeout=1)
                 return httpx2.Response(401)
             return httpx2.Response(200, json=_search_payload())
 
@@ -69,6 +75,7 @@ def test_concurrent_unauthorized_responses_share_one_refresh() -> None:
 
         assert first == second
         assert token_calls == 1
+        assert client.session.refresh_generation == 1
 
     asyncio.run(run())
 
@@ -105,6 +112,7 @@ def test_get_rate_limit_above_cap_raises_without_retry() -> None:
             await _search_with_handler(handler)
 
         assert error.value.retry_after_seconds == 6
+        assert not isinstance(error.value, CatalogQuotaExhaustedError)
         assert api_calls == 1
 
     asyncio.run(run())
@@ -136,10 +144,11 @@ def test_quota_exhaustion_raises_without_retry_or_sleep(
             )
 
         monkeypatch.setattr("app.adapters.spotify.client.asyncio.sleep", fake_sleep)
-        with pytest.raises(SpotifyRateLimitedError) as error:
+        with pytest.raises(SpotifyQuotaExhaustedError) as error:
             await _search_with_handler(handler)
 
         assert error.value.reason == "QUOTA_EXCEEDED"
+        assert isinstance(error.value, CatalogQuotaExhaustedError)
         assert api_calls == 1
         assert sleep_calls == 0
 
@@ -172,6 +181,46 @@ def test_authentication_then_rate_limit_retries_each_policy_once() -> None:
     asyncio.run(run())
 
 
+def test_rate_limit_retry_landing_on_unauthorized_refreshes_once() -> None:
+    async def run() -> None:
+        token_calls = 0
+        api_calls = 0
+
+        async def handler(request: httpx2.Request) -> httpx2.Response:
+            nonlocal api_calls, token_calls
+            if request.url.host == "accounts.spotify.com":
+                token_calls += 1
+                return _token_response()
+            api_calls += 1
+            if api_calls == 1:
+                return httpx2.Response(429, headers={"Retry-After": "0"})
+            if api_calls == 2:
+                return httpx2.Response(401)
+            return httpx2.Response(200, json=_search_payload())
+
+        tracks = await _search_with_handler(handler)
+
+        assert len(tracks) == 1
+        assert token_calls == 1
+        assert api_calls == 3
+
+    asyncio.run(run())
+
+
+def test_transport_error_becomes_spotify_unavailable() -> None:
+    async def run() -> None:
+        async def handler(request: httpx2.Request) -> httpx2.Response:
+            raise httpx2.ConnectError("connection failed", request=request)
+
+        with pytest.raises(SpotifyUnavailableError) as error:
+            await _search_with_handler(handler)
+
+        assert error.value.status_code == 504
+        assert str(error.value) == "Spotify request failed"
+
+    asyncio.run(run())
+
+
 def test_request_error_includes_parsed_spotify_message() -> None:
     async def run() -> None:
         async def handler(request: httpx2.Request) -> httpx2.Response:
@@ -267,8 +316,8 @@ def _client(http: httpx2.AsyncClient) -> SpotifyClient:
     return SpotifyClient(http, session, Settings(spotify_client_id="client"))
 
 
-def _token_response() -> httpx2.Response:
+def _token_response(access_token: str = "new-access") -> httpx2.Response:
-    return httpx2.Response(200, json={"access_token": "new-access", "expires_in": 3600})
+    return httpx2.Response(200, json={"access_token": access_token, "expires_in": 3600})
 
 
 def _search_payload() -> dict[str, object]:

backend/uv.lock

@@ -132,8 +132,10 @@ source = { virtual = "." }
 dependencies = [
     { name = "anthropic" },
     { name = "fastapi" },
+    { name = "httpx" },
     { name = "httpx2" },
     { name = "pydantic-settings" },
+    { name = "pyyaml" },
     { name = "structlog" },
     { name = "uvicorn", extra = ["standard"] },
 ]
@@ -143,14 +145,17 @@ dev = [
     { name = "mypy" },
     { name = "pytest" },
     { name = "ruff" },
+    { name = "types-pyyaml" },
 ]
 
 [package.metadata]
 requires-dist = [
     { name = "anthropic", specifier = "==0.121.0" },
     { name = "fastapi", specifier = ">=0.116" },
+    { name = "httpx", specifier = ">=0.28" },
     { name = "httpx2", specifier = ">=2.10" },
     { name = "pydantic-settings", specifier = ">=2.10" },
+    { name = "pyyaml", specifier = ">=6.0" },
     { name = "structlog", specifier = ">=25.4" },
     { name = "uvicorn", extras = ["standard"], specifier = ">=0.35" },
 ]
@@ -160,6 +165,7 @@ dev = [
     { name = "mypy", specifier = ">=1.17" },
     { name = "pytest", specifier = ">=8.4" },
     { name = "ruff", specifier = ">=0.12" },
+    { name = "types-pyyaml", specifier = ">=6.0" },
 ]
 
 [[package]]
@@ -580,6 +586,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/19/97/56608b2249fe206a67cd573bc93cd9896e1efb9e98bce9c163bcdc704b88/truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981", size = 18660, upload-time = "2025-08-12T18:49:01.46Z" },
 ]
 
+[[package]]
+name = "types-pyyaml"
+version = "6.0.12.20260724"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/3f/6f/a28f44bcd56bebed42b028a2894c79853e2f5e6b5279e633cb3f287a05e7/types_pyyaml-6.0.12.20260724.tar.gz", hash = "sha256:3c1ce1bb73cd5ec02e90390c2b1f00e810d241d8825fd73ff359696839271b6b", size = 17893, upload-time = "2026-07-24T04:58:43.453Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/8b/42/0337fefc615e20ee55d1c8f71b774a9b2b734a04669139c20753b27a2a3a/types_pyyaml-6.0.12.20260724-py3-none-any.whl", hash = "sha256:d57db930a4b2efbc57cf430ec8882765d246929432fa253092f383902329a453", size = 20312, upload-time = "2026-07-24T04:58:42.486Z" },
+]
+
 [[package]]
 name = "typing-extensions"
 version = "4.16.0"

docker-compose.yml

@@ -3,6 +3,7 @@ services:
     build:
       context: .
       dockerfile: backend/Dockerfile
+      target: demo
     env_file:
       - path: .env
         required: false

docs/logboek.md

@@ -2,7 +2,9 @@
 
 Bijgehouden tijdens de bouw. Per stap: wat ik deed, waarom, wat ik heb laten
 vallen.
+
 ## Dag 1 - korte sessie in de avond
+
 ### Opzet
 
 Wat ik deed:
@@ -35,6 +37,7 @@ Wat ik heb laten vallen of uitgesteld:
 - Geen apart beslisdocument. De motivering staat in de README en hier.
 
 ## Dag 2
+
 ### Spotify-koppeling
 
 Wat ik deed:
@@ -273,3 +276,199 @@ Waarom:
 - Machine-geconverteerde UI-code krijgt dezelfde behandeling als de
   backend: een review-pass met concrete findings en een test-suite die de
   contractkritische randen vastzet, voordat het richting main gaat.
+
+## Dag 3 - avond
+
+### Live eval run
+
+Wat ik deed:
+
+- Alle 8 scenario's in 1 run live gedraaid, plus de baseline arm (bare
+  search) voor de comparison table.
+- Het Spotify-dagquotum was eerder al een keer vol; daarom vooraf
+  maatregelen genomen: scenario's sequentieel, geen retries, de server-boot
+  zelf als quota-check, en live meegekeken op 429's (niet gehit).
+- Uitkomst: 4 scenario's pass, 3 vallen alleen op een te strenge
+  synonym-check in de eval zelf (gefixt), en discover-new faalt echt:
+  call 1 verzint bij familiarity=new titels die niet bestaan, dus er
+  overleeft niets de grounding. Fix volgt in de intent prompt.
+- De refinement turn deed live 0 Spotify calls: pool reuse werkt.
+
+Waarom:
+
+- De comparison table in de README wil ik op echte metingen baseren, en het
+  quotum maakt herhalen duur.
+
+Wat ik heb laten vallen of uitgesteld:
+
+- discover-new opnieuw opnemen wacht op de prompt fix.
+
+### Frontend: presentatie
+
+Wat ik deed:
+
+- Cards komen rustig gestaggerd binnen (met reduced-motion pad), results
+  en chips scrollen in eigen panelen zodat de summary zichtbaar blijft,
+  klik op het logo start een nieuwe chat, fonts self-hosted.
+- Een error die voor de metadata binnenkomt is nu een echte foutmelding
+  in de chat, geen transport failure. Playlist-namen hadden een dubbele
+  prefix; de frontend stuurt nu de kale query. 10 nieuwe tests.
+
+Waarom:
+
+- Dit is wat de reviewer als eerste ziet; rustige presentatie en eerlijke
+  foutmeldingen gaan voor extra features.
+
+Wat ik heb laten vallen of uitgesteld:
+
+- Verdere styling; de tijd gaat naar eval, demo mode en de README.
+
+### Backend hardening
+
+Wat ik deed:
+
+- Een hardening pass over de backend randgevallen, deels gevonden via een
+  adversarial review: fouten per candidate ingedamd zodat 1 kapotte
+  candidate nooit de hele request breekt, quota exhaustion apart herkend
+  van gewone 429's, token refresh single-flight per sessie-generatie,
+  typed errors voor de intent stap, en de seed-sessie kan de boot niet
+  meer laten crashen.
+- De request deadline start nu bij binnenkomst van de request en dekt
+  alles tot en met grounding. De gestreamde rerank heeft bewust een eigen
+  timeout: een totaalbudget zou een gezonde stream halverwege afkappen.
+- Concurrency op de grounding fan-out is nu process-wide begrensd in
+  plaats van per request.
+
+Waarom:
+
+- Dit zijn precies de randgevallen die je in een demo niet wilt zien; ze
+  nu dichtzetten is goedkoper dan er straks 1 in een review tegenkomen.
+
+Wat ik heb laten vallen of uitgesteld:
+
+- Een totaalbudget over de hele request heen; de afweging staat hierboven
+  en komt ook in de README.
+
+### Discover-new gefixt in de intent prompt
+
+Wat ik deed:
+
+- Het discover-new scenario faalde live steeds op de grounding: call 1
+  verzon voor echte, goed gekozen artiesten generieke titels die niet
+  bestaan ("Vibes", "Drift", "Shine"), en de matcher liet terecht niets
+  door. Niet de matcher aangepast maar de prompt: bij familiarity=new
+  alleen signature tracks en scene-anthems noemen die het model met
+  zekerheid kan spellen; nieuw-voor-de-luisteraar wordt toch al
+  afgedwongen door de exact-ID exclusion na grounding.
+- Live gevalideerd: het scenario levert nu 15 tracks van 16 artiesten.
+  Van de 32 kandidaten vielen er 12 als verzinsel af en resolvede de
+  rest; drop-over-substitute deed precies wat het moet doen.
+
+Waarom:
+
+- Titel-recall van een LLM stort in zodra je bewust om onbekend werk
+  vraagt; om obscuriteit vragen is dan het verkeerde gereedschap. Nieuw
+  en obscuur zijn geen synoniemen.
+
+Wat ik heb laten vallen of uitgesteld:
+
+- Een repair loop die gefaalde kandidaten opnieuw aan het model
+  voorlegt; blijft staan als potentiele vervolgstap.
+
+### Evaluatie en fixtures
+
+Wat ik deed:
+
+- Een eval-script dat de app de 8 standaardvragen stelt en de antwoorden
+  controleert: komen de events in de juiste volgorde, zijn alle tracks
+  uniek, heeft elke track een reden, kwam de eerste kaart binnen het
+  tijdsbudget, en gaat het antwoord echt over wat er gevraagd werd.
+- Dezelfde 8 vragen ook aan de kale Spotify search gesteld; de
+  vergelijking tussen die twee wordt de tabel in de README.
+- Elk scenario 1 keer opgenomen tegen de echte API's en opgeslagen als
+  cassettes; demo mode speelt die af, dus de app werkt straks ook zonder
+  keys. Persoonlijke data gaat er bij het opnemen uit (nepprofiel, geen
+  tokens); de ruwe opnames blijven buiten git.
+- Dit draait ook in CI, zonder keys.
+
+Waarom:
+
+- De claim dat de LLM-laag iets toevoegt moet meetbaar zijn, niet
+  beweerd. En de reviewer moet de app kunnen starten zonder eigen keys.
+
+Wat ik heb laten vallen of uitgesteld:
+
+- Byte-snapshots per pipeline-stap; de checks hierboven en de cassettes
+  zijn nu het bewijs.
+
+## Dag 3 - afronding
+
+### Demo mode
+
+Wat ik deed:
+
+- Demo mode af: zonder keys speelt de app de opgenomen cassettes af door
+  dezelfde pipeline en hetzelfde streamingpad als live. Onbekende vragen
+  krijgen het dichtstbijzijnde scenario, met een banner die dat eerlijk
+  benoemt; playlist-acties zijn gesimuleerd en zo gelabeld.
+- Een eerste versie deelde replay-state tussen gelijktijdige requests;
+  dat kon elkaars refinement verstoren. Nu krijgt elke request zijn
+  eigen replay-pool, met een test die de botsing naspeelt.
+- Het docker image bestaat nu in twee smaken: live zonder fixtures, demo
+  met. docker compose bouwt standaard de demo-smaak, dus een verse clone
+  zonder keys werkt direct. Een .dockerignore bracht de build context
+  van ~350 MB terug naar ~7 MB.
+
+Waarom:
+
+- De reviewer moet de app kunnen starten met alleen docker compose,
+  zonder accounts of keys; demo mode is ook het bewijs dat de
+  ports-and-adapters opzet echt is (dezelfde poorten, andere adapter).
+
+Wat ik heb laten vallen of uitgesteld:
+
+- Meer demo-scenario's; de 8 opgenomen zijn de suggestion chips en dat
+  is genoeg voor het verhaal.
+
+### Opruimen
+
+Wat ik deed:
+
+- De suggestion chips terug naar de wrapped look; de scrollbare box uit de
+  vorige batch stond lelijker dan wat er eerst was.
+- Ongebruikte OAuth scopes weggehaald (player en recently-played; nergens
+  in de code gebruikt), .env.example compleet gemaakt en AGENTS.md
+  gelijkgetrokken met hoe de repo er echt uitziet.
+
+Waarom:
+
+- Minder scopes vragen dan je gebruikt is netter richting de reviewer en
+  richting Spotify.
+
+### Documentatie
+
+Wat ik deed:
+
+- De README geschreven: wat het is, drie manieren om het te draaien, hoe
+  de pipeline werkt, de keuzes, eerlijke performance-cijfers (eerste kaart
+  20 tot 30 s, gemeten via de UI en de eval), een stukje van de
+  vergelijking met kale search, en een "where to look" tabel per
+  beoordelingscriterium.
+- docs/workflow.md toegevoegd: hoe ik werk. De server, Forgejo met een
+  GitHub-mirror, Komodo en Caddy voor de gehoste instantie, en hoe ik
+  agents inzet: implementatie parallel en goedkoop, ontwerp, review en
+  verificatie serieel en bij mij.
+- De gehoste preview bouwt nu bij elke merge opnieuw vanaf main.
+
+Waarom:
+
+- De reviewer leest de README het eerst; die moet kort zijn en naar de
+  rest wijzen in plaats van alles zelf te vertellen. En de werkwijze
+  uitleggen is eerlijker dan hem laten raden waarom commits in bursts
+  binnenkomen.
+
+Wat ik heb laten vallen of uitgesteld:
+
+- Een sneller model op de intent call (zou de wachttijd flink verlagen;
+  de fabrication rate is de afweging om dan te meten) staat als vervolg
+  in de README, niet gebouwd.

docs/workflow.md

@@ -0,0 +1,62 @@
+# How I work
+
+A short map of the setup behind this repo, because the process says as much
+as the code.
+
+## The environment
+
+Everything was built over SSH from a laptop (I was house-sitting for most
+of it) against my home server, a Debian box that runs my self-hosted
+infrastructure. The server carries the whole toolchain: uv and node for
+fast local feedback, Docker for the deliverable, and the deployment stack
+below. No cloud dev environment involved.
+
+## Source control and CI
+
+- Origin is my self-hosted Forgejo instance, with a push mirror to GitHub;
+  the repo you are reading is the mirror and shows the same history and
+  the same green CI.
+- CI runs on every push: ruff, mypy and pytest for the backend; typecheck,
+  lint, build and vitest for the frontend. It has been green since the
+  first commit.
+
+## Deployment
+
+- The hosted instance is a single container on the same server, built from
+  this repo's Dockerfile and managed with Komodo (a self-hosted container
+  control plane). It is rebuilt from main as the build advances.
+- Routing and TLS come from caddy-docker-proxy: the container carries its
+  route as labels, Caddy picks them up. The streaming endpoint needed one
+  deliberate setting there (no response buffering), verified through the
+  public URL.
+- HTTP basic auth sits at the edge because the instance runs against my
+  personal Spotify account.
+
+## AI-assisted building
+
+I work with coding agents, and this project was built that way end to end:
+
+- One orchestrating session (Claude Code harness) holds the plan and the
+  state. It briefs implementation agents (codex CLI, Opus 5 subagents) that
+  build scoped tracks in their own git worktrees, in parallel.
+- Contracts come first: the API schemas and the event protocol were frozen
+  before frontend and backend lanes ran in parallel against them.
+- Parallel lanes buy time where it matters. While I was working on the
+  backend, a separate lane generated a ready-to-implement UI (design
+  tokens, component structure, copy) against the frozen event protocol, so
+  frontend implementation started from a settled design instead of a blank
+  page.
+- Everything that lands is reviewed by me, commit by commit, in my editor
+  before it goes in. Voice-carrying text (this file, the README, the
+  logboek) I write or rewrite myself.
+- Verification against the real APIs is deliberate and budgeted: the
+  Spotify development quota is a daily account-level budget, so live test
+  runs are planned, sequential and measured rather than sprayed.
+- Review also runs as a tool: adversarial review passes over the diffs
+  produced findings that became fix batches; the useful findings and the
+  rejected ones are both traceable in the logboek.
+
+The result is that implementation is cheap and parallel, while design,
+review and verification stay serial and human. The logboek
+([logboek.md](logboek.md)) records that rhythm as it happened, including
+what was dropped under time pressure.

eval/baseline.py

@@ -0,0 +1,114 @@
+"""Record bare Spotify search results and render side-by-side comparisons."""
+
+import json
+from pathlib import Path
+
+import httpx
+from scenario import Scenario
+
+TOKEN_URL = "https://accounts.spotify.com/api/token"
+SEARCH_URL = "https://api.spotify.com/v1/search"
+
+
+async def record_baselines(
+    scenarios: tuple[Scenario, ...],
+    client_id: str,
+    client_secret: str,
+    snapshot_root: Path,
+) -> None:
+    """Snapshot the top ten results from bare searches for every query."""
+    snapshot_root.mkdir(parents=True, exist_ok=True)
+    async with httpx.AsyncClient(timeout=20.0) as client:
+        token_response = await client.post(
+            TOKEN_URL,
+            data={"grant_type": "client_credentials"},
+            auth=(client_id, client_secret),
+        )
+        token_response.raise_for_status()
+        access_token = token_response.json()["access_token"]
+        for scenario in scenarios:
+            response = await client.get(
+                SEARCH_URL,
+                params={"q": scenario.query, "type": "track", "limit": 10},
+                headers={"Authorization": f"Bearer {access_token}"},
+            )
+            response.raise_for_status()
+            tracks = response.json().get("tracks", {}).get("items", [])[:10]
+            _write_json(
+                snapshot_root / f"{scenario.key}.json",
+                {"key": scenario.key, "query": scenario.query, "tracks": tracks},
+            )
+
+
+def write_comparison(
+    scenarios: tuple[Scenario, ...],
+    snapshot_root: Path,
+    report_root: Path,
+    top_n: int,
+) -> Path:
+    """Write a markdown comparison for scenarios with both result arms."""
+    report_root.mkdir(parents=True, exist_ok=True)
+    lines = ["# Pipeline and bare search comparison", ""]
+    compared = 0
+    for scenario in scenarios:
+        baseline_path = snapshot_root / f"{scenario.key}.json"
+        pipeline_path = report_root / f"{scenario.key}.json"
+        if not baseline_path.exists() or not pipeline_path.exists():
+            continue
+        baseline = json.loads(baseline_path.read_text(encoding="ascii"))
+        pipeline = json.loads(pipeline_path.read_text(encoding="ascii"))
+        lines.extend(_comparison_section(scenario, pipeline, baseline, top_n))
+        compared += 1
+    if compared == 0:
+        lines.extend(["No scenario has results from both arms yet.", ""])
+    output_path = report_root / "baseline-comparison.md"
+    output_path.write_text("\n".join(lines), encoding="utf-8")
+    return output_path
+
+
+def _comparison_section(
+    scenario: Scenario,
+    pipeline: object,
+    baseline: object,
+    top_n: int,
+) -> list[str]:
+    pipeline_tracks = pipeline.get("tracks", []) if isinstance(pipeline, dict) else []
+    baseline_tracks = baseline.get("tracks", []) if isinstance(baseline, dict) else []
+    lines = [f"## {scenario.chip}", "", "| Rank | Pipeline | Bare search |", "| ---: | --- | --- |"]
+    for index in range(top_n):
+        pipeline_label = _pipeline_label(pipeline_tracks, index)
+        baseline_label = _spotify_label(baseline_tracks, index)
+        lines.append(f"| {index + 1} | {pipeline_label} | {baseline_label} |")
+    lines.append("")
+    return lines
+
+
+def _pipeline_label(tracks: object, index: int) -> str:
+    if not isinstance(tracks, list) or index >= len(tracks):
+        return ""
+    event = tracks[index]
+    track = event.get("track") if isinstance(event, dict) else None
+    return _track_label(track, "title")
+
+
+def _spotify_label(tracks: object, index: int) -> str:
+    if not isinstance(tracks, list) or index >= len(tracks):
+        return ""
+    return _track_label(tracks[index], "name")
+
+
+def _track_label(track: object, title_key: str) -> str:
+    if not isinstance(track, dict):
+        return ""
+    artists = track.get("artists", [])
+    names = [
+        artist.get("name", "") if isinstance(artist, dict) else str(artist) for artist in artists
+    ]
+    return f"{track.get(title_key, '')} by {', '.join(names)}"
+
+
+def _write_json(path: Path, payload: object) -> None:
+    path.write_text(
+        json.dumps(payload, ensure_ascii=True, indent=2, sort_keys=True) + "\n",
+        encoding="ascii",
+    )

eval/live.py

@@ -0,0 +1,284 @@
+"""Exercise the live recommendation stream and assert behavior properties."""
+
+import asyncio
+import json
+import time
+from dataclasses import dataclass, field
+from pathlib import Path
+
+import httpx
+from scenario import Scenario
+from wire import validate_event
+
+FACET_TERMS: dict[tuple[str, str], tuple[str, ...]] = {
+    ("activity", "coding"): ("coding", "programming"),
+    ("activity", "workout"): ("workout", "gym"),
+    ("familiarity", "mix"): ("mix", "familiar", "new", "discover"),
+    ("familiarity", "new"): ("new", "unfamiliar", "discover", "surprise"),
+    ("language", "nl"): ("dutch", "nederlandse"),
+    ("era", "1990s"): ("1990", "nineties", "90s"),
+    ("mood", "energetic"): ("energetic", "high-energy", "high energy"),
+    ("mood", "relaxed"): ("relaxed", "laid-back", "laid back", "couch-friendly"),
+    ("mood", "subdued"): ("subdued", "background", "unobtrusive", "not too present"),
+}
+
+
+@dataclass(frozen=True)
+class LiveLimits:
+    """Quality and latency thresholds for one live request."""
+
+    minimum_tracks: int
+    maximum_tracks: int
+    minimum_artists: int
+    first_track_budget_ms: int
+    total_budget_ms: int
+
+
+@dataclass
+class ScenarioResult:
+    """Serializable observations and property failures for one scenario."""
+
+    scenario: Scenario
+    events: list[dict[str, object]] = field(default_factory=list)
+    failures: list[str] = field(default_factory=list)
+    checks: dict[str, bool] = field(default_factory=dict)
+    first_track_ms: int | None = None
+    total_ms: int | None = None
+
+    @property
+    def tracks(self) -> list[dict[str, object]]:
+        """Return track event payloads in streamed order."""
+        return [event for event in self.events if event.get("type") == "track"]
+
+    @property
+    def passed(self) -> bool:
+        """Return whether all required properties passed."""
+        return not self.failures
+
+    def as_report(self) -> dict[str, object]:
+        """Render the stable JSON report shape."""
+        return {
+            "key": self.scenario.key,
+            "query": self.scenario.query,
+            "after": self.scenario.after,
+            "status": "passed" if self.passed else "failed",
+            "first_track_ms": self.first_track_ms,
+            "total_ms": self.total_ms,
+            "checks": self.checks,
+            "failures": self.failures,
+            "events": self.events,
+            "tracks": self.tracks,
+        }
+
+
+async def run_live_scenarios(
+    scenarios: tuple[Scenario, ...],
+    base_url: str,
+    report_root: Path,
+    limits: LiveLimits,
+) -> tuple[ScenarioResult, ...]:
+    """Run all scenarios sequentially and write one report for each."""
+    report_root.mkdir(parents=True, exist_ok=True)
+    results_by_key: dict[str, ScenarioResult] = {}
+    async with httpx.AsyncClient(base_url=base_url, timeout=None) as client:
+        for scenario in scenarios:
+            parent = results_by_key.get(scenario.after) if scenario.after is not None else None
+            result = await run_live_scenario(client, scenario, limits, parent)
+            _write_report(report_root / f"{scenario.key}.json", result.as_report())
+            results_by_key[scenario.key] = result
+    return tuple(results_by_key[scenario.key] for scenario in scenarios)
+
+
+async def run_live_scenario(
+    client: httpx.AsyncClient,
+    scenario: Scenario,
+    limits: LiveLimits,
+    parent: ScenarioResult | None = None,
+) -> ScenarioResult:
+    """Run one strict NDJSON request and evaluate its properties."""
+    result = ScenarioResult(scenario)
+    if scenario.after is not None and (parent is None or not parent.tracks):
+        result.failures.append("parent scenario did not produce usable recommendations")
+        return result
+    payload = build_request(scenario, parent)
+    started_at = time.monotonic()
+    try:
+        async with asyncio.timeout(limits.total_budget_ms / 1000):
+            async with client.stream("POST", "/api/recommendations", json=payload) as response:
+                if response.status_code != 200:
+                    body = (await response.aread()).decode("utf-8", errors="replace")
+                    result.failures.append(f"HTTP {response.status_code}: {body}")
+                else:
+                    await _consume_response(response, result, started_at)
+    except TimeoutError:
+        result.failures.append("stream exceeded the configured total latency budget")
+    except (httpx.HTTPError, ValueError) as error:
+        result.failures.append(f"stream failed: {error}")
+    result.total_ms = _elapsed_ms(started_at)
+    _evaluate_result(result, limits)
+    return result
+
+
+def build_request(scenario: Scenario, parent: ScenarioResult | None) -> dict[str, object]:
+    """Build the frozen request shape, including client-owned turn context."""
+    payload: dict[str, object] = {"schema_version": 1, "query": scenario.query}
+    if parent is None:
+        return payload
+    prior = []
+    labels = []
+    for event in parent.tracks:
+        track = event["track"]
+        if not isinstance(track, dict):
+            continue
+        rank = event["rank"]
+        artists = track["artists"]
+        prior_artists = list(artists)[:10] if isinstance(artists, list) else []
+        prior.append(
+            {
+                "rank": rank,
+                "track_id": track["id"],
+                "title": track["title"],
+                "artists": prior_artists,
+            }
+        )
+        labels.append(
+            f"{rank}. {track['title']} by {', '.join(str(name) for name in prior_artists)}"
+        )
+    payload["history"] = [
+        {"role": "user", "content": parent.scenario.query},
+        {"role": "assistant", "content": "\n".join(labels)[:2000]},
+    ]
+    payload["prior_recommendations"] = prior
+    return payload
+
+
+def _evaluate_result(result: ScenarioResult, limits: LiveLimits) -> None:
+    event_types = [str(event.get("type")) for event in result.events]
+    terminal_type = event_types[-1] if event_types else None
+    order_is_valid = (
+        bool(event_types)
+        and event_types[0] == "metadata"
+        and terminal_type in {"done", "error"}
+        and all(event_type in {"track", "warning"} for event_type in event_types[1:-1])
+        and event_types.count("metadata") == 1
+    )
+    _record_check(result, "event_order", order_is_valid, "event order is invalid")
+    if terminal_type == "error":
+        terminal = result.events[-1]
+        result.failures.append(f"terminal error: {terminal.get('code', 'unknown')}")
+
+    tracks = result.tracks
+    if terminal_type == "done":
+        done_count = result.events[-1].get("track_count")
+        _record_check(
+            result,
+            "done_track_count",
+            done_count == len(tracks),
+            "done event track count did not match streamed tracks",
+        )
+    track_ids = [_track_field(event, "id") for event in tracks]
+    _record_check(
+        result,
+        "unique_track_ids",
+        len(track_ids) == len(set(track_ids)),
+        "duplicate track ids were returned",
+    )
+    justifications = [event.get("justification") for event in tracks]
+    _record_check(
+        result,
+        "non_empty_justifications",
+        all(isinstance(value, str) and bool(value.strip()) for value in justifications),
+        "a track justification was empty",
+    )
+    _record_check(
+        result,
+        "track_count",
+        limits.minimum_tracks <= len(tracks) <= limits.maximum_tracks,
+        f"track count {len(tracks)} is outside configured bounds",
+    )
+    artists = {str(artist).casefold() for event in tracks for artist in _track_artists(event)}
+    _record_check(
+        result,
+        "distinct_artists",
+        len(artists) >= limits.minimum_artists,
+        f"distinct artist count {len(artists)} is below configured minimum",
+    )
+    _record_check(
+        result,
+        "first_track_latency",
+        result.first_track_ms is not None and result.first_track_ms <= limits.first_track_budget_ms,
+        "time to first track exceeded the configured budget",
+    )
+    _record_check(
+        result,
+        "total_latency",
+        result.total_ms is not None and result.total_ms <= limits.total_budget_ms,
+        "total latency exceeded the configured budget",
+    )
+    _evaluate_facets(result)
+
+
+def _evaluate_facets(result: ScenarioResult) -> None:
+    metadata = next(
+        (event for event in result.events if event.get("type") == "metadata"),
+        None,
+    )
+    summary = str(metadata.get("intent_summary", "")).casefold() if metadata else ""
+    for name, value in result.scenario.facets.items():
+        if not isinstance(value, str):
+            continue
+        terms = FACET_TERMS.get((name, value), (value.casefold(),))
+        _record_check(
+            result,
+            f"facet_{name}",
+            any(term in summary for term in terms),
+            f"intent summary did not express {name}={value}",
+        )
+
+
+async def _consume_response(
+    response: httpx.Response,
+    result: ScenarioResult,
+    started_at: float,
+) -> None:
+    content_type = response.headers.get("content-type", "").split(";", 1)[0]
+    if content_type != "application/x-ndjson":
+        result.failures.append(f"unexpected content type: {content_type or 'missing'}")
+        return
+    async for line in response.aiter_lines():
+        if not line:
+            result.failures.append("NDJSON stream contained an empty line")
+            continue
+        event = validate_event(line)
+        event_payload = event.model_dump(mode="json")
+        result.events.append(event_payload)
+        if event_payload["type"] == "track" and result.first_track_ms is None:
+            result.first_track_ms = _elapsed_ms(started_at)
+
+
+def _record_check(result: ScenarioResult, name: str, passed: bool, failure: str) -> None:
+    result.checks[name] = passed
+    if not passed:
+        result.failures.append(failure)
+
+
+def _track_field(event: dict[str, object], key: str) -> str:
+    track = event.get("track")
+    return str(track.get(key, "")) if isinstance(track, dict) else ""
+
+
+def _track_artists(event: dict[str, object]) -> list[object]:
+    track = event.get("track")
+    artists = track.get("artists") if isinstance(track, dict) else None
+    return artists if isinstance(artists, list) else []
+
+
+def _elapsed_ms(started_at: float) -> int:
+    return round((time.monotonic() - started_at) * 1000)
+
+
+def _write_report(path: Path, report: dict[str, object]) -> None:
+    path.write_text(
+        json.dumps(report, ensure_ascii=True, indent=2, sort_keys=True) + "\n",
+        encoding="ascii",
+    )

eval/record_fixtures.py

@@ -0,0 +1,142 @@
+"""Record one scenario through a locally constructed application."""
+
+import argparse
+import os
+import sys
+from importlib import import_module
+from pathlib import Path
+from typing import cast
+
+import httpx
+import httpx2
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+from live import ScenarioResult, build_request
+from recording import Httpx2RecordingTransport, HttpxRecordingTransport, write_cassettes
+from scenario import SCENARIO_PATH, Scenario, load_scenarios
+from wire import BACKEND_ROOT, validate_event
+
+EVAL_ROOT = Path(__file__).parent
+
+
+def main() -> int:
+    """Record the selected scenario and persist raw plus safe cassettes."""
+    arguments = _parse_arguments()
+    scenarios = load_scenarios(arguments.scenarios)
+    selected = next((item for item in scenarios if item.key == arguments.scenario), None)
+    if selected is None:
+        raise SystemExit(f"Unknown scenario: {arguments.scenario}")
+    spotify_client_id = _required_environment("SPOTIFY_CLIENT_ID")
+    spotify_refresh_token = _required_environment("SPOTIFY_SEED_REFRESH_TOKEN")
+    anthropic_api_key = _required_environment("ANTHROPIC_API_KEY")
+
+    spotify_transport = Httpx2RecordingTransport(httpx2.AsyncHTTPTransport())
+    anthropic_transport = HttpxRecordingTransport(httpx.AsyncHTTPTransport())
+    anthropic_http_client = httpx.AsyncClient(transport=anthropic_transport)
+    app = _create_recording_app(
+        spotify_client_id,
+        spotify_refresh_token,
+        anthropic_api_key,
+        spotify_transport,
+        anthropic_http_client,
+    )
+    try:
+        with TestClient(app, base_url=arguments.base_url) as client:
+            _record_selected(client, scenarios, selected, spotify_transport)
+    finally:
+        raw_root, redacted_root = write_cassettes(
+            arguments.fixture_dir,
+            selected.key,
+            {
+                "spotify": spotify_transport.interactions,
+                "anthropic": anthropic_transport.interactions,
+            },
+        )
+    print(f"Raw cassettes: {raw_root}")
+    print(f"Redacted cassettes: {redacted_root}")
+    return 0
+
+
+def _record_selected(
+    client: TestClient,
+    scenarios: tuple[Scenario, ...],
+    selected: Scenario,
+    spotify_transport: Httpx2RecordingTransport,
+) -> None:
+    if selected.after is not None:
+        parent_scenario = next(item for item in scenarios if item.key == selected.after)
+        parent = _send_turn(client, parent_scenario, None)
+        search_count = _search_count(spotify_transport)
+        _send_turn(client, selected, parent)
+        if _search_count(spotify_transport) != search_count:
+            raise RuntimeError("Refinement issued new Spotify searches")
+    else:
+        _send_turn(client, selected, None)
+
+
+def _send_turn(
+    client: TestClient,
+    scenario: Scenario,
+    parent: ScenarioResult | None,
+) -> ScenarioResult:
+    response = client.post("/api/recommendations", json=build_request(scenario, parent))
+    response.raise_for_status()
+    result = ScenarioResult(scenario)
+    for line in response.text.splitlines():
+        event = validate_event(line)
+        result.events.append(cast(dict[str, object], event.model_dump(mode="json")))
+    if not result.events or result.events[-1].get("type") != "done":
+        raise RuntimeError(f"Scenario {scenario.key} did not complete")
+    return result
+
+
+def _create_recording_app(
+    spotify_client_id: str,
+    spotify_refresh_token: str,
+    anthropic_api_key: str,
+    spotify_transport: Httpx2RecordingTransport,
+    anthropic_http_client: httpx.AsyncClient,
+) -> FastAPI:
+    backend_path = str(BACKEND_ROOT)
+    if backend_path not in sys.path:
+        sys.path.insert(0, backend_path)
+    config_module = import_module("app.config")
+    main_module = import_module("app.main")
+    application_settings = config_module.Settings(
+        app_mode=config_module.AppMode.LIVE,
+        spotify_client_id=spotify_client_id,
+        spotify_seed_refresh_token=spotify_refresh_token,
+        anthropic_api_key=anthropic_api_key,
+    )
+    return cast(
+        FastAPI,
+        main_module.create_app(
+            application_settings=application_settings,
+            http_transport=spotify_transport,
+            anthropic_http_client=anthropic_http_client,
+        ),
+    )
+
+
+def _search_count(transport: Httpx2RecordingTransport) -> int:
+    return sum("/search" in interaction.url for interaction in transport.interactions)
+
+
+def _required_environment(name: str) -> str:
+    value = os.environ.get(name)
+    if not value:
+        raise SystemExit(f"{name} is required for recording")
+    return value
+
+
+def _parse_arguments() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--scenario", required=True)
+    parser.add_argument("--base-url", required=True)
+    parser.add_argument("--scenarios", type=Path, default=SCENARIO_PATH)
+    parser.add_argument("--fixture-dir", type=Path, default=EVAL_ROOT / "fixtures")
+    return parser.parse_args()
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

eval/recording/__init__.py

@@ -0,0 +1,14 @@
+"""Record and redact external HTTP interactions for fixture replay."""
+
+from recording.model import RecordedInteraction
+from recording.redaction import redact_cassette
+from recording.storage import write_cassettes
+from recording.transport import Httpx2RecordingTransport, HttpxRecordingTransport
+
+__all__ = [
+    "Httpx2RecordingTransport",
+    "HttpxRecordingTransport",
+    "RecordedInteraction",
+    "redact_cassette",
+    "write_cassettes",
+]

eval/recording/model.py

@@ -0,0 +1,30 @@
+"""Transport-neutral cassette interaction model."""
+
+import base64
+from dataclasses import dataclass
+
+
+@dataclass(frozen=True)
+class RecordedInteraction:
+    """One completed HTTP exchange with exact response chunks."""
+
+    method: str
+    url: str
+    status: int
+    request_body: bytes
+    response_chunks: tuple[bytes, ...]
+
+    def as_dict(self) -> dict[str, object]:
+        """Encode byte fields losslessly for JSON storage."""
+        return {
+            "method": self.method,
+            "url": self.url,
+            "status": self.status,
+            "request_body_base64": _encode(self.request_body),
+            "response_body_base64": _encode(b"".join(self.response_chunks)),
+            "response_chunks_base64": [_encode(chunk) for chunk in self.response_chunks],
+        }
+
+
+def _encode(value: bytes) -> str:
+    return base64.b64encode(value).decode("ascii")

eval/recording/redaction.py

@@ -0,0 +1,219 @@
+"""Remove credentials and listener identity from persisted cassettes."""
+
+import base64
+import copy
+import json
+import re
+from dataclasses import dataclass
+from pathlib import Path
+from typing import cast
+from urllib.parse import urlparse
+
+SYNTHETIC_ACCOUNT_ID = "synthetic-account"
+SYNTHETIC_DISPLAY_NAME = "Synthetic Listener"
+SYNTHETIC_TASTE_PATH = Path(__file__).with_name("synthetic_taste.json")
+BEARER_PATTERN = re.compile(r"Bearer\s+[A-Za-z0-9._~+/=-]+", re.IGNORECASE)
+SENSITIVE_PATH_PARTS = {"authorize", "oauth", "token"}
+
+
+@dataclass(frozen=True)
+class _ListenerIdentities:
+    account_ids: frozenset[str]
+    display_names: frozenset[str]
+
+
+def is_sensitive_url(url: str) -> bool:
+    """Return whether an authentication exchange must never be recorded."""
+    parsed = urlparse(url)
+    path_parts = {part.casefold() for part in parsed.path.split("/") if part}
+    return parsed.hostname == "accounts.spotify.com" or bool(path_parts & SENSITIVE_PATH_PARTS)
+
+
+def redact_cassette(interactions: list[dict[str, object]]) -> list[dict[str, object]]:
+    """Return safe interactions without mutating or persisting the source."""
+    safe = [
+        _drop_headers(copy.deepcopy(interaction))
+        for interaction in interactions
+        if not is_sensitive_url(str(interaction.get("url", "")))
+    ]
+    identities = _collect_identities(safe)
+    synthetic_taste = _load_synthetic_taste()
+    return [_redact_interaction(interaction, identities, synthetic_taste) for interaction in safe]
+
+
+def _drop_headers(value: object) -> object:
+    if isinstance(value, dict):
+        return {
+            key: _drop_headers(item)
+            for key, item in value.items()
+            if not str(key).casefold().endswith("headers")
+        }
+    if isinstance(value, list):
+        return [_drop_headers(item) for item in value]
+    return value
+
+
+def _collect_identities(interactions: list[object]) -> _ListenerIdentities:
+    account_ids: set[str] = set()
+    display_names: set[str] = set()
+    for interaction in interactions:
+        if not isinstance(interaction, dict) or not _is_current_user_url(
+            str(interaction.get("url", ""))
+        ):
+            continue
+        payload = _decode_json(str(interaction.get("response_body_base64", "")))
+        if not isinstance(payload, dict):
+            continue
+        for key in ("id", "account_id"):
+            value = payload.get(key)
+            if isinstance(value, str) and value:
+                account_ids.add(value)
+        display_name = payload.get("display_name")
+        if isinstance(display_name, str) and display_name:
+            display_names.add(display_name)
+    return _ListenerIdentities(frozenset(account_ids), frozenset(display_names))
+
+
+def _redact_interaction(
+    interaction: object,
+    identities: _ListenerIdentities,
+    synthetic_taste: dict[str, object],
+) -> dict[str, object]:
+    if not isinstance(interaction, dict):
+        raise ValueError("Cassette interaction must be a mapping")
+    redacted = {str(key): value for key, value in interaction.items()}
+    url = _replace_text(str(redacted.get("url", "")), identities)
+    redacted["url"] = url
+    redacted["request_body_base64"] = _redact_body(
+        str(redacted.get("request_body_base64", "")), identities, None, synthetic_taste
+    )
+    replacement = _taste_replacement(url, synthetic_taste)
+    original_body = str(redacted.get("response_body_base64", ""))
+    response_body = _redact_body(original_body, identities, replacement, synthetic_taste)
+    redacted["response_body_base64"] = response_body
+    chunks = redacted.get("response_chunks_base64", [])
+    if response_body != original_body:
+        redacted["response_chunks_base64"] = [response_body] if response_body else []
+    elif isinstance(chunks, list):
+        redacted["response_chunks_base64"] = [
+            _redact_body(str(chunk), identities, None, synthetic_taste) for chunk in chunks
+        ]
+    return cast(dict[str, object], _redact_value(redacted, identities, synthetic_taste))
+
+
+def _redact_body(
+    encoded: str,
+    identities: _ListenerIdentities,
+    replacement: object | None,
+    synthetic_taste: dict[str, object],
+) -> str:
+    try:
+        raw = base64.b64decode(encoded, validate=True)
+    except ValueError:
+        raw = b""
+    if replacement is not None:
+        value = replacement
+    else:
+        try:
+            value = json.loads(raw)
+        except (json.JSONDecodeError, UnicodeDecodeError):
+            text = raw.decode("utf-8", errors="replace")
+            return _encode(_replace_text(text, identities).encode())
+    redacted = _redact_value(value, identities, synthetic_taste)
+    return _encode(json.dumps(redacted, ensure_ascii=True, separators=(",", ":")).encode("ascii"))
+
+
+def _redact_value(
+    value: object,
+    identities: _ListenerIdentities,
+    synthetic_taste: dict[str, object],
+) -> object:
+    if isinstance(value, dict):
+        return {
+            str(key): _taste_summary(synthetic_taste)
+            if str(key) == "taste_profile"
+            else _synthetic_identity(item, identities, synthetic_taste)
+            for key, item in value.items()
+            if not str(key).casefold().endswith("headers")
+        }
+    if isinstance(value, list):
+        return [_redact_value(item, identities, synthetic_taste) for item in value]
+    if isinstance(value, str):
+        text = _replace_text(value, identities)
+        try:
+            nested = json.loads(text)
+        except json.JSONDecodeError:
+            return text
+        if isinstance(nested, (dict, list)):
+            return json.dumps(
+                _redact_value(nested, identities, synthetic_taste),
+                ensure_ascii=True,
+                separators=(",", ":"),
+            )
+        return text
+    return value
+
+
+def _synthetic_identity(
+    value: object,
+    identities: _ListenerIdentities,
+    synthetic_taste: dict[str, object],
+) -> object:
+    if isinstance(value, str) and value in identities.display_names:
+        return SYNTHETIC_DISPLAY_NAME
+    if isinstance(value, str) and value in identities.account_ids:
+        return SYNTHETIC_ACCOUNT_ID
+    return _redact_value(value, identities, synthetic_taste)
+
+
+def _replace_text(value: str, identities: _ListenerIdentities) -> str:
+    redacted = BEARER_PATTERN.sub("[REDACTED]", value)
+    replacements = {
+        **{identity: SYNTHETIC_ACCOUNT_ID for identity in identities.account_ids},
+        **{identity: SYNTHETIC_DISPLAY_NAME for identity in identities.display_names},
+    }
+    for identity, replacement in sorted(
+        replacements.items(), key=lambda item: len(item[0]), reverse=True
+    ):
+        redacted = redacted.replace(identity, replacement)
+    return redacted
+
+
+def _taste_replacement(url: str, synthetic_taste: dict[str, object]) -> object | None:
+    path = urlparse(url).path.rstrip("/")
+    if path.endswith("/me/top/artists"):
+        return synthetic_taste.get("top_artists")
+    if path.endswith("/me/top/tracks"):
+        return synthetic_taste.get("top_tracks")
+    if path.endswith("/me/tracks"):
+        return synthetic_taste.get("saved_tracks")
+    return None
+
+
+def _is_current_user_url(url: str) -> bool:
+    return urlparse(url).path.rstrip("/").endswith("/v1/me")
+
+
+def _taste_summary(synthetic_taste: dict[str, object]) -> str:
+    summary = synthetic_taste.get("summary")
+    if not isinstance(summary, str):
+        raise ValueError("Synthetic taste fixture must contain a summary")
+    return summary
+
+
+def _load_synthetic_taste() -> dict[str, object]:
+    payload: object = json.loads(SYNTHETIC_TASTE_PATH.read_text(encoding="ascii"))
+    if not isinstance(payload, dict):
+        raise ValueError("Synthetic taste fixture must be a mapping")
+    return cast(dict[str, object], payload)
+
+
+def _decode_json(encoded: str) -> object:
+    try:
+        return json.loads(base64.b64decode(encoded, validate=True))
+    except (ValueError, json.JSONDecodeError, UnicodeDecodeError):
+        return None
+
+
+def _encode(value: bytes) -> str:
+    return base64.b64encode(value).decode("ascii")

eval/recording/storage.py

@@ -0,0 +1,32 @@
+"""Persist raw and redacted cassette stores."""
+
+import json
+from pathlib import Path
+
+from recording.model import RecordedInteraction
+from recording.redaction import redact_cassette
+
+
+def write_cassettes(
+    fixture_root: Path,
+    scenario_key: str,
+    recordings: dict[str, list[RecordedInteraction]],
+) -> tuple[Path, Path]:
+    """Write ignored raw data first, then separately mapped safe cassettes."""
+    raw_root = fixture_root / "raw" / scenario_key
+    redacted_root = fixture_root / scenario_key
+    raw_root.mkdir(parents=True, exist_ok=True)
+    redacted_root.mkdir(parents=True, exist_ok=True)
+    for service, interactions in recordings.items():
+        raw_payload = [interaction.as_dict() for interaction in interactions]
+        _write_json(raw_root / f"{service}.json", raw_payload)
+        redacted_payload = redact_cassette(raw_payload)
+        _write_json(redacted_root / f"{service}.json", redacted_payload)
+    return raw_root, redacted_root
+
+
+def _write_json(path: Path, payload: object) -> None:
+    path.write_text(
+        json.dumps(payload, ensure_ascii=True, indent=2, sort_keys=True) + "\n",
+        encoding="ascii",
+    )

eval/recording/synthetic_taste.json

@@ -0,0 +1,95 @@
+{
+  "saved_tracks": {
+    "href": "https://api.spotify.com/v1/me/tracks",
+    "items": [
+      {
+        "added_at": "2026-01-01T00:00:00Z",
+        "track": {
+          "album": {
+            "images": [],
+            "name": "Synthetic Saved Album"
+          },
+          "artists": [
+            {
+              "name": "Synthetic Saved Artist"
+            }
+          ],
+          "external_urls": {
+            "spotify": "https://open.spotify.com/track/synthetic-saved-track"
+          },
+          "id": "synthetic-saved-track",
+          "name": "Synthetic Saved Track",
+          "uri": "spotify:track:synthetic-saved-track"
+        }
+      }
+    ],
+    "limit": 50,
+    "next": null,
+    "offset": 0,
+    "previous": null,
+    "total": 1
+  },
+  "summary": "Short-term top artists: Synthetic Focus Artist; Long-term top artists: Synthetic Jazz Artist; Short-term top tracks: Synthetic Focus Track by Synthetic Focus Artist; Long-term top tracks: Synthetic Jazz Track by Synthetic Jazz Artist; Saved-track sample: Synthetic Saved Track by Synthetic Saved Artist",
+  "top_artists": {
+    "href": "https://api.spotify.com/v1/me/top/artists",
+    "items": [
+      {
+        "id": "synthetic-focus-artist",
+        "name": "Synthetic Focus Artist"
+      },
+      {
+        "id": "synthetic-jazz-artist",
+        "name": "Synthetic Jazz Artist"
+      }
+    ],
+    "limit": 50,
+    "next": null,
+    "offset": 0,
+    "previous": null,
+    "total": 2
+  },
+  "top_tracks": {
+    "href": "https://api.spotify.com/v1/me/top/tracks",
+    "items": [
+      {
+        "album": {
+          "images": [],
+          "name": "Synthetic Focus Album"
+        },
+        "artists": [
+          {
+            "name": "Synthetic Focus Artist"
+          }
+        ],
+        "external_urls": {
+          "spotify": "https://open.spotify.com/track/synthetic-focus-track"
+        },
+        "id": "synthetic-focus-track",
+        "name": "Synthetic Focus Track",
+        "uri": "spotify:track:synthetic-focus-track"
+      },
+      {
+        "album": {
+          "images": [],
+          "name": "Synthetic Jazz Album"
+        },
+        "artists": [
+          {
+            "name": "Synthetic Jazz Artist"
+          }
+        ],
+        "external_urls": {
+          "spotify": "https://open.spotify.com/track/synthetic-jazz-track"
+        },
+        "id": "synthetic-jazz-track",
+        "name": "Synthetic Jazz Track",
+        "uri": "spotify:track:synthetic-jazz-track"
+      }
+    ],
+    "limit": 50,
+    "next": null,
+    "offset": 0,
+    "previous": null,
+    "total": 2
+  }
+}

eval/recording/transport.py

@@ -0,0 +1,154 @@
+"""Recording transports for Spotify's httpx2 and Anthropic's httpx."""
+
+from collections.abc import AsyncIterator, Callable
+from typing import cast
+
+import httpx
+import httpx2
+
+from recording.model import RecordedInteraction
+from recording.redaction import is_sensitive_url
+
+
+class HttpxRecordingTransport(httpx.AsyncBaseTransport):
+    """Wrap an httpx transport and retain completed response chunk sequences."""
+
+    def __init__(self, transport: httpx.AsyncBaseTransport) -> None:
+        """Bind the real transport and start an empty in-memory cassette."""
+        self.transport = transport
+        self.interactions: list[RecordedInteraction] = []
+
+    async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
+        """Forward one request and wrap its response stream for capture."""
+        request.headers["Accept-Encoding"] = "identity"
+        request_body = await request.aread()
+        response = await self.transport.handle_async_request(request)
+        if is_sensitive_url(str(request.url)):
+            return response
+        stream = _HttpxRecordingStream(
+            cast(httpx.AsyncByteStream, response.stream),
+            lambda chunks: self._finish(request, request_body, response.status_code, chunks),
+        )
+        return httpx.Response(
+            response.status_code,
+            headers=response.headers,
+            stream=stream,
+            extensions=response.extensions,
+            request=request,
+        )
+
+    async def aclose(self) -> None:
+        """Close the wrapped transport."""
+        await self.transport.aclose()
+
+    def _finish(
+        self,
+        request: httpx.Request,
+        request_body: bytes,
+        status: int,
+        chunks: tuple[bytes, ...],
+    ) -> None:
+        self.interactions.append(
+            RecordedInteraction(request.method, str(request.url), status, request_body, chunks)
+        )
+
+
+class Httpx2RecordingTransport(httpx2.AsyncBaseTransport):
+    """Wrap an httpx2 transport and retain completed response chunk sequences."""
+
+    def __init__(self, transport: httpx2.AsyncBaseTransport) -> None:
+        """Bind the real transport and start an empty in-memory cassette."""
+        self.transport = transport
+        self.interactions: list[RecordedInteraction] = []
+
+    async def handle_async_request(self, request: httpx2.Request) -> httpx2.Response:
+        """Forward one request and wrap its response stream for capture."""
+        request.headers["Accept-Encoding"] = "identity"
+        request_body = await request.aread()
+        response = await self.transport.handle_async_request(request)
+        if is_sensitive_url(str(request.url)):
+            return response
+        stream = _Httpx2RecordingStream(
+            cast(httpx2.AsyncByteStream, response.stream),
+            lambda chunks: self._finish(request, request_body, response.status_code, chunks),
+        )
+        return httpx2.Response(
+            response.status_code,
+            headers=response.headers,
+            stream=stream,
+            extensions=response.extensions,
+            request=request,
+        )
+
+    async def aclose(self) -> None:
+        """Close the wrapped transport."""
+        await self.transport.aclose()
+
+    def _finish(
+        self,
+        request: httpx2.Request,
+        request_body: bytes,
+        status: int,
+        chunks: tuple[bytes, ...],
+    ) -> None:
+        self.interactions.append(
+            RecordedInteraction(request.method, str(request.url), status, request_body, chunks)
+        )
+
+
+class _HttpxRecordingStream(httpx.AsyncByteStream):
+    def __init__(
+        self,
+        stream: httpx.AsyncByteStream,
+        finish: Callable[[tuple[bytes, ...]], None],
+    ) -> None:
+        self.stream = stream
+        self.finish = finish
+        self.chunks: list[bytes] = []
+        self.is_finished = False
+
+    async def __aiter__(self) -> AsyncIterator[bytes]:
+        try:
+            async for chunk in self.stream:
+                self.chunks.append(chunk)
+                yield chunk
+        finally:
+            self._finish()
+
+    async def aclose(self) -> None:
+        await self.stream.aclose()
+        self._finish()
+
+    def _finish(self) -> None:
+        if not self.is_finished:
+            self.is_finished = True
+            self.finish(tuple(self.chunks))
+
+
+class _Httpx2RecordingStream(httpx2.AsyncByteStream):
+    def __init__(
+        self,
+        stream: httpx2.AsyncByteStream,
+        finish: Callable[[tuple[bytes, ...]], None],
+    ) -> None:
+        self.stream = stream
+        self.finish = finish
+        self.chunks: list[bytes] = []
+        self.is_finished = False
+
+    async def __aiter__(self) -> AsyncIterator[bytes]:
+        try:
+            async for chunk in self.stream:
+                self.chunks.append(chunk)
+                yield chunk
+        finally:
+            self._finish()
+
+    async def aclose(self) -> None:
+        await self.stream.aclose()
+        self._finish()
+
+    def _finish(self) -> None:
+        if not self.is_finished:
+            self.is_finished = True
+            self.finish(tuple(self.chunks))

eval/run_eval.py

@@ -0,0 +1,98 @@
+"""Command-line entry point for live and baseline evaluation."""
+
+import argparse
+import asyncio
+import os
+from pathlib import Path
+
+from baseline import record_baselines, write_comparison
+from live import LiveLimits, ScenarioResult, run_live_scenarios
+from scenario import SCENARIO_PATH, load_scenarios
+
+EVAL_ROOT = Path(__file__).parent
+
+
+def main() -> int:
+    """Run the selected evaluation arms and return a process status."""
+    arguments = _parse_arguments()
+    scenarios = load_scenarios(arguments.scenarios)
+    exit_code = 0
+    if arguments.base_url is None:
+        raise SystemExit("--base-url is required")
+    limits = LiveLimits(
+        minimum_tracks=arguments.min_tracks,
+        maximum_tracks=arguments.max_tracks,
+        minimum_artists=arguments.min_artists,
+        first_track_budget_ms=arguments.first_track_budget_ms,
+        total_budget_ms=arguments.total_budget_ms,
+    )
+    results = asyncio.run(
+        run_live_scenarios(scenarios, arguments.base_url, arguments.report_dir, limits)
+    )
+    _print_summary(results)
+    if not all(result.passed for result in results):
+        exit_code = 1
+
+    if arguments.baseline:
+        client_id = os.environ.get("SPOTIFY_CLIENT_ID")
+        client_secret = os.environ.get("SPOTIFY_CLIENT_SECRET")
+        if not client_id or not client_secret:
+            print("Spotify client credentials are absent; skipping baseline.")
+        else:
+            baseline_root = arguments.snapshot_dir / "baseline"
+            asyncio.run(record_baselines(scenarios, client_id, client_secret, baseline_root))
+        comparison_path = write_comparison(
+            scenarios,
+            arguments.snapshot_dir / "baseline",
+            arguments.report_dir,
+            arguments.comparison_top_n,
+        )
+        print(f"Comparison: {comparison_path}")
+    return exit_code
+
+
+def _parse_arguments() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--base-url")
+    parser.add_argument("--baseline", action="store_true")
+    parser.add_argument("--scenarios", type=Path, default=SCENARIO_PATH)
+    parser.add_argument("--snapshot-dir", type=Path, default=EVAL_ROOT / "snapshots")
+    parser.add_argument("--report-dir", type=Path, default=EVAL_ROOT / "reports")
+    parser.add_argument("--min-tracks", type=int, default=8)
+    parser.add_argument("--max-tracks", type=int, default=15)
+    parser.add_argument("--min-artists", type=int, default=5)
+    parser.add_argument("--first-track-budget-ms", type=int, default=20_000)
+    parser.add_argument("--total-budget-ms", type=int, default=30_000)
+    parser.add_argument("--comparison-top-n", type=int, default=10)
+    arguments = parser.parse_args()
+    if arguments.min_tracks < 0 or arguments.max_tracks < arguments.min_tracks:
+        parser.error("track bounds are invalid")
+    if min(arguments.min_artists, arguments.first_track_budget_ms, arguments.total_budget_ms) < 0:
+        parser.error("artist and latency limits must be non-negative")
+    if arguments.comparison_top_n < 1:
+        parser.error("comparison top N must be positive")
+    return arguments
+
+
+def _print_summary(results: tuple[ScenarioResult, ...]) -> None:
+    print("| Scenario | Status | Tracks | Artists | First track | Total |")
+    print("| --- | --- | ---: | ---: | ---: | ---: |")
+    for result in results:
+        artists = {str(artist).casefold() for event in result.tracks for artist in _artists(event)}
+        first = f"{result.first_track_ms} ms" if result.first_track_ms is not None else "n/a"
+        total = f"{result.total_ms} ms" if result.total_ms is not None else "n/a"
+        status = "PASS" if result.passed else "FAIL"
+        print(
+            f"| {result.scenario.key} | {status} | {len(result.tracks)} | "
+            f"{len(artists)} | {first} | {total} |"
+        )
+
+
+def _artists(event: dict[str, object]) -> list[object]:
+    track = event.get("track")
+    artists = track.get("artists") if isinstance(track, dict) else None
+    return artists if isinstance(artists, list) else []
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

eval/scenario.py

@@ -0,0 +1,79 @@
+"""Load the shared evaluation scenario catalog."""
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import cast
+
+import yaml
+
+SCENARIO_PATH = Path(__file__).with_name("scenarios.yaml")
+FacetValue = str | bool
+
+
+@dataclass(frozen=True)
+class Scenario:
+    """One golden query and its expected intent facets."""
+
+    key: str
+    chip: str
+    query: str
+    facets: dict[str, FacetValue]
+    after: str | None = None
+
+    @property
+    def is_refinement(self) -> bool:
+        """Return whether this query follows another scenario."""
+        return self.after is not None
+
+
+def load_scenarios(path: Path = SCENARIO_PATH) -> tuple[Scenario, ...]:
+    """Read and validate the shared YAML scenario catalog."""
+    payload = yaml.safe_load(path.read_text(encoding="ascii"))
+    if not isinstance(payload, dict) or payload.get("version") != 1:
+        raise ValueError("Scenario catalog must have version 1")
+    entries = payload.get("scenarios")
+    if not isinstance(entries, list) or not entries:
+        raise ValueError("Scenario catalog must contain scenarios")
+
+    scenarios = tuple(_parse_scenario(entry) for entry in entries)
+    keys = [scenario.key for scenario in scenarios]
+    if len(keys) != len(set(keys)):
+        raise ValueError("Scenario keys must be unique")
+    known_keys = set(keys)
+    for scenario in scenarios:
+        if scenario.after is not None and scenario.after not in known_keys:
+            raise ValueError(f"Unknown parent scenario: {scenario.after}")
+    return scenarios
+
+
+def _parse_scenario(payload: object) -> Scenario:
+    if not isinstance(payload, dict):
+        raise ValueError("Each scenario must be a mapping")
+    required = {"key", "chip", "query", "facets"}
+    allowed = required | {"after"}
+    if set(payload) - allowed or not required <= set(payload):
+        raise ValueError("Scenario fields must match the catalog schema")
+    facets = payload["facets"]
+    if not isinstance(facets, dict) or not facets:
+        raise ValueError("Scenario facets must be a non-empty mapping")
+    if not all(
+        isinstance(key, str) and isinstance(value, (str, bool)) for key, value in facets.items()
+    ):
+        raise ValueError("Scenario facets must contain string or boolean values")
+    after = payload.get("after")
+    if after is not None and not isinstance(after, str):
+        raise ValueError("Scenario after must be a key")
+    return Scenario(
+        key=_required_string(payload, "key"),
+        chip=_required_string(payload, "chip"),
+        query=_required_string(payload, "query"),
+        facets=cast(dict[str, FacetValue], facets),
+        after=after,
+    )
+
+
+def _required_string(payload: dict[object, object], key: str) -> str:
+    value = payload.get(key)
+    if not isinstance(value, str) or not value:
+        raise ValueError(f"Scenario {key} must be a non-empty string")
+    return value

eval/scenarios.yaml

@@ -0,0 +1,49 @@
+version: 1
+scenarios:
+  - key: focus-coding
+    chip: Focus while coding
+    query: something calm for while I am programming, but not boring
+    facets:
+      activity: coding
+      familiarity: mix
+  - key: workout-energy
+    chip: Energy for the gym
+    query: high-energy music for a heavy workout
+    facets:
+      activity: workout
+      mood: energetic
+  - key: dutch-chill
+    chip: Dutch and relaxed
+    query: relaxed Dutch-language music for the couch
+    facets:
+      language: nl
+      mood: relaxed
+  - key: nineties-nostalgia
+    chip: 90s nostalgia
+    query: take me back to the nineties
+    facets:
+      era: 1990s
+  - key: discover-new
+    chip: Surprise me with something new
+    query: something I do not know yet but will probably like
+    facets:
+      familiarity: new
+  - key: rainy-sunday
+    chip: Rainy Sunday
+    query: melancholic but warm, for a rainy Sunday morning
+    facets:
+      mood: melancholic
+  - key: dinner-background
+    chip: Background for dinner
+    query: something jazzy for during dinner, not too present
+    facets:
+      genre: jazz
+      mood: subdued
+  - key: focus-coding-refine
+    chip: More electronic
+    query: a bit more electronic, and drop number 3
+    after: focus-coding
+    facets:
+      refinement: true
+      pool_reuse: true
+      re_grounding: false

eval/tests/test_live.py

@@ -0,0 +1,78 @@
+"""Tests for strict live stream property evaluation."""
+
+import asyncio
+import json
+
+import httpx
+import pytest
+from fastapi import FastAPI
+from fastapi.responses import StreamingResponse
+from live import LiveLimits, run_live_scenario
+from scenario import Scenario
+from wire import validate_event
+
+
+def test_live_scenario_accepts_a_valid_property_stream() -> None:
+    async def run() -> None:
+        app = FastAPI()
+
+        @app.post("/api/recommendations")
+        async def recommendations() -> StreamingResponse:
+            events = [
+                {
+                    "type": "metadata",
+                    "request_id": "request",
+                    "intent_summary": "Calm music for coding with a familiar discovery mix.",
+                    "candidate_count": 10,
+                },
+                {
+                    "type": "track",
+                    "rank": 1,
+                    "track": {
+                        "id": "track",
+                        "uri": "spotify:track:track",
+                        "title": "Track",
+                        "artists": ["Artist"],
+                        "album_name": "Album",
+                        "album_art_url": None,
+                        "external_url": None,
+                    },
+                    "justification": "A focused fit.",
+                },
+                {"type": "done", "track_count": 1, "total_ms": 1},
+            ]
+            body = "".join(json.dumps(event) + "\n" for event in events)
+            return StreamingResponse(iter((body,)), media_type="application/x-ndjson")
+
+        transport = httpx.ASGITransport(app=app)
+        async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
+            result = await run_live_scenario(
+                client,
+                Scenario(
+                    "focus-coding",
+                    "Focus while coding",
+                    "calm coding music",
+                    {"activity": "coding", "familiarity": "mix"},
+                ),
+                LiveLimits(1, 1, 1, 1_000, 1_000),
+            )
+
+        assert result.passed
+        assert result.checks["event_order"]
+        assert result.checks["done_track_count"]
+
+    asyncio.run(run())
+
+
+def test_wire_validation_rejects_extra_fields() -> None:
+    line = json.dumps(
+        {
+            "type": "done",
+            "track_count": 1,
+            "total_ms": 1,
+            "unexpected": True,
+        }
+    )
+
+    with pytest.raises(ValueError, match="fields do not match"):
+        validate_event(line)

eval/tests/test_recording.py

@@ -0,0 +1,134 @@
+"""Tests for cassette transport fidelity and redaction."""
+
+import asyncio
+import base64
+import json
+from collections.abc import AsyncIterator
+
+import httpx
+import httpx2
+from recording import Httpx2RecordingTransport, HttpxRecordingTransport, redact_cassette
+
+
+class HttpxChunks(httpx.AsyncByteStream):
+    """Yield fixed httpx byte chunks."""
+
+    async def __aiter__(self) -> AsyncIterator[bytes]:
+        yield b"first-"
+        yield b"second"
+
+
+class Httpx2Chunks(httpx2.AsyncByteStream):
+    """Yield fixed httpx2 byte chunks."""
+
+    async def __aiter__(self) -> AsyncIterator[bytes]:
+        yield b"first-"
+        yield b"second"
+
+
+def test_httpx_recording_transport_preserves_chunk_boundaries() -> None:
+    async def run() -> None:
+        async def handler(request: httpx.Request) -> httpx.Response:
+            return httpx.Response(200, stream=HttpxChunks())
+
+        transport = HttpxRecordingTransport(httpx.MockTransport(handler))
+        async with httpx.AsyncClient(transport=transport) as client:
+            response = await client.get("https://api.anthropic.com/v1/messages")
+
+        assert response.content == b"first-second"
+        assert transport.interactions[0].response_chunks == (b"first-", b"second")
+
+    asyncio.run(run())
+
+
+def test_httpx2_recording_transport_preserves_chunk_boundaries() -> None:
+    async def run() -> None:
+        async def handler(request: httpx2.Request) -> httpx2.Response:
+            return httpx2.Response(200, stream=Httpx2Chunks())
+
+        transport = Httpx2RecordingTransport(httpx2.MockTransport(handler))
+        async with httpx2.AsyncClient(transport=transport) as client:
+            response = await client.get("https://api.spotify.com/v1/search?q=test")
+
+        assert response.content == b"first-second"
+        assert transport.interactions[0].response_chunks == (b"first-", b"second")
+
+    asyncio.run(run())
+
+
+def test_redaction_removes_headers_tokens_identity_and_taste() -> None:
+    interactions = [
+        {
+            "method": "POST",
+            "url": "https://accounts.spotify.com/api/token",
+            "request_headers": {"Authorization": "Bearer raw-token"},
+            "request_body_base64": _encode(b"client_secret=secret"),
+            "response_body_base64": _encode(b'{"access_token":"raw-token"}'),
+            "response_chunks_base64": [],
+            "status": 200,
+        },
+        {
+            "method": "GET",
+            "url": "https://api.spotify.com/v1/me",
+            "response_headers": {"Set-Cookie": "secret"},
+            "request_body_base64": "",
+            "response_body_base64": _encode(
+                b'{"id":"real-account-id","display_name":"Real Listener"}'
+            ),
+            "response_chunks_base64": [],
+            "status": 200,
+        },
+        {
+            "method": "GET",
+            "url": "https://api.spotify.com/v1/me/top/artists?limit=50",
+            "headers": {"Authorization": "Bearer raw-token"},
+            "request_body_base64": "",
+            "response_body_base64": _encode(b'{"items":[{"name":"Private Artist"}]}'),
+            "response_chunks_base64": [],
+            "status": 200,
+        },
+        {
+            "method": "POST",
+            "url": "https://api.anthropic.com/v1/messages",
+            "request_body_base64": _encode(
+                json.dumps(
+                    {
+                        "messages": [
+                            {
+                                "content": json.dumps(
+                                    {
+                                        "taste_profile": "Private Artist",
+                                        "account": "real-account-id",
+                                    }
+                                )
+                            }
+                        ]
+                    }
+                ).encode()
+            ),
+            "response_body_base64": _encode(b"Bearer raw-token"),
+            "response_chunks_base64": [_encode(b"Bearer raw-token")],
+            "status": 200,
+        },
+    ]
+
+    redacted = redact_cassette(interactions)
+    rendered = json.dumps(redacted)
+    decoded_bodies = " ".join(
+        base64.b64decode(str(item.get(field, ""))).decode(errors="replace")
+        for item in redacted
+        for field in ("request_body_base64", "response_body_base64")
+    )
+
+    assert len(redacted) == 3
+    assert "headers" not in rendered.casefold()
+    assert "raw-token" not in rendered + decoded_bodies
+    assert "real-account-id" not in rendered + decoded_bodies
+    assert "Real Listener" not in rendered + decoded_bodies
+    assert "Private Artist" not in rendered + decoded_bodies
+    assert "Synthetic Focus Artist" in decoded_bodies
+    assert "accounts.spotify.com" not in rendered
+
+
+def _encode(value: bytes) -> str:
+    return base64.b64encode(value).decode("ascii")

eval/wire.py

@@ -0,0 +1,46 @@
+"""Validate NDJSON events against the application-owned wire schemas."""
+
+import json
+import sys
+from importlib import import_module
+from pathlib import Path
+from types import ModuleType
+
+from pydantic import BaseModel, TypeAdapter
+
+BACKEND_ROOT = Path(__file__).parents[1] / "backend"
+EVENT_CLASS_NAMES = {
+    "metadata": "MetadataEvent",
+    "track": "TrackEvent",
+    "warning": "WarningEvent",
+    "error": "ErrorEvent",
+    "done": "DoneEvent",
+}
+
+
+def validate_event(line: str) -> BaseModel:
+    """Parse one complete line with exact fields and strict value types."""
+    schemas = _load_schemas()
+    payload = json.loads(line)
+    if not isinstance(payload, dict):
+        raise ValueError("NDJSON event must be an object")
+    event_type = payload.get("type")
+    if not isinstance(event_type, str) or event_type not in EVENT_CLASS_NAMES:
+        raise ValueError("NDJSON event has an unknown type")
+    event_class = getattr(schemas, EVENT_CLASS_NAMES[event_type])
+    if set(payload) != set(event_class.model_fields):
+        raise ValueError(f"{event_type} event fields do not match the wire schema")
+    if event_type == "track":
+        track = payload.get("track")
+        track_class = schemas.TrackCard
+        if not isinstance(track, dict) or set(track) != set(track_class.model_fields):
+            raise ValueError("track card fields do not match the wire schema")
+    adapter: TypeAdapter[BaseModel] = TypeAdapter(schemas.StreamEvent)
+    return adapter.validate_json(line, strict=True)
+
+
+def _load_schemas() -> ModuleType:
+    backend_path = str(BACKEND_ROOT)
+    if backend_path not in sys.path:
+        sys.path.insert(0, backend_path)
+    return import_module("app.api.schemas")

frontend/package-lock.json

@@ -8,6 +8,9 @@
       "name": "frontend",
       "version": "0.0.0",
       "dependencies": {
+        "@fontsource/bricolage-grotesque": "^5.3.0",
+        "@fontsource/jetbrains-mono": "^5.3.0",
+        "@fontsource/public-sans": "^5.3.0",
         "vue": "^3.5.40"
       },
       "devDependencies": {
@@ -156,6 +159,30 @@
         "node": "^20.19.0 || ^22.13.0 || >=24"
       }
     },
+    "node_modules/@fontsource/bricolage-grotesque": {
+      "version": "5.3.0",
+      "resolved": "https://registry.npmjs.org/@fontsource/bricolage-grotesque/-/bricolage-grotesque-5.3.0.tgz",
+      "integrity": "sha512-MdOVb/5in11IfN/IQJOExMuu6AroCIQbirl0yX7NUvKk+h0HYIKfGaVAXRS/gCPil4aFxWhjZnOw28tUwv7tFw==",
+      "funding": {
+        "url": "https://github.com/sponsors/ayuhito"
+      }
+    },
+    "node_modules/@fontsource/jetbrains-mono": {
+      "version": "5.3.0",
+      "resolved": "https://registry.npmjs.org/@fontsource/jetbrains-mono/-/jetbrains-mono-5.3.0.tgz",
+      "integrity": "sha512-fqDfB5I9f1p1TV486aUgB9t8zP84P0O1FtQR5Ol9vjwPy+S+EIGlVYm1cvj2W5shcZMTg2nZFdVMoH5wFu8a1A==",
+      "funding": {
+        "url": "https://github.com/sponsors/ayuhito"
+      }
+    },
+    "node_modules/@fontsource/public-sans": {
+      "version": "5.3.0",
+      "resolved": "https://registry.npmjs.org/@fontsource/public-sans/-/public-sans-5.3.0.tgz",
+      "integrity": "sha512-kjODI0S3zdv0mBYCIQ8TbBayaiqszpc2UbhJiO3bjIqVVXzcWfHSt2o3WBCLOY3juaGaQoy4MoWCcgmfI5hCuA==",
+      "funding": {
+        "url": "https://github.com/sponsors/ayuhito"
+      }
+    },
     "node_modules/@humanfs/core": {
       "version": "0.19.2",
       "resolved": "https://registry.npmjs.org/@humanfs/core/-/core-0.19.2.tgz",

frontend/package.json

@@ -13,6 +13,9 @@
     "typecheck": "vue-tsc --noEmit -p tsconfig.app.json"
   },
   "dependencies": {
+    "@fontsource/bricolage-grotesque": "^5.3.0",
+    "@fontsource/jetbrains-mono": "^5.3.0",
+    "@fontsource/public-sans": "^5.3.0",
     "vue": "^3.5.40"
   },
   "devDependencies": {

frontend/src/components/AppHeader.vue

@@ -9,7 +9,7 @@ const props = defineProps<{
   mode: AppMode | null
   healthFailed: boolean
 }>()
-const emit = defineEmits<{ logout: []; toggleDev: [opener: HTMLElement] }>()
+const emit = defineEmits<{ logout: []; reset: []; toggleDev: [opener: HTMLElement] }>()
 
 function openDevPanel(event: MouseEvent): void {
   if (event.currentTarget instanceof HTMLElement) {
@@ -29,11 +29,18 @@ const modeLabel = computed(() => {
   <header class="header">
     <div class="inner">
       <div class="brand">
+        <button
+          class="brand-control"
+          type="button"
+          :aria-label="messages.appHeaderBrandResetLabel"
+          @click="$emit('reset')"
+        >
           <span class="wordmark"
             >{{ messages.appHeaderBrandPrefix
             }}<span class="accent">{{ messages.appHeaderBrandAccent }}</span
             >{{ messages.appHeaderBrandSuffix }}</span
           >
+        </button>
         <span class="kicker">{{ messages.appHeaderKicker }}</span>
       </div>
       <div class="controls">
@@ -78,6 +85,20 @@ const modeLabel = computed(() => {
   min-width: 0;
 }
 
+.brand-control {
+  padding: 0;
+  color: inherit;
+  cursor: pointer;
+  background: transparent;
+  border: 0;
+  border-radius: var(--r-sm);
+}
+
+.brand-control:focus-visible {
+  outline: 2px solid var(--c-accent);
+  outline-offset: 4px;
+}
+
 .wordmark {
   font-family: var(--f-display);
   font-size: var(--t-brand);

frontend/src/components/AssistantMessage.vue

@@ -42,8 +42,6 @@ const isEmptyResult = computed(
     <div class="column">
       <p v-if="turn.intentSummary" class="intent">{{ turn.intentSummary }}</p>
 
-      <ResultSet v-if="turn.tracks.length" :tracks="turn.tracks" />
-
       <ThinkingIndicator v-if="turn.status === 'streaming'" :label="thinkingLabel" />
 
       <StreamWarning
@@ -85,6 +83,8 @@ const isEmptyResult = computed(
         :url="turn.playlist.url"
         :track-count="turn.tracks.length"
       />
+
+      <ResultSet v-if="turn.tracks.length" :tracks="turn.tracks" />
     </div>
   </div>
 </template>

frontend/src/components/ChatView.vue

@@ -112,6 +112,7 @@ onUnmounted(() => window.removeEventListener('keydown', onShortcut))
         :mode="mode"
         :health-failed="health.status === 'failed'"
         @logout="logout"
+        @reset="reset"
         @toggle-dev="openDevPanel"
       />
 

frontend/src/components/MessageList.vue

@@ -69,7 +69,13 @@ async function scrollToLatest(): Promise<void> {
   if (!pinned.value) return
   await nextTick()
   requestAnimationFrame(() => {
-    if (list.value) list.value.scrollTop = list.value.scrollHeight
+    const element = list.value
+    if (!element) return
+    if (latestAssistant.value?.status === 'done') {
+      element.querySelector<HTMLElement>('.turn:last-child')?.scrollIntoView({ block: 'start' })
+      return
+    }
+    element.scrollTop = element.scrollHeight
   })
 }
 

frontend/src/components/ResultSet.vue

@@ -1,14 +1,28 @@
 <script setup lang="ts">
-import { messages } from '../lib/messages'
+import { computed } from 'vue'
+import { formatMessage } from '../lib/messages'
 import type { TrackEvent } from '../lib/types'
 import TrackCard from './TrackCard.vue'
 
-defineProps<{ tracks: TrackEvent[] }>()
+const props = defineProps<{ tracks: TrackEvent[] }>()
+const regionLabel = computed(() =>
+  formatMessage(
+    props.tracks.length === 1
+      ? 'resultSetRecommendedTrackLabel'
+      : 'resultSetRecommendedTracksLabel',
+    { count: props.tracks.length },
+  ),
+)
 </script>
 
 <template>
-  <div class="set" :aria-label="messages.resultSetRecommendedTracksLabel">
+  <div class="set" role="region" :aria-label="regionLabel" tabindex="0">
-    <TrackCard v-for="event in tracks" :key="event.track.id" :event="event" />
+    <TrackCard
+      v-for="(event, index) in tracks"
+      :key="event.track.id"
+      :event="event"
+      :style="`--card-order: ${index}`"
+    />
   </div>
 </template>
 
@@ -17,5 +31,18 @@ defineProps<{ tracks: TrackEvent[] }>()
   display: flex;
   flex-direction: column;
   gap: var(--s-2);
+  max-height: min(42dvh, 480px);
+  padding: var(--s-2) var(--s-3) var(--s-2) 0;
+  overflow-y: auto;
+  overscroll-behavior-y: contain;
+  border-block: 1px solid var(--c-line);
+  scrollbar-color: var(--c-line-strong) transparent;
+  scrollbar-width: thin;
+}
+
+@media (max-width: 560px) {
+  .set {
+    max-height: min(34dvh, 360px);
+  }
 }
 </style>

frontend/src/components/SuggestionChips.vue

@@ -1,10 +1,12 @@
 <script setup lang="ts">
+import { messages } from '../lib/messages'
+
 defineProps<{ suggestions: readonly string[] }>()
 defineEmits<{ pick: [suggestion: string] }>()
 </script>
 
 <template>
-  <div class="chips">
+  <div class="chips" role="region" :aria-label="messages.suggestionChipsRegionLabel">
     <button
       v-for="suggestion in suggestions"
       :key="suggestion"
@@ -25,13 +27,12 @@ defineEmits<{ pick: [suggestion: string] }>()
 }
 
 .chip {
-  flex: none;
   padding: var(--s-3) 15px;
   color: var(--c-text-chip);
   font-size: var(--t-small);
   white-space: nowrap;
   cursor: pointer;
-  background: var(--c-surface-raised);
+  background: var(--c-surface);
   border: 1px solid var(--c-line);
   border-radius: var(--r-pill);
   transition: all var(--dur-fast) ease;
@@ -42,18 +43,4 @@ defineEmits<{ pick: [suggestion: string] }>()
   border-color: var(--c-accent);
   transform: translateY(-1px);
 }
-
-@media (max-width: 560px) {
-  .chips {
-    flex-wrap: nowrap;
-    padding: 0 var(--gutter);
-    margin: 0 calc(-1 * var(--gutter));
-    overflow-x: auto;
-    scrollbar-width: none;
-  }
-
-  .chips::-webkit-scrollbar {
-    display: none;
-  }
-}
 </style>

frontend/src/components/TrackCard.vue

@@ -53,6 +53,7 @@ const artworkAlt = computed(() =>
     border-color var(--dur-fast) ease,
     background var(--dur-fast) ease;
   animation: card-in var(--dur-card) var(--ease) both;
+  animation-delay: calc(var(--card-order, 0) * var(--dur-card-stagger));
 }
 
 .card:hover {

frontend/src/composables/useChatStream.ts

@@ -7,7 +7,7 @@ import {
   PRIOR_RECOMMENDATIONS_MAX_TRACKS,
   QUERY_MAX_LENGTH,
 } from '../lib/constants'
-import { formatMessage, messages } from '../lib/messages'
+import { messages } from '../lib/messages'
 import type { AssistantTurn, ChatTurn, EventLogEntry, TransportFailure } from '../lib/models'
 import { EMPTY_PLAYLIST_STATE } from '../lib/models'
 import {
@@ -101,9 +101,7 @@ function reduceEvent(turn: AssistantTurn, event: StreamEvent): AssistantTurn {
 }
 
 function playlistName(query: string): string {
-  return formatMessage('useChatStreamPlaylistName', { query })
+  return query.slice(0, PLAYLIST_NAME_MAX_LENGTH).trim()
-    .slice(0, PLAYLIST_NAME_MAX_LENGTH)
-    .trim()
 }
 
 function createTurnId(): string {

frontend/src/lib/messages.ts

@@ -2,6 +2,7 @@
 
 export const messages = {
   appHeaderBrandPrefix: 'discovery',
+  appHeaderBrandResetLabel: 'discovery-by-llm: start a new chat',
   appHeaderBrandAccent: '-by-',
   appHeaderBrandSuffix: 'llm',
   appHeaderKicker: 'proof of concept',
@@ -90,10 +91,13 @@ export const messages = {
   resultActionsConnectSpotify: 'Connect Spotify to save',
   resultActionsSummary: '{count} verified tracks ready for a private playlist',
 
-  resultSetRecommendedTracksLabel: 'Recommended tracks',
+  resultSetRecommendedTrackLabel: '{count} recommended track',
+  resultSetRecommendedTracksLabel: '{count} recommended tracks',
 
   streamErrorRetry: 'Edit and retry',
 
+  suggestionChipsRegionLabel: 'Listening suggestions',
+
   trackCardArtworkAlt: '{album} album artwork',
   trackCardOpenSpotify: 'Open in Spotify',
 
@@ -110,7 +114,6 @@ export const messages = {
   useApiPlaylistCreationFailed: 'Playlist creation returned {status}.',
 
   useChatStreamTrackCountMismatch: 'The final track count did not match the streamed results.',
-  useChatStreamPlaylistName: '[discovery-by-llm] {query}',
   useChatStreamRequestReplaced: 'This request was replaced by a newer request.',
   useChatStreamUnexpectedFailure: 'The recommendation request failed unexpectedly.',
   useChatStreamTransportEvent: 'transport',

frontend/src/lib/recommendationStream.ts

@@ -36,6 +36,7 @@ function parseLine(line: string): StreamEvent {
 function advancePhase(phase: StreamPhase, event: StreamEvent): StreamPhase {
   if (phase === 'metadata') {
     if (event.type === 'metadata') return 'events'
+    if (event.type === 'error') return 'terminal'
     throw new StreamTransportError('protocol', messages.recommendationStreamMissingMetadata)
   }
   if (phase === 'events') {

frontend/src/main.ts

@@ -1,4 +1,8 @@
 import { createApp } from 'vue'
+import '@fontsource/bricolage-grotesque/600.css'
+import '@fontsource/jetbrains-mono/400.css'
+import '@fontsource/public-sans/400.css'
+import '@fontsource/public-sans/500.css'
 import './style.css'
 import App from './App.vue'
 

frontend/src/style.css

@@ -74,6 +74,11 @@
 
   --shadow-drawer: -24px 0 60px rgba(0, 0, 0, 0.45);
   --ease: cubic-bezier(0.2, 0.8, 0.3, 1);
+  --dur-fast: 0.16s;
+  --dur-card: 0.4s;
+  --dur-card-stagger: 0.08s;
+  --dur-reduced-motion: 0.001s;
+
   color-scheme: dark;
   color: var(--c-text);
   background: var(--c-bg);
@@ -151,7 +156,9 @@ a:hover {
   *::before,
   *::after {
     scroll-behavior: auto !important;
+    animation-delay: var(--dur-reduced-motion) !important;
     animation-duration: var(--dur-reduced-motion) !important;
+    transition-delay: var(--dur-reduced-motion) !important;
     transition-duration: var(--dur-reduced-motion) !important;
   }
 }

frontend/tests/recommendationStream.test.ts

@@ -128,8 +128,23 @@ describe('streamRecommendations', () => {
     expect(wasCancelled).toBe(true)
   })
 
+  it('accepts a terminal error as the only event', async () => {
+    const error = { type: 'error', code: 'first_stage_failed', message: 'Planning failed.' }
+    stubResponse(streamResponse([encoder.encode(`${JSON.stringify(error)}\n`)]))
+    const events: StreamEvent[] = []
+
+    await streamRecommendations(
+      { schema_version: 1, query: 'focus', history: [], prior_recommendations: [] },
+      new AbortController().signal,
+      (event) => events.push(event),
+    )
+
+    expect(events).toEqual([error])
+  })
+
   it.each([
     ['an event before metadata', [{ type: 'warning', code: 'early', message: 'No metadata.' }]],
+    ['done before metadata', [{ type: 'done', track_count: 0, total_ms: 4 }]],
     [
       'duplicate metadata',
       [

frontend/tests/resultPresentation.test.ts

@@ -0,0 +1,186 @@
+import { createApp, defineComponent, h, nextTick, ref } from 'vue'
+import type { Component } from 'vue'
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import AppHeader from '../src/components/AppHeader.vue'
+import AssistantMessage from '../src/components/AssistantMessage.vue'
+import MessageList from '../src/components/MessageList.vue'
+import SuggestionChips from '../src/components/SuggestionChips.vue'
+import type { AssistantTurn, ChatTurn } from '../src/lib/models'
+import type { TrackEvent } from '../src/lib/types'
+
+const unmountCallbacks: Array<() => void> = []
+
+function track(rank: number): TrackEvent {
+  return {
+    type: 'track',
+    rank,
+    track: {
+      id: `track-${rank}`,
+      uri: `spotify:track:${rank}`,
+      title: `Track ${rank}`,
+      artists: ['Artist'],
+      album_name: 'Album',
+      album_art_url: null,
+      external_url: null,
+    },
+    justification: 'It fits.',
+  }
+}
+
+function doneTurn(): AssistantTurn {
+  return {
+    id: 'assistant-1',
+    role: 'assistant',
+    query: 'Focused listening',
+    status: 'done',
+    requestId: 'request-1',
+    intentSummary: 'Calm music for focused work.',
+    candidateCount: 20,
+    tracks: [track(1), track(2)],
+    warnings: [{ type: 'warning', code: 'limited_pool', message: 'The pool was limited.' }],
+    error: null,
+    transportFailure: null,
+    completion: { type: 'done', track_count: 2, total_ms: 12 },
+    playlist: { status: 'idle', name: null, url: null, message: null },
+  }
+}
+
+function mountComponent(component: Component, props: Record<string, unknown>): HTMLElement {
+  const root = document.createElement('div')
+  document.body.append(root)
+  const app = createApp(
+    defineComponent({
+      render: () => h(component, props),
+    }),
+  )
+  app.mount(root)
+  unmountCallbacks.push(() => {
+    app.unmount()
+    root.remove()
+  })
+  return root
+}
+
+afterEach(() => {
+  while (unmountCallbacks.length) unmountCallbacks.pop()?.()
+  vi.restoreAllMocks()
+})
+
+describe('result presentation', () => {
+  it('uses the app title as a fresh-chat control', () => {
+    const onReset = vi.fn()
+    const root = mountComponent(AppHeader, {
+      auth: { status: 'anonymous', user: null, message: null },
+      mode: 'demo',
+      healthFailed: false,
+      onReset,
+    })
+    const button = root.querySelector<HTMLButtonElement>(
+      'button[aria-label="discovery-by-llm: start a new chat"]',
+    )
+
+    expect(button?.type).toBe('button')
+    expect(button?.textContent).toContain('discovery-by-llm')
+    button?.click()
+    expect(onReset).toHaveBeenCalledOnce()
+  })
+
+  it('keeps completion context outside the focusable track region', () => {
+    const root = mountComponent(AssistantMessage, { turn: doneTurn(), canSave: true })
+    const region = root.querySelector<HTMLElement>(
+      '[role="region"][aria-label="2 recommended tracks"]',
+    )
+    const summary = root.querySelector<HTMLElement>('.intent')
+    const warning = root.querySelector<HTMLElement>('[data-warning-code="limited_pool"]')
+    const action = root.querySelector<HTMLButtonElement>('button')
+
+    expect(region?.tabIndex).toBe(0)
+    expect(region?.querySelectorAll('article')).toHaveLength(2)
+    expect(summary?.textContent).toContain('Calm music')
+    expect(warning?.textContent).toContain('pool was limited')
+    expect(action?.textContent).toContain('Save as playlist')
+    expect(region?.contains(summary ?? null)).toBe(false)
+    expect(region?.contains(warning ?? null)).toBe(false)
+    expect(region?.contains(action ?? null)).toBe(false)
+  })
+
+  it('assigns a deliberate stagger order to arriving cards', () => {
+    const root = mountComponent(AssistantMessage, { turn: doneTurn(), canSave: true })
+    const cards = root.querySelectorAll<HTMLElement>('article')
+
+    expect(cards[0]?.style.getPropertyValue('--card-order')).toBe('0')
+    expect(cards[1]?.style.getPropertyValue('--card-order')).toBe('1')
+  })
+
+  it('anchors a completed response at the top of the conversation viewport', async () => {
+    const scrollIntoView = vi
+      .spyOn(Element.prototype, 'scrollIntoView')
+      .mockImplementation(() => undefined)
+    vi.spyOn(window, 'requestAnimationFrame').mockImplementation((callback) => {
+      callback(0)
+      return 1
+    })
+    const complete = doneTurn()
+    const turns = ref<ChatTurn[]>([
+      { id: 'user-1', role: 'user', text: 'Focused listening' },
+      { ...complete, status: 'streaming', completion: null },
+    ])
+    const root = document.createElement('div')
+    document.body.append(root)
+    const app = createApp(
+      defineComponent({
+        render: () =>
+          h(MessageList, {
+            turns: turns.value,
+            suggestions: [],
+            canSave: true,
+          }),
+      }),
+    )
+    app.mount(root)
+    unmountCallbacks.push(() => {
+      app.unmount()
+      root.remove()
+    })
+
+    turns.value = [turns.value[0] as ChatTurn, complete]
+    await nextTick()
+    await nextTick()
+
+    expect(scrollIntoView).toHaveBeenCalledWith({ block: 'start' })
+  })
+
+  it('keeps suggestions in a labeled region', () => {
+    const onPick = vi.fn()
+    const suggestions = ['Focus', 'Energy', 'Surprise']
+    const root = mountComponent(SuggestionChips, { suggestions, onPick })
+    const region = root.querySelector<HTMLElement>('[aria-label="Listening suggestions"]')
+    const buttons = root.querySelectorAll<HTMLButtonElement>('button')
+
+    expect(region).not.toBeNull()
+    expect(buttons).toHaveLength(suggestions.length)
+    expect([...buttons].map((button) => button.textContent?.trim())).toEqual(suggestions)
+
+    buttons[0]?.click()
+    expect(onPick).toHaveBeenCalledWith('Focus')
+  })
+
+  it('renders a first-stage stream error with its code and message', () => {
+    const turn: AssistantTurn = {
+      ...doneTurn(),
+      status: 'error',
+      requestId: null,
+      intentSummary: '',
+      tracks: [],
+      warnings: [],
+      error: { type: 'error', code: 'first_stage_failed', message: 'Planning failed.' },
+      transportFailure: null,
+      completion: null,
+    }
+    const root = mountComponent(AssistantMessage, { turn, canSave: false })
+    const alert = root.querySelector<HTMLElement>('[role="alert"]')
+
+    expect(alert?.dataset.errorCode).toBe('first_stage_failed')
+    expect(alert?.textContent).toContain('Planning failed.')
+  })
+})

frontend/tests/useChatStream.test.ts

@@ -13,14 +13,16 @@ vi.mock('../src/lib/recommendationStream', async (importOriginal) => {
 const streamMock = vi.mocked(streamRecommendations)
 const unmountCallbacks: Array<() => void> = []
 
-function mountChat() {
+function mountChat(
+  createPlaylist: Parameters<typeof useChatStream>[0] = vi.fn(async () => ({ url: null })),
+) {
   let chat: ReturnType<typeof useChatStream> | undefined
   const root = document.createElement('div')
   document.body.append(root)
   const app = createApp(
     defineComponent({
       setup() {
-        chat = useChatStream(vi.fn(async () => ({ url: null })))
+        chat = useChatStream(createPlaylist)
         return () => h('div')
       },
     }),
@@ -106,6 +108,22 @@ describe('useChatStream', () => {
     expect(turn?.transportFailure).toBeNull()
   })
 
+  it('stores a metadata-free terminal error as a normal stream error', async () => {
+    streamMock.mockImplementation(async (_request, _signal, onEvent) => {
+      onEvent({ type: 'error', code: 'first_stage_failed', message: 'Planning failed.' })
+    })
+    const chat = mountChat()
+
+    await chat.send('error')
+
+    expect(assistantTurns(chat)[0]).toMatchObject({
+      status: 'error',
+      requestId: null,
+      error: { code: 'first_stage_failed', message: 'Planning failed.' },
+      transportFailure: null,
+    })
+  })
+
   it('stores an incomplete stream as a transport failure', async () => {
     streamMock.mockRejectedValue(
       new StreamTransportError('unexpected_eof', 'The stream ended early.'),
@@ -239,4 +257,26 @@ describe('useChatStream', () => {
 
     expect(receivedQuery).toHaveLength(1000)
   })
+
+  it('uses the bare query as the capped playlist name', async () => {
+    const createPlaylist = vi.fn(async () => ({ url: null }))
+    streamMock.mockImplementation(async (_request, _signal, onEvent) => {
+      onEvent(metadata('playlist'))
+      onEvent(track(1))
+      onEvent({ type: 'done', track_count: 1, total_ms: 5 })
+    })
+    const chat = mountChat(createPlaylist)
+    const query = `Late-night instrumental focus ${'x'.repeat(100)}`
+
+    await chat.send(query)
+    const turn = assistantTurns(chat)[0]
+    if (!turn) throw new Error('Assistant turn was not created.')
+    await chat.savePlaylist(turn.id)
+
+    expect(createPlaylist).toHaveBeenCalledWith({
+      schema_version: 1,
+      name: query.slice(0, 100),
+      track_uris: ['spotify:track:1'],
+    })
+  })
 })