justinzeus/discovery-by-llm
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: write the README and map the build workflow
Some checks are pending
ci / backend (push) Waiting to run
Details
ci / frontend (push) Waiting to run
Details

Browse source
This commit is contained in:
Justin Visser 2026-08-11 09:47:22 +02:00
parent 5080e9a609
commit fa356ba5c5
3 changed files with 245 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

161
README.md
View file

@ -1,35 +1,168 @@
# discovery-by-llm
This is a demo that serves as a proof of concept for LLM utilization for music discovery, specifically using the Spotify API. The form-factor is an LLM-chat like experience, with direct Spotify integration.
Music discovery in a chat form: describe the moment, get verified Spotify
tracks with a reason per track, save the result as a playlist.
> Work in progress. This file is filled in during the build.
> Chronological build log (Dutch): [docs/logboek.md](docs/logboek.md).
Spotify removed its recommendation and audio-intelligence endpoints for new
apps, so in this PoC the LLM fills that role. It interprets the request and
proposes candidates; Spotify verifies them and supplies the taste profile,
album art and playback. The eval suite compares the output against bare
Spotify search on the same questions.
## Demo
> Chronological build log, including what was dropped and why (Dutch):
> [docs/logboek.md](docs/logboek.md).
*(to follow: hosted instance + local `docker compose up --build`, with and
without API keys)*
## Running it
**Hosted instance** (credentials in the submission mail): a live deployment
bound to my own Spotify account, with real personalisation and playlist
writes. Spotify dev mode caps an app at 5 allowlisted users, so a reviewer
cannot log in with their own account; I can allowlist a reviewer account on
request.
**Without any keys:**
```
docker compose up --build
```
Demo mode replays recorded API cassettes through the same pipeline and
streaming path as live mode. The suggestion chips map to the recorded
scenarios; any other input replays the nearest scenario, with a banner
naming it. Playlist writes are simulated and labeled.
**Live with your own keys:** copy `.env.example` to `.env`, set
`APP_MODE=live`, a Spotify client id (redirect URI
`http://127.0.0.1:8888/callback`) and an Anthropic API key, then
`docker compose up --build` and log in via Spotify.
## How it works
*(to follow: pipeline diagram and module map)*
```
query ──► call 1: intent + candidates ──► grounding ──► call 2: rerank ──► cards
(LLM, structured output) (Spotify (LLM, streamed
▲ search per track)
taste profile fan-out) │
(cached, /me/top + saved) │ playlist write
verified pool
```
- **Call 1** interprets the request (mood, activity, era, language,
familiarity) and proposes 35 candidate tracks, informed by a compressed
taste profile fetched once per session.
- **Grounding** resolves every candidate against Spotify search in a
bounded concurrent fan-out with early stop, a request deadline and a
name-to-id cache. Unverified candidates are dropped. Misses and
mismatches are logged as separate rates.
- **Call 2** reranks the verified pool and streams one justification per
track as NDJSON events; cards render as they arrive.
- Only track ids from the verified pool can reach the user. In live
testing at familiarity `new`, the model fabricated 12 of 32 candidates;
the matcher dropped all 12 and the user still received 15 verified
tracks.
- An honest limit on discovery depth: "new to you" means "not in your top
or saved tracks", and the intent prompt favors well-known tracks from
adjacent scenes because those verify reliably (chasing obscure work is
where the model starts inventing titles). A listener deep in those
scenes will recognize a fair share of the results. Going deeper needs a
repair loop and a better novelty signal; see the last section.
- Refinement turns rerank the existing pool; turn 2 makes zero Spotify
calls.
## Choices
*(to follow)*
- **Two LLM calls around a deterministic grounding step**, no agent loop:
predictable latency and every stage testable on its own.
- **Hand-rolled Spotify client** (~200 lines over httpx). The endpoint
surface is small and the retry, rate-limit and caching behavior is where
the engineering lives. Retry policy is per endpoint: bounded Retry-After
honor on reads, one token refresh, and no retries on playlist writes
(side effects would be ambiguous).
- **Ports and adapters.** Demo mode is a second implementation of the same
ports. The domain model never sees Spotify JSON.
- **NDJSON over a streamed POST.** The request has a body and EventSource
cannot POST. Typed events (metadata / track / warning / error / done)
with a strict client-side phase machine.
- **Degradation is explicit.** Below the grounding floor the app returns
what resolved plus a warning; quota exhaustion (QUOTA_EXCEEDED) is
recognized and never retried; live mode never falls back to fixtures.
- **Hosted seed session.** The public instance installs a session from an
escrowed refresh token at startup, so every visitor shares my account,
including playlist writes. HTTP basic auth at the edge is what makes
that acceptable. Playlists carry a fixed name prefix for bulk cleanup.
- **No embeddings.** Search is the only entry point into the catalog; an
index does not pay for itself inside a PoC.
## Performance and optimisations
## Performance
*(to follow)*
Time to first card is 20 to 30 seconds, measured through the UI and in the
eval suite; the spread is LLM latency. Streaming keeps the wait honest:
metadata arrives early, cards render one by one, progress states name what
is happening.
Under the hood:
- Search `limit` is capped at 10 since Feb 2026, so resolving 35 candidates
requires a fan-out: bounded concurrency, early stop once the rerank pool
is full, request deadline, partial results on quota exhaustion.
- Name-to-id resolution cache and a session-scoped taste-profile cache.
- Anthropic prompt caching between calls.
- Per-request counters (Spotify calls, cache hits, LLM tokens) in
structured logs, surfaced in the dev panel.
Bare search matches words; it has no notion of mood, era, energy or who is
asking. The pipeline answers from the listener's actual scenes. Across all
8 eval scenarios the pipeline delivered 11 to 15 verified tracks from 8 to
20 distinct artists; on the discovery scenario, all 15 were outside the
listener's top and saved tracks on Spotify. A sample (full table via
`eval/run_eval.py --baseline`; the eval checks are plain code assertions,
no LLM judging):
| "Energy for the gym" | Pipeline | Bare search |
| -------------------- | ---------------------------------------- | -------------------------------------------------------------- |
| 1 | Baddadan by Chase & Status, Bou, Flowdan | Redbone (with GloRilla) by Lil Baby |
| 2 | Tough Talk by Chase & Status, Kwengface | Rock That Body by Black Eyed Peas |
| 3 | Solar System by Sub Focus | Instigator by Inpatient, Ren, Chris Webby |
| 8 | DJ Turn It Up by Dimension | Gym Power Beat by Ultimate Fitness Playlist Power Workout Trax |
## Where to look
*(to follow)*
| Criterion | Where |
| -------------------------- | ------------------------------------------------------------------------------------------------------ |
| Code quality and structure | `backend/app/` (domain / ports / adapters / pipeline / api), `AGENTS.md` |
| AI application | `backend/app/prompts.py`, `backend/app/adapters/anthropic/llm.py`, `backend/app/pipeline/grounding.py` |
| Spotify integration | `backend/app/adapters/spotify/` (PKCE, retry policy, mapping) |
| Performance | `backend/app/pipeline/grounding.py` (fan-out, cache, early stop), `backend/app/observability/` |
| Streaming contract | `backend/app/api/recommendations.py`, `frontend/src/lib/recommendationStream.ts` |
| Evaluation | `eval/` (runner, scenarios, baseline arm, recorded fixtures) |
| Pragmatic cuts over time | `docs/logboek.md` |
## Method
*(to follow)*
About one hour of preparation (API research, a design spike) and about
eight hours of hands-on build time across three days. The build was
agent-assisted: coding agents implemented scoped tracks from briefs I
wrote, while I owned the architecture, the API contracts, every code
review, every merge and all live verification against the real APIs. Work
landed in reviewed batches, which is why commits arrive in bursts. The
logboek documents the process as it happened; the setup behind it (server,
Forgejo/GitHub mirror, deployment, how the agents are used) is mapped in
[docs/workflow.md](docs/workflow.md).
## What I cut / what I would do next
## What I cut, and what would come next
*(to follow)*
- A faster model on the intent call, which would cut most of the time to
first card; the grounding stage already measures the trade-off to watch
(fabrication rate), so the experiment is cheap to run safely.
- Deeper discovery. A repair loop that re-prompts the model for candidates
that failed verification would let the prompt chase less obvious work
without losing the pool, and a novelty signal beyond exact-id exclusion
(artist-level familiarity, recently played) would push results past
"famous in an adjacent scene".
- Refinement turns that re-ground; today turn 2 reranks the existing pool.
- Chat history persistence. Conversations are client-side only and the
server keeps no conversation state; sessions and caches are in-process,
Redis is the named production step.
- Playback queue control, an i18n framework (copy is already centralized),
per-stage byte snapshots in the eval, and a semantic index over grounded
candidates.