First product

planning/mvp_implementation_reminders.md

@@ -0,0 +1,92 @@
+# MVP Implementation Reminders (From Scholar Probe)
+
+Last validated probe run: `planning/scholar_probe_tmp/notes/probe_report_run_20260216T182334Z.md`.
+
+## Decision: Are we ready?
+
+Yes, for the scoped MVP.
+
+- We have stable selectors and field coverage for first-page profile parsing.
+- We have explicit failure modes for blocked/inaccessible pages.
+- We have enough fixtures to start parser + ingestion tests immediately.
+
+Not guaranteed (and intentionally out of MVP):
+
+- Full historical completeness from profile pagination.
+- Robots currently disallows `citations?*cstart=`; treat deep pagination as out of scope unless policy changes.
+
+## Non-negotiables
+
+- Keep app container-first (`docker compose`) and test while developing.
+- Keep feature scope small; prefer deterministic behavior over clever scraping.
+- Never crash a whole run because one scholar page fails.
+- Preserve strict tenant isolation for all run/output/read-state data.
+
+## Scraping Contract
+
+Target URL:
+
+- `https://scholar.google.com/citations?hl=en&user=<scholar_id>`
+
+Primary parse markers:
+
+- Row: `tr.gsc_a_tr`
+- Title/link: `a.gsc_a_at`
+- Citation count: `a.gsc_a_ac`
+- Year: `span.gsc_a_h` (fallback year regex)
+- Metadata lines: first/second `div.gs_gray` => authors/venue
+- Cluster id from `citation_for_view=<user>:<cluster_id>`
+
+## Required Parse States
+
+Use and persist one of:
+
+- `ok`
+- `no_results`
+- `blocked_or_captcha`
+- `layout_changed`
+- `network_error`
+
+Plus these page-level flags:
+
+- `has_show_more_button`
+- `articles_range`
+
+## Data Quality Expectations
+
+Observed from probe sample:
+
+- `title`: 100%
+- `cluster_id`: 100%
+- `citation_count`: 100%
+- `authors_text`: 100%
+- `year`: 98.2%
+- `venue_text`: 94.7%
+
+Implications:
+
+- `year` and `venue_text` must remain nullable.
+- Dedupe must not depend solely on `year`/`venue_text` presence.
+
+## Run Semantics
+
+- First successful run per scholar = baseline.
+- If `has_show_more_button=true`, label result as partial in run status/UI.
+- Invalid/inaccessible scholar IDs are expected and should become structured run errors, not exceptions.
+
+## Testing Priorities (Do Before Feature Expansion)
+
+- Fixture parser unit tests using probe HTML snapshots.
+- Parse-state classification tests (ok/blocked/layout/no_results).
+- Dedupe integration tests (`cluster_id` first, fingerprint fallback).
+- Tenant isolation tests for run records and read-state.
+- Smoke test for manual run path through Docker.
+
+## Stop Conditions
+
+Pause implementation and re-probe if:
+
+- Selector markers drop out unexpectedly.
+- Login/redirect pages become frequent for valid IDs.
+- Robots policy for profile endpoints changes.
+