- Extract _load_unenriched_publications, _enrich_batch, _flush_and_sweep_duplicates from enrich_pending_publications
- Extract _sanitize_titles shared helper for title cleaning
- Extract _run_iteration_and_complete and _inline_enrich_and_finalize from run_for_user
- Extract _run_first_pass and _run_depth_pass from run_scholar_iteration
- Extract _finalize_successful_queue_job and _finalize_failed_queue_job from _finalize_queue_job_after_run
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rename _int_or_default → int_or_default (public API, was using private convention)
- Extract shared pdf_queue utilities (utcnow, event_row, queued_job, status constants) to pdf_queue_common.py to avoid circular imports
- Consolidate _effective_request_delay_seconds into queue_runner.py with explicit floor parameter
- Update tests to match new function locations and names
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split app/services/ingestion/application.py (2,955 lines) into three
focused modules:
- page_fetch.py (219 lines): PageFetcher class for single-page fetch,
parse-or-layout-error handling, and retry with backoff
- pagination.py (486 lines): PaginationEngine class for multi-page
orchestration, loop state management, and short-circuit detection
- publication_upsert.py (239 lines): module-level functions for
resolve_publication, upsert_profile_publications, and all
find/create/update helpers
Also fixes two external import paths in portability/ that incorrectly
sourced normalize_title and build_publication_url from application.py
instead of fingerprints.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add white logo variant at .github/logo-dark.png using <picture> tag
with prefers-color-scheme media query
- Update mermaid diagram to show Vue<->FastAPI REST+SSE connection
and bidirectional API<->DB relationship
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ruff and mypy to dev dependencies with configuration in pyproject.toml.
Add a lint CI job that runs ruff check, ruff format --check, and mypy.
Auto-fix import sorting and formatting across the codebase. Exclude
alembic/versions from linting (auto-generated migrations). Ignore B008
(FastAPI Depends pattern) and RUF001 (unicode in user-facing strings).
21 ruff lint errors and 50 mypy errors remain for manual review.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove 114 tracked build/lib/ files (stale setuptools artifact that
broke the repo-hygiene CI guard)
- Add build/ to .gitignore
- Remove AGENTS.MD from .gitignore (track agents.md instead)
- Delete ad-hoc scripts: diag_pubs.py, test_enrich.py
- Delete orphaned frontend/docs/website/package-lock.json
- Replace npm install with npm ci in ci.yml and docs-pages.yml for
deterministic lockfile installs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
