Commit graph

4 commits

Author SHA1 Message Date
d9be57a6c1 fix: URL-length-aware OpenAlex batching and eager scholar metadata hydration
- Replace fixed batch_size=25 with URL-byte-length-aware chunking for
  OpenAlex title.search queries. Long/Unicode titles caused URLs to
  exceed server limits, resulting in 500 errors.
- Always hydrate scholar profile metadata (name, image) on creation,
  even when an initial scrape job is queued, so the UI shows profile
  info immediately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 23:42:15 +01:00
a5165dc3ba after audit 2026-03-01 18:14:22 +01:00
2a6e573a52 refactor: extract helpers from long functions in ingestion core
- Extract _load_unenriched_publications, _enrich_batch, _flush_and_sweep_duplicates from enrich_pending_publications
- Extract _sanitize_titles shared helper for title cleaning
- Extract _run_iteration_and_complete and _inline_enrich_and_finalize from run_for_user
- Extract _run_first_pass and _run_depth_pass from run_scholar_iteration
- Extract _finalize_successful_queue_job and _finalize_failed_queue_job from _finalize_queue_job_after_run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 16:18:21 +01:00
6379c97e90 decompose application.py into enrichment, scholar processing, run completion 2026-02-27 14:19:39 +01:00