Commit graph

6 commits

Author SHA1 Message Date
4620978dce feat(ingestion,frontend): add live scholar progress counter to dashboard
Emit scholar_progress SSE events during two-pass ingestion so the
Activity Monitor shows "X / N visited · Y finished · Z new publications"
in real time instead of static post-run counts.

- scholar_processing: add on_progress callback to _run_first_pass and
  _run_depth_pass; build _emit closure in run_scholar_iteration that
  publishes visited/finished/total via run_events; batch-emit scholars
  finished in first pass before depth pass; skip emission on abort/cancel
- run_status store: add scholarProgress state, reset on stream open and
  reset(); subscribe to scholar_progress SSE events with safe parsing
- DashboardPage: show live counter during run, fall back to static text
  when run is not active or no progress received yet

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 17:36:43 +01:00
f50b609a36 fix(db,frontend): reserve DB connections for API and reduce frontend polling
Background tasks (ingestion, PDF resolution, scheduler) now acquire an
asyncio.Semaphore before checking out a DB connection, guaranteeing at
least N connections remain available for API request handlers. Removes
duplicate polling loop from PublicationsPage, debounces publication
reload on run-status changes, and increases poll interval from 5s to 15s
since SSE handles live discovery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 19:09:06 +01:00
3d4cfeff1a Intermediate commit 2026-02-26 16:09:57 +01:00
0e9e49df16 temp commit 2026-02-26 12:54:19 +01:00
d7404abf18 arden scrape safety telemetry and polish run state UX 2026-02-19 21:59:36 +01:00
110e246a1c Added run state front end support
Updated scholars layout to match dashboard
2026-02-19 20:07:08 +01:00