- Add missing blank line after imports in application.py (ruff I001)
- Use full path string instead of template literal for export API call
to avoid false positive in API contract drift check
- Update pdf-queue test to expect 200 for non-admin listing
Add Set<number>-based selection state to ScholarsPage with checkboxes
in both table and card views, select-all toggle, and stale-key pruning.
Backend: POST /scholars/bulk-delete, POST /scholars/bulk-toggle, and
GET /scholars/export?ids= filter. Service functions use user-scoped
queries. Structured logging for bulk operations.
Frontend: useScholarBulkActions composable extracts selection and bulk
action logic. AppSelect-based action dropdown with dynamic options.
Delete prompts window.confirm; enable/disable applies immediately.
Export downloads filtered JSON.
Includes integration tests for all bulk endpoints and frontend unit
tests for selection toggle, select-all, and stale pruning.
- Convert check interval setting from minutes to hours in the UI
- Show PDF queue tab to all authenticated users (not admin-only); requeue
actions remain admin-only in both backend and frontend
- Fix users tab not loading on first visit by removing the AsyncStateGate
wrapper that prevented child component refs from being populated before
onMounted fired
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove the auth.isAdmin guard from the Start check and Cancel check
buttons in the Activity Monitor. Admin-only route links (check history,
diagnostics) are unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues with the initial implementation:
- Counter only appeared after the first SSE event arrived (scholarProgress null
kept the v-else static template showing until a scholar completed)
- First events were often dropped because the EventSource hadn't subscribed yet
Fix:
- Backend: emit an initial scholar_progress kickoff event (0/0/N) at the
start of run_scholar_iteration so the frontend receives total early
- Frontend: gate on isLikelyRunning alone (not scholarProgress); use optional
chaining with fallbacks so "0 / … visited · 0 finished" shows immediately
when a run starts, before any SSE events arrive
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Emit scholar_progress SSE events during two-pass ingestion so the
Activity Monitor shows "X / N visited · Y finished · Z new publications"
in real time instead of static post-run counts.
- scholar_processing: add on_progress callback to _run_first_pass and
_run_depth_pass; build _emit closure in run_scholar_iteration that
publishes visited/finished/total via run_events; batch-emit scholars
finished in first pass before depth pass; skip emission on abort/cancel
- run_status store: add scholarProgress state, reset on stream open and
reset(); subscribe to scholar_progress SSE events with safe parsing
- DashboardPage: show live counter during run, fall back to static text
when run is not active or no progress received yet
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Background tasks (ingestion, PDF resolution, scheduler) now acquire an
asyncio.Semaphore before checking out a DB connection, guaranteeing at
least N connections remain available for API request handlers. Removes
duplicate polling loop from PublicationsPage, debounces publication
reload on run-status changes, and increases poll interval from 5s to 15s
since SSE handles live discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unused type:ignore comment in scholars.py
- Fix 51 mypy errors across test files (Any return types, cast(), protocol stubs)
- Fix arxiv unit test failures caused by session isolation (patch_session_factory fixture)
- Fix integration test rate-limit bleed between search and create
- Add scripts/premerge.sh for local CI verification
- Add docs/website/.docusaurus/ to .gitignore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace fixed batch_size=25 with URL-byte-length-aware chunking for
OpenAlex title.search queries. Long/Unicode titles caused URLs to
exceed server limits, resulting in 500 errors.
- Always hydrate scholar profile metadata (name, image) on creation,
even when an initial scrape job is queued, so the UI shows profile
info immediately.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The upsert rolls back the entire transaction on partial failure,
so new_pub_count and link count should be consistent (both 0),
not assuming the first publication survived.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Extract _load_unenriched_publications, _enrich_batch, _flush_and_sweep_duplicates from enrich_pending_publications
- Extract _sanitize_titles shared helper for title cleaning
- Extract _run_iteration_and_complete and _inline_enrich_and_finalize from run_for_user
- Extract _run_first_pass and _run_depth_pass from run_scholar_iteration
- Extract _finalize_successful_queue_job and _finalize_failed_queue_job from _finalize_queue_job_after_run
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rename _int_or_default → int_or_default (public API, was using private convention)
- Extract shared pdf_queue utilities (utcnow, event_row, queued_job, status constants) to pdf_queue_common.py to avoid circular imports
- Consolidate _effective_request_delay_seconds into queue_runner.py with explicit floor parameter
- Update tests to match new function locations and names
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split app/services/ingestion/application.py (2,955 lines) into three
focused modules:
- page_fetch.py (219 lines): PageFetcher class for single-page fetch,
parse-or-layout-error handling, and retry with backoff
- pagination.py (486 lines): PaginationEngine class for multi-page
orchestration, loop state management, and short-circuit detection
- publication_upsert.py (239 lines): module-level functions for
resolve_publication, upsert_profile_publications, and all
find/create/update helpers
Also fixes two external import paths in portability/ that incorrectly
sourced normalize_title and build_publication_url from application.py
instead of fingerprints.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
