justinzeus/scholarr
1
0
Fork 0
Code Issues 1 Pull requests 18 Projects Releases Packages Wiki Activity Actions
84 commits 41 branches 1 tag 2.5 MiB
Commit graph

3 commits

Author SHA1 Message Date
Justin Visser
a5165dc3ba after audit 2026-03-01 18:14:22 +01:00
Justin Visser
72bc046215 refactor: extract helpers from remaining long functions 
- Extract _start_manual_run and _spawn_background_execution from run_manual route
- Extract _upsert_page_publications to deduplicate pagination loop
- Extract _classify_attempt from fetch_and_parse_with_retry
- Extract _log_alert_threshold_warnings from complete_run_for_user
- Functions 3f-3j accepted as kwargs-exception (body ≤50 lines excluding signatures)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-27 16:21:49 +01:00
Justin Visser
9a7fa5004e refactor: extract pagination engine and publication upsert from ingestion service 
Split app/services/ingestion/application.py (2,955 lines) into three
focused modules:

- page_fetch.py (219 lines): PageFetcher class for single-page fetch,
  parse-or-layout-error handling, and retry with backoff
- pagination.py (486 lines): PaginationEngine class for multi-page
  orchestration, loop state management, and short-circuit detection
- publication_upsert.py (239 lines): module-level functions for
  resolve_publication, upsert_profile_publications, and all
  find/create/update helpers

Also fixes two external import paths in portability/ that incorrectly
sourced normalize_title and build_publication_url from application.py
instead of fingerprints.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-27 13:47:23 +01:00