ci: add ruff linting and mypy type checking

Add ruff and mypy to dev dependencies with configuration in pyproject.toml.
Add a lint CI job that runs ruff check, ruff format --check, and mypy.
Auto-fix import sorting and formatting across the codebase. Exclude
alembic/versions from linting (auto-generated migrations). Ignore B008
(FastAPI Depends pattern) and RUF001 (unicode in user-facing strings).

21 ruff lint errors and 50 mypy errors remain for manual review.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

.github/workflows/ci.yml

@@ -20,10 +20,28 @@ jobs:
       - name: Enforce env contract parity
         run: python3 scripts/check_env_contract.py
 
+  lint:
+    runs-on: ubuntu-latest
+    needs: [repo-hygiene]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - uses: astral-sh/setup-uv@v4
+      - run: uv sync --extra dev
+      - name: Ruff check
+        run: uv run ruff check .
+      - name: Ruff format check
+        run: uv run ruff format --check .
+      - name: Mypy
+        run: uv run mypy app/ --ignore-missing-imports
+
   test:
     runs-on: ubuntu-latest
     needs:
       - repo-hygiene
+      - lint
     services:
       postgres:
         image: postgres:15

MODERNIZATION_PLAN.md

@@ -0,0 +1,628 @@
+# Scholarr Modernization Plan — Slices 2-10
+
+> **Slice 1 is complete** (commit `7736cab`): removed 114 stale `build/` files, fixed `.gitignore`, committed pending deletions, replaced `npm install` with `npm ci` in CI.
+>
+> Each slice below is a self-contained prompt designed for an LLM executor. Execute in order — each slice depends on the prior ones being complete.
+
+---
+
+## Slice 2: Create `structured_log()` Utility
+
+**Why:** The codebase has 19 `_log_*` helper functions (422 lines) and 138 instances of duplicated `"event"` keys in logging calls. A single utility eliminates all of it.
+
+**Files to create:**
+- `app/logging_utils.py`
+
+**Files to modify:**
+- `tests/unit/test_logging.py`
+
+**Context files (read but don't modify):**
+- `app/logging_config.py` — line 88 has `"event": getattr(record, "event", record.getMessage())` which means when no `event` key is in `extra`, it falls back to `getMessage()`. This is the mechanism that makes `structured_log()` work without duplicating the event name.
+- `app/logging_context.py`
+
+**Prompt:**
+```
+You are working on the scholarr repository. The current logging pattern has a DRY problem:
+
+1. Every log call duplicates the event name as both the message and in extra["event"]:
+   `logger.info("event.name", extra={"event": "event.name", "key": value})`
+
+2. There are 19 `_log_*` helper functions (422 lines total) that just wrap logger calls with typed signatures to build extra dicts.
+
+3. Metrics data (metric_name/metric_value) is mixed into log extra dicts but has no consumer.
+
+Create `app/logging_utils.py`:
+
+```python
+"""Structured logging utility — eliminates boilerplate across domain services."""
+
+from __future__ import annotations
+
+import logging
+from typing import Any
+
+
+def structured_log(
+    logger: logging.Logger,
+    level: str,
+    event: str,
+    /,
+    **fields: Any,
+) -> None:
+    """Emit a structured log entry.
+
+    The event name is passed as the log message. The JsonLogFormatter in
+    logging_config.py extracts it via record.getMessage() when no explicit
+    'event' key exists in extra — so we do NOT duplicate it.
+
+    Usage:
+        structured_log(logger, "info", "ingestion.run_started", user_id=1, scholar_count=5)
+    """
+    fields.pop("metric_name", None)
+    fields.pop("metric_value", None)
+
+    log_method = getattr(logger, level.lower())
+    log_method(event, extra=fields)
+```
+
+Verify that `app/logging_config.py` line 88 already handles this correctly — when no `event` key is in extra, it falls back to getMessage() which returns the event string passed as the first argument. No changes to logging_config.py should be needed.
+
+Add tests in `tests/unit/test_logging.py` (append to existing file):
+- Test that `structured_log()` produces a log record where the JsonLogFormatter outputs the event name correctly
+- Test that `structured_log()` works with ConsoleLogFormatter
+- Test that metric_name/metric_value fields are stripped from output
+- Test that extra fields (user_id, scholar_id, etc.) appear in the formatted output
+
+Commit message: "refactor: add structured_log utility to eliminate logging boilerplate"
+```
+
+---
+
+## Slice 3: Migrate Ingestion Logging to `structured_log()`
+
+**Why:** `app/services/domains/ingestion/application.py` is 3,089 lines. 8 `_log_*` helpers consume ~239 lines. 34 inline logger calls duplicate event names and include dead metric fields. This slice removes ~300 lines.
+
+**Files to modify:**
+- `app/services/domains/ingestion/application.py`
+
+**Context files:**
+- `app/logging_utils.py` (created in Slice 2)
+
+**Prompt:**
+```
+You are working on the scholarr repository. The file `app/services/domains/ingestion/application.py` (3,089 lines) has 8 `_log_*` helper functions consuming ~239 lines, plus 34 inline logger calls with duplicated `"event":` keys and mixed `metric_name`/`metric_value` fields.
+
+Migrate ALL logging in this file to use `structured_log()` from `app.logging_utils`.
+
+1. Add `from app.logging_utils import structured_log` at the top.
+
+2. Delete these 8 helper methods entirely:
+   - `_log_request_delay_coercion` (line ~123, 21 lines)
+   - `_log_run_started` (line ~310, 36 lines)
+   - `_log_scholar_parsed` (line ~383, 28 lines)
+   - `_log_alert_thresholds` (line ~1013, 47 lines)
+   - `_log_safety_transition` (line ~1093, 42 lines)
+   - `_log_run_completed` (line ~1178, 28 lines)
+   - `_attempt_log_entry` (line ~1837, 16 lines)
+   - `_page_log_entry` (line ~1999, 21 lines)
+
+3. Replace every call to these deleted helpers with an inline `structured_log()` call. Example:
+
+   Before:
+   ```python
+   self._log_run_started(
+       user_id=user_id,
+       trigger_type=trigger_type,
+       scholar_count=scholar_count,
+       ...
+   )
+   ```
+
+   After:
+   ```python
+   structured_log(
+       logger, "info", "ingestion.run_started",
+       user_id=user_id,
+       trigger_type=trigger_type.value,
+       scholar_count=scholar_count,
+       ...
+   )
+   ```
+
+4. For ALL remaining inline `logger.info/warning/debug/exception()` calls:
+   - Replace with `structured_log()` where the call uses `extra={"event": ..., ...}` pattern
+   - Do NOT convert `logger.exception()` calls — keep them but remove the duplicated `"event"` key from their extra dict
+   - Remove `metric_name` and `metric_value` from all calls
+
+5. Do NOT change any business logic. Only logging call sites.
+
+Expected outcome: ~300 lines removed. All existing tests must pass unchanged.
+
+Commit message: "refactor: migrate ingestion service logging to structured_log"
+```
+
+---
+
+## Slice 4: Migrate All Remaining Services to `structured_log()`
+
+**Why:** 11 more `_log_*` helpers remain across 8 files (183 lines total), plus inline logger calls with the same duplication pattern in ~20 files.
+
+**Files to modify (all have `_log_*` helpers to delete):**
+- `app/services/domains/ingestion/scheduler.py` — `_log_queue_item_resolved` (line ~518, 21 lines)
+- `app/services/domains/arxiv/rate_limit.py` — `_log_request_scheduled` (199), `_log_request_completed` (216), `_log_cooldown_activated` (235)
+- `app/services/domains/arxiv/client.py` — `_log_cache_event` (283), `_log_request_skipped_for_cooldown` (299)
+- `app/services/domains/publications/pdf_resolution_pipeline.py` — `_log_arxiv_skip` (148)
+- `app/services/domains/unpaywall/application.py` — `_log_resolution_summary` (194)
+- `app/api/routers/publications.py` — `_log_retry_pdf_result` (343)
+- `app/api/routers/settings.py` — `_log_settings_update` (103)
+- `app/main.py` — `_log_startup_build_marker` (53)
+
+**Files to modify (inline `extra={"event":...}` pattern only — no helpers to delete):**
+- `app/http/middleware.py`
+- `app/db/session.py`
+- `app/auth/runtime.py`
+- `app/security/csrf.py`
+- `app/api/routers/scholars.py`
+- `app/api/routers/runs.py`
+- `app/api/routers/admin_dbops.py`
+- `app/api/routers/admin.py`
+- `app/api/routers/auth.py`
+- `app/api/routers/publications.py`
+- `app/services/domains/scholars/application.py`
+- `app/services/domains/runs/events.py`
+- `app/services/domains/openalex/client.py`
+- `app/services/domains/openalex/matching.py`
+- `app/services/domains/crossref/application.py`
+- `app/services/domains/publications/dedup.py`
+- `app/services/domains/publications/enrichment.py`
+- `app/services/domains/publications/pdf_queue.py`
+- `app/services/domains/scholar/source.py`
+- `app/services/domains/arxiv/gateway.py`
+
+**Prompt:**
+```
+You are working on the scholarr repository. Slice 3 migrated `ingestion/application.py` to `structured_log()`. Now do the same for ALL remaining files.
+
+1. Delete these `_log_*` helper functions and replace their call sites with inline `structured_log()`:
+
+   - `app/services/domains/ingestion/scheduler.py:~518` — `_log_queue_item_resolved` (21 lines)
+   - `app/services/domains/arxiv/rate_limit.py:~199` — `_log_request_scheduled` (17 lines)
+   - `app/services/domains/arxiv/rate_limit.py:~216` — `_log_request_completed` (19 lines)
+   - `app/services/domains/arxiv/rate_limit.py:~235` — `_log_cooldown_activated` (13 lines)
+   - `app/services/domains/arxiv/client.py:~283` — `_log_cache_event` (16 lines)
+   - `app/services/domains/arxiv/client.py:~299` — `_log_request_skipped_for_cooldown` (13 lines)
+   - `app/services/domains/publications/pdf_resolution_pipeline.py:~148` — `_log_arxiv_skip` (11 lines)
+   - `app/services/domains/unpaywall/application.py:~194` — `_log_resolution_summary` (21 lines)
+   - `app/api/routers/publications.py:~343` — `_log_retry_pdf_result` (24 lines)
+   - `app/api/routers/settings.py:~103` — `_log_settings_update` (15 lines)
+   - `app/main.py:~53` — `_log_startup_build_marker` (13 lines)
+
+2. In ALL files under `app/` that have inline `logger.info/warning/debug("event.name", extra={"event": "event.name", ...})` calls:
+   - Replace with `structured_log(logger, "level", "event.name", key=value, ...)`
+   - Add `from app.logging_utils import structured_log` import
+   - Remove `metric_name`/`metric_value` from all calls
+   - Keep `logger.exception()` calls but remove the duplicated `"event"` key from their extra dicts
+
+3. In `app/services/domains/openalex/client.py`, the lines that log `response.text` (lines ~87, ~137):
+   - Truncate: `response.text[:500]`
+
+4. Do NOT change any business logic. Only logging call sites.
+
+Verify: `grep -rn '"event":' app/ --include="*.py" | grep -v __pycache__` should return 0 matches.
+Verify: `grep -rn 'def _log_' app/ --include="*.py"` should return 0 matches.
+
+Commit message: "refactor: migrate all remaining logging to structured_log, remove boilerplate helpers"
+```
+
+---
+
+## Slice 5: Logging Relevance Audit — Reduce Noise, Improve Readability
+
+**Why:** Self-hosted users need clear, actionable logs. Current logging has verbose debug noise (per-HTTP-request, per-publication), overly long event names, and no console display optimization.
+
+**Files to modify:**
+- `app/services/domains/scholar/source.py`
+- `app/services/domains/ingestion/application.py`
+- `app/logging_config.py`
+- Any files with event names longer than ~40 chars
+
+**Prompt:**
+```
+You are working on the scholarr repository. The logging has been migrated to `structured_log()` (slices 2-4). Now audit log RELEVANCE and READABILITY for self-hosted users running this in Docker.
+
+1. **Remove redundant debug logs:**
+   - `app/services/domains/scholar/source.py`: Remove the 3 debug-level HTTP fetch events (fetch_started, search_fetch_started, publication_fetch_started). Keep only `fetch_succeeded` at DEBUG. The HTTP middleware already logs request.started/completed.
+   - `app/services/domains/ingestion/application.py`: Remove per-publication `publication.discovered` and `publication.created` DEBUG logs. The run_completed summary already reports totals.
+
+2. **Shorten overly long event names** (search all structured_log calls in app/):
+   - `ingestion.request_delay_coerced_to_policy_floor` → `ingestion.delay_coerced`
+   - `ingestion.safety_cooldown_cleared` → `ingestion.cooldown_cleared`
+   - Any other names longer than ~40 chars — shorten while preserving meaning
+
+3. **Improve console readability** in `app/logging_config.py` `ConsoleLogFormatter.format()`:
+   - Add a short-name mapping for common context fields in console output ONLY (JSON keeps full names):
+     ```python
+     _CONSOLE_SHORT_KEYS = {
+         "user_id": "user",
+         "scholar_id": "scholar",
+         "crawl_run_id": "run",
+         "run_id": "run",
+     }
+     ```
+   - Apply in the `for key in sorted(payload.keys())` loop
+
+4. Do NOT remove any WARNING or ERROR level logs. Only remove/demote DEBUG/INFO that are redundant.
+5. Do NOT change business logic.
+
+Commit message: "refactor: reduce log noise, improve event naming and console readability"
+```
+
+---
+
+## Slice 6: Add Ruff + Mypy to CI
+
+**Why:** No Python linting or type checking in CI. Code quality regressions go undetected.
+
+**Files to modify:**
+- `pyproject.toml`
+- `.github/workflows/ci.yml`
+
+**Prompt:**
+```
+You are working on the scholarr repository. Add Python linting and type checking to CI.
+
+1. In `pyproject.toml`, add ruff configuration:
+   ```toml
+   [tool.ruff]
+   target-version = "py312"
+   line-length = 120
+
+   [tool.ruff.lint]
+   select = ["E", "F", "W", "I", "UP", "B", "SIM", "RUF"]
+   ignore = ["E501"]
+
+   [tool.ruff.lint.isort]
+   known-first-party = ["app"]
+   ```
+
+2. In `pyproject.toml`, add mypy configuration:
+   ```toml
+   [tool.mypy]
+   python_version = "3.12"
+   ignore_missing_imports = true
+   check_untyped_defs = false
+   warn_unused_ignores = true
+   ```
+
+3. In `.github/workflows/ci.yml`, add a `lint` job after `repo-hygiene` and before `test`:
+   ```yaml
+   lint:
+     runs-on: ubuntu-latest
+     needs: [repo-hygiene]
+     steps:
+       - uses: actions/checkout@v4
+       - uses: actions/setup-python@v5
+         with:
+           python-version: "3.12"
+       - uses: astral-sh/setup-uv@v4
+       - run: uv sync --extra dev
+       - name: Ruff check
+         run: uv run ruff check .
+       - name: Ruff format check
+         run: uv run ruff format --check .
+       - name: Mypy
+         run: uv run mypy app/ --ignore-missing-imports
+   ```
+
+4. Update the `test` job's `needs` to include `lint`.
+
+5. Add `ruff` and `mypy` to dev dependencies in `pyproject.toml` under `[project.optional-dependencies]`.
+
+6. Run `ruff check .` locally and fix auto-fixable issues with `ruff check --fix .`. For unfixable issues, add targeted `noqa` comments only if fixing would change behavior.
+
+Commit message: "ci: add ruff linting and mypy type checking"
+```
+
+---
+
+## Slice 7: Add CodeQL + Dependabot
+
+**Why:** No security scanning. Missed vulnerabilities, no automated dependency updates.
+
+**Files to create:**
+- `.github/workflows/codeql.yml`
+- `.github/dependabot.yml`
+
+**Prompt:**
+```
+You are working on the scholarr repository. Add security scanning.
+
+1. Create `.github/workflows/codeql.yml`:
+   ```yaml
+   name: CodeQL
+
+   on:
+     push:
+       branches: [main]
+     pull_request:
+       branches: [main]
+     schedule:
+       - cron: "30 5 * * 1"
+
+   jobs:
+     analyze:
+       runs-on: ubuntu-latest
+       permissions:
+         security-events: write
+         contents: read
+       strategy:
+         matrix:
+           language: [python, javascript]
+       steps:
+         - uses: actions/checkout@v4
+         - uses: github/codeql-action/init@v3
+           with:
+             languages: ${{ matrix.language }}
+         - uses: github/codeql-action/autobuild@v3
+         - uses: github/codeql-action/analyze@v3
+   ```
+
+2. Create `.github/dependabot.yml`:
+   ```yaml
+   version: 2
+   updates:
+     - package-ecosystem: pip
+       directory: /
+       schedule:
+         interval: weekly
+     - package-ecosystem: npm
+       directory: /frontend
+       schedule:
+         interval: weekly
+     - package-ecosystem: github-actions
+       directory: /
+       schedule:
+         interval: weekly
+   ```
+
+Commit message: "ci: add CodeQL security scanning and Dependabot"
+```
+
+---
+
+## Slice 8: Fix Version Remnants + Adopt Semantic Release
+
+**Why:** Version `0.1.0` is hardcoded in 5 places (including `crossref/application.py:294`). `0.0.1` exists in `docs/website/package.json`. No CHANGELOG, no GitHub Releases.
+
+**Files to modify:**
+- `pyproject.toml`
+- `app/services/domains/crossref/application.py`
+
+**Files to create:**
+- `.github/workflows/release.yml`
+
+**Prompt:**
+```
+You are working on the scholarr repository. Set up automated versioning with python-semantic-release.
+
+1. Fix hardcoded version in `app/services/domains/crossref/application.py` (line ~294):
+   ```python
+   # Before:
+   etiquette = Etiquette(settings.app_name, "0.1.0", "https://scholarr.local", email)
+
+   # After:
+   from importlib.metadata import version as pkg_version
+   _APP_VERSION = pkg_version("scholarr")
+   # then in function:
+   etiquette = Etiquette(settings.app_name, _APP_VERSION, "https://scholarr.local", email)
+   ```
+
+2. In `pyproject.toml`, add semantic-release config:
+   ```toml
+   [tool.semantic_release]
+   version_toml = ["pyproject.toml:project.version"]
+   version_variables = ["frontend/package.json:version"]
+   branch = "main"
+   build_command = ""
+   commit_message = "chore(release): v{version}"
+   ```
+
+3. Create `.github/workflows/release.yml`:
+   ```yaml
+   name: Release
+
+   on:
+     push:
+       branches: [main]
+
+   jobs:
+     release:
+       runs-on: ubuntu-latest
+       if: github.repository == 'JustinZeus/scholarr'
+       permissions:
+         contents: write
+         id-token: write
+       steps:
+         - uses: actions/checkout@v4
+           with:
+             fetch-depth: 0
+             token: ${{ secrets.GITHUB_TOKEN }}
+         - uses: actions/setup-python@v5
+           with:
+             python-version: "3.12"
+         - run: pip install python-semantic-release
+         - name: Semantic Release
+           id: release
+           run: semantic-release publish
+           env:
+             GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+   ```
+
+4. Add `python-semantic-release` to dev dependencies in `pyproject.toml`.
+
+Note: `docs/website/package.json` version (`0.0.1`) is addressed in Slice 9 (docs rebuild). `frontend/package-lock.json` and `uv.lock` versions auto-update on next `npm install`/`uv sync`.
+
+Commit message: "ci: adopt python-semantic-release, fix hardcoded version strings"
+```
+
+---
+
+## Slice 9: Docs — Delete and Rebuild from Scratch
+
+**Why:** Current docs have split authority (two api-contract.md files), orphaned pages, stale version (`0.0.1`), and missing critical docs (testing, deployment, configuration reference). Clean slate is faster than migration.
+
+**Files to delete:**
+- `docs/` (entire directory)
+
+**Files to create:**
+- Complete new `docs/` tree
+
+**Prompt:**
+```
+You are working on the scholarr repository. Delete the entire `docs/` directory and rebuild from scratch.
+
+Target IA:
+
+docs/
+├── index.md                          # Landing page: what is scholarr, quick links
+├── user/
+│   ├── overview.md                   # What scholarr does, key concepts
+│   ├── getting-started.md            # Install (docker compose), first run, add first scholar
+│   └── configuration.md              # ALL env vars from .env.example, organized by category
+├── developer/
+│   ├── overview.md                   # Dev quickstart
+│   ├── architecture.md               # FastAPI + SQLAlchemy + Vue 3, domain service boundaries
+│   ├── local-development.md          # docker-compose.dev.yml, hot reload, running tests
+│   ├── contributing.md               # PR process, conventional commits, code standards from agents.md
+│   ├── ingestion.md                  # Ingestion pipeline: parsing, rate limiting, safety gates
+│   ├── frontend-theme-inventory.md   # Theme tokens, Tailwind integration
+│   └── testing.md                    # Test tiers (unit/integration/smoke), markers, fixtures, how to run
+├── operations/
+│   ├── overview.md                   # Ops quickstart
+│   ├── deployment.md                 # Production Docker, scaling, health checks
+│   ├── database-runbook.md           # Backup, restore, integrity checks, migration procedures
+│   ├── scrape-safety-runbook.md      # Rate limiting, cooldowns, CAPTCHA handling
+│   └── arxiv-runbook.md              # ArXiv rate limits, cache, query patterns
+├── reference/
+│   ├── overview.md                   # Reference index
+│   ├── api.md                        # CANONICAL: envelope spec + endpoints + DTO contract
+│   ├── environment.md                # Env var quick-reference (links to user/configuration.md)
+│   └── changelog.md                  # Placeholder: "auto-generated by semantic-release"
+└── website/
+    ├── docusaurus.config.js
+    ├── sidebars.js                   # ALL pages listed, no orphans
+    ├── package.json                  # version: "0.1.0"
+    └── src/css/custom.css
+
+Content guidelines:
+- Self-contained, scannable docs (headers, bullets, tables)
+- Frontmatter: `title`, `sidebar_position`
+- `configuration.md`: read `.env.example`, organize every variable with description, type, default, example
+- `testing.md`: pytest markers from pyproject.toml (`integration`, `db`, `migrations`, `schema`, `smoke`), container-based runner from agents.md, fixture org under tests/fixtures/
+- `api.md`: merge old `developer/api-contract.md` (DTO structure) and `reference/api-contract.md` (envelope spec). Include exact envelope shapes:
+  - Success: `{"data": ..., "meta": {"request_id": "..."}}`
+  - Error: `{"error": {"code": "...", "message": "...", "details": ...}, "meta": {"request_id": "..."}}`
+  - Publication semantics (modes, pagination, identifiers)
+  - Scholar portability endpoints
+- `deployment.md`: practical Docker commands from docker-compose.yml
+- `database-runbook.md`: reference scripts/db/ (backup_full.sh, restore_dump.sh, check_integrity.py, repair_publication_links.py)
+
+Docusaurus config:
+- url: "https://justinzeus.github.io"
+- baseUrl: "/scholarr/"
+- organizationName: "JustinZeus"
+- projectName: "scholarr"
+- onBrokenLinks: "throw"
+- docs path: ".." (reads from docs/ parent)
+- Exclude: "website/**", "README.md"
+
+Commit message: "docs: rebuild documentation from scratch with clean IA"
+```
+
+---
+
+## Slice 10: Begin `ingestion/application.py` Decomposition
+
+**Why:** After logging cleanup (slices 3+5), the file is ~2,700-2,800 lines. Still the largest source file by 3x. This begins decomposition with the safest extract: safety gate logic.
+
+**Files to modify:**
+- `app/services/domains/ingestion/application.py`
+
+**Files to create:**
+- `app/services/domains/ingestion/safety.py`
+
+**Prompt:**
+```
+You are working on the scholarr repository. `app/services/domains/ingestion/application.py` is ~2,700-2,800 lines (after logging cleanup). It contains the entire ingestion orchestration in a single class.
+
+Extract the FIRST logical chunk: safety gate logic.
+
+1. Read `app/services/domains/ingestion/application.py` fully.
+
+2. Identify safety/cooldown methods:
+   - `_enforce_safety_gate`
+   - `_raise_safety_blocked_start`
+   - Cooldown activation/clearing methods
+   - Alert threshold evaluation methods
+
+3. Create `app/services/domains/ingestion/safety.py`:
+   - Move identified methods to standalone functions or a small class
+   - Accept explicit parameters (no implicit self.xxx state)
+   - Keep same function signatures where possible
+
+4. Update `application.py` to import from `safety.py`.
+
+5. Run full test suite: `docker compose -f docker-compose.yml -f docker-compose.dev.yml run --rm app pytest tests/`
+   All tests must pass unchanged.
+
+Constraints from `agents.md`:
+- Max 50 lines per function
+- No flat files in `app/services/` root — all within `app/services/domains/ingestion/`
+- Fail fast, early returns, guard clauses
+
+Commit message: "refactor: extract ingestion safety gate logic to dedicated module"
+```
+
+---
+
+## Verification (after all slices)
+
+Run these checks to confirm everything is clean:
+
+```bash
+# 1. No tracked build artifacts
+git ls-files build/ | wc -l  # expect: 0
+
+# 2. No duplicated event keys in logging
+grep -rn '"event":' app/ --include="*.py" | grep -v __pycache__ | wc -l  # expect: 0
+
+# 3. No _log_ helper functions remain
+grep -rn 'def _log_' app/ --include="*.py" | wc -l  # expect: 0
+
+# 4. No stale version strings
+grep -rn '0\.0\.1' . --include="*.py" --include="*.json" --include="*.toml" \
+  | grep -v node_modules | grep -v .venv | grep -v uv.lock | wc -l  # expect: 0
+
+# 5. Docs build
+npm --prefix docs/website run build  # expect: success, no broken links
+
+# 6. All tests pass
+docker compose -f docker-compose.yml -f docker-compose.dev.yml run --rm app pytest tests/
+```
+
+---
+
+## Slice Summary
+
+| # | Slice | Files Changed | Estimated Impact |
+|---|-------|---------------|------------------|
+| ~~1~~ | ~~Repo hygiene~~ | ~~120 files~~ | ~~-18,380 lines~~ **DONE** |
+| 2 | `structured_log()` utility | 2 files | +50 lines |
+| 3 | Migrate ingestion logging | 1 file | -300 lines |
+| 4 | Migrate remaining logging | ~20 files | -180 lines |
+| 5 | Logging noise/readability | ~5 files | -30 lines, better output |
+| 6 | Ruff + mypy in CI | 2 files | +50 lines config |
+| 7 | CodeQL + Dependabot | 2 files | +40 lines config |
+| 8 | Version fix + semantic-release | 3 files | +40 lines config |
+| 9 | Docs rebuild | full replacement | ~20 new files |
+| 10 | Ingestion decomposition | 2 files | net zero (move) |

alembic/env.py

@@ -1,13 +1,13 @@
-from logging.config import fileConfig
 import os
+from logging.config import fileConfig
 
-from alembic import context
 from sqlalchemy import pool
 from sqlalchemy.engine import Connection
 from sqlalchemy.ext.asyncio import async_engine_from_config
 
-from app.db.base import metadata
+from alembic import context
 from app.db import models  # noqa: F401
+from app.db.base import metadata
 
 config = context.config
 

alembic/versions/20260216_0001_multi_user_schema.py

@@ -8,10 +8,11 @@ Create Date: 2026-02-16 17:10:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects import postgresql
 
+from alembic import op
+
 # revision identifiers, used by Alembic.
 revision: str = "20260216_0001"
 down_revision: str | Sequence[str] | None = None

alembic/versions/20260216_0002_add_user_admin_flag.py

@@ -7,9 +7,10 @@ Create Date: 2026-02-16 17:40:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
+
 # revision identifiers, used by Alembic.
 revision: str = "20260216_0002"
 down_revision: str | Sequence[str] | None = "20260216_0001"
@@ -31,4 +32,3 @@ def upgrade() -> None:
 
 def downgrade() -> None:
     op.drop_column("users", "is_admin")
-

alembic/versions/20260216_0003_add_ingestion_queue.py

@@ -7,9 +7,10 @@ Create Date: 2026-02-16 23:45:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
+
 # revision identifiers, used by Alembic.
 revision: str = "20260216_0003"
 down_revision: str | Sequence[str] | None = "20260216_0002"

alembic/versions/20260217_0004_queue_status_fields.py

@@ -7,9 +7,10 @@ Create Date: 2026-02-17 10:15:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
+
 # revision identifiers, used by Alembic.
 revision: str = "20260217_0004"
 down_revision: str | Sequence[str] | None = "20260216_0003"
@@ -22,9 +23,7 @@ def upgrade() -> None:
     inspector = sa.inspect(bind)
     columns = {column["name"] for column in inspector.get_columns("ingestion_queue_items")}
     indexes = {index["name"] for index in inspector.get_indexes("ingestion_queue_items")}
-    checks = {
-        constraint["name"] for constraint in inspector.get_check_constraints("ingestion_queue_items")
-    }
+    checks = {constraint["name"] for constraint in inspector.get_check_constraints("ingestion_queue_items")}
 
     if "status" not in columns:
         op.add_column(
@@ -80,9 +79,7 @@ def downgrade() -> None:
     inspector = sa.inspect(bind)
     columns = {column["name"] for column in inspector.get_columns("ingestion_queue_items")}
     indexes = {index["name"] for index in inspector.get_indexes("ingestion_queue_items")}
-    checks = {
-        constraint["name"] for constraint in inspector.get_check_constraints("ingestion_queue_items")
-    }
+    checks = {constraint["name"] for constraint in inspector.get_check_constraints("ingestion_queue_items")}
 
     if "ix_ingestion_queue_status_next_attempt" in indexes:
         op.drop_index("ix_ingestion_queue_status_next_attempt", table_name="ingestion_queue_items")

alembic/versions/20260217_0005_manual_run_idempotency.py

@@ -7,9 +7,10 @@ Create Date: 2026-02-17 16:20:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
+
 # revision identifiers, used by Alembic.
 revision: str = "20260217_0005"
 down_revision: str | Sequence[str] | None = "20260217_0004"
@@ -70,9 +71,7 @@ def upgrade() -> None:
             "crawl_runs",
             ["user_id", "idempotency_key"],
             unique=True,
-            postgresql_where=sa.text(
-                "idempotency_key IS NOT NULL AND trigger_type = 'manual'::run_trigger_type"
-            ),
+            postgresql_where=sa.text("idempotency_key IS NOT NULL AND trigger_type = 'manual'::run_trigger_type"),
         )
 
 

alembic/versions/20260217_0006_scholar_profile_images.py

@@ -7,9 +7,10 @@ Create Date: 2026-02-17 18:30:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
+
 # revision identifiers, used by Alembic.
 revision: str = "20260217_0006"
 down_revision: str | Sequence[str] | None = "20260217_0005"

alembic/versions/20260217_0007_scholar_initial_page_fingerprint.py

@@ -7,9 +7,10 @@ Create Date: 2026-02-17 18:55:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
+
 # revision identifiers, used by Alembic.
 revision: str = "20260217_0007"
 down_revision: str | Sequence[str] | None = "20260217_0006"

alembic/versions/20260219_0008_user_settings_nav_visible_pages.py

@@ -7,10 +7,10 @@ Create Date: 2026-02-19 14:58:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects import postgresql
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260219_0008"
@@ -21,7 +21,7 @@ depends_on: str | Sequence[str] | None = None
 
 TABLE_NAME = "user_settings"
 DEFAULT_NAV_VISIBLE_PAGES_SQL = (
-    "'[\"dashboard\",\"scholars\",\"publications\",\"settings\",\"style-guide\",\"runs\",\"users\"]'::jsonb"
+    '\'["dashboard","scholars","publications","settings","style-guide","runs","users"]\'::jsonb'
 )
 
 

alembic/versions/20260219_0009_user_settings_scrape_safety_state.py

@@ -7,10 +7,10 @@ Create Date: 2026-02-19 21:20:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects import postgresql
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260219_0009"

alembic/versions/20260219_0010_author_search_shared_state.py

@@ -7,10 +7,10 @@ Create Date: 2026-02-19 22:20:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects import postgresql
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260219_0010"

alembic/versions/20260220_0011_add_publication_doi.py

@@ -7,9 +7,9 @@ Create Date: 2026-02-20 13:35:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 revision: str = "20260220_0011"
 down_revision: str | Sequence[str] | None = "20260219_0010"

alembic/versions/20260220_0012_add_data_repair_jobs.py

@@ -7,10 +7,10 @@ Create Date: 2026-02-20 23:35:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects import postgresql
 
+from alembic import op
 
 revision: str = "20260220_0012"
 down_revision: str | Sequence[str] | None = "20260220_0011"

alembic/versions/20260221_0013_add_publication_pdf_jobs.py

@@ -7,9 +7,9 @@ Create Date: 2026-02-21 12:25:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 revision: str = "20260221_0013"
 down_revision: str | Sequence[str] | None = "20260220_0012"

alembic/versions/20260221_0014_add_scholar_publication_favorite_state.py

@@ -7,9 +7,9 @@ Create Date: 2026-02-21 14:35:00
 
 from __future__ import annotations
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision = "20260221_0014"

alembic/versions/20260221_0015_enforce_request_delay_floor.py

@@ -9,9 +9,9 @@ from __future__ import annotations
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260221_0015"
@@ -30,11 +30,7 @@ LEGACY_CHECK_NAME_MIN_2 = "ck_user_settings_request_delay_seconds_min_2"
 def _check_constraints() -> set[str]:
     bind = op.get_bind()
     inspector = sa.inspect(bind)
-    return {
-        str(item.get("name"))
-        for item in inspector.get_check_constraints(TABLE_NAME)
-        if item.get("name")
-    }
+    return {str(item.get("name")) for item in inspector.get_check_constraints(TABLE_NAME) if item.get("name")}
 
 
 def _drop_check_if_exists(name: str) -> None:
@@ -48,12 +44,7 @@ def _drop_check_if_exists(name: str) -> None:
 
 
 def upgrade() -> None:
-    op.execute(
-        sa.text(
-            "UPDATE user_settings SET request_delay_seconds = 2 "
-            "WHERE request_delay_seconds < 2"
-        )
-    )
+    op.execute(sa.text("UPDATE user_settings SET request_delay_seconds = 2 WHERE request_delay_seconds < 2"))
 
     _drop_check_if_exists(CHECK_NAME_MIN_1)
     _drop_check_if_exists(LEGACY_CHECK_NAME_MIN_1)

alembic/versions/20260222_0016_add_publication_identifiers.py

@@ -7,9 +7,9 @@ Create Date: 2026-02-22 12:45:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 revision: str = "20260222_0016"
 down_revision: str | Sequence[str] | None = "20260221_0015"

alembic/versions/20260224_0018_add_openalex_enriched_to_publications.py

@@ -5,26 +5,29 @@ Revises: fa0c5e3262d5
 Create Date: 2026-02-24 19:10:00.000000
 
 """
+
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
-revision: str = '20260224_0018'
-down_revision: str | Sequence[str] | None = 'fa0c5e3262d5'
+revision: str = "20260224_0018"
+down_revision: str | Sequence[str] | None = "fa0c5e3262d5"
 branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 
 
 def upgrade() -> None:
     # ### commands auto generated by Alembic - please adjust! ###
-    op.add_column('publications', sa.Column('openalex_enriched', sa.Boolean(), server_default=sa.text('false'), nullable=False))
+    op.add_column(
+        "publications", sa.Column("openalex_enriched", sa.Boolean(), server_default=sa.text("false"), nullable=False)
+    )
     # ### end Alembic commands ###
 
 
 def downgrade() -> None:
     # ### commands auto generated by Alembic - please adjust! ###
-    op.drop_column('publications', 'openalex_enriched')
+    op.drop_column("publications", "openalex_enriched")
     # ### end Alembic commands ###

alembic/versions/20260224_0019_add_resolving_to_runstatus.py

@@ -5,14 +5,14 @@ Revises: 20260224_0018
 Create Date: 2026-02-24 22:20:00.000000
 
 """
+
 from collections.abc import Sequence
 
 from alembic import op
 
-
 # revision identifiers, used by Alembic.
-revision: str = '20260224_0019'
-down_revision: str | Sequence[str] | None = '20260224_0018'
+revision: str = "20260224_0019"
+down_revision: str | Sequence[str] | None = "20260224_0018"
 branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 

alembic/versions/20260224_0020_add_openalex_last_attempt_at.py

@@ -5,22 +5,23 @@ Revises: 20260224_0019
 Create Date: 2026-02-24 22:50:00.000000
 
 """
+
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
-revision: str = '20260224_0020'
-down_revision: str | Sequence[str] | None = '20260224_0019'
+revision: str = "20260224_0020"
+down_revision: str | Sequence[str] | None = "20260224_0019"
 branch_labels: str | Sequence[str] | None = None
 depends_on: str | Sequence[str] | None = None
 
 
 def upgrade() -> None:
-    op.add_column('publications', sa.Column('openalex_last_attempt_at', sa.DateTime(timezone=True), nullable=True))
+    op.add_column("publications", sa.Column("openalex_last_attempt_at", sa.DateTime(timezone=True), nullable=True))
 
 
 def downgrade() -> None:
-    op.drop_column('publications', 'openalex_last_attempt_at')
+    op.drop_column("publications", "openalex_last_attempt_at")

alembic/versions/20260225_0021_enforce_single_active_run_per_user.py

@@ -7,9 +7,9 @@ Create Date: 2026-02-25 09:10:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260225_0021"
@@ -56,9 +56,7 @@ def upgrade() -> None:
             "crawl_runs",
             ["user_id"],
             unique=True,
-            postgresql_where=sa.text(
-                "status IN ('running'::run_status, 'resolving'::run_status)"
-            ),
+            postgresql_where=sa.text("status IN ('running'::run_status, 'resolving'::run_status)"),
         )
 
 

alembic/versions/20260225_0022_add_canonical_title_hash_to_publications.py

@@ -7,9 +7,9 @@ Create Date: 2026-02-25 10:00:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260225_0022"

alembic/versions/20260226_0023_add_arxiv_runtime_state.py

@@ -7,9 +7,9 @@ Create Date: 2026-02-26 12:40:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260226_0023"

alembic/versions/20260226_0024_add_arxiv_query_cache_entries.py

@@ -7,10 +7,10 @@ Create Date: 2026-02-26 14:20:00.000000
 
 from collections.abc import Sequence
 
-from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects import postgresql
 
+from alembic import op
 
 # revision identifiers, used by Alembic.
 revision: str = "20260226_0024"

app/api/__init__.py

@@ -1,2 +1 @@
 from __future__ import annotations
-

app/api/errors.py

@@ -82,4 +82,3 @@ def register_api_exception_handlers(app: FastAPI) -> None:
             message="Request validation failed.",
             details=exc.errors(),
         )
-

app/api/routers/__init__.py

@@ -1,2 +1 @@
 from __future__ import annotations
-

app/api/routers/admin.py

@@ -86,7 +86,14 @@ async def create_user(
             message=str(exc),
         ) from exc
 
-    structured_log(logger, "info", "api.admin.user_created", admin_user_id=int(admin_user.id), target_user_id=int(created_user.id), target_is_admin=bool(created_user.is_admin))
+    structured_log(
+        logger,
+        "info",
+        "api.admin.user_created",
+        admin_user_id=int(admin_user.id),
+        target_user_id=int(created_user.id),
+        target_is_admin=bool(created_user.is_admin),
+    )
     return success_payload(
         request,
         data=_serialize_user(created_user),
@@ -111,11 +118,7 @@ async def set_user_active(
             code="user_not_found",
             message="User not found.",
         )
-    if (
-        int(target_user.id) == int(admin_user.id)
-        and bool(target_user.is_active)
-        and not bool(payload.is_active)
-    ):
+    if int(target_user.id) == int(admin_user.id) and bool(target_user.is_active) and not bool(payload.is_active):
         raise ApiException(
             status_code=400,
             code="cannot_deactivate_self",
@@ -126,7 +129,14 @@ async def set_user_active(
         user=target_user,
         is_active=bool(payload.is_active),
     )
-    structured_log(logger, "info", "api.admin.user_active_updated", admin_user_id=int(admin_user.id), target_user_id=int(updated_user.id), is_active=bool(updated_user.is_active))
+    structured_log(
+        logger,
+        "info",
+        "api.admin.user_active_updated",
+        admin_user_id=int(admin_user.id),
+        target_user_id=int(updated_user.id),
+        is_active=bool(updated_user.is_active),
+    )
     return success_payload(
         request,
         data=_serialize_user(updated_user),
@@ -166,7 +176,13 @@ async def reset_user_password(
         user=target_user,
         password_hash=auth_service.hash_password(validated_password),
     )
-    structured_log(logger, "info", "api.admin.user_password_reset", admin_user_id=int(admin_user.id), target_user_id=int(target_user.id))
+    structured_log(
+        logger,
+        "info",
+        "api.admin.user_password_reset",
+        admin_user_id=int(admin_user.id),
+        target_user_id=int(target_user.id),
+    )
     return success_payload(
         request,
         data={"message": f"Password reset: {target_user.email}"},

app/api/routers/admin_dbops.py

@@ -10,14 +10,14 @@ from app.api.errors import ApiException
 from app.api.responses import success_payload
 from app.api.schemas import (
     AdminDbIntegrityEnvelope,
-    AdminPdfQueueBulkEnqueueEnvelope,
-    AdminPdfQueueRequeueEnvelope,
-    AdminPdfQueueEnvelope,
     AdminDbRepairJobsEnvelope,
-    AdminRepairPublicationNearDuplicatesEnvelope,
-    AdminRepairPublicationNearDuplicatesRequest,
+    AdminPdfQueueBulkEnqueueEnvelope,
+    AdminPdfQueueEnvelope,
+    AdminPdfQueueRequeueEnvelope,
     AdminRepairPublicationLinksEnvelope,
     AdminRepairPublicationLinksRequest,
+    AdminRepairPublicationNearDuplicatesEnvelope,
+    AdminRepairPublicationNearDuplicatesRequest,
 )
 from app.db.models import DataRepairJob, User
 from app.db.session import get_db_session
@@ -25,8 +25,8 @@ from app.logging_utils import structured_log
 from app.services.domains.dbops import (
     collect_integrity_report,
     list_repair_jobs,
-    run_publication_near_duplicate_repair,
     run_publication_link_repair,
+    run_publication_near_duplicate_repair,
 )
 from app.services.domains.publications import application as publication_service
 
@@ -136,7 +136,15 @@ async def get_integrity_report(
     admin_user: User = Depends(get_api_admin_user),
 ):
     report = await collect_integrity_report(db_session)
-    structured_log(logger, "info", "api.admin.db.integrity_checked", admin_user_id=int(admin_user.id), status=report.get("status"), failure_count=len(report.get("failures", [])), warning_count=len(report.get("warnings", [])))
+    structured_log(
+        logger,
+        "info",
+        "api.admin.db.integrity_checked",
+        admin_user_id=int(admin_user.id),
+        status=report.get("status"),
+        failure_count=len(report.get("failures", [])),
+        warning_count=len(report.get("warnings", [])),
+    )
     return success_payload(request, data=report)
 
 
@@ -151,7 +159,14 @@ async def get_repair_jobs(
     admin_user: User = Depends(get_api_admin_user),
 ):
     jobs = await list_repair_jobs(db_session, limit=limit)
-    structured_log(logger, "info", "api.admin.db.repair_jobs_listed", admin_user_id=int(admin_user.id), limit=int(limit), job_count=len(jobs))
+    structured_log(
+        logger,
+        "info",
+        "api.admin.db.repair_jobs_listed",
+        admin_user_id=int(admin_user.id),
+        limit=int(limit),
+        job_count=len(jobs),
+    )
     return success_payload(
         request,
         data={"jobs": [_serialize_repair_job(job) for job in jobs]},
@@ -186,7 +201,9 @@ async def get_pdf_queue(
         status=normalized_status,
     )
     structured_log(
-        logger, "info", "api.admin.db.pdf_queue_listed",
+        logger,
+        "info",
+        "api.admin.db.pdf_queue_listed",
         admin_user_id=int(admin_user.id),
         page=int(resolved_page),
         page_size=int(resolved_limit),
@@ -232,7 +249,14 @@ async def requeue_pdf_lookup(
             message="Publication not found.",
         )
     status, message = _requeue_response_state(queued=result.queued)
-    structured_log(logger, "info", "api.admin.db.pdf_requeued", admin_user_id=int(admin_user.id), publication_id=int(publication_id), queued=bool(result.queued))
+    structured_log(
+        logger,
+        "info",
+        "api.admin.db.pdf_requeued",
+        admin_user_id=int(admin_user.id),
+        publication_id=int(publication_id),
+        queued=bool(result.queued),
+    )
     return success_payload(
         request,
         data={
@@ -260,7 +284,15 @@ async def requeue_all_missing_pdfs(
         request_email=admin_user.email,
         limit=limit,
     )
-    structured_log(logger, "info", "api.admin.db.pdf_queue_requeue_all", admin_user_id=int(admin_user.id), limit=int(limit), requested_count=int(result.requested_count), queued_count=int(result.queued_count))
+    structured_log(
+        logger,
+        "info",
+        "api.admin.db.pdf_queue_requeue_all",
+        admin_user_id=int(admin_user.id),
+        limit=int(limit),
+        requested_count=int(result.requested_count),
+        queued_count=int(result.queued_count),
+    )
     return success_payload(
         request,
         data={
@@ -301,7 +333,9 @@ async def trigger_publication_link_repair(
             message=str(exc),
         ) from exc
     structured_log(
-        logger, "info", "api.admin.db.publication_link_repair_triggered",
+        logger,
+        "info",
+        "api.admin.db.publication_link_repair_triggered",
         admin_user_id=int(admin_user.id),
         scope_mode=payload.scope_mode,
         target_user_id=int(payload.user_id) if payload.user_id is not None else None,
@@ -340,7 +374,16 @@ async def trigger_publication_near_duplicate_repair(
             code="invalid_near_duplicate_repair_request",
             message=str(exc),
         ) from exc
-    structured_log(logger, "info", "api.admin.db.dedup_repair_triggered", admin_user_id=int(admin_user.id), dry_run=bool(payload.dry_run), selected_cluster_count=len(payload.selected_cluster_keys), job_id=int(result["job_id"]), status=result["status"])
+    structured_log(
+        logger,
+        "info",
+        "api.admin.db.dedup_repair_triggered",
+        admin_user_id=int(admin_user.id),
+        dry_run=bool(payload.dry_run),
+        selected_cluster_count=len(payload.selected_cluster_keys),
+        job_id=int(result["job_id"]),
+        status=result["status"],
+    )
     return success_payload(request, data=result)
 
 
@@ -381,12 +424,17 @@ async def drop_all_publications(
     await db_session.execute(delete(PublicationPdfJobEvent))
     await db_session.execute(delete(PublicationPdfJob))
     await db_session.execute(delete(Publication))
-    await db_session.execute(
-        update(ScholarProfile).values(baseline_completed=False)
-    )
+    await db_session.execute(update(ScholarProfile).values(baseline_completed=False))
     await db_session.commit()
 
-    structured_log(logger, "warning", "api.admin.db.all_publications_dropped", admin_user_id=int(admin_user.id), admin_email=admin_user.email, deleted_count=int(total_publications))
+    structured_log(
+        logger,
+        "warning",
+        "api.admin.db.all_publications_dropped",
+        admin_user_id=int(admin_user.id),
+        admin_email=admin_user.email,
+        deleted_count=int(total_publications),
+    )
     return success_payload(
         request,
         data={

app/api/routers/auth.py

@@ -5,8 +5,8 @@ import logging
 from fastapi import APIRouter, Depends, Request
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.api.errors import ApiException
 from app.api.deps import get_api_current_user
+from app.api.errors import ApiException
 from app.api.responses import success_payload
 from app.api.schemas import (
     AuthMeEnvelope,
@@ -16,14 +16,14 @@ from app.api.schemas import (
     LoginRequest,
     MessageEnvelope,
 )
-from app.auth.deps import get_auth_service, get_login_rate_limiter
-from app.logging_utils import structured_log
-from app.auth.rate_limit import SlidingWindowRateLimiter
 from app.auth import runtime as auth_runtime
+from app.auth.deps import get_auth_service, get_login_rate_limiter
+from app.auth.rate_limit import SlidingWindowRateLimiter
 from app.auth.service import AuthService
 from app.auth.session import set_session_user
 from app.db.models import User
 from app.db.session import get_db_session
+from app.logging_utils import structured_log
 from app.security.csrf import ensure_csrf_token
 from app.services.domains.users import application as user_service
 
@@ -37,7 +37,13 @@ def _login_limiter_key_and_email(request: Request, payload: LoginRequest) -> tup
 
 
 def _raise_rate_limited(normalized_email: str, retry_after_seconds: int) -> None:
-    structured_log(logger, "warning", "api.auth.login_rate_limited", email=normalized_email, retry_after_seconds=retry_after_seconds)
+    structured_log(
+        logger,
+        "warning",
+        "api.auth.login_rate_limited",
+        email=normalized_email,
+        retry_after_seconds=retry_after_seconds,
+    )
     raise ApiException(
         status_code=429,
         code="rate_limited",
@@ -200,7 +206,9 @@ async def logout(
 ):
     current_user = await auth_runtime.get_authenticated_user(request, db_session)
     auth_runtime.invalidate_session(request)
-    structured_log(logger, "info", "api.auth.logout", user_id=int(current_user.id) if current_user is not None else None)
+    structured_log(
+        logger, "info", "api.auth.logout", user_id=int(current_user.id) if current_user is not None else None
+    )
     return success_payload(
         request,
         data={

app/api/routers/publications.py

@@ -1,7 +1,7 @@
 from __future__ import annotations
 
-from datetime import datetime, timezone
 import logging
+from datetime import UTC, datetime
 from typing import Literal
 
 from fastapi import APIRouter, Depends, Path, Query, Request
@@ -183,7 +183,7 @@ def _resolve_publications_snapshot(
     snapshot: str | None,
 ) -> tuple[datetime, str]:
     if snapshot is None:
-        now_utc = datetime.now(timezone.utc)
+        now_utc = datetime.now(UTC)
         return now_utc, now_utc.isoformat()
     try:
         parsed = datetime.fromisoformat(snapshot)
@@ -194,8 +194,8 @@ def _resolve_publications_snapshot(
             message="Invalid publications snapshot cursor.",
         ) from exc
     if parsed.tzinfo is None:
-        parsed = parsed.replace(tzinfo=timezone.utc)
-    normalized = parsed.astimezone(timezone.utc)
+        parsed = parsed.replace(tzinfo=UTC)
+    normalized = parsed.astimezone(UTC)
     return normalized, normalized.isoformat()
 
 
@@ -420,7 +420,9 @@ async def mark_all_publications_read(
         db_session,
         user_id=current_user.id,
     )
-    structured_log(logger, "info", "api.publications.mark_all_read", user_id=current_user.id, updated_count=updated_count)
+    structured_log(
+        logger, "info", "api.publications.mark_all_read", user_id=current_user.id, updated_count=updated_count
+    )
     return success_payload(
         request,
         data={
@@ -440,18 +442,20 @@ async def mark_selected_publications_read(
     db_session: AsyncSession = Depends(get_db_session),
     current_user: User = Depends(get_api_current_user),
 ):
-    selection_pairs = sorted(
-        {
-            (int(item.scholar_profile_id), int(item.publication_id))
-            for item in payload.selections
-        }
-    )
+    selection_pairs = sorted({(int(item.scholar_profile_id), int(item.publication_id)) for item in payload.selections})
     updated_count = await publication_service.mark_selected_as_read_for_user(
         db_session,
         user_id=current_user.id,
         selections=selection_pairs,
     )
-    structured_log(logger, "info", "api.publications.mark_selected_read", user_id=current_user.id, requested_count=len(selection_pairs), updated_count=updated_count)
+    structured_log(
+        logger,
+        "info",
+        "api.publications.mark_selected_read",
+        user_id=current_user.id,
+        requested_count=len(selection_pairs),
+        updated_count=updated_count,
+    )
     return success_payload(
         request,
         data={
@@ -486,7 +490,9 @@ async def retry_publication_pdf(
         pdf_status=current.pdf_status,
     )
     structured_log(
-        logger, "info", "api.publications.retry_pdf",
+        logger,
+        "info",
+        "api.publications.retry_pdf",
         user_id=current_user.id,
         scholar_profile_id=payload.scholar_profile_id,
         publication_id=publication_id,
@@ -523,7 +529,15 @@ async def toggle_publication_favorite(
         publication_id=publication_id,
         is_favorite=payload.is_favorite,
     )
-    structured_log(logger, "info", "api.publications.favorite", user_id=current_user.id, scholar_profile_id=payload.scholar_profile_id, publication_id=publication_id, is_favorite=bool(payload.is_favorite))
+    structured_log(
+        logger,
+        "info",
+        "api.publications.favorite",
+        user_id=current_user.id,
+        scholar_profile_id=payload.scholar_profile_id,
+        publication_id=publication_id,
+        is_favorite=bool(payload.is_favorite),
+    )
     return success_payload(
         request,
         data={

app/api/routers/runs.py

@@ -5,13 +5,14 @@ import re
 from typing import Any
 
 from fastapi import APIRouter, Depends, Query, Request
+from fastapi.responses import StreamingResponse
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.api.deps import get_api_current_user
 from app.api.errors import ApiException
 from app.api.responses import success_payload
-from fastapi.responses import StreamingResponse
+from app.api.runtime_deps import get_ingestion_service
 from app.api.schemas import (
     ManualRunEnvelope,
     QueueClearEnvelope,
@@ -29,7 +30,6 @@ from app.services.domains.runs import application as run_service
 from app.services.domains.runs.events import event_generator
 from app.services.domains.settings import application as user_settings_service
 from app.settings import settings
-from app.api.runtime_deps import get_ingestion_service
 
 logger = logging.getLogger(__name__)
 
@@ -77,10 +77,7 @@ def _normalize_idempotency_key(raw_value: str | None) -> str | None:
         raise ApiException(
             status_code=400,
             code="invalid_idempotency_key",
-            message=(
-                "Invalid Idempotency-Key. Use 8-128 characters from: "
-                "A-Z a-z 0-9 . _ : -"
-            ),
+            message=("Invalid Idempotency-Key. Use 8-128 characters from: A-Z a-z 0-9 . _ : -"),
         )
     return candidate
 
@@ -130,11 +127,7 @@ def _normalize_attempt_log(value: Any) -> list[dict[str, Any]]:
                 "cstart": _int_value(item.get("cstart"), 0),
                 "state": _str_value(item.get("state")),
                 "state_reason": _str_value(item.get("state_reason")),
-                "status_code": (
-                    _int_value(item.get("status_code"))
-                    if item.get("status_code") is not None
-                    else None
-                ),
+                "status_code": (_int_value(item.get("status_code")) if item.get("status_code") is not None else None),
                 "fetch_error": _str_value(item.get("fetch_error")),
             }
         )
@@ -155,19 +148,12 @@ def _normalize_page_logs(value: Any) -> list[dict[str, Any]]:
                 "cstart": _int_value(item.get("cstart"), 0),
                 "state": _str_value(item.get("state")) or "unknown",
                 "state_reason": _str_value(item.get("state_reason")),
-                "status_code": (
-                    _int_value(item.get("status_code"))
-                    if item.get("status_code") is not None
-                    else None
-                ),
+                "status_code": (_int_value(item.get("status_code")) if item.get("status_code") is not None else None),
                 "publication_count": _int_value(item.get("publication_count"), 0),
                 "attempt_count": _int_value(item.get("attempt_count"), 0),
                 "has_show_more_button": _bool_value(item.get("has_show_more_button"), False),
                 "articles_range": _str_value(item.get("articles_range")),
-                "warning_codes": [
-                    str(code)
-                    for code in (warning_codes if isinstance(warning_codes, list) else [])
-                ],
+                "warning_codes": [str(code) for code in (warning_codes if isinstance(warning_codes, list) else [])],
             }
         )
     return normalized
@@ -179,19 +165,11 @@ def _normalize_debug(value: Any) -> dict[str, Any] | None:
     marker_counts = value.get("marker_counts_nonzero")
     warning_codes = value.get("warning_codes")
     return {
-        "status_code": (
-            _int_value(value.get("status_code"))
-            if value.get("status_code") is not None
-            else None
-        ),
+        "status_code": (_int_value(value.get("status_code")) if value.get("status_code") is not None else None),
         "final_url": _str_value(value.get("final_url")),
         "fetch_error": _str_value(value.get("fetch_error")),
         "body_sha256": _str_value(value.get("body_sha256")),
-        "body_length": (
-            _int_value(value.get("body_length"))
-            if value.get("body_length") is not None
-            else None
-        ),
+        "body_length": (_int_value(value.get("body_length")) if value.get("body_length") is not None else None),
         "has_show_more_button": (
             _bool_value(value.get("has_show_more_button"), False)
             if value.get("has_show_more_button") is not None
@@ -199,10 +177,7 @@ def _normalize_debug(value: Any) -> dict[str, Any] | None:
         ),
         "articles_range": _str_value(value.get("articles_range")),
         "state_reason": _str_value(value.get("state_reason")),
-        "warning_codes": [
-            str(code)
-            for code in (warning_codes if isinstance(warning_codes, list) else [])
-        ],
+        "warning_codes": [str(code) for code in (warning_codes if isinstance(warning_codes, list) else [])],
         "marker_counts_nonzero": {
             str(key): _int_value(count, 0)
             for key, count in (marker_counts.items() if isinstance(marker_counts, dict) else [])
@@ -241,9 +216,7 @@ def _normalize_scholar_result(value: Any) -> dict[str, Any]:
         "publication_count": _int_value(value.get("publication_count"), 0),
         "start_cstart": _int_value(value.get("start_cstart"), 0),
         "continuation_cstart": (
-            _int_value(value.get("continuation_cstart"))
-            if value.get("continuation_cstart") is not None
-            else None
+            _int_value(value.get("continuation_cstart")) if value.get("continuation_cstart") is not None else None
         ),
         "continuation_enqueued": _bool_value(value.get("continuation_enqueued"), False),
         "continuation_cleared": _bool_value(value.get("continuation_cleared"), False),
@@ -289,7 +262,9 @@ async def _load_safety_state(
         await db_session.commit()
         await db_session.refresh(user_settings)
         structured_log(
-            logger, "info", "api.runs.safety_cooldown_cleared",
+            logger,
+            "info",
+            "api.runs.safety_cooldown_cleared",
             user_id=user_id,
             reason=previous_safety_state.get("cooldown_reason"),
             cooldown_until=previous_safety_state.get("cooldown_until"),
@@ -299,7 +274,9 @@ async def _load_safety_state(
 
 def _raise_manual_runs_disabled(*, user_id: int, safety_state: dict[str, Any]) -> None:
     structured_log(
-        logger, "warning", "api.runs.manual_blocked_policy",
+        logger,
+        "warning",
+        "api.runs.manual_blocked_policy",
         user_id=user_id,
         policy={"manual_run_allowed": False},
         safety_state=safety_state,
@@ -463,7 +440,9 @@ async def _execute_manual_run(
 
 def _raise_manual_blocked_safety(*, exc, user_id: int) -> None:
     structured_log(
-        logger, "info", "api.runs.manual_blocked_safety",
+        logger,
+        "info",
+        "api.runs.manual_blocked_safety",
         user_id=user_id,
         reason=exc.safety_state.get("cooldown_reason"),
         cooldown_until=exc.safety_state.get("cooldown_until"),
@@ -613,12 +592,12 @@ async def cancel_run(
     scholar_results = error_log.get("scholar_results")
     if not isinstance(scholar_results, list):
         scholar_results = []
-        
+
     safety_state = await _load_safety_state(
         db_session,
         user_id=current_user.id,
     )
-    
+
     return success_payload(
         request,
         data={
@@ -657,7 +636,7 @@ async def run_manual(
 
     try:
         user_settings = await user_settings_service.get_or_create_settings(db_session, user_id=current_user.id)
-        
+
         # Initialize run (creates the record and performs safety checks)
         run, scholars, start_cstart_map = await ingest_service.initialize_run(
             db_session,
@@ -673,13 +652,14 @@ async def run_manual(
             alert_network_failure_threshold=settings.ingestion_alert_network_failure_threshold,
             alert_retry_scheduled_threshold=settings.ingestion_alert_retry_scheduled_threshold,
         )
-        
+
         await db_session.commit()
-        
+
         # Kick off background execution
-        from app.db.session import get_session_factory
         import asyncio
-        
+
+        from app.db.session import get_session_factory
+
         asyncio.create_task(
             ingest_service.execute_run(
                 session_factory=get_session_factory(),
@@ -700,7 +680,7 @@ async def run_manual(
                 idempotency_key=idempotency_key,
             )
         )
-        
+
         return success_payload(
             request,
             data={
@@ -714,7 +694,7 @@ async def run_manual(
                 "reused_existing_run": False,
                 "idempotency_key": idempotency_key,
                 "safety_state": await _load_safety_state(db_session, user_id=current_user.id),
-            }
+            },
         )
     except ingestion_service.RunBlockedBySafetyPolicyError as exc:
         await db_session.rollback()
@@ -740,7 +720,6 @@ async def run_manual(
         _raise_manual_failed(exc=exc, user_id=current_user.id)
 
 
-
 @router.get(
     "/queue/items",
     response_model=QueueListEnvelope,
@@ -891,7 +870,4 @@ async def stream_run_events(
             code="run_not_found",
             message="Run not found.",
         )
-    return StreamingResponse(
-        event_generator(run_id),
-        media_type="text/event-stream"
-    )
+    return StreamingResponse(event_generator(run_id), media_type="text/event-stream")

app/api/routers/scholars.py

@@ -27,8 +27,8 @@ from app.logging_utils import structured_log
 from app.services.domains.ingestion import queue as ingestion_queue_service
 from app.services.domains.portability import application as import_export_service
 from app.services.domains.scholar import rate_limit as scholar_rate_limit
-from app.services.domains.scholars import application as scholar_service
 from app.services.domains.scholar.source import ScholarSource
+from app.services.domains.scholars import application as scholar_service
 from app.settings import settings
 
 logger = logging.getLogger(__name__)
@@ -82,14 +82,18 @@ async def _enqueue_initial_scrape_job_for_scholar(
     except Exception:
         await db_session.rollback()
         structured_log(
-            logger, "warning", "api.scholars.initial_scrape_enqueue_failed",
+            logger,
+            "warning",
+            "api.scholars.initial_scrape_enqueue_failed",
             user_id=user_id,
             scholar_profile_id=profile.id,
         )
         return False
 
     structured_log(
-        logger, "info", "api.scholars.initial_scrape_enqueued",
+        logger,
+        "info",
+        "api.scholars.initial_scrape_enqueued",
         user_id=user_id,
         scholar_profile_id=profile.id,
         reason=INITIAL_SCHOLAR_SCRAPE_QUEUE_REASON,
@@ -120,9 +124,7 @@ def _serialize_scholar(profile) -> dict[str, object]:
         "is_enabled": bool(profile.is_enabled),
         "baseline_completed": bool(profile.baseline_completed),
         "last_run_dt": profile.last_run_dt,
-        "last_run_status": (
-            profile.last_run_status.value if profile.last_run_status is not None else None
-        ),
+        "last_run_status": (profile.last_run_status.value if profile.last_run_status is not None else None),
     }
 
 
@@ -139,7 +141,9 @@ async def _hydrate_scholar_metadata_if_needed(
     should_skip, remaining_seconds = _is_create_hydration_rate_limited()
     if should_skip:
         structured_log(
-            logger, "info", "api.scholars.create_metadata_hydration_skipped",
+            logger,
+            "info",
+            "api.scholars.create_metadata_hydration_skipped",
             reason="scholar_request_throttle_active",
             user_id=user_id,
             scholar_profile_id=profile.id,
@@ -158,14 +162,18 @@ async def _hydrate_scholar_metadata_if_needed(
         )
     except TimeoutError:
         structured_log(
-            logger, "info", "api.scholars.create_metadata_hydration_skipped",
+            logger,
+            "info",
+            "api.scholars.create_metadata_hydration_skipped",
             reason="create_timeout",
             user_id=user_id,
             scholar_profile_id=profile.id,
         )
     except Exception:
         structured_log(
-            logger, "warning", "api.scholars.create_metadata_hydration_failed",
+            logger,
+            "warning",
+            "api.scholars.create_metadata_hydration_failed",
             user_id=user_id,
             scholar_profile_id=profile.id,
         )
@@ -185,9 +193,7 @@ def _search_kwargs() -> dict[str, object]:
         "cooldown_block_threshold": settings.scholar_name_search_cooldown_block_threshold,
         "cooldown_seconds": settings.scholar_name_search_cooldown_seconds,
         "retry_alert_threshold": settings.scholar_name_search_alert_retry_count_threshold,
-        "cooldown_rejection_alert_threshold": (
-            settings.scholar_name_search_alert_cooldown_rejections_threshold
-        ),
+        "cooldown_rejection_alert_threshold": (settings.scholar_name_search_alert_cooldown_rejections_threshold),
     }
 
 
@@ -299,8 +305,7 @@ async def import_scholars_and_publications(
             status_code=400,
             code="invalid_import_schema_version",
             message=(
-                "Import schema version is not supported. "
-                f"Expected {import_export_service.EXPORT_SCHEMA_VERSION}."
+                f"Import schema version is not supported. Expected {import_export_service.EXPORT_SCHEMA_VERSION}."
             ),
         )
     try:
@@ -393,7 +398,15 @@ async def search_scholars(
             message=str(exc),
         ) from exc
 
-    structured_log(logger, "info", "api.scholars.search.completed", user_id=current_user.id, query=query.strip(), candidate_count=len(parsed.candidates), state=parsed.state.value)
+    structured_log(
+        logger,
+        "info",
+        "api.scholars.search.completed",
+        user_id=current_user.id,
+        query=query.strip(),
+        candidate_count=len(parsed.candidates),
+        state=parsed.state.value,
+    )
     return success_payload(
         request,
         data=_search_response_data(query, parsed),
@@ -422,7 +435,14 @@ async def toggle_scholar(
             message="Scholar not found.",
         )
     updated = await scholar_service.toggle_scholar_enabled(db_session, profile=profile)
-    structured_log(logger, "info", "api.scholars.toggled", user_id=current_user.id, scholar_profile_id=updated.id, is_enabled=updated.is_enabled)
+    structured_log(
+        logger,
+        "info",
+        "api.scholars.toggled",
+        user_id=current_user.id,
+        scholar_profile_id=updated.id,
+        is_enabled=updated.is_enabled,
+    )
     return success_payload(
         request,
         data=_serialize_scholar(updated),
@@ -455,7 +475,9 @@ async def delete_scholar(
         profile=profile,
         upload_dir=settings.scholar_image_upload_dir,
     )
-    structured_log(logger, "info", "api.scholars.deleted", user_id=current_user.id, scholar_profile_id=scholar_profile_id)
+    structured_log(
+        logger, "info", "api.scholars.deleted", user_id=current_user.id, scholar_profile_id=scholar_profile_id
+    )
     return success_payload(
         request,
         data={"message": "Scholar deleted."},
@@ -499,7 +521,9 @@ async def update_scholar_image_url(
             message=str(exc),
         ) from exc
 
-    structured_log(logger, "info", "api.scholars.image_url_updated", user_id=current_user.id, scholar_profile_id=updated.id)
+    structured_log(
+        logger, "info", "api.scholars.image_url_updated", user_id=current_user.id, scholar_profile_id=updated.id
+    )
     return success_payload(
         request,
         data=_serialize_scholar(updated),
@@ -541,7 +565,15 @@ async def upload_scholar_image(
         ) from exc
 
     image_size = len(image_bytes)
-    structured_log(logger, "info", "api.scholars.image_uploaded", user_id=current_user.id, scholar_profile_id=updated.id, content_type=image.content_type, size_bytes=image_size)
+    structured_log(
+        logger,
+        "info",
+        "api.scholars.image_uploaded",
+        user_id=current_user.id,
+        scholar_profile_id=updated.id,
+        content_type=image.content_type,
+        size_bytes=image_size,
+    )
     return success_payload(
         request,
         data=_serialize_scholar(updated),

app/api/routers/settings.py

@@ -85,7 +85,9 @@ async def _clear_expired_cooldown_with_log(
     await db_session.commit()
     await db_session.refresh(user_settings)
     structured_log(
-        logger, "info", "api.settings.safety_cooldown_cleared",
+        logger,
+        "info",
+        "api.settings.safety_cooldown_cleared",
         user_id=user_id,
         reason=previous_safety_state.get("cooldown_reason"),
         cooldown_until=previous_safety_state.get("cooldown_until"),
@@ -164,7 +166,9 @@ async def update_settings(
         user_settings=updated,
     )
     structured_log(
-        logger, "info", "api.settings.updated",
+        logger,
+        "info",
+        "api.settings.updated",
         user_id=current_user.id,
         auto_run_enabled=updated.auto_run_enabled,
         run_interval_minutes=updated.run_interval_minutes,

app/api/schemas.py

@@ -671,7 +671,7 @@ class AdminRepairPublicationLinksRequest(BaseModel):
     model_config = ConfigDict(extra="forbid")
 
     @model_validator(mode="after")
-    def validate_scope(self) -> "AdminRepairPublicationLinksRequest":
+    def validate_scope(self) -> AdminRepairPublicationLinksRequest:
         if self.scope_mode == "single_user" and self.user_id is None:
             raise ValueError("user_id is required when scope_mode=single_user.")
         if self.scope_mode == "all_users" and self.user_id is not None:
@@ -680,10 +680,7 @@ class AdminRepairPublicationLinksRequest(BaseModel):
             expected = "REPAIR ALL USERS"
             provided = (self.confirmation_text or "").strip()
             if provided != expected:
-                raise ValueError(
-                    "confirmation_text must equal 'REPAIR ALL USERS' "
-                    "when applying a repair to all users."
-                )
+                raise ValueError("confirmation_text must equal 'REPAIR ALL USERS' when applying a repair to all users.")
         return self
 
 
@@ -735,7 +732,7 @@ class AdminRepairPublicationNearDuplicatesRequest(BaseModel):
     model_config = ConfigDict(extra="forbid")
 
     @model_validator(mode="after")
-    def validate_apply_mode(self) -> "AdminRepairPublicationNearDuplicatesRequest":
+    def validate_apply_mode(self) -> AdminRepairPublicationNearDuplicatesRequest:
         if self.dry_run:
             return self
         if not self.selected_cluster_keys:
@@ -744,8 +741,7 @@ class AdminRepairPublicationNearDuplicatesRequest(BaseModel):
         provided = (self.confirmation_text or "").strip()
         if provided != expected:
             raise ValueError(
-                "confirmation_text must equal 'MERGE SELECTED DUPLICATES' "
-                "when applying near-duplicate merges."
+                "confirmation_text must equal 'MERGE SELECTED DUPLICATES' when applying near-duplicate merges."
             )
         return self
 
@@ -787,7 +783,7 @@ class SettingsData(BaseModel):
     nav_visible_pages: list[str]
     policy: SettingsPolicyData
     safety_state: ScrapeSafetyStateData
-    
+
     openalex_api_key: str | None = None
     crossref_api_token: str | None = None
     crossref_api_mailto: str | None = None

app/auth/__init__.py

@@ -1,2 +1 @@
 """Authentication package for scholarr."""
-

app/auth/deps.py

@@ -24,4 +24,3 @@ def get_login_rate_limiter() -> SlidingWindowRateLimiter:
         max_attempts=settings.login_rate_limit_attempts,
         window_seconds=settings.login_rate_limit_window_seconds,
     )
-

app/auth/runtime.py

@@ -6,8 +6,8 @@ from fastapi import Request
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.auth.session import clear_session_user, get_session_user, set_session_user
-from app.logging_utils import structured_log
 from app.db.models import User
+from app.logging_utils import structured_log
 from app.security.csrf import CSRF_SESSION_KEY
 from app.services.domains.users import application as user_service
 

app/auth/security.py

@@ -16,4 +16,3 @@ class PasswordService:
             return bool(self._hasher.verify(password_hash, password))
         except (InvalidHashError, VerificationError):
             return False
-

app/auth/service.py

@@ -21,9 +21,7 @@ class AuthService:
         normalized_email = email.strip().lower()
         if not normalized_email or not password:
             return None
-        result = await db_session.execute(
-            select(User).where(User.email == normalized_email)
-        )
+        result = await db_session.execute(select(User).where(User.email == normalized_email))
         user = result.scalar_one_or_none()
         if user is None or not user.is_active:
             return None

app/auth/session.py

@@ -4,7 +4,6 @@ from dataclasses import dataclass
 
 from starlette.requests import Request
 
-
 SESSION_USER_ID_KEY = "auth_user_id"
 SESSION_USER_EMAIL_KEY = "auth_user_email"
 SESSION_USER_IS_ADMIN_KEY = "auth_user_is_admin"

app/db/base.py

@@ -1,9 +1,8 @@
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 
 from sqlalchemy import MetaData, event
 from sqlalchemy.orm import DeclarativeBase
 
-
 NAMING_CONVENTION = {
     "ix": "ix_%(column_0_label)s",
     "uq": "uq_%(table_name)s_%(column_0_name)s",
@@ -21,7 +20,7 @@ class Base(DeclarativeBase):
 def _set_updated_at_before_update(_mapper, _connection, target) -> None:
     # Keep audit timestamps current for ORM-managed updates.
     if hasattr(target, "updated_at"):
-        target.updated_at = datetime.now(timezone.utc)
+        target.updated_at = datetime.now(UTC)
 
 
 metadata = Base.metadata

app/db/models.py

@@ -62,18 +62,10 @@ class User(Base):
     id: Mapped[int] = mapped_column(primary_key=True)
     email: Mapped[str] = mapped_column(String(255), nullable=False, unique=True)
     password_hash: Mapped[str] = mapped_column(String(255), nullable=False)
-    is_active: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("true")
-    )
-    is_admin: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("false")
-    )
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("true"))
+    is_admin: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class UserSetting(Base):
@@ -89,18 +81,10 @@ class UserSetting(Base):
         ),
     )
 
-    user_id: Mapped[int] = mapped_column(
-        ForeignKey("users.id", ondelete="CASCADE"), primary_key=True
-    )
-    auto_run_enabled: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("false")
-    )
-    run_interval_minutes: Mapped[int] = mapped_column(
-        Integer, nullable=False, server_default=text("1440")
-    )
-    request_delay_seconds: Mapped[int] = mapped_column(
-        Integer, nullable=False, server_default=text("10")
-    )
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), primary_key=True)
+    auto_run_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))
+    run_interval_minutes: Mapped[int] = mapped_column(Integer, nullable=False, server_default=text("1440"))
+    request_delay_seconds: Mapped[int] = mapped_column(Integer, nullable=False, server_default=text("10"))
     nav_visible_pages: Mapped[list[str]] = mapped_column(
         JSONB,
         nullable=False,
@@ -115,17 +99,13 @@ class UserSetting(Base):
     )
     scrape_cooldown_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
     scrape_cooldown_reason: Mapped[str | None] = mapped_column(String(64))
-    
+
     openalex_api_key: Mapped[str | None] = mapped_column(String(255))
     crossref_api_token: Mapped[str | None] = mapped_column(String(255))
     crossref_api_mailto: Mapped[str | None] = mapped_column(String(255))
 
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class ScholarProfile(Base):
@@ -136,9 +116,7 @@ class ScholarProfile(Base):
     )
 
     id: Mapped[int] = mapped_column(primary_key=True)
-    user_id: Mapped[int] = mapped_column(
-        ForeignKey("users.id", ondelete="CASCADE"), nullable=False
-    )
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), nullable=False)
     scholar_id: Mapped[str] = mapped_column(String(64), nullable=False)
     display_name: Mapped[str | None] = mapped_column(String(255))
     profile_image_url: Mapped[str | None] = mapped_column(Text)
@@ -146,22 +124,14 @@ class ScholarProfile(Base):
     profile_image_upload_path: Mapped[str | None] = mapped_column(Text)
     last_initial_page_fingerprint_sha256: Mapped[str | None] = mapped_column(String(64))
     last_initial_page_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    is_enabled: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("true")
-    )
-    baseline_completed: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("false")
-    )
+    is_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("true"))
+    baseline_completed: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))
     last_run_dt: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
     last_run_status: Mapped[RunStatus | None] = mapped_column(
         RUN_STATUS_DB_ENUM,
     )
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class CrawlRun(Base):
@@ -172,51 +142,29 @@ class CrawlRun(Base):
             "uq_crawl_runs_user_active",
             "user_id",
             unique=True,
-            postgresql_where=text(
-                "status IN ('running'::run_status, 'resolving'::run_status)"
-            ),
+            postgresql_where=text("status IN ('running'::run_status, 'resolving'::run_status)"),
         ),
         Index(
             "uq_crawl_runs_user_manual_idempotency_key",
             "user_id",
             "idempotency_key",
             unique=True,
-            postgresql_where=text(
-                "idempotency_key IS NOT NULL AND trigger_type = 'manual'::run_trigger_type"
-            ),
+            postgresql_where=text("idempotency_key IS NOT NULL AND trigger_type = 'manual'::run_trigger_type"),
         ),
     )
 
     id: Mapped[int] = mapped_column(primary_key=True)
-    user_id: Mapped[int] = mapped_column(
-        ForeignKey("users.id", ondelete="CASCADE"), nullable=False
-    )
-    trigger_type: Mapped[RunTriggerType] = mapped_column(
-        RUN_TRIGGER_TYPE_DB_ENUM, nullable=False
-    )
-    status: Mapped[RunStatus] = mapped_column(
-        RUN_STATUS_DB_ENUM, nullable=False
-    )
-    start_dt: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), nullable=False)
+    trigger_type: Mapped[RunTriggerType] = mapped_column(RUN_TRIGGER_TYPE_DB_ENUM, nullable=False)
+    status: Mapped[RunStatus] = mapped_column(RUN_STATUS_DB_ENUM, nullable=False)
+    start_dt: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
     end_dt: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    scholar_count: Mapped[int] = mapped_column(
-        Integer, nullable=False, server_default=text("0")
-    )
-    new_pub_count: Mapped[int] = mapped_column(
-        Integer, nullable=False, server_default=text("0")
-    )
+    scholar_count: Mapped[int] = mapped_column(Integer, nullable=False, server_default=text("0"))
+    new_pub_count: Mapped[int] = mapped_column(Integer, nullable=False, server_default=text("0"))
     idempotency_key: Mapped[str | None] = mapped_column(String(128))
-    error_log: Mapped[dict] = mapped_column(
-        JSONB, nullable=False, server_default=text("'{}'::jsonb")
-    )
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    error_log: Mapped[dict] = mapped_column(JSONB, nullable=False, server_default=text("'{}'::jsonb"))
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class Publication(Base):
@@ -237,28 +185,16 @@ class Publication(Base):
     title_raw: Mapped[str] = mapped_column(Text, nullable=False)
     title_normalized: Mapped[str] = mapped_column(Text, nullable=False)
     year: Mapped[int | None] = mapped_column(Integer)
-    citation_count: Mapped[int] = mapped_column(
-        Integer, nullable=False, server_default=text("0")
-    )
+    citation_count: Mapped[int] = mapped_column(Integer, nullable=False, server_default=text("0"))
     author_text: Mapped[str | None] = mapped_column(Text)
     venue_text: Mapped[str | None] = mapped_column(Text)
     pub_url: Mapped[str | None] = mapped_column(Text)
     pdf_url: Mapped[str | None] = mapped_column(Text)
-    canonical_title_hash: Mapped[str | None] = mapped_column(
-        String(64), nullable=True, index=True
-    )
-    openalex_enriched: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("false")
-    )
-    openalex_last_attempt_at: Mapped[datetime | None] = mapped_column(
-        DateTime(timezone=True), nullable=True
-    )
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    canonical_title_hash: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
+    openalex_enriched: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))
+    openalex_last_attempt_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class PublicationIdentifier(Base):
@@ -300,12 +236,8 @@ class PublicationIdentifier(Base):
         server_default=text("0"),
     )
     evidence_url: Mapped[str | None] = mapped_column(Text)
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class PublicationPdfJob(Base):
@@ -343,12 +275,8 @@ class PublicationPdfJob(Base):
     last_requested_by_user_id: Mapped[int | None] = mapped_column(
         ForeignKey("users.id", ondelete="SET NULL"),
     )
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class PublicationPdfJobEvent(Base):
@@ -372,9 +300,7 @@ class PublicationPdfJobEvent(Base):
     source: Mapped[str | None] = mapped_column(String(32))
     failure_reason: Mapped[str | None] = mapped_column(String(64))
     message: Mapped[str | None] = mapped_column(Text)
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class ScholarPublication(Base):
@@ -392,18 +318,10 @@ class ScholarPublication(Base):
         ForeignKey("publications.id", ondelete="CASCADE"),
         primary_key=True,
     )
-    is_read: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("false")
-    )
-    is_favorite: Mapped[bool] = mapped_column(
-        Boolean, nullable=False, server_default=text("false")
-    )
-    first_seen_run_id: Mapped[int | None] = mapped_column(
-        ForeignKey("crawl_runs.id", ondelete="SET NULL")
-    )
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    is_read: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))
+    is_favorite: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))
+    first_seen_run_id: Mapped[int | None] = mapped_column(ForeignKey("crawl_runs.id", ondelete="SET NULL"))
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class IngestionQueueItem(Base):
@@ -458,12 +376,8 @@ class IngestionQueueItem(Base):
     last_error: Mapped[str | None] = mapped_column(Text)
     dropped_reason: Mapped[str | None] = mapped_column(String(128))
     dropped_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class DataRepairJob(Base):
@@ -503,12 +417,8 @@ class DataRepairJob(Base):
     error_text: Mapped[str | None] = mapped_column(Text)
     started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
     finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class AuthorSearchRuntimeState(Base):
@@ -532,12 +442,8 @@ class AuthorSearchRuntimeState(Base):
         nullable=False,
         server_default=text("false"),
     )
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class ArxivRuntimeState(Base):
@@ -546,12 +452,8 @@ class ArxivRuntimeState(Base):
     state_key: Mapped[str] = mapped_column(String(64), primary_key=True)
     next_allowed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
     cooldown_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class ArxivQueryCacheEntry(Base):
@@ -570,12 +472,8 @@ class ArxivQueryCacheEntry(Base):
         DateTime(timezone=True),
         nullable=False,
     )
-    cached_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    cached_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
 
 
 class AuthorSearchCacheEntry(Base):
@@ -594,9 +492,5 @@ class AuthorSearchCacheEntry(Base):
         DateTime(timezone=True),
         nullable=False,
     )
-    cached_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
+    cached_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())

app/db/session.py

@@ -1,6 +1,6 @@
-from collections.abc import AsyncIterator
 import logging
 import os
+from collections.abc import AsyncIterator
 
 from sqlalchemy import text
 from sqlalchemy.ext.asyncio import (
@@ -32,7 +32,9 @@ def _normalized_pool_mode(raw_mode: str) -> str:
     if mode in {"null", "queue"}:
         return mode
     structured_log(
-        logger, "warning", "db.invalid_pool_mode_fallback",
+        logger,
+        "warning",
+        "db.invalid_pool_mode_fallback",
         database_pool_mode=raw_mode,
         fallback_mode="queue",
     )
@@ -55,7 +57,9 @@ def get_engine() -> AsyncEngine:
 
         _engine = create_async_engine(settings.database_url, **engine_kwargs)
         structured_log(
-            logger, "info", "db.engine_initialized",
+            logger,
+            "info",
+            "db.engine_initialized",
             pool_mode=pool_mode,
         )
     return _engine

app/http/middleware.py

@@ -1,8 +1,8 @@
 from __future__ import annotations
 
-from secrets import token_urlsafe
 import logging
 import time
+from secrets import token_urlsafe
 
 from starlette.middleware.base import BaseHTTPMiddleware
 from starlette.requests import Request
@@ -63,7 +63,9 @@ class RequestLoggingMiddleware(BaseHTTPMiddleware):
         should_log = self._log_requests and not self._is_skipped_path(request.url.path)
         if should_log:
             structured_log(
-                logger, "debug", "request.started",
+                logger,
+                "debug",
+                "request.started",
                 method=request.method,
                 path=request.url.path,
             )
@@ -86,7 +88,9 @@ class RequestLoggingMiddleware(BaseHTTPMiddleware):
             response.headers[REQUEST_ID_HEADER] = request_id
             if should_log:
                 structured_log(
-                    logger, "debug", "request.completed",
+                    logger,
+                    "debug",
+                    "request.completed",
                     method=request.method,
                     path=request.url.path,
                     status_code=response.status_code,
@@ -164,11 +168,7 @@ class SecurityHeadersMiddleware(BaseHTTPMiddleware):
 
         csp_policy = self._csp_policy_for_path(request.url.path)
         if self._csp_enabled and csp_policy:
-            csp_header = (
-                "Content-Security-Policy-Report-Only"
-                if self._csp_report_only
-                else "Content-Security-Policy"
-            )
+            csp_header = "Content-Security-Policy-Report-Only" if self._csp_report_only else "Content-Security-Policy"
             response.headers.setdefault(csp_header, csp_policy)
 
         hsts = self._strict_transport_security_value()

app/logging_config.py

@@ -1,9 +1,9 @@
 from __future__ import annotations
 
-from datetime import datetime, timezone
 import json
 import logging
 import sys
+from datetime import UTC, datetime
 from typing import Any
 
 from app.logging_context import get_request_id
@@ -102,7 +102,9 @@ class JsonLogFormatter(logging.Formatter):
         if key.lower() in self._redact_fields:
             return "[REDACTED]"
         if isinstance(value, dict):
-            return {nested_key: self._redact_value(nested_key, nested_value) for nested_key, nested_value in value.items()}
+            return {
+                nested_key: self._redact_value(nested_key, nested_value) for nested_key, nested_value in value.items()
+            }
         if isinstance(value, (list, tuple)):
             return [self._redact_value(key, item) for item in value]
         return value
@@ -177,7 +179,7 @@ def _normalize_level(level: str) -> int:
 
 
 def _format_timestamp(created_ts: float) -> str:
-    dt = datetime.fromtimestamp(created_ts, tz=timezone.utc)
+    dt = datetime.fromtimestamp(created_ts, tz=UTC)
     return dt.strftime("%Y-%m-%d %H:%M:%SZ")
 
 

app/logging_context.py

@@ -2,7 +2,6 @@ from __future__ import annotations
 
 from contextvars import ContextVar
 
-
 _request_id_ctx: ContextVar[str | None] = ContextVar("request_id", default=None)
 
 
@@ -12,4 +11,3 @@ def get_request_id() -> str | None:
 
 def set_request_id(value: str | None) -> None:
     _request_id_ctx.set(value)
-

app/main.py

@@ -1,7 +1,7 @@
 from __future__ import annotations
 
-from contextlib import asynccontextmanager
 import logging
+from contextlib import asynccontextmanager
 from pathlib import Path
 
 from fastapi import FastAPI, HTTPException
@@ -12,9 +12,7 @@ from starlette.middleware.sessions import SessionMiddleware
 from app.api.errors import register_api_exception_handlers
 from app.api.media import router as media_router
 from app.api.router import router as api_router
-from app.api.runtime_deps import get_ingestion_service, get_scholar_source
-from app.db.session import check_database
-from app.db.session import close_engine
+from app.db.session import check_database, close_engine
 from app.http.middleware import (
     RequestLoggingMiddleware,
     SecurityHeadersMiddleware,
@@ -54,19 +52,25 @@ scheduler_service = SchedulerService(
 @asynccontextmanager
 async def lifespan(_: FastAPI):
     structured_log(
-        logger, "info", "app.startup_build_marker",
+        logger,
+        "info",
+        "app.startup_build_marker",
         build_marker=BUILD_MARKER,
         frontend_enabled=settings.frontend_enabled,
         scheduler_enabled=settings.scheduler_enabled,
         log_format=settings.log_format,
     )
-    
-    from app.db.session import get_session_factory
+
     from sqlalchemy import text
+
+    from app.db.session import get_session_factory
+
     try:
         session_factory = get_session_factory()
         async with session_factory() as session:
-            await session.execute(text("UPDATE crawl_runs SET status = 'failed' WHERE status::text IN ('running', 'resolving')"))
+            await session.execute(
+                text("UPDATE crawl_runs SET status = 'failed' WHERE status::text IN ('running', 'resolving')")
+            )
             await session.commit()
             structured_log(logger, "info", "app.startup_orphaned_runs_cleaned")
     except Exception as e:
@@ -107,9 +111,7 @@ app.add_middleware(
     content_security_policy_report_only=settings.security_csp_report_only,
     strict_transport_security_enabled=settings.security_strict_transport_security_enabled,
     strict_transport_security_max_age=settings.security_strict_transport_security_max_age,
-    strict_transport_security_include_subdomains=(
-        settings.security_strict_transport_security_include_subdomains
-    ),
+    strict_transport_security_include_subdomains=(settings.security_strict_transport_security_include_subdomains),
     strict_transport_security_preload=settings.security_strict_transport_security_preload,
 )
 app.include_router(api_router)

app/security/__init__.py

@@ -1,2 +1 @@
 """Security middleware and helpers for scholarr."""
-

app/security/csrf.py

@@ -38,7 +38,9 @@ class CSRFMiddleware(BaseHTTPMiddleware):
         session_token = request.session.get(CSRF_SESSION_KEY)
         if not session_token:
             structured_log(
-                logger, "warning", "csrf.missing_session_token",
+                logger,
+                "warning",
+                "csrf.missing_session_token",
                 method=request.method,
                 path=request.url.path,
             )
@@ -56,7 +58,9 @@ class CSRFMiddleware(BaseHTTPMiddleware):
 
         if not request_token or not compare_digest(str(session_token), str(request_token)):
             structured_log(
-                logger, "warning", "csrf.invalid_token",
+                logger,
+                "warning",
+                "csrf.invalid_token",
                 method=request.method,
                 path=request.url.path,
             )

app/services/__init__.py

@@ -1,2 +1 @@
 """Service layer for scholarr application workflows."""
-

app/services/domains/arxiv/application.py

@@ -1,11 +1,11 @@
 from __future__ import annotations
 
 from typing import TYPE_CHECKING
+
 from app.services.domains.arxiv.gateway import (
     build_arxiv_query,
     get_arxiv_gateway,
 )
-from app.services.domains.arxiv.errors import ArxivRateLimitError
 
 if TYPE_CHECKING:
     from app.services.domains.publications.types import PublicationListItem, UnreadPublicationItem

app/services/domains/arxiv/cache.py

@@ -1,11 +1,11 @@
 from __future__ import annotations
 
 import asyncio
-from collections.abc import Awaitable, Callable, Mapping
-from dataclasses import asdict
-from datetime import datetime, timedelta, timezone
 import hashlib
 import json
+from collections.abc import Awaitable, Callable, Mapping
+from dataclasses import asdict
+from datetime import UTC, datetime, timedelta
 from typing import Any
 
 from sqlalchemy import delete, func, select
@@ -33,15 +33,12 @@ async def get_cached_feed(
 ) -> ArxivFeed | None:
     timestamp = _as_utc(now_utc)
     session_factory = get_session_factory()
-    async with session_factory() as db_session:
-        async with db_session.begin():
-            result = await db_session.execute(
-                select(ArxivQueryCacheEntry).where(
-                    ArxivQueryCacheEntry.query_fingerprint == query_fingerprint
-                )
-            )
-            entry = result.scalar_one_or_none()
-            return await _validate_cached_entry(db_session, entry=entry, now_utc=timestamp)
+    async with session_factory() as db_session, db_session.begin():
+        result = await db_session.execute(
+            select(ArxivQueryCacheEntry).where(ArxivQueryCacheEntry.query_fingerprint == query_fingerprint)
+        )
+        entry = result.scalar_one_or_none()
+        return await _validate_cached_entry(db_session, entry=entry, now_utc=timestamp)
 
 
 async def set_cached_feed(
@@ -54,16 +51,15 @@ async def set_cached_feed(
 ) -> None:
     timestamp = _as_utc(now_utc)
     session_factory = get_session_factory()
-    async with session_factory() as db_session:
-        async with db_session.begin():
-            await _write_cached_entry(
-                db_session,
-                query_fingerprint=query_fingerprint,
-                feed=feed,
-                ttl_seconds=ttl_seconds,
-                max_entries=max_entries,
-                now_utc=timestamp,
-            )
+    async with session_factory() as db_session, db_session.begin():
+        await _write_cached_entry(
+            db_session,
+            query_fingerprint=query_fingerprint,
+            feed=feed,
+            ttl_seconds=ttl_seconds,
+            max_entries=max_entries,
+            now_utc=timestamp,
+        )
 
 
 async def run_with_inflight_dedupe(
@@ -142,9 +138,7 @@ async def _write_cached_entry(
 ) -> None:
     ttl = max(float(ttl_seconds), 0.0)
     result = await db_session.execute(
-        select(ArxivQueryCacheEntry).where(
-            ArxivQueryCacheEntry.query_fingerprint == query_fingerprint
-        )
+        select(ArxivQueryCacheEntry).where(ArxivQueryCacheEntry.query_fingerprint == query_fingerprint)
     )
     existing = result.scalar_one_or_none()
     if ttl <= 0.0:
@@ -177,30 +171,22 @@ async def _prune_cache_entries(
     now_utc: datetime,
     max_entries: int,
 ) -> None:
-    await db_session.execute(
-        delete(ArxivQueryCacheEntry).where(ArxivQueryCacheEntry.expires_at <= now_utc)
-    )
+    await db_session.execute(delete(ArxivQueryCacheEntry).where(ArxivQueryCacheEntry.expires_at <= now_utc))
     bounded_max_entries = int(max_entries)
     if bounded_max_entries <= 0:
         return
-    count_result = await db_session.execute(
-        select(func.count()).select_from(ArxivQueryCacheEntry)
-    )
+    count_result = await db_session.execute(select(func.count()).select_from(ArxivQueryCacheEntry))
     entry_count = int(count_result.scalar_one() or 0)
     overflow = max(0, entry_count - bounded_max_entries)
     if overflow <= 0:
         return
     stale_result = await db_session.execute(
-        select(ArxivQueryCacheEntry.query_fingerprint)
-        .order_by(ArxivQueryCacheEntry.cached_at.asc())
-        .limit(overflow)
+        select(ArxivQueryCacheEntry.query_fingerprint).order_by(ArxivQueryCacheEntry.cached_at.asc()).limit(overflow)
     )
     stale_keys = [str(row[0]) for row in stale_result.all()]
     if stale_keys:
         await db_session.execute(
-            delete(ArxivQueryCacheEntry).where(
-                ArxivQueryCacheEntry.query_fingerprint.in_(stale_keys)
-            )
+            delete(ArxivQueryCacheEntry).where(ArxivQueryCacheEntry.query_fingerprint.in_(stale_keys))
         )
 
 
@@ -275,9 +261,9 @@ def _as_string_list(value: object) -> list[str]:
 
 def _as_utc(value: datetime | None) -> datetime:
     if value is None:
-        return datetime.now(timezone.utc)
+        return datetime.now(UTC)
     if value.tzinfo is None:
-        return value.replace(tzinfo=timezone.utc)
+        return value.replace(tzinfo=UTC)
     return value
 
 

app/services/domains/arxiv/client.py

@@ -1,7 +1,7 @@
 from __future__ import annotations
 
-from collections.abc import Awaitable, Callable
 import logging
+from collections.abc import Awaitable, Callable
 
 import httpx
 
@@ -103,9 +103,13 @@ class ArxivClient:
         if self._cache_enabled:
             cached = await get_cached_feed(query_fingerprint=query_fingerprint)
             if cached is not None:
-                structured_log(logger, "info", "arxiv.cache_hit", query_fingerprint=query_fingerprint, source_path=source_path)
+                structured_log(
+                    logger, "info", "arxiv.cache_hit", query_fingerprint=query_fingerprint, source_path=source_path
+                )
                 return cached
-            structured_log(logger, "info", "arxiv.cache_miss", query_fingerprint=query_fingerprint, source_path=source_path)
+            structured_log(
+                logger, "info", "arxiv.cache_miss", query_fingerprint=query_fingerprint, source_path=source_path
+            )
         return await run_with_inflight_dedupe(
             query_fingerprint=query_fingerprint,
             fetch_feed=lambda: self._fetch_live_feed(
@@ -216,13 +220,13 @@ async def _request_arxiv_feed(
     cooldown_status = await get_arxiv_cooldown_status()
     if cooldown_status.is_active:
         structured_log(
-            logger, "warning", "arxiv.request_skipped_cooldown",
+            logger,
+            "warning",
+            "arxiv.request_skipped_cooldown",
             source_path=source_path,
             cooldown_remaining_seconds=float(cooldown_status.remaining_seconds),
         )
-        raise ArxivRateLimitError(
-            f"arXiv global cooldown active ({cooldown_status.remaining_seconds:.0f}s remaining)"
-        )
+        raise ArxivRateLimitError(f"arXiv global cooldown active ({cooldown_status.remaining_seconds:.0f}s remaining)")
 
     async def _fetch() -> httpx.Response:
         timeout_value = _timeout_seconds(timeout_seconds)
@@ -272,5 +276,3 @@ def _source_path_from_params(params: dict[str, object]) -> str:
     if "id_list" in params:
         return ARXIV_SOURCE_PATH_LOOKUP_IDS
     return ARXIV_SOURCE_PATH_UNKNOWN
-
-

app/services/domains/arxiv/gateway.py

@@ -2,8 +2,8 @@ from __future__ import annotations
 
 import logging
 import re
-from typing import TYPE_CHECKING, Protocol
 import unicodedata
+from typing import TYPE_CHECKING, Protocol
 
 from app.logging_utils import structured_log
 from app.services.domains.arxiv.client import ArxivClient

app/services/domains/arxiv/rate_limit.py

@@ -1,10 +1,10 @@
 from __future__ import annotations
 
 import asyncio
+import logging
 from collections.abc import Awaitable, Callable
 from dataclasses import dataclass
-from datetime import datetime, timedelta, timezone
-import logging
+from datetime import UTC, datetime, timedelta
 
 import httpx
 from sqlalchemy import select, text
@@ -47,13 +47,11 @@ async def run_with_global_arxiv_limit(
 
 
 async def get_arxiv_cooldown_status(*, now_utc: datetime | None = None) -> ArxivCooldownStatus:
-    timestamp = _normalize_datetime(now_utc) or datetime.now(timezone.utc)
+    timestamp = _normalize_datetime(now_utc) or datetime.now(UTC)
     session_factory = get_session_factory()
     async with session_factory() as db_session:
         result = await db_session.execute(
-            select(ArxivRuntimeState.cooldown_until).where(
-                ArxivRuntimeState.state_key == ARXIV_RUNTIME_STATE_KEY
-            )
+            select(ArxivRuntimeState.cooldown_until).where(ArxivRuntimeState.state_key == ARXIV_RUNTIME_STATE_KEY)
         )
     cooldown_until = _normalize_datetime(result.scalar_one_or_none())
     remaining_seconds = _cooldown_remaining_seconds(cooldown_until, now_utc=timestamp)
@@ -85,10 +83,14 @@ async def _run_serialized_fetch(
                 source_path=source_path,
             )
             structured_log(
-                logger, "info", "arxiv.request_completed",
+                logger,
+                "info",
+                "arxiv.request_completed",
                 status_code=int(response.status_code),
                 wait_seconds=wait_seconds,
-                cooldown_remaining_seconds=_cooldown_remaining_seconds(runtime_state.cooldown_until, now_utc=datetime.now(timezone.utc)),
+                cooldown_remaining_seconds=_cooldown_remaining_seconds(
+                    runtime_state.cooldown_until, now_utc=datetime.now(UTC)
+                ),
                 source_path=source_path,
             )
             return response, hit_rate_limit
@@ -106,9 +108,7 @@ async def _acquire_arxiv_lock(db_session: AsyncSession) -> None:
 
 async def _load_runtime_state_for_update(db_session: AsyncSession) -> ArxivRuntimeState:
     result = await db_session.execute(
-        select(ArxivRuntimeState)
-        .where(ArxivRuntimeState.state_key == ARXIV_RUNTIME_STATE_KEY)
-        .with_for_update()
+        select(ArxivRuntimeState).where(ArxivRuntimeState.state_key == ARXIV_RUNTIME_STATE_KEY).with_for_update()
     )
     state = result.scalar_one_or_none()
     if state is not None:
@@ -124,13 +124,27 @@ async def _wait_for_allowed_slot_or_raise(
     *,
     source_path: str,
 ) -> float:
-    now_utc = datetime.now(timezone.utc)
+    now_utc = datetime.now(UTC)
     cooldown_seconds = _cooldown_remaining_seconds(runtime_state.cooldown_until, now_utc=now_utc)
     if cooldown_seconds > 0:
-        structured_log(logger, "info", "arxiv.request_scheduled", wait_seconds=0.0, source_path=source_path, cooldown_remaining_seconds=cooldown_seconds)
+        structured_log(
+            logger,
+            "info",
+            "arxiv.request_scheduled",
+            wait_seconds=0.0,
+            source_path=source_path,
+            cooldown_remaining_seconds=cooldown_seconds,
+        )
         raise ArxivRateLimitError(f"arXiv global cooldown active ({cooldown_seconds:.0f}s remaining)")
     wait_seconds = _next_allowed_wait_seconds(runtime_state.next_allowed_at, now_utc=now_utc)
-    structured_log(logger, "info", "arxiv.request_scheduled", wait_seconds=wait_seconds, source_path=source_path, cooldown_remaining_seconds=0.0)
+    structured_log(
+        logger,
+        "info",
+        "arxiv.request_scheduled",
+        wait_seconds=wait_seconds,
+        source_path=source_path,
+        cooldown_remaining_seconds=0.0,
+    )
     if wait_seconds > 0:
         await asyncio.sleep(wait_seconds)
     return wait_seconds
@@ -142,13 +156,15 @@ def _record_post_response_state(
     response_status: int,
     source_path: str,
 ) -> bool:
-    now_utc = datetime.now(timezone.utc)
+    now_utc = datetime.now(UTC)
     runtime_state.next_allowed_at = now_utc + timedelta(seconds=_min_interval_seconds())
     if response_status == 429:
         cooldown_seconds = _cooldown_seconds()
         runtime_state.cooldown_until = now_utc + timedelta(seconds=cooldown_seconds)
         structured_log(
-            logger, "warning", "arxiv.cooldown_activated",
+            logger,
+            "warning",
+            "arxiv.cooldown_activated",
             cooldown_remaining_seconds=cooldown_seconds,
             source_path=source_path,
         )
@@ -176,7 +192,7 @@ def _normalize_datetime(value: datetime | None) -> datetime | None:
     if value is None:
         return None
     if value.tzinfo is None:
-        return value.replace(tzinfo=timezone.utc)
+        return value.replace(tzinfo=UTC)
     return value
 
 
@@ -186,5 +202,3 @@ def _min_interval_seconds() -> float:
 
 def _cooldown_seconds() -> float:
     return max(float(settings.arxiv_rate_limit_cooldown_seconds), 0.0)
-
-

app/services/domains/crossref/application.py

@@ -9,8 +9,8 @@ from typing import TYPE_CHECKING
 
 from crossref.restful import Etiquette, Works
 
-from app.services.domains.doi.normalize import normalize_doi
 from app.logging_utils import structured_log
+from app.services.domains.doi.normalize import normalize_doi
 from app.settings import settings
 
 if TYPE_CHECKING:
@@ -38,7 +38,7 @@ def _rate_limit_wait(min_interval_seconds: float) -> None:
 
 
 def _normalized_tokens(value: str) -> list[str]:
-    lowered = value.lower().replace("’", "'").replace("“", "\"").replace("”", "\"")
+    lowered = value.lower().replace("’", "'").replace("“", '"').replace("”", '"')
     lowered = NON_ALNUM_RE.sub(" ", lowered)
     return [token for token in TOKEN_RE.findall(lowered) if len(token) >= 3]
 

app/services/domains/dbops/__init__.py

@@ -1,8 +1,8 @@
 from app.services.domains.dbops.application import run_publication_link_repair
+from app.services.domains.dbops.integrity import collect_integrity_report
 from app.services.domains.dbops.near_duplicate_repair import (
     run_publication_near_duplicate_repair,
 )
-from app.services.domains.dbops.integrity import collect_integrity_report
 from app.services.domains.dbops.query import list_repair_jobs
 
 __all__ = [

app/services/domains/dbops/application.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from typing import Any
 
 from sqlalchemy import delete, exists, func, select, update
@@ -8,7 +8,6 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.models import DataRepairJob, IngestionQueueItem, Publication, ScholarProfile, ScholarPublication
 
-
 REPAIR_STATUS_PLANNED = "planned"
 REPAIR_STATUS_RUNNING = "running"
 REPAIR_STATUS_COMPLETED = "completed"
@@ -18,7 +17,7 @@ SCOPE_MODE_ALL_USERS = "all_users"
 
 
 def _utcnow() -> datetime:
-    return datetime.now(timezone.utc)
+    return datetime.now(UTC)
 
 
 def _normalize_scope_mode(scope_mode: str) -> str:
@@ -105,11 +104,7 @@ async def _count_orphan_publications(db_session: AsyncSession) -> int:
     stmt = (
         select(func.count())
         .select_from(Publication)
-        .where(
-            ~exists(
-                select(1).where(ScholarPublication.publication_id == Publication.id)
-            )
-        )
+        .where(~exists(select(1).where(ScholarPublication.publication_id == Publication.id)))
     )
     result = await db_session.execute(stmt)
     return int(result.scalar_one() or 0)
@@ -158,9 +153,7 @@ def _job_summary(
 
 async def _delete_links_for_targets(db_session: AsyncSession, *, target_scholar_profile_ids: list[int]) -> int:
     result = await db_session.execute(
-        delete(ScholarPublication).where(
-            ScholarPublication.scholar_profile_id.in_(target_scholar_profile_ids)
-        )
+        delete(ScholarPublication).where(ScholarPublication.scholar_profile_id.in_(target_scholar_profile_ids))
     )
     return int(result.rowcount or 0)
 
@@ -171,9 +164,7 @@ async def _delete_queue_for_targets(
     user_id: int | None,
     target_scholar_profile_ids: list[int],
 ) -> int:
-    stmt = delete(IngestionQueueItem).where(
-        IngestionQueueItem.scholar_profile_id.in_(target_scholar_profile_ids)
-    )
+    stmt = delete(IngestionQueueItem).where(IngestionQueueItem.scholar_profile_id.in_(target_scholar_profile_ids))
     if user_id is not None:
         stmt = stmt.where(IngestionQueueItem.user_id == user_id)
     result = await db_session.execute(stmt)
@@ -203,9 +194,7 @@ async def _reset_scholar_tracking_state(
 
 async def _delete_orphan_publications(db_session: AsyncSession) -> int:
     result = await db_session.execute(
-        delete(Publication).where(
-            ~exists(select(1).where(ScholarPublication.publication_id == Publication.id))
-        )
+        delete(Publication).where(~exists(select(1).where(ScholarPublication.publication_id == Publication.id)))
     )
     return int(result.rowcount or 0)
 

app/services/domains/dbops/integrity.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from typing import Any
 
 from sqlalchemy import func, select, text
@@ -168,7 +168,7 @@ async def collect_integrity_report(db_session: AsyncSession) -> dict[str, Any]:
 
     return {
         "status": _status_from_issues(failures=failures, warnings=warnings),
-        "checked_at": datetime.now(timezone.utc).isoformat(),
+        "checked_at": datetime.now(UTC).isoformat(),
         "failures": failures,
         "warnings": warnings,
         "checks": checks,

app/services/domains/dbops/near_duplicate_repair.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from typing import Any
 
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -17,7 +17,7 @@ NEAR_DUP_DEFAULT_MAX_CLUSTERS = 25
 
 
 def _utcnow() -> datetime:
-    return datetime.now(timezone.utc)
+    return datetime.now(UTC)
 
 
 def _normalized_cluster_keys(values: list[str] | None) -> list[str]:

app/services/domains/dbops/query.py

@@ -16,7 +16,5 @@ async def list_repair_jobs(
     limit: int = 50,
 ) -> list[DataRepairJob]:
     bounded = _bounded_limit(limit)
-    result = await db_session.execute(
-        select(DataRepairJob).order_by(DataRepairJob.created_at.desc()).limit(bounded)
-    )
+    result = await db_session.execute(select(DataRepairJob).order_by(DataRepairJob.created_at.desc()).limit(bounded))
     return list(result.scalars())

app/services/domains/ingestion/application.py

@@ -1,10 +1,10 @@
 from __future__ import annotations
 
 import asyncio
-from datetime import datetime, timedelta, timezone
 import hashlib
 import logging
 import re
+from datetime import UTC, datetime, timedelta
 from typing import Any
 
 from sqlalchemy import or_, select, text
@@ -19,6 +19,11 @@ from app.db.models import (
     ScholarProfile,
     ScholarPublication,
 )
+from app.logging_utils import structured_log
+from app.services.domains.arxiv.errors import ArxivRateLimitError
+from app.services.domains.doi.normalize import first_doi_from_texts
+from app.services.domains.ingestion import queue as queue_service
+from app.services.domains.ingestion import safety as run_safety_service
 from app.services.domains.ingestion.constants import (
     FAILED_STATES,
     FAILURE_BUCKET_BLOCKED,
@@ -30,9 +35,6 @@ from app.services.domains.ingestion.constants import (
     RESUMABLE_PARTIAL_REASONS,
     RUN_LOCK_NAMESPACE,
 )
-from app.services.domains.arxiv.errors import ArxivRateLimitError
-from app.services.domains.doi.normalize import first_doi_from_texts
-from app.services.domains.publication_identifiers import application as identifier_service
 from app.services.domains.ingestion.fingerprints import (
     _build_body_excerpt,
     _dedupe_publication_candidates,
@@ -43,8 +45,6 @@ from app.services.domains.ingestion.fingerprints import (
     canonical_title_for_dedup,
     normalize_title,
 )
-from app.services.domains.ingestion import queue as queue_service
-from app.services.domains.ingestion import safety as run_safety_service
 from app.services.domains.ingestion.types import (
     PagedLoopState,
     PagedParseResult,
@@ -56,17 +56,17 @@ from app.services.domains.ingestion.types import (
     RunProgress,
     ScholarProcessingOutcome,
 )
-from app.services.domains.settings import application as user_settings_service
+from app.services.domains.publication_identifiers import application as identifier_service
 from app.services.domains.runs.events import run_events
 from app.services.domains.scholar.parser import (
-    ParseState,
     ParsedProfilePage,
+    ParseState,
     PublicationCandidate,
     ScholarParserError,
     parse_profile_page,
 )
 from app.services.domains.scholar.source import FetchResult, ScholarSource
-from app.logging_utils import structured_log
+from app.services.domains.settings import application as user_settings_service
 from app.settings import settings
 
 logger = logging.getLogger(__name__)
@@ -147,18 +147,20 @@ class ScholarIngestionService:
         user_id: int,
         trigger_type: RunTriggerType,
     ) -> None:
-        now_utc = datetime.now(timezone.utc)
+        now_utc = datetime.now(UTC)
         previous = run_safety_service.get_safety_event_context(user_settings, now_utc=now_utc)
         if run_safety_service.clear_expired_cooldown(user_settings, now_utc=now_utc):
             await db_session.commit()
             await db_session.refresh(user_settings)
             structured_log(
-                logger, "info", "ingestion.cooldown_cleared",
+                logger,
+                "info",
+                "ingestion.cooldown_cleared",
                 user_id=user_id,
                 reason=previous.get("cooldown_reason"),
                 cooldown_until=previous.get("cooldown_until"),
             )
-            now_utc = datetime.now(timezone.utc)
+            now_utc = datetime.now(UTC)
         if run_safety_service.is_cooldown_active(user_settings, now_utc=now_utc):
             await self._raise_safety_blocked_start(
                 db_session,
@@ -183,7 +185,9 @@ class ScholarIngestionService:
         )
         await db_session.commit()
         structured_log(
-            logger, "warning", "ingestion.safety_policy_blocked_run_start",
+            logger,
+            "warning",
+            "ingestion.safety_policy_blocked_run_start",
             user_id=user_id,
             trigger_type=trigger_type.value,
             reason=safety_state.get("cooldown_reason"),
@@ -204,14 +208,9 @@ class ScholarIngestionService:
         start_cstart_by_scholar_id: dict[int, int] | None,
     ) -> tuple[set[int] | None, dict[int, int]]:
         filtered_scholar_ids = (
-            {int(value) for value in scholar_profile_ids}
-            if scholar_profile_ids is not None
-            else None
+            {int(value) for value in scholar_profile_ids} if scholar_profile_ids is not None else None
         )
-        start_cstart_map = {
-            int(key): max(0, int(value))
-            for key, value in (start_cstart_by_scholar_id or {}).items()
-        }
+        start_cstart_map = {int(key): max(0, int(value)) for key, value in (start_cstart_by_scholar_id or {}).items()}
         return filtered_scholar_ids, start_cstart_map
 
     async def _load_target_scholars(
@@ -521,7 +520,9 @@ class ScholarIngestionService:
             page_logs=paged_parse_result.page_logs,
         )
         structured_log(
-            logger, "warning", "ingestion.scholar_parse_failed",
+            logger,
+            "warning",
+            "ingestion.scholar_parse_failed",
             scholar_profile_id=scholar.id,
             scholar_id=scholar.scholar_id,
             state=paged_parse_result.parsed_page.state.value,
@@ -679,7 +680,7 @@ class ScholarIngestionService:
         max_pages_per_scholar: int,
         page_size: int,
     ) -> tuple[datetime, PagedParseResult, dict[str, Any]]:
-        run_dt = datetime.now(timezone.utc)
+        run_dt = datetime.now(UTC)
         paged_parse_result = await self._fetch_and_parse_all_pages_with_retry(
             scholar=scholar,
             run=run,
@@ -698,7 +699,9 @@ class ScholarIngestionService:
         self._apply_first_page_profile_metadata(scholar=scholar, paged_parse_result=paged_parse_result, run_dt=run_dt)
         parsed_page = paged_parse_result.parsed_page
         structured_log(
-            logger, "info", "ingestion.scholar_parsed",
+            logger,
+            "info",
+            "ingestion.scholar_parsed",
             user_id=user_id,
             crawl_run_id=run.id,
             scholar_profile_id=scholar.id,
@@ -796,9 +799,7 @@ class ScholarIngestionService:
             scholar_results=progress.scholar_results,
             scholar_profile_id=scholar_profile_id,
         )
-        next_succeeded, next_failed, next_partial = ScholarIngestionService._result_counters(
-            outcome.result_entry
-        )
+        next_succeeded, next_failed, next_partial = ScholarIngestionService._result_counters(outcome.result_entry)
         if prior_index is None:
             progress.scholar_results.append(outcome.result_entry)
             ScholarIngestionService._adjust_progress_counts(
@@ -809,9 +810,7 @@ class ScholarIngestionService:
             )
             return
         previous_entry = progress.scholar_results[prior_index]
-        prev_succeeded, prev_failed, prev_partial = ScholarIngestionService._result_counters(
-            previous_entry
-        )
+        prev_succeeded, prev_failed, prev_partial = ScholarIngestionService._result_counters(previous_entry)
         progress.scholar_results[prior_index] = outcome.result_entry
         ScholarIngestionService._adjust_progress_counts(
             progress=progress,
@@ -829,7 +828,7 @@ class ScholarIngestionService:
         exc: Exception,
     ) -> ScholarProcessingOutcome:
         scholar.last_run_status = RunStatus.FAILED
-        scholar.last_run_dt = datetime.now(timezone.utc)
+        scholar.last_run_dt = datetime.now(UTC)
         logger.exception(
             "ingestion.scholar_unexpected_failure",
             extra={
@@ -933,7 +932,7 @@ class ScholarIngestionService:
     ) -> None:
         pre_apply_state = run_safety_service.get_safety_event_context(
             user_settings,
-            now_utc=datetime.now(timezone.utc),
+            now_utc=datetime.now(UTC),
         )
         safety_state, cooldown_trigger_reason = run_safety_service.apply_run_safety_outcome(
             user_settings,
@@ -944,11 +943,13 @@ class ScholarIngestionService:
             network_failure_threshold=alert_summary.network_failure_threshold,
             blocked_cooldown_seconds=settings.ingestion_safety_cooldown_blocked_seconds,
             network_cooldown_seconds=settings.ingestion_safety_cooldown_network_seconds,
-            now_utc=datetime.now(timezone.utc),
+            now_utc=datetime.now(UTC),
         )
         if cooldown_trigger_reason is not None:
             structured_log(
-                logger, "warning", "ingestion.safety_cooldown_entered",
+                logger,
+                "warning",
+                "ingestion.safety_cooldown_entered",
                 user_id=user_id,
                 crawl_run_id=int(run.id),
                 reason=cooldown_trigger_reason,
@@ -962,7 +963,9 @@ class ScholarIngestionService:
             )
         elif pre_apply_state.get("cooldown_active") and not safety_state.get("cooldown_active"):
             structured_log(
-                logger, "info", "ingestion.cooldown_cleared",
+                logger,
+                "info",
+                "ingestion.cooldown_cleared",
                 user_id=user_id,
                 crawl_run_id=int(run.id),
                 reason=pre_apply_state.get("cooldown_reason"),
@@ -979,7 +982,7 @@ class ScholarIngestionService:
         alert_summary: RunAlertSummary,
         idempotency_key: str | None,
     ) -> None:
-        run.end_dt = datetime.now(timezone.utc)
+        run.end_dt = datetime.now(UTC)
         if run.status != RunStatus.CANCELED:
             run.status = self._resolve_run_status(
                 scholar_count=len(scholars),
@@ -1062,7 +1065,26 @@ class ScholarIngestionService:
             new_publication_count=run.new_pub_count,
         )
 
-    async def _initialize_run_for_user(self, db_session: AsyncSession, *, user_id: int, trigger_type: RunTriggerType, scholar_profile_ids: set[int] | None, start_cstart_by_scholar_id: dict[int, int] | None, request_delay_seconds: int, network_error_retries: int, retry_backoff_seconds: float, rate_limit_retries: int, rate_limit_backoff_seconds: float, max_pages_per_scholar: int, page_size: int, idempotency_key: str | None, alert_blocked_failure_threshold: int, alert_network_failure_threshold: int, alert_retry_scheduled_threshold: int) -> tuple[Any, CrawlRun, list[ScholarProfile], dict[int, int]]:
+    async def _initialize_run_for_user(
+        self,
+        db_session: AsyncSession,
+        *,
+        user_id: int,
+        trigger_type: RunTriggerType,
+        scholar_profile_ids: set[int] | None,
+        start_cstart_by_scholar_id: dict[int, int] | None,
+        request_delay_seconds: int,
+        network_error_retries: int,
+        retry_backoff_seconds: float,
+        rate_limit_retries: int,
+        rate_limit_backoff_seconds: float,
+        max_pages_per_scholar: int,
+        page_size: int,
+        idempotency_key: str | None,
+        alert_blocked_failure_threshold: int,
+        alert_network_failure_threshold: int,
+        alert_retry_scheduled_threshold: int,
+    ) -> tuple[Any, CrawlRun, list[ScholarProfile], dict[int, int]]:
         user_settings = await self._load_user_settings_for_run(
             db_session,
             user_id=user_id,
@@ -1116,7 +1138,9 @@ class ScholarIngestionService:
         alert_retry_scheduled_threshold: int,
     ) -> CrawlRun:
         structured_log(
-            logger, "info", "ingestion.run_started",
+            logger,
+            "info",
+            "ingestion.run_started",
             user_id=user_id,
             trigger_type=trigger_type.value,
             scholar_count=len(scholars),
@@ -1143,9 +1167,7 @@ class ScholarIngestionService:
         except IntegrityError as exc:
             if _is_active_run_integrity_error(exc):
                 await db_session.rollback()
-                raise RunAlreadyInProgressError(
-                    f"Run already in progress for user_id={user_id}."
-                ) from exc
+                raise RunAlreadyInProgressError(f"Run already in progress for user_id={user_id}.") from exc
             raise
         return run
 
@@ -1177,8 +1199,11 @@ class ScholarIngestionService:
             await db_session.refresh(run)
             if run.status == RunStatus.CANCELED:
                 structured_log(
-                    logger, "info", "ingestion.run_canceled",
-                    run_id=run.id, user_id=user_id,
+                    logger,
+                    "info",
+                    "ingestion.run_canceled",
+                    run_id=run.id,
+                    user_id=user_id,
                 )
                 return progress
             await self._wait_between_scholars(index=index, request_delay_seconds=request_delay_seconds)
@@ -1217,8 +1242,11 @@ class ScholarIngestionService:
             await db_session.refresh(run)
             if run.status == RunStatus.CANCELED:
                 structured_log(
-                    logger, "info", "ingestion.run_canceled",
-                    run_id=run.id, user_id=user_id,
+                    logger,
+                    "info",
+                    "ingestion.run_canceled",
+                    run_id=run.id,
+                    user_id=user_id,
                 )
                 break
             await self._wait_between_scholars(index=index, request_delay_seconds=request_delay_seconds)
@@ -1263,7 +1291,9 @@ class ScholarIngestionService:
         )
         if alert_summary.alert_flags["blocked_failure_threshold_exceeded"]:
             structured_log(
-                logger, "warning", "ingestion.alert_blocked_failure_threshold_exceeded",
+                logger,
+                "warning",
+                "ingestion.alert_blocked_failure_threshold_exceeded",
                 user_id=user_id,
                 crawl_run_id=int(run.id),
                 blocked_failure_count=alert_summary.blocked_failure_count,
@@ -1271,7 +1301,9 @@ class ScholarIngestionService:
             )
         if alert_summary.alert_flags["network_failure_threshold_exceeded"]:
             structured_log(
-                logger, "warning", "ingestion.alert_network_failure_threshold_exceeded",
+                logger,
+                "warning",
+                "ingestion.alert_network_failure_threshold_exceeded",
                 user_id=user_id,
                 crawl_run_id=int(run.id),
                 network_failure_count=alert_summary.network_failure_count,
@@ -1279,7 +1311,9 @@ class ScholarIngestionService:
             )
         if alert_summary.alert_flags["retry_scheduled_threshold_exceeded"]:
             structured_log(
-                logger, "warning", "ingestion.alert_retry_scheduled_threshold_exceeded",
+                logger,
+                "warning",
+                "ingestion.alert_retry_scheduled_threshold_exceeded",
                 user_id=user_id,
                 crawl_run_id=int(run.id),
                 retries_scheduled_count=failure_summary.retries_scheduled_count,
@@ -1319,7 +1353,9 @@ class ScholarIngestionService:
         effective_delay = self._effective_request_delay_seconds(request_delay_seconds)
         if effective_delay != _int_or_default(request_delay_seconds, effective_delay):
             structured_log(
-                logger, "warning", "ingestion.delay_coerced",
+                logger,
+                "warning",
+                "ingestion.delay_coerced",
                 user_id=user_id,
                 requested_request_delay_seconds=_int_or_default(request_delay_seconds, 0),
                 effective_request_delay_seconds=effective_delay,
@@ -1332,8 +1368,12 @@ class ScholarIngestionService:
             request_delay_seconds=effective_delay,
             network_error_retries=network_error_retries,
             retry_backoff_seconds=retry_backoff_seconds,
-            rate_limit_retries=rate_limit_retries if rate_limit_retries is not None else settings.ingestion_rate_limit_retries,
-            rate_limit_backoff_seconds=rate_limit_backoff_seconds if rate_limit_backoff_seconds is not None else settings.ingestion_rate_limit_backoff_seconds,
+            rate_limit_retries=rate_limit_retries
+            if rate_limit_retries is not None
+            else settings.ingestion_rate_limit_retries,
+            rate_limit_backoff_seconds=rate_limit_backoff_seconds
+            if rate_limit_backoff_seconds is not None
+            else settings.ingestion_rate_limit_backoff_seconds,
             max_pages_per_scholar=max_pages_per_scholar,
             page_size=page_size,
         )
@@ -1383,13 +1423,17 @@ class ScholarIngestionService:
                 run, user_settings, attached_scholars = await self._prepare_execute_run(
                     db_session, run_id=run_id, user_id=user_id, scholars=scholars
                 )
-                
+
                 paging_kwargs = self._paging_kwargs(
                     request_delay_seconds=request_delay_seconds,
                     network_error_retries=network_error_retries,
                     retry_backoff_seconds=retry_backoff_seconds,
-                    rate_limit_retries=rate_limit_retries if rate_limit_retries is not None else settings.ingestion_rate_limit_retries,
-                    rate_limit_backoff_seconds=rate_limit_backoff_seconds if rate_limit_backoff_seconds is not None else settings.ingestion_rate_limit_backoff_seconds,
+                    rate_limit_retries=rate_limit_retries
+                    if rate_limit_retries is not None
+                    else settings.ingestion_rate_limit_retries,
+                    rate_limit_backoff_seconds=rate_limit_backoff_seconds
+                    if rate_limit_backoff_seconds is not None
+                    else settings.ingestion_rate_limit_backoff_seconds,
                     max_pages_per_scholar=max_pages_per_scholar,
                     page_size=page_size,
                 )
@@ -1427,7 +1471,9 @@ class ScholarIngestionService:
                     run.status = RunStatus.RESOLVING
                 await db_session.commit()
                 structured_log(
-                    logger, "info", "ingestion.run_completed",
+                    logger,
+                    "info",
+                    "ingestion.run_completed",
                     user_id=user_id,
                     crawl_run_id=run.id,
                     status=run.status.value,
@@ -1471,7 +1517,8 @@ class ScholarIngestionService:
 
         scholar_ids = [s.id for s in scholars]
         scholars_result = await db_session.execute(
-            select(ScholarProfile).where(ScholarProfile.id.in_(scholar_ids))
+            select(ScholarProfile)
+            .where(ScholarProfile.id.in_(scholar_ids))
             .order_by(ScholarProfile.created_at.asc(), ScholarProfile.id.asc())
         )
         return run, user_settings, list(scholars_result.scalars().all())
@@ -1481,7 +1528,7 @@ class ScholarIngestionService:
             run_to_fail = await cleanup_session.get(CrawlRun, run_id)
             if run_to_fail:
                 run_to_fail.status = RunStatus.FAILED
-                run_to_fail.end_dt = datetime.now(timezone.utc)
+                run_to_fail.end_dt = datetime.now(UTC)
                 run_to_fail.error_log["terminal_exception"] = str(exc)
                 await cleanup_session.commit()
 
@@ -1568,17 +1615,21 @@ class ScholarIngestionService:
             alert_network_failure_threshold=alert_network_failure_threshold,
             alert_retry_scheduled_threshold=alert_retry_scheduled_threshold,
         )
-        
+
         paging_kwargs = self._paging_kwargs(
             request_delay_seconds=self._effective_request_delay_seconds(request_delay_seconds),
             network_error_retries=network_error_retries,
             retry_backoff_seconds=retry_backoff_seconds,
-            rate_limit_retries=rate_limit_retries if rate_limit_retries is not None else settings.ingestion_rate_limit_retries,
-            rate_limit_backoff_seconds=rate_limit_backoff_seconds if rate_limit_backoff_seconds is not None else settings.ingestion_rate_limit_backoff_seconds,
+            rate_limit_retries=rate_limit_retries
+            if rate_limit_retries is not None
+            else settings.ingestion_rate_limit_retries,
+            rate_limit_backoff_seconds=rate_limit_backoff_seconds
+            if rate_limit_backoff_seconds is not None
+            else settings.ingestion_rate_limit_backoff_seconds,
             max_pages_per_scholar=max_pages_per_scholar,
             page_size=page_size,
         )
-        
+
         threshold_kwargs = self._threshold_kwargs(
             alert_blocked_failure_threshold=alert_blocked_failure_threshold,
             alert_network_failure_threshold=alert_network_failure_threshold,
@@ -1628,7 +1679,9 @@ class ScholarIngestionService:
         await db_session.commit()
 
         structured_log(
-            logger, "info", "ingestion.run_completed",
+            logger,
+            "info",
+            "ingestion.run_completed",
             user_id=user_id,
             crawl_run_id=run.id,
             status=run.status.value,
@@ -1663,8 +1716,7 @@ class ScholarIngestionService:
                 return await self._source.fetch_profile_html(scholar_id)
             return FetchResult(
                 requested_url=(
-                    "https://scholar.google.com/citations"
-                    f"?hl=en&user={scholar_id}&cstart={cstart}&pagesize={page_size}"
+                    f"https://scholar.google.com/citations?hl=en&user={scholar_id}&cstart={cstart}&pagesize={page_size}"
                 ),
                 status_code=None,
                 final_url=None,
@@ -1682,8 +1734,7 @@ class ScholarIngestionService:
             )
             return FetchResult(
                 requested_url=(
-                    "https://scholar.google.com/citations"
-                    f"?hl=en&user={scholar_id}&cstart={cstart}&pagesize={page_size}"
+                    f"https://scholar.google.com/citations?hl=en&user={scholar_id}&cstart={cstart}&pagesize={page_size}"
                 ),
                 status_code=None,
                 final_url=None,
@@ -1702,7 +1753,10 @@ class ScholarIngestionService:
     ) -> bool:
         if parsed_page.state == ParseState.NETWORK_ERROR:
             return network_attempt_count <= network_error_retries
-        if parsed_page.state == ParseState.BLOCKED_OR_CAPTCHA and parsed_page.state_reason == "blocked_http_429_rate_limited":
+        if (
+            parsed_page.state == ParseState.BLOCKED_OR_CAPTCHA
+            and parsed_page.state_reason == "blocked_http_429_rate_limited"
+        ):
             return rate_limit_attempt_count <= rate_limit_retries
         return False
 
@@ -1725,7 +1779,9 @@ class ScholarIngestionService:
             attempt_label = network_attempt_count
 
         structured_log(
-            logger, "warning", "ingestion.scholar_retry_scheduled",
+            logger,
+            "warning",
+            "ingestion.scholar_retry_scheduled",
             scholar_id=scholar_id,
             cstart=cstart,
             attempt_count=attempt_label,
@@ -1759,24 +1815,29 @@ class ScholarIngestionService:
                 page_size=page_size,
             )
             parsed_page = self._parse_profile_page_or_layout_error(fetch_result=fetch_result)
-            
+
             if parsed_page.state == ParseState.NETWORK_ERROR:
                 network_attempts += 1
                 total_attempts = network_attempts
-            elif parsed_page.state == ParseState.BLOCKED_OR_CAPTCHA and parsed_page.state_reason == "blocked_http_429_rate_limited":
+            elif (
+                parsed_page.state == ParseState.BLOCKED_OR_CAPTCHA
+                and parsed_page.state_reason == "blocked_http_429_rate_limited"
+            ):
                 rate_limit_attempts += 1
                 total_attempts = rate_limit_attempts
             else:
                 total_attempts = network_attempts + rate_limit_attempts + 1
 
-            attempt_log.append({
-                "attempt": total_attempts,
-                "cstart": cstart,
-                "state": parsed_page.state.value,
-                "state_reason": parsed_page.state_reason,
-                "status_code": fetch_result.status_code,
-                "fetch_error": fetch_result.error,
-            })
+            attempt_log.append(
+                {
+                    "attempt": total_attempts,
+                    "cstart": cstart,
+                    "state": parsed_page.state.value,
+                    "state_reason": parsed_page.state_reason,
+                    "status_code": fetch_result.status_code,
+                    "fetch_error": fetch_result.error,
+                }
+            )
 
             if not self._should_retry_page_fetch(
                 parsed_page=parsed_page,
@@ -1995,18 +2056,20 @@ class ScholarIngestionService:
     ) -> None:
         state.pages_attempted += 1
         state.attempt_log.extend(page_attempt_log)
-        state.page_logs.append({
-            "page": state.pages_attempted,
-            "cstart": state.current_cstart,
-            "state": parsed_page.state.value,
-            "state_reason": parsed_page.state_reason,
-            "status_code": fetch_result.status_code,
-            "publication_count": len(parsed_page.publications),
-            "articles_range": parsed_page.articles_range,
-            "has_show_more_button": parsed_page.has_show_more_button,
-            "warning_codes": parsed_page.warnings,
-            "attempt_count": len(page_attempt_log),
-        })
+        state.page_logs.append(
+            {
+                "page": state.pages_attempted,
+                "cstart": state.current_cstart,
+                "state": parsed_page.state.value,
+                "state_reason": parsed_page.state_reason,
+                "status_code": fetch_result.status_code,
+                "publication_count": len(parsed_page.publications),
+                "articles_range": parsed_page.articles_range,
+                "has_show_more_button": parsed_page.has_show_more_button,
+                "warning_codes": parsed_page.warnings,
+                "attempt_count": len(page_attempt_log),
+            }
+        )
         state.fetch_result = fetch_result
         state.parsed_page = parsed_page
 
@@ -2052,18 +2115,20 @@ class ScholarIngestionService:
         )
         first_page_fingerprint_sha256 = build_initial_page_fingerprint(parsed_page)
         attempt_log = list(first_attempt_log)
-        page_logs = [{
-            "page": 1,
-            "cstart": start_cstart,
-            "state": parsed_page.state.value,
-            "state_reason": parsed_page.state_reason,
-            "status_code": fetch_result.status_code,
-            "publication_count": len(parsed_page.publications),
-            "articles_range": parsed_page.articles_range,
-            "has_show_more_button": parsed_page.has_show_more_button,
-            "warning_codes": parsed_page.warnings,
-            "attempt_count": len(first_attempt_log),
-        }]
+        page_logs = [
+            {
+                "page": 1,
+                "cstart": start_cstart,
+                "state": parsed_page.state.value,
+                "state_reason": parsed_page.state_reason,
+                "status_code": fetch_result.status_code,
+                "publication_count": len(parsed_page.publications),
+                "articles_range": parsed_page.articles_range,
+                "has_show_more_button": parsed_page.has_show_more_button,
+                "warning_codes": parsed_page.warnings,
+                "attempt_count": len(first_attempt_log),
+            }
+        ]
         return fetch_result, parsed_page, first_page_fingerprint_sha256, attempt_log, page_logs
 
     async def _paginate_loop(
@@ -2098,7 +2163,9 @@ class ScholarIngestionService:
             await db_session.refresh(run)
             if run.status == RunStatus.CANCELED:
                 structured_log(
-                    logger, "info", "ingestion.pagination_canceled",
+                    logger,
+                    "info",
+                    "ingestion.pagination_canceled",
                     run_id=run.id,
                 )
                 self._set_truncated_state(
@@ -2217,12 +2284,17 @@ class ScholarIngestionService:
         rate_limit_backoff_seconds: float,
         max_pages: int,
         page_size: int,
-        previous_initial_page_fingerprint_sha256: str | None = None
+        previous_initial_page_fingerprint_sha256: str | None = None,
     ) -> PagedParseResult:
         bounded_max_pages = max(1, int(max_pages))
         bounded_page_size = max(1, int(page_size))
-        fetch_result, parsed_page, first_page_fingerprint_sha256, attempt_log, page_logs = (
-            await self._fetch_initial_page_context(
+        (
+            fetch_result,
+            parsed_page,
+            first_page_fingerprint_sha256,
+            attempt_log,
+            page_logs,
+        ) = await self._fetch_initial_page_context(
             scholar_id=scholar.scholar_id,
             start_cstart=start_cstart,
             bounded_page_size=bounded_page_size,
@@ -2230,7 +2302,7 @@ class ScholarIngestionService:
             retry_backoff_seconds=retry_backoff_seconds,
             rate_limit_retries=rate_limit_retries,
             rate_limit_backoff_seconds=rate_limit_backoff_seconds,
-        ))
+        )
         shortcut_result = self._short_circuit_initial_page(
             start_cstart=start_cstart,
             previous_initial_page_fingerprint_sha256=previous_initial_page_fingerprint_sha256,
@@ -2275,9 +2347,7 @@ class ScholarIngestionService:
         *,
         run_id: int,
     ) -> bool:
-        check_result = await db_session.execute(
-            select(CrawlRun.status).where(CrawlRun.id == run_id)
-        )
+        check_result = await db_session.execute(select(CrawlRun.status).where(CrawlRun.id == run_id))
         status = check_result.scalar_one_or_none()
         if status is None:
             raise RuntimeError(f"Missing crawl_run for run_id={run_id}.")
@@ -2302,12 +2372,10 @@ class ScholarIngestionService:
         )
         from app.services.domains.openalex.matching import find_best_match
 
-        run_result = await db_session.execute(
-            select(CrawlRun.user_id).where(CrawlRun.id == run_id)
-        )
+        run_result = await db_session.execute(select(CrawlRun.user_id).where(CrawlRun.id == run_id))
         user_id = run_result.scalar_one()
 
-        now = datetime.now(timezone.utc)
+        now = datetime.now(UTC)
         cooldown_threshold = now - timedelta(days=7)
 
         stmt = (
@@ -2355,21 +2423,28 @@ class ScholarIngestionService:
                 )
             except OpenAlexBudgetExhaustedError:
                 structured_log(
-                    logger, "warning", "ingestion.openalex_budget_exhausted",
+                    logger,
+                    "warning",
+                    "ingestion.openalex_budget_exhausted",
                     run_id=run_id,
                 )
                 break  # Stop all enrichment — budget won't reset until midnight UTC
             except OpenAlexRateLimitError:
                 structured_log(
-                    logger, "warning", "ingestion.openalex_rate_limited",
+                    logger,
+                    "warning",
+                    "ingestion.openalex_rate_limited",
                     run_id=run_id,
                 )
                 await asyncio.sleep(60)
                 continue
             except Exception as e:
                 structured_log(
-                    logger, "warning", "ingestion.openalex_enrichment_failed",
-                    error=str(e), run_id=run_id,
+                    logger,
+                    "warning",
+                    "ingestion.openalex_enrichment_failed",
+                    error=str(e),
+                    run_id=run_id,
                 )
                 continue
 
@@ -2406,7 +2481,9 @@ class ScholarIngestionService:
         merge_count = await sweep_identifier_duplicates(db_session)
         if merge_count:
             structured_log(
-                logger, "info", "ingestion.identifier_dedup_sweep",
+                logger,
+                "info",
+                "ingestion.identifier_dedup_sweep",
                 merged_count=merge_count,
                 run_id=run_id,
             )
@@ -2444,7 +2521,9 @@ class ScholarIngestionService:
             return True
         except ArxivRateLimitError:
             structured_log(
-                logger, "warning", "ingestion.arxiv_rate_limited",
+                logger,
+                "warning",
+                "ingestion.arxiv_rate_limited",
                 run_id=run_id,
                 publication_id=int(publication.id),
                 detail="arXiv temporarily disabled for remaining enrichment pass",
@@ -2498,17 +2577,18 @@ class ScholarIngestionService:
 
         from app.services.domains.openalex.client import OpenAlexClient
         from app.services.domains.openalex.matching import find_best_match
-        
+
         client = OpenAlexClient(api_key=settings.openalex_api_key, mailto=settings.crossref_api_mailto)
-        
+
         # Batch requests by 25 to stay within URL length limits
         batch_size = 25
         enriched: list[PublicationCandidate] = []
-        
+
         import re
+
         for i in range(0, len(publications), batch_size):
-            batch = publications[i:i + batch_size]
-            
+            batch = publications[i : i + batch_size]
+
             titles = []
             for p in batch:
                 if not p.title:
@@ -2519,29 +2599,27 @@ class ScholarIngestionService:
                 safe_title = " ".join(safe_title.split())
                 if safe_title:
                     titles.append(safe_title)
-            
+
             if not titles:
                 enriched.extend(batch)
                 continue
-            
+
             query = "|".join(t for t in titles)
             try:
-                openalex_works = await client.get_works_by_filter(
-                    {"title.search": query}, limit=batch_size * 3
-                )
+                openalex_works = await client.get_works_by_filter({"title.search": query}, limit=batch_size * 3)
             except Exception as e:
                 logger.warning(
                     "ingestion.openalex_enrichment_failed",
-                    extra={"error": str(e), "batch_size": len(batch), "scholar_id": scholar.id}
+                    extra={"error": str(e), "batch_size": len(batch), "scholar_id": scholar.id},
                 )
                 openalex_works = []
-                
+
             for p in batch:
                 match = find_best_match(
                     target_title=p.title,
                     target_year=p.year,
                     target_authors=p.authors_text or (scholar.display_name or scholar.scholar_id),
-                    candidates=openalex_works
+                    candidates=openalex_works,
                 )
                 if match:
                     new_p = PublicationCandidate(
@@ -2627,7 +2705,7 @@ class ScholarIngestionService:
                 "pub_url": publication.pub_url,
                 "scholar_profile_id": scholar.id,
                 "scholar_label": scholar.display_name or scholar.scholar_id,
-                "first_seen_at": datetime.now(timezone.utc).isoformat(),
+                "first_seen_at": datetime.now(UTC).isoformat(),
                 "new_publication_count": int(run.new_pub_count or 0),
             },
         )
@@ -2647,9 +2725,7 @@ class ScholarIngestionService:
     ) -> Publication | None:
         if not cluster_id:
             return None
-        result = await db_session.execute(
-            select(Publication).where(Publication.cluster_id == cluster_id)
-        )
+        result = await db_session.execute(select(Publication).where(Publication.cluster_id == cluster_id))
         return result.scalar_one_or_none()
 
     async def _find_publication_by_fingerprint(
@@ -2658,9 +2734,7 @@ class ScholarIngestionService:
         *,
         fingerprint: str,
     ) -> Publication | None:
-        result = await db_session.execute(
-            select(Publication).where(Publication.fingerprint_sha256 == fingerprint)
-        )
+        result = await db_session.execute(select(Publication).where(Publication.fingerprint_sha256 == fingerprint))
         return result.scalar_one_or_none()
 
     @staticmethod
@@ -2685,9 +2759,7 @@ class ScholarIngestionService:
         canonical_title_hash: str,
     ) -> Publication | None:
         result = await db_session.execute(
-            select(Publication).where(
-                Publication.canonical_title_hash == canonical_title_hash
-            )
+            select(Publication).where(Publication.canonical_title_hash == canonical_title_hash)
         )
         return result.scalar_one_or_none()
 
@@ -2816,9 +2888,7 @@ class ScholarIngestionService:
     ) -> tuple[str | None, int]:
         if outcome == "partial":
             reason = (pagination_truncated_reason or "").strip()
-            if reason in RESUMABLE_PARTIAL_REASONS or reason.startswith(
-                RESUMABLE_PARTIAL_REASON_PREFIXES
-            ):
+            if reason in RESUMABLE_PARTIAL_REASONS or reason.startswith(RESUMABLE_PARTIAL_REASON_PREFIXES):
                 return reason, queue_service.normalize_cstart(
                     continuation_cstart if continuation_cstart is not None else fallback_cstart
                 )
@@ -2852,13 +2922,9 @@ class ScholarIngestionService:
             "has_show_more_button": parsed_page.has_show_more_button,
             "has_operation_error_banner": parsed_page.has_operation_error_banner,
             "warning_codes": parsed_page.warnings,
-            "marker_counts_nonzero": {
-                key: value for key, value in parsed_page.marker_counts.items() if value > 0
-            },
+            "marker_counts_nonzero": {key: value for key, value in parsed_page.marker_counts.items() if value > 0},
             "body_length": len(fetch_result.body),
-            "body_sha256": hashlib.sha256(fetch_result.body.encode("utf-8")).hexdigest()
-            if fetch_result.body
-            else None,
+            "body_sha256": hashlib.sha256(fetch_result.body.encode("utf-8")).hexdigest() if fetch_result.body else None,
             "body_excerpt": _build_body_excerpt(fetch_result.body),
             "attempt_log": attempt_log,
         }
@@ -2876,9 +2942,7 @@ class ScholarIngestionService:
         user_id: int,
     ) -> bool:
         result = await db_session.execute(
-            text(
-                "SELECT pg_try_advisory_xact_lock(:namespace, :user_key)"
-            ),
+            text("SELECT pg_try_advisory_xact_lock(:namespace, :user_key)"),
             {
                 "namespace": RUN_LOCK_NAMESPACE,
                 "user_key": int(user_id),

app/services/domains/ingestion/fingerprints.py

@@ -3,8 +3,8 @@ from __future__ import annotations
 import hashlib
 import json
 import re
-from typing import Any
 import unicodedata
+from typing import Any
 from urllib.parse import urljoin
 
 from app.services.domains.ingestion.constants import (
@@ -14,7 +14,7 @@ from app.services.domains.ingestion.constants import (
     TITLE_ALNUM_RE,
     WORD_RE,
 )
-from app.services.domains.scholar.parser import ParseState, ParsedProfilePage, PublicationCandidate
+from app.services.domains.scholar.parser import ParsedProfilePage, ParseState, PublicationCandidate
 
 # Scholar-specific noise patterns stripped before canonical comparison.
 # Applied in order; each targets a different Scholar metadata injection style.
@@ -381,4 +381,4 @@ def _build_body_excerpt(body: str, *, max_chars: int = 220) -> str | None:
         return None
     if len(flattened) <= max_chars:
         return flattened
-    return f"{flattened[:max_chars - 1]}..."
+    return f"{flattened[: max_chars - 1]}..."

app/services/domains/ingestion/queue.py

@@ -1,7 +1,7 @@
 from __future__ import annotations
 
 from dataclasses import dataclass
-from datetime import datetime, timedelta, timezone
+from datetime import UTC, datetime, timedelta
 
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -115,7 +115,7 @@ async def upsert_job(
     run_id: int | None,
     delay_seconds: int,
 ) -> IngestionQueueItem:
-    now = datetime.now(timezone.utc)
+    now = datetime.now(UTC)
     next_attempt_dt = now + timedelta(seconds=max(0, int(delay_seconds)))
     item = await _get_item_for_user_scholar(
         db_session,
@@ -171,9 +171,7 @@ async def delete_job_by_id(
     *,
     job_id: int,
 ) -> bool:
-    result = await db_session.execute(
-        select(IngestionQueueItem).where(IngestionQueueItem.id == job_id)
-    )
+    result = await db_session.execute(select(IngestionQueueItem).where(IngestionQueueItem.id == job_id))
     item = result.scalar_one_or_none()
     if item is None:
         return False
@@ -222,10 +220,8 @@ async def increment_attempt_count(
     *,
     job_id: int,
 ) -> IngestionQueueItem | None:
-    now = datetime.now(timezone.utc)
-    result = await db_session.execute(
-        select(IngestionQueueItem).where(IngestionQueueItem.id == job_id)
-    )
+    now = datetime.now(UTC)
+    result = await db_session.execute(select(IngestionQueueItem).where(IngestionQueueItem.id == job_id))
     item = result.scalar_one_or_none()
     if item is None:
         return None
@@ -239,10 +235,8 @@ async def reset_attempt_count(
     *,
     job_id: int,
 ) -> IngestionQueueItem | None:
-    now = datetime.now(timezone.utc)
-    result = await db_session.execute(
-        select(IngestionQueueItem).where(IngestionQueueItem.id == job_id)
-    )
+    now = datetime.now(UTC)
+    result = await db_session.execute(select(IngestionQueueItem).where(IngestionQueueItem.id == job_id))
     item = result.scalar_one_or_none()
     if item is None:
         return None
@@ -259,10 +253,8 @@ async def reschedule_job(
     reason: str,
     error: str | None = None,
 ) -> IngestionQueueItem | None:
-    now = datetime.now(timezone.utc)
-    result = await db_session.execute(
-        select(IngestionQueueItem).where(IngestionQueueItem.id == job_id)
-    )
+    now = datetime.now(UTC)
+    result = await db_session.execute(select(IngestionQueueItem).where(IngestionQueueItem.id == job_id))
     item = result.scalar_one_or_none()
     if item is None:
         return None
@@ -282,10 +274,8 @@ async def mark_retrying(
     job_id: int,
     reason: str | None = None,
 ) -> IngestionQueueItem | None:
-    now = datetime.now(timezone.utc)
-    result = await db_session.execute(
-        select(IngestionQueueItem).where(IngestionQueueItem.id == job_id)
-    )
+    now = datetime.now(UTC)
+    result = await db_session.execute(select(IngestionQueueItem).where(IngestionQueueItem.id == job_id))
     item = result.scalar_one_or_none()
     if item is None:
         return None
@@ -305,10 +295,8 @@ async def mark_dropped(
     reason: str,
     error: str | None = None,
 ) -> IngestionQueueItem | None:
-    now = datetime.now(timezone.utc)
-    result = await db_session.execute(
-        select(IngestionQueueItem).where(IngestionQueueItem.id == job_id)
-    )
+    now = datetime.now(UTC)
+    result = await db_session.execute(select(IngestionQueueItem).where(IngestionQueueItem.id == job_id))
     item = result.scalar_one_or_none()
     if item is None:
         return None
@@ -329,10 +317,8 @@ async def mark_queued_now(
     reason: str,
     reset_attempt_count: bool = False,
 ) -> IngestionQueueItem | None:
-    now = datetime.now(timezone.utc)
-    result = await db_session.execute(
-        select(IngestionQueueItem).where(IngestionQueueItem.id == job_id)
-    )
+    now = datetime.now(UTC)
+    result = await db_session.execute(select(IngestionQueueItem).where(IngestionQueueItem.id == job_id))
     item = result.scalar_one_or_none()
     if item is None:
         return None

app/services/domains/ingestion/safety.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-from datetime import datetime, timedelta, timezone
+from datetime import UTC, datetime, timedelta
 from typing import Any
 
 from app.db.models import UserSetting
@@ -18,7 +18,7 @@ _COUNTER_LAST_EVALUATED_RUN_ID = "last_evaluated_run_id"
 
 
 def _utcnow() -> datetime:
-    return datetime.now(timezone.utc)
+    return datetime.now(UTC)
 
 
 def _safe_int(value: Any, default: int = 0) -> int:
@@ -41,8 +41,8 @@ def _normalize_datetime(value: datetime | None) -> datetime | None:
     if value is None:
         return None
     if value.tzinfo is None:
-        return value.replace(tzinfo=timezone.utc)
-    return value.astimezone(timezone.utc)
+        return value.replace(tzinfo=UTC)
+    return value.astimezone(UTC)
 
 
 def _state_dict(settings: UserSetting) -> dict[str, Any]:
@@ -100,9 +100,7 @@ def _recommended_action(reason: str | None) -> str | None:
             "increase request delay, and avoid repeated manual retries."
         )
     if reason == COOLDOWN_REASON_NETWORK_FAILURE_THRESHOLD:
-        return (
-            "Network failures crossed the threshold. Verify connectivity and retry after cooldown."
-        )
+        return "Network failures crossed the threshold. Verify connectivity and retry after cooldown."
     return None
 
 
@@ -159,14 +157,10 @@ def _update_run_counters(
     counters[_COUNTER_LAST_NETWORK_FAILURE_COUNT] = bounded_network_failures
     counters[_COUNTER_LAST_EVALUATED_RUN_ID] = int(run_id)
     counters[_COUNTER_CONSECUTIVE_BLOCKED_RUNS] = (
-        int(counters[_COUNTER_CONSECUTIVE_BLOCKED_RUNS]) + 1
-        if bounded_blocked_failures > 0
-        else 0
+        int(counters[_COUNTER_CONSECUTIVE_BLOCKED_RUNS]) + 1 if bounded_blocked_failures > 0 else 0
     )
     counters[_COUNTER_CONSECUTIVE_NETWORK_RUNS] = (
-        int(counters[_COUNTER_CONSECUTIVE_NETWORK_RUNS]) + 1
-        if bounded_network_failures > 0
-        else 0
+        int(counters[_COUNTER_CONSECUTIVE_NETWORK_RUNS]) + 1 if bounded_network_failures > 0 else 0
     )
     return bounded_blocked_failures, bounded_network_failures
 

app/services/domains/ingestion/scheduler.py

@@ -1,9 +1,9 @@
 from __future__ import annotations
 
 import asyncio
-from dataclasses import dataclass
-from datetime import datetime, timedelta, timezone
 import logging
+from dataclasses import dataclass
+from datetime import UTC, datetime, timedelta
 
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -17,24 +17,22 @@ from app.db.models import (
     UserSetting,
 )
 from app.db.session import get_session_factory
+from app.logging_utils import structured_log
 from app.services.domains.ingestion import queue as queue_service
 from app.services.domains.ingestion.application import (
     RunAlreadyInProgressError,
     RunBlockedBySafetyPolicyError,
     ScholarIngestionService,
 )
-from app.logging_utils import structured_log
-from app.services.domains.settings import application as user_settings_service
 from app.services.domains.scholar.source import LiveScholarSource
+from app.services.domains.settings import application as user_settings_service
 from app.settings import settings
 
 logger = logging.getLogger(__name__)
 
 
 def _request_delay_floor_seconds() -> int:
-    return user_settings_service.resolve_request_delay_minimum(
-        settings.ingestion_min_request_delay_seconds
-    )
+    return user_settings_service.resolve_request_delay_minimum(settings.ingestion_min_request_delay_seconds)
 
 
 def _effective_request_delay_seconds(value: int | None) -> int:
@@ -96,7 +94,9 @@ class SchedulerService:
             return
         self._task = asyncio.create_task(self._run_loop(), name="scholarr-scheduler")
         structured_log(
-            logger, "info", "scheduler.started",
+            logger,
+            "info",
+            "scheduler.started",
             tick_seconds=self._tick_seconds,
             network_error_retries=self._network_error_retries,
             retry_backoff_seconds=self._retry_backoff_seconds,
@@ -137,13 +137,13 @@ class SchedulerService:
     async def _tick_once(self) -> None:
         if self._continuation_queue_enabled:
             await self._drain_continuation_queue()
-        
+
         await self._drain_pdf_queue()
 
         candidates = await self._load_candidates()
         if not candidates:
             return
-        now = datetime.now(timezone.utc)
+        now = datetime.now(UTC)
         for candidate in candidates:
             if not await self._is_due(candidate, now=now):
                 continue
@@ -170,10 +170,12 @@ class SchedulerService:
     def _candidate_from_row(row: tuple, *, now_utc: datetime) -> _AutoRunCandidate | None:
         user_id, run_interval_minutes, request_delay_seconds, cooldown_until, cooldown_reason = row
         if cooldown_until is not None and cooldown_until.tzinfo is None:
-            cooldown_until = cooldown_until.replace(tzinfo=timezone.utc)
+            cooldown_until = cooldown_until.replace(tzinfo=UTC)
         if cooldown_until is not None and cooldown_until > now_utc:
             structured_log(
-                logger, "info", "scheduler.run_skipped_safety_cooldown_precheck",
+                logger,
+                "info",
+                "scheduler.run_skipped_safety_cooldown_precheck",
                 user_id=int(user_id),
                 reason=cooldown_reason,
                 cooldown_until=cooldown_until,
@@ -192,7 +194,7 @@ class SchedulerService:
         if not settings.ingestion_automation_allowed:
             return []
         rows = await self._load_candidate_rows()
-        now_utc = datetime.now(timezone.utc)
+        now_utc = datetime.now(UTC)
         candidates: list[_AutoRunCandidate] = []
         for row in rows:
             candidate = self._candidate_from_row(row, now_utc=now_utc)
@@ -216,9 +218,7 @@ class SchedulerService:
         if last_run is None:
             return True
 
-        next_due_dt = last_run + timedelta(
-            minutes=candidate.run_interval_minutes
-        )
+        next_due_dt = last_run + timedelta(minutes=candidate.run_interval_minutes)
         return now >= next_due_dt
 
     async def _run_candidate_ingestion(
@@ -252,7 +252,9 @@ class SchedulerService:
             except RunBlockedBySafetyPolicyError as exc:
                 await session.rollback()
                 structured_log(
-                    logger, "info", "scheduler.run_skipped_safety_cooldown",
+                    logger,
+                    "info",
+                    "scheduler.run_skipped_safety_cooldown",
                     user_id=candidate.user_id,
                     reason=exc.safety_state.get("cooldown_reason"),
                     cooldown_until=exc.safety_state.get("cooldown_until"),
@@ -269,7 +271,9 @@ class SchedulerService:
         if run_summary is None:
             return
         structured_log(
-            logger, "info", "scheduler.run_completed",
+            logger,
+            "info",
+            "scheduler.run_completed",
             user_id=candidate.user_id,
             run_id=run_summary.crawl_run_id,
             status=run_summary.status.value,
@@ -278,7 +282,7 @@ class SchedulerService:
         )
 
     async def _drain_continuation_queue(self) -> None:
-        now = datetime.now(timezone.utc)
+        now = datetime.now(UTC)
         session_factory = get_session_factory()
         async with session_factory() as session:
             jobs = await queue_service.list_due_jobs(
@@ -305,7 +309,9 @@ class SchedulerService:
             await session.commit()
         if dropped is not None:
             structured_log(
-                logger, "warning", "scheduler.queue_item_dropped_max_attempts",
+                logger,
+                "warning",
+                "scheduler.queue_item_dropped_max_attempts",
                 queue_item_id=job.id,
                 user_id=job.user_id,
                 scholar_profile_id=job.scholar_profile_id,
@@ -352,7 +358,9 @@ class SchedulerService:
             await session.commit()
         if dropped is not None:
             structured_log(
-                logger, "info", "scheduler.queue_item_dropped_scholar_unavailable",
+                logger,
+                "info",
+                "scheduler.queue_item_dropped_scholar_unavailable",
                 queue_item_id=job.id,
                 user_id=job.user_id,
                 scholar_profile_id=job.scholar_profile_id,
@@ -371,8 +379,11 @@ class SchedulerService:
             )
             await recovery_session.commit()
         structured_log(
-            logger, "info", "scheduler.queue_item_deferred_lock",
-            queue_item_id=job.id, user_id=job.user_id,
+            logger,
+            "info",
+            "scheduler.queue_item_deferred_lock",
+            queue_item_id=job.id,
+            user_id=job.user_id,
         )
 
     async def _reschedule_queue_job_safety_cooldown(
@@ -395,7 +406,9 @@ class SchedulerService:
             )
             await recovery_session.commit()
         structured_log(
-            logger, "info", "scheduler.queue_item_deferred_safety_cooldown",
+            logger,
+            "info",
+            "scheduler.queue_item_deferred_safety_cooldown",
             queue_item_id=job.id,
             user_id=job.user_id,
             reason=exc.safety_state.get("cooldown_reason"),
@@ -423,8 +436,12 @@ class SchedulerService:
                 )
                 await recovery_session.commit()
                 structured_log(
-                    logger, "warning", "scheduler.queue_item_dropped_after_exception",
-                    queue_item_id=job.id, user_id=job.user_id, attempt_count=queue_item.attempt_count,
+                    logger,
+                    "warning",
+                    "scheduler.queue_item_dropped_after_exception",
+                    queue_item_id=job.id,
+                    user_id=job.user_id,
+                    attempt_count=queue_item.attempt_count,
                 )
                 return
             delay_seconds = queue_service.compute_backoff_seconds(
@@ -489,15 +506,23 @@ class SchedulerService:
                 await session.commit()
                 if queue_item is None:
                     structured_log(
-                        logger, "info", "scheduler.queue_item_resolved",
-                        queue_item_id=job.id, user_id=job.user_id,
-                        run_id=run_summary.crawl_run_id, status=run_summary.status.value,
+                        logger,
+                        "info",
+                        "scheduler.queue_item_resolved",
+                        queue_item_id=job.id,
+                        user_id=job.user_id,
+                        run_id=run_summary.crawl_run_id,
+                        status=run_summary.status.value,
                     )
                     return
                 structured_log(
-                    logger, "info", "scheduler.queue_item_progressed",
-                    queue_item_id=job.id, user_id=job.user_id,
-                    run_id=run_summary.crawl_run_id, status=run_summary.status.value,
+                    logger,
+                    "info",
+                    "scheduler.queue_item_progressed",
+                    queue_item_id=job.id,
+                    user_id=job.user_id,
+                    run_id=run_summary.crawl_run_id,
+                    status=run_summary.status.value,
                     attempt_count=int(queue_item.attempt_count),
                 )
                 return
@@ -505,28 +530,50 @@ class SchedulerService:
             if queue_item is None:
                 await session.commit()
                 structured_log(
-                    logger, "info", "scheduler.queue_item_resolved",
-                    queue_item_id=job.id, user_id=job.user_id,
-                    run_id=run_summary.crawl_run_id, status=run_summary.status.value,
+                    logger,
+                    "info",
+                    "scheduler.queue_item_resolved",
+                    queue_item_id=job.id,
+                    user_id=job.user_id,
+                    run_id=run_summary.crawl_run_id,
+                    status=run_summary.status.value,
                 )
                 return
             if int(queue_item.attempt_count) >= self._continuation_max_attempts:
                 await queue_service.mark_dropped(session, job_id=job.id, reason="max_attempts_after_run")
                 await session.commit()
                 structured_log(
-                    logger, "warning", "scheduler.queue_item_dropped_max_attempts_after_run",
-                    queue_item_id=job.id, user_id=job.user_id,
+                    logger,
+                    "warning",
+                    "scheduler.queue_item_dropped_max_attempts_after_run",
+                    queue_item_id=job.id,
+                    user_id=job.user_id,
                     attempt_count=queue_item.attempt_count,
-                    run_id=run_summary.crawl_run_id, status=run_summary.status.value,
+                    run_id=run_summary.crawl_run_id,
+                    status=run_summary.status.value,
                 )
                 return
-            delay_seconds = queue_service.compute_backoff_seconds(base_seconds=self._continuation_base_delay_seconds, attempt_count=int(queue_item.attempt_count), max_seconds=self._continuation_max_delay_seconds)
-            await queue_service.reschedule_job(session, job_id=job.id, delay_seconds=delay_seconds, reason=queue_item.reason, error=queue_item.last_error)
+            delay_seconds = queue_service.compute_backoff_seconds(
+                base_seconds=self._continuation_base_delay_seconds,
+                attempt_count=int(queue_item.attempt_count),
+                max_seconds=self._continuation_max_delay_seconds,
+            )
+            await queue_service.reschedule_job(
+                session,
+                job_id=job.id,
+                delay_seconds=delay_seconds,
+                reason=queue_item.reason,
+                error=queue_item.last_error,
+            )
             await session.commit()
         structured_log(
-            logger, "info", "scheduler.queue_item_rescheduled_failed",
-            queue_item_id=job.id, user_id=job.user_id,
-            run_id=run_summary.crawl_run_id, status=run_summary.status.value,
+            logger,
+            "info",
+            "scheduler.queue_item_rescheduled_failed",
+            queue_item_id=job.id,
+            user_id=job.user_id,
+            run_id=run_summary.crawl_run_id,
+            status=run_summary.status.value,
             attempt_count=int(queue_item.attempt_count),
             delay_seconds=delay_seconds,
         )
@@ -545,7 +592,7 @@ class SchedulerService:
 
     async def _drain_pdf_queue(self) -> None:
         from app.services.domains.publications.pdf_queue import drain_ready_jobs
-        
+
         session_factory = get_session_factory()
         async with session_factory() as session:
             try:
@@ -556,7 +603,9 @@ class SchedulerService:
                 )
                 if processed > 0:
                     structured_log(
-                        logger, "info", "scheduler.pdf_queue_drain_completed",
+                        logger,
+                        "info",
+                        "scheduler.pdf_queue_drain_completed",
                         processed_count=processed,
                     )
             except Exception:

app/services/domains/openalex/client.py

@@ -1,6 +1,4 @@
-import asyncio
 import logging
-from typing import Any, Mapping
 
 import httpx
 from tenacity import (
@@ -23,11 +21,13 @@ class OpenAlexClientError(Exception):
 
 class OpenAlexRateLimitError(OpenAlexClientError):
     """Transient rate limit (too many requests per second)."""
+
     pass
 
 
 class OpenAlexBudgetExhaustedError(OpenAlexClientError):
     """Daily API budget exhausted — retrying is futile until midnight UTC."""
+
     pass
 
 
@@ -64,7 +64,7 @@ class OpenAlexClient:
             return None
 
         url = f"{OPENALEX_BASE_URL}/works/{clean_doi}"
-        
+
         headers = {}
         if self.mailto:
             headers["User-Agent"] = f"scholar-scraper/1.0 (mailto:{self.mailto})"
@@ -79,9 +79,7 @@ class OpenAlexClient:
         if response.status_code == 429:
             remaining = response.headers.get("X-RateLimit-Remaining-USD", "")
             if remaining == "0" or remaining.startswith("-"):
-                raise OpenAlexBudgetExhaustedError(
-                    "Daily API budget exhausted; retrying won't help until midnight UTC"
-                )
+                raise OpenAlexBudgetExhaustedError("Daily API budget exhausted; retrying won't help until midnight UTC")
             raise OpenAlexRateLimitError("Rate limit exceeded fetching OpenAlex work by DOI")
         if response.status_code >= 400:
             logger.warning("OpenAlex API error: %s %s", response.status_code, response.text[:500])
@@ -110,13 +108,13 @@ class OpenAlexClient:
 
         # Example: {"doi": "10.foo|10.bar", "title.search": "query"}
         filter_str = ",".join(f"{k}:{v}" for k, v in filters.items())
-        
+
         params = self._base_params.copy()
         params["filter"] = filter_str
         params["per-page"] = str(limit)
 
         url = f"{OPENALEX_BASE_URL}/works"
-        
+
         headers = {}
         if self.mailto:
             headers["User-Agent"] = f"scholar-scraper/1.0 (mailto:{self.mailto})"
@@ -129,9 +127,7 @@ class OpenAlexClient:
         if response.status_code == 429:
             remaining = response.headers.get("X-RateLimit-Remaining-USD", "")
             if remaining == "0" or remaining.startswith("-"):
-                raise OpenAlexBudgetExhaustedError(
-                    "Daily API budget exhausted; retrying won't help until midnight UTC"
-                )
+                raise OpenAlexBudgetExhaustedError("Daily API budget exhausted; retrying won't help until midnight UTC")
             raise OpenAlexRateLimitError("Rate limit exceeded fetching OpenAlex works list")
         if response.status_code >= 400:
             logger.warning("OpenAlex API error (filters=%s): %s %s", filters, response.status_code, response.text[:500])
@@ -139,7 +135,7 @@ class OpenAlexClient:
 
         data = response.json()
         results = data.get("results") or []
-        
+
         parsed_works = []
         for raw_work in results:
             try:

app/services/domains/openalex/matching.py

@@ -24,11 +24,11 @@ def _clean_string(s: str | None) -> str:
 def _author_overlap_score(target_authors: str | None, candidate_authors: list[str]) -> bool:
     if not target_authors or not candidate_authors:
         return False
-    
+
     target_clean = _clean_string(target_authors)
     if not target_clean:
         return False
-        
+
     for candidate in candidate_authors:
         cand_clean = _clean_string(candidate)
         if cand_clean and (cand_clean in target_clean or target_clean in cand_clean):
@@ -36,7 +36,7 @@ def _author_overlap_score(target_authors: str | None, candidate_authors: list[st
         # Alternatively check rapidfuzz token_set_ratio
         if cand_clean and fuzz.token_set_ratio(target_clean, cand_clean) > 80:
             return True
-            
+
     return False
 
 
@@ -62,12 +62,12 @@ def find_best_match(
     for cand in candidates:
         if not cand.title:
             continue
-            
+
         clean_cand = _clean_string(cand.title)
-        
+
         # Primary sort: string similarity ratio
         score = fuzz.ratio(clean_target, clean_cand)
-        
+
         if score >= TITLE_MATCH_THRESHOLD:
             scored_candidates.append((score, cand))
 
@@ -76,13 +76,12 @@ def find_best_match(
 
     # Sort descending by score
     scored_candidates.sort(key=lambda x: x[0], reverse=True)
-    
+
     best_score = scored_candidates[0][0]
-    
+
     # Extract all candidates within the tiebreaker margin
     top_scored_candidates = [
-        (score, cand) for score, cand in scored_candidates 
-        if best_score - score <= TIEBREAKER_MARGIN
+        (score, cand) for score, cand in scored_candidates if best_score - score <= TIEBREAKER_MARGIN
     ]
 
     if len(top_scored_candidates) == 1:
@@ -91,18 +90,18 @@ def find_best_match(
     # We have a tie or near-tie. Use year and author overlap to break the tie.
     # Score candidates: +1 for year match (within 1 year), +1 for author overlap
     tiebreaker_scores: list[tuple[int, float, OpenAlexWork]] = []
-    
+
     for original_score, cand in top_scored_candidates:
         tb_score = 0
         if target_year is not None and cand.publication_year is not None:
             if abs(target_year - cand.publication_year) <= 1:
                 tb_score += 1
-                
+
         candidate_author_names = [a.display_name for a in cand.authors if a.display_name]
         if _author_overlap_score(target_authors, candidate_author_names):
             tb_score += 1
-            
+
         tiebreaker_scores.append((tb_score, original_score, cand))
-        
+
     tiebreaker_scores.sort(key=lambda x: (x[0], x[1]), reverse=True)
     return tiebreaker_scores[0][2]

app/services/domains/openalex/types.py

@@ -1,8 +1,8 @@
 from __future__ import annotations
 
+from collections.abc import Mapping
 from dataclasses import dataclass, field
-from datetime import datetime
-from typing import Any, Mapping
+from typing import Any
 
 
 @dataclass(frozen=True)
@@ -28,24 +28,24 @@ class OpenAlexWork:
     @classmethod
     def from_api_dict(cls, data: Mapping[str, Any]) -> OpenAlexWork:
         ids = data.get("ids") or {}
-        
+
         # Extract DOI without the https://doi.org/ prefix
         doi = ids.get("doi")
         if doi and doi.startswith("https://doi.org/"):
             doi = doi[16:]
-            
+
         # Extract PMID without the url prefix
         pmid = ids.get("pmid")
         if pmid and pmid.startswith("https://pubmed.ncbi.nlm.nih.gov/"):
             pmid = pmid[32:]
-            
+
         # Extract PMCID without the url prefix
         pmcid = ids.get("pmcid")
         if pmcid and pmcid.startswith("https://www.ncbi.nlm.nih.gov/pmc/articles/"):
             pmcid = pmcid[42:]
 
         open_access = data.get("open_access") or {}
-        
+
         authors = []
         for authorship in data.get("authorships") or []:
             author_data = authorship.get("author") or {}

app/services/domains/portability/application.py

@@ -18,7 +18,7 @@ from app.services.domains.portability.publication_import import (
     _upsert_imported_publication,
 )
 from app.services.domains.portability.scholar_import import _upsert_imported_scholars
-from app.services.domains.portability.types import ImportExportError, ImportedPublicationInput
+from app.services.domains.portability.types import ImportedPublicationInput, ImportExportError
 
 
 async def import_user_data(
@@ -58,8 +58,8 @@ async def import_user_data(
 
 __all__ = [
     "EXPORT_SCHEMA_VERSION",
-    "MAX_IMPORT_SCHOLARS",
     "MAX_IMPORT_PUBLICATIONS",
+    "MAX_IMPORT_SCHOLARS",
     "ImportExportError",
     "ImportedPublicationInput",
     "export_user_data",

app/services/domains/portability/exporting.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from typing import Any
 
 from sqlalchemy import select
@@ -11,7 +11,7 @@ from app.services.domains.portability.constants import EXPORT_SCHEMA_VERSION
 
 
 def _exported_at_iso() -> str:
-    return datetime.now(timezone.utc).replace(microsecond=0).isoformat()
+    return datetime.now(UTC).replace(microsecond=0).isoformat()
 
 
 def _serialize_export_scholar(profile: ScholarProfile) -> dict[str, Any]:
@@ -58,9 +58,7 @@ async def export_user_data(
     user_id: int,
 ) -> dict[str, Any]:
     scholars_result = await db_session.execute(
-        select(ScholarProfile)
-        .where(ScholarProfile.user_id == user_id)
-        .order_by(ScholarProfile.id.asc())
+        select(ScholarProfile).where(ScholarProfile.user_id == user_id).order_by(ScholarProfile.id.asc())
     )
     publication_result = await db_session.execute(
         select(

app/services/domains/portability/normalize.py

@@ -104,6 +104,4 @@ def _validate_import_sizes(
     if len(scholars) > MAX_IMPORT_SCHOLARS:
         raise ImportExportError(f"Import exceeds max scholars ({MAX_IMPORT_SCHOLARS}).")
     if len(publications) > MAX_IMPORT_PUBLICATIONS:
-        raise ImportExportError(
-            f"Import exceeds max publications ({MAX_IMPORT_PUBLICATIONS})."
-        )
+        raise ImportExportError(f"Import exceeds max publications ({MAX_IMPORT_PUBLICATIONS}).")

app/services/domains/portability/publication_import.py

@@ -6,8 +6,8 @@ from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.models import Publication, ScholarProfile, ScholarPublication
-from app.services.domains.ingestion.application import build_publication_url, normalize_title
 from app.services.domains.doi.normalize import normalize_doi
+from app.services.domains.ingestion.application import build_publication_url, normalize_title
 from app.services.domains.portability.normalize import (
     _normalize_citation_count,
     _normalize_optional_text,
@@ -22,9 +22,7 @@ async def _find_publication_by_cluster(
     *,
     cluster_id: str,
 ) -> Publication | None:
-    result = await db_session.execute(
-        select(Publication).where(Publication.cluster_id == cluster_id)
-    )
+    result = await db_session.execute(select(Publication).where(Publication.cluster_id == cluster_id))
     return result.scalar_one_or_none()
 
 
@@ -33,9 +31,7 @@ async def _find_publication_by_fingerprint(
     *,
     fingerprint_sha256: str,
 ) -> Publication | None:
-    result = await db_session.execute(
-        select(Publication).where(Publication.fingerprint_sha256 == fingerprint_sha256)
-    )
+    result = await db_session.execute(select(Publication).where(Publication.fingerprint_sha256 == fingerprint_sha256))
     return result.scalar_one_or_none()
 
 

app/services/domains/portability/scholar_import.py

@@ -15,9 +15,7 @@ async def _load_user_scholar_map(
     *,
     user_id: int,
 ) -> dict[str, ScholarProfile]:
-    result = await db_session.execute(
-        select(ScholarProfile).where(ScholarProfile.user_id == user_id)
-    )
+    result = await db_session.execute(select(ScholarProfile).where(ScholarProfile.user_id == user_id))
     profiles = list(result.scalars().all())
     return {profile.scholar_id: profile for profile in profiles}
 
@@ -70,9 +68,7 @@ async def _upsert_imported_scholars(
     for item in scholars:
         try:
             scholar_id = scholar_service.validate_scholar_id(str(item["scholar_id"]))
-            display_name = scholar_service.normalize_display_name(
-                str(item.get("display_name") or "")
-            )
+            display_name = scholar_service.normalize_display_name(str(item.get("display_name") or ""))
             override_url = scholar_service.normalize_profile_image_url(
                 _normalize_optional_text(item.get("profile_image_override_url"))
             )

app/services/domains/publication_identifiers/__init__.py

@@ -1,7 +1,7 @@
 from app.services.domains.publication_identifiers.application import (
     DisplayIdentifier,
-    display_identifier_for_publication_id,
     derive_display_identifier_from_values,
+    display_identifier_for_publication_id,
     overlay_pdf_queue_items_with_display_identifiers,
     overlay_publication_items_with_display_identifiers,
     sync_identifiers_for_publication_fields,
@@ -10,8 +10,8 @@ from app.services.domains.publication_identifiers.application import (
 
 __all__ = [
     "DisplayIdentifier",
-    "display_identifier_for_publication_id",
     "derive_display_identifier_from_values",
+    "display_identifier_for_publication_id",
     "overlay_pdf_queue_items_with_display_identifiers",
     "overlay_publication_items_with_display_identifiers",
     "sync_identifiers_for_publication_fields",

app/services/domains/publication_identifiers/application.py

@@ -55,7 +55,9 @@ def _fallback_candidates_from_values(
     if doi:
         normalized_doi = normalize_doi(doi)
         if normalized_doi:
-            candidates.append(_candidate(IdentifierKind.DOI, doi, normalized_doi, "legacy_doi", CONFIDENCE_HIGH, pub_url))
+            candidates.append(
+                _candidate(IdentifierKind.DOI, doi, normalized_doi, "legacy_doi", CONFIDENCE_HIGH, pub_url)
+            )
     candidates.extend(_url_identifier_candidates(values=values, source="legacy_urls"))
     return _dedup_candidates(candidates)
 
@@ -332,10 +334,13 @@ async def _existing_identifier_by_kind(
     kind: str,
 ) -> PublicationIdentifier | None:
     result = await db_session.execute(
-        select(PublicationIdentifier).where(
+        select(PublicationIdentifier)
+        .where(
             PublicationIdentifier.publication_id == publication_id,
             PublicationIdentifier.kind == kind,
-        ).order_by(PublicationIdentifier.confidence_score.desc()).limit(1)
+        )
+        .order_by(PublicationIdentifier.confidence_score.desc())
+        .limit(1)
     )
     return result.scalar_one_or_none()
 
@@ -380,7 +385,9 @@ def _overlay_publication_item(
     item: PublicationListItem,
     display_identifier: DisplayIdentifier | None,
 ) -> PublicationListItem:
-    fallback = display_identifier or derive_display_identifier_from_values(doi=None, pub_url=item.pub_url, pdf_url=item.pdf_url)
+    fallback = display_identifier or derive_display_identifier_from_values(
+        doi=None, pub_url=item.pub_url, pdf_url=item.pdf_url
+    )
     return replace(item, display_identifier=fallback)
 
 
@@ -447,8 +454,7 @@ def _best_display_identifier_map(rows: list[PublicationIdentifier]) -> dict[int,
     return {
         publication_id: display
         for publication_id, display in (
-            (publication_id, _best_display_identifier(candidates))
-            for publication_id, candidates in grouped.items()
+            (publication_id, _best_display_identifier(candidates)) for publication_id, candidates in grouped.items()
         )
         if display is not None
     }

app/services/domains/publications/application.py

@@ -1,21 +1,21 @@
 from __future__ import annotations
 
 from app.services.domains.publications.counts import (
-    count_for_user,
     count_favorite_for_user,
+    count_for_user,
     count_latest_for_user,
     count_unread_for_user,
 )
-from app.services.domains.publications.listing import (
-    list_for_user,
-    retry_pdf_for_user,
-    list_unread_for_user,
-)
 from app.services.domains.publications.enrichment import (
     hydrate_pdf_enrichment_state,
     schedule_missing_pdf_enrichment_for_user,
     schedule_retry_pdf_enrichment_for_row,
 )
+from app.services.domains.publications.listing import (
+    list_for_user,
+    list_unread_for_user,
+    retry_pdf_for_user,
+)
 from app.services.domains.publications.modes import (
     MODE_ALL,
     MODE_LATEST,
@@ -23,6 +23,13 @@ from app.services.domains.publications.modes import (
     MODE_UNREAD,
     resolve_publication_view_mode,
 )
+from app.services.domains.publications.pdf_queue import (
+    count_pdf_queue_items,
+    enqueue_all_missing_pdf_jobs,
+    enqueue_retry_pdf_job_for_publication_id,
+    list_pdf_queue_items,
+    list_pdf_queue_page,
+)
 from app.services.domains.publications.queries import (
     get_latest_run_id_for_user,
     get_publication_item_for_user,
@@ -33,42 +40,35 @@ from app.services.domains.publications.read_state import (
     mark_selected_as_read_for_user,
     set_publication_favorite_for_user,
 )
-from app.services.domains.publications.pdf_queue import (
-    count_pdf_queue_items,
-    enqueue_all_missing_pdf_jobs,
-    enqueue_retry_pdf_job_for_publication_id,
-    list_pdf_queue_page,
-    list_pdf_queue_items,
-)
 from app.services.domains.publications.types import PublicationListItem, UnreadPublicationItem
 
 __all__ = [
     "MODE_ALL",
-    "MODE_UNREAD",
     "MODE_LATEST",
     "MODE_NEW",
+    "MODE_UNREAD",
     "PublicationListItem",
     "UnreadPublicationItem",
-    "resolve_publication_view_mode",
-    "get_latest_run_id_for_user",
-    "publications_query",
-    "get_publication_item_for_user",
-    "list_for_user",
-    "list_unread_for_user",
-    "retry_pdf_for_user",
-    "hydrate_pdf_enrichment_state",
-    "schedule_retry_pdf_enrichment_for_row",
-    "list_pdf_queue_items",
-    "list_pdf_queue_page",
+    "count_favorite_for_user",
+    "count_for_user",
+    "count_latest_for_user",
     "count_pdf_queue_items",
+    "count_unread_for_user",
     "enqueue_all_missing_pdf_jobs",
     "enqueue_retry_pdf_job_for_publication_id",
-    "schedule_missing_pdf_enrichment_for_user",
-    "count_for_user",
-    "count_favorite_for_user",
-    "count_unread_for_user",
-    "count_latest_for_user",
+    "get_latest_run_id_for_user",
+    "get_publication_item_for_user",
+    "hydrate_pdf_enrichment_state",
+    "list_for_user",
+    "list_pdf_queue_items",
+    "list_pdf_queue_page",
+    "list_unread_for_user",
     "mark_all_unread_as_read_for_user",
     "mark_selected_as_read_for_user",
+    "publications_query",
+    "resolve_publication_view_mode",
+    "retry_pdf_for_user",
+    "schedule_missing_pdf_enrichment_for_user",
+    "schedule_retry_pdf_enrichment_for_row",
     "set_publication_favorite_for_user",
 ]

app/services/domains/publications/dedup.py

@@ -1,9 +1,9 @@
 from __future__ import annotations
 
-from dataclasses import dataclass
 import hashlib
 import logging
-from typing import Iterable
+from collections.abc import Iterable
+from dataclasses import dataclass
 
 from sqlalchemy import delete, select
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -113,9 +113,7 @@ async def _load_publication(
     *,
     publication_id: int,
 ) -> Publication | None:
-    result = await db_session.execute(
-        select(Publication).where(Publication.id == publication_id)
-    )
+    result = await db_session.execute(select(Publication).where(Publication.id == publication_id))
     return result.scalar_one_or_none()
 
 
@@ -160,9 +158,7 @@ async def _migrate_scholar_links(
     dup_links = dup_links_result.scalars().all()
 
     winner_profiles_result = await db_session.execute(
-        select(ScholarPublication.scholar_profile_id).where(
-            ScholarPublication.publication_id == winner_id
-        )
+        select(ScholarPublication.scholar_profile_id).where(ScholarPublication.publication_id == winner_id)
     )
     winner_profiles: set[int] = {row for (row,) in winner_profiles_result}
 
@@ -346,11 +342,7 @@ def _candidate_from_row(
 
 
 def _normalized_tokens(tokens: Iterable[str]) -> set[str]:
-    return {
-        token
-        for token in tokens
-        if len(token) >= NEAR_DUP_MIN_TOKEN_LENGTH and token not in NEAR_DUP_STOPWORDS
-    }
+    return {token for token in tokens if len(token) >= NEAR_DUP_MIN_TOKEN_LENGTH and token not in NEAR_DUP_STOPWORDS}
 
 
 def _cluster_candidate_groups(

app/services/domains/publications/enrichment.py

@@ -4,12 +4,12 @@ import logging
 
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.logging_utils import structured_log
 from app.services.domains.publications.pdf_queue import (
     enqueue_missing_pdf_jobs,
     enqueue_retry_pdf_job,
     overlay_pdf_job_state,
 )
-from app.logging_utils import structured_log
 from app.services.domains.publications.types import PublicationListItem
 
 logger = logging.getLogger(__name__)
@@ -30,7 +30,9 @@ async def schedule_missing_pdf_enrichment_for_user(
         rows=items,
         max_items=max_items,
     )
-    structured_log(logger, "info", "publications.enrichment.scheduled", user_id=user_id, publication_count=len(queued_ids))
+    structured_log(
+        logger, "info", "publications.enrichment.scheduled", user_id=user_id, publication_count=len(queued_ids)
+    )
     return len(queued_ids)
 
 
@@ -47,7 +49,14 @@ async def schedule_retry_pdf_enrichment_for_row(
         request_email=request_email,
         row=item,
     )
-    structured_log(logger, "info", "publications.enrichment.retry_scheduled", user_id=user_id, publication_id=item.publication_id, queued=queued)
+    structured_log(
+        logger,
+        "info",
+        "publications.enrichment.retry_scheduled",
+        user_id=user_id,
+        publication_id=item.publication_id,
+        queued=queued,
+    )
     return queued
 
 

app/services/domains/publications/listing.py

@@ -51,10 +51,7 @@ async def list_for_user(
             snapshot_before=snapshot_before,
         )
     )
-    rows = [
-        publication_list_item_from_row(row, latest_run_id=latest_run_id)
-        for row in result.all()
-    ]
+    rows = [publication_list_item_from_row(row, latest_run_id=latest_run_id) for row in result.all()]
     return await identifier_service.overlay_publication_items_with_display_identifiers(
         db_session,
         items=rows,

app/services/domains/publications/pdf_queue.py

@@ -1,9 +1,9 @@
 from __future__ import annotations
 
 import asyncio
-from dataclasses import dataclass
-from datetime import datetime, timedelta, timezone
 import logging
+from dataclasses import dataclass
+from datetime import UTC, datetime, timedelta
 
 from sqlalchemy import Select, and_, func, literal, or_, select, union_all
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -17,6 +17,7 @@ from app.db.models import (
     User,
 )
 from app.db.session import get_session_factory
+from app.logging_utils import structured_log
 from app.services.domains.publication_identifiers import application as identifier_service
 from app.services.domains.publication_identifiers.types import DisplayIdentifier
 from app.services.domains.publications.pdf_resolution_pipeline import (
@@ -27,7 +28,6 @@ from app.services.domains.unpaywall.application import (
     FAILURE_RESOLUTION_EXCEPTION,
     OaResolutionOutcome,
 )
-from app.logging_utils import structured_log
 from app.settings import settings
 
 PDF_STATUS_UNTRACKED = "untracked"
@@ -85,7 +85,7 @@ class PdfQueuePage:
 
 
 def _utcnow() -> datetime:
-    return datetime.now(timezone.utc)
+    return datetime.now(UTC)
 
 
 def _publication_ids(rows: list[PublicationListItem]) -> list[int]:
@@ -472,7 +472,13 @@ async def _resolve_publication_row(
         # Propagate upward so the batch loop can stop immediately.
         raise
     except Exception as exc:  # pragma: no cover - defensive network boundary
-        structured_log(logger, "warning", "publications.pdf_queue.resolve_failed", publication_id=row.publication_id, error=str(exc))
+        structured_log(
+            logger,
+            "warning",
+            "publications.pdf_queue.resolve_failed",
+            publication_id=row.publication_id,
+            error=str(exc),
+        )
         outcome = _failed_outcome(row=row)
         arxiv_rate_limited = False
     await _persist_outcome(
@@ -514,9 +520,19 @@ async def _run_resolution_task(
             )
             if arxiv_rate_limited and arxiv_lookup_allowed:
                 arxiv_lookup_allowed = False
-                structured_log(logger, "warning", "pdf_queue.arxiv_batch_disabled", detail="arXiv temporarily disabled for remaining batch after rate limit")
+                structured_log(
+                    logger,
+                    "warning",
+                    "pdf_queue.arxiv_batch_disabled",
+                    detail="arXiv temporarily disabled for remaining batch after rate limit",
+                )
         except OpenAlexBudgetExhaustedError:
-            structured_log(logger, "warning", "pdf_queue.budget_exhausted", detail="Stopping PDF resolution batch — OpenAlex daily budget exhausted")
+            structured_log(
+                logger,
+                "warning",
+                "pdf_queue.budget_exhausted",
+                detail="Stopping PDF resolution batch — OpenAlex daily budget exhausted",
+            )
             break
 
 
@@ -679,7 +695,7 @@ async def _missing_pdf_candidates(
     limit: int,
 ) -> list[PublicationListItem]:
     bounded_limit = max(1, min(int(limit), 5000))
-    now = datetime.now(timezone.utc)
+    now = datetime.now(UTC)
     cooldown_threshold = now - timedelta(days=7)
 
     result = await db_session.execute(
@@ -701,10 +717,7 @@ async def _missing_pdf_candidates(
         .order_by(Publication.updated_at.desc(), Publication.id.desc())
         .limit(bounded_limit)
     )
-    return [
-        _queue_candidate_from_publication(publication)
-        for publication in result.scalars()
-    ]
+    return [_queue_candidate_from_publication(publication) for publication in result.scalars()]
 
 
 async def enqueue_all_missing_pdf_jobs(
@@ -902,9 +915,7 @@ async def count_pdf_queue_items(
     if normalized_status == PDF_STATUS_UNTRACKED:
         result = await db_session.execute(_untracked_queue_count_select())
         return int(result.scalar_one() or 0)
-    tracked_result = await db_session.execute(
-        _tracked_queue_count_select(status=normalized_status)
-    )
+    tracked_result = await db_session.execute(_tracked_queue_count_select(status=normalized_status))
     tracked_count = int(tracked_result.scalar_one() or 0)
     if normalized_status is not None:
         return tracked_count
@@ -946,9 +957,7 @@ async def drain_ready_jobs(
     limit: int,
     max_attempts: int,
 ) -> int:
-    result = await db_session.execute(
-        select(User.id).where(User.is_active.is_(True)).order_by(User.id.asc()).limit(1)
-    )
+    result = await db_session.execute(select(User.id).where(User.is_active.is_(True)).order_by(User.id.asc()).limit(1))
     system_user_id = result.scalar_one_or_none()
     if system_user_id is None:
         return 0

app/services/domains/publications/pdf_resolution_pipeline.py

@@ -1,15 +1,15 @@
 from __future__ import annotations
 
-from dataclasses import dataclass
 import logging
+from dataclasses import dataclass
 from typing import Any
 
+from app.logging_utils import structured_log
 from app.services.domains.arxiv.application import ArxivRateLimitError
 from app.services.domains.arxiv.guards import arxiv_skip_reason_for_item
 from app.services.domains.openalex.client import OpenAlexBudgetExhaustedError
 from app.services.domains.publications.types import PublicationListItem
 from app.services.domains.unpaywall.application import OaResolutionOutcome, resolve_publication_oa_outcomes
-from app.logging_utils import structured_log
 from app.settings import settings
 
 logger = logging.getLogger(__name__)
@@ -66,6 +66,7 @@ async def _openalex_outcome(
         return None
 
     import re
+
     safe_title = re.sub(r"[^\w\s]", " ", row.title)
     safe_title = " ".join(safe_title.split())
     if not safe_title:
@@ -107,12 +108,24 @@ async def _arxiv_outcome(
     from app.services.domains.arxiv.application import discover_arxiv_id_for_publication
 
     if not allow_lookup:
-        structured_log(logger, "info", "pdf_resolution.arxiv_skipped", publication_id=int(row.publication_id), skip_reason="batch_arxiv_cooldown_active")
+        structured_log(
+            logger,
+            "info",
+            "pdf_resolution.arxiv_skipped",
+            publication_id=int(row.publication_id),
+            skip_reason="batch_arxiv_cooldown_active",
+        )
         return None
 
     skip_reason = arxiv_skip_reason_for_item(item=row)
     if skip_reason is not None:
-        structured_log(logger, "info", "pdf_resolution.arxiv_skipped", publication_id=int(row.publication_id), skip_reason=skip_reason)
+        structured_log(
+            logger,
+            "info",
+            "pdf_resolution.arxiv_skipped",
+            publication_id=int(row.publication_id),
+            skip_reason=skip_reason,
+        )
         return None
 
     try:
@@ -134,7 +147,6 @@ async def _arxiv_outcome(
     return None
 
 
-
 async def _oa_outcome(
     *,
     row: PublicationListItem,

app/services/domains/publications/queries.py

@@ -228,9 +228,7 @@ def publication_list_item_from_row(
         is_read=bool(is_read),
         is_favorite=bool(is_favorite),
         first_seen_at=created_at,
-        is_new_in_latest_run=(
-            latest_run_id is not None and int(first_seen_run_id or 0) == latest_run_id
-        ),
+        is_new_in_latest_run=(latest_run_id is not None and int(first_seen_run_id or 0) == latest_run_id),
     )
 
 

app/services/domains/publications/read_state.py

@@ -17,11 +17,7 @@ def _normalized_selection_pairs(selections: list[tuple[int, int]]) -> set[tuple[
 
 
 def _scoped_scholar_ids_query(*, user_id: int):
-    return (
-        select(ScholarProfile.id)
-        .where(ScholarProfile.user_id == user_id)
-        .scalar_subquery()
-    )
+    return select(ScholarProfile.id).where(ScholarProfile.user_id == user_id).scalar_subquery()
 
 
 async def mark_all_unread_as_read_for_user(

app/services/domains/runs/application.py

@@ -25,21 +25,21 @@ from app.services.domains.runs.types import (
 )
 
 __all__ = [
+    "QUEUE_STATUS_DROPPED",
     "QUEUE_STATUS_QUEUED",
     "QUEUE_STATUS_RETRYING",
-    "QUEUE_STATUS_DROPPED",
-    "QueueListItem",
     "QueueClearResult",
+    "QueueListItem",
     "QueueTransitionError",
+    "clear_queue_item_for_user",
+    "drop_queue_item_for_user",
     "extract_run_summary",
+    "get_manual_run_by_idempotency_key",
+    "get_queue_item_for_user",
+    "get_run_for_user",
+    "list_queue_items_for_user",
     "list_recent_runs_for_user",
     "list_runs_for_user",
-    "get_run_for_user",
-    "get_manual_run_by_idempotency_key",
-    "list_queue_items_for_user",
-    "get_queue_item_for_user",
-    "retry_queue_item_for_user",
-    "drop_queue_item_for_user",
-    "clear_queue_item_for_user",
     "queue_status_counts_for_user",
+    "retry_queue_item_for_user",
 ]

app/services/domains/runs/events.py

@@ -1,14 +1,16 @@
 import asyncio
 import json
 import logging
-from typing import Any, AsyncGenerator, Dict, Set
+from collections.abc import AsyncGenerator
+from typing import Any
 
 logger = logging.getLogger(__name__)
 
+
 class RunEventPublisher:
     def __init__(self) -> None:
         # Maps run_id to a set of subscriber queues
-        self._subscribers: Dict[int, Set[asyncio.Queue]] = {}
+        self._subscribers: dict[int, set[asyncio.Queue]] = {}
 
     def subscribe(self, run_id: int) -> asyncio.Queue:
         if run_id not in self._subscribers:
@@ -27,12 +29,9 @@ class RunEventPublisher:
     async def publish(self, run_id: int, event_type: str, data: dict[str, Any]) -> None:
         if run_id not in self._subscribers:
             return
-        
-        message = {
-            "type": event_type,
-            "data": data
-        }
-        
+
+        message = {"type": event_type, "data": data}
+
         # Fan-out to all active subscribers for this run
         for queue in list(self._subscribers[run_id]):
             try:
@@ -40,8 +39,10 @@ class RunEventPublisher:
             except asyncio.QueueFull:
                 logger.warning(f"Subscriber queue full for run {run_id}, dropping message")
 
+
 run_events = RunEventPublisher()
 
+
 async def event_generator(run_id: int) -> AsyncGenerator[str, None]:
     queue = run_events.subscribe(run_id)
     try:

app/services/domains/runs/queue_service.py

@@ -41,9 +41,7 @@ async def get_queue_item_for_user(
     queue_item_id: int,
 ) -> QueueListItem | None:
     result = await db_session.execute(
-        queue_item_select(user_id=user_id)
-        .where(IngestionQueueItem.id == queue_item_id)
-        .limit(1)
+        queue_item_select(user_id=user_id).where(IngestionQueueItem.id == queue_item_id).limit(1)
     )
     row = result.one_or_none()
     if row is None:

app/services/domains/runs/runs_service.py

@@ -35,9 +35,7 @@ async def list_runs_for_user(
         .limit(limit)
     )
     if failed_only:
-        stmt = stmt.where(
-            CrawlRun.status.in_([RunStatus.FAILED, RunStatus.PARTIAL_FAILURE])
-        )
+        stmt = stmt.where(CrawlRun.status.in_([RunStatus.FAILED, RunStatus.PARTIAL_FAILURE]))
     result = await db_session.execute(stmt)
     return list(result.scalars().all())
 

app/services/domains/runs/summary.py

@@ -24,9 +24,7 @@ def _summary_int_dict(summary: dict[str, Any], key: str) -> dict[str, int]:
     if not isinstance(value, dict):
         return {}
     return {
-        str(item_key): _safe_int(item_value, 0)
-        for item_key, item_value in value.items()
-        if isinstance(item_key, str)
+        str(item_key): _safe_int(item_value, 0) for item_key, item_value in value.items() if isinstance(item_key, str)
     }
 
 
@@ -34,11 +32,7 @@ def _summary_bool_dict(summary: dict[str, Any], key: str) -> dict[str, bool]:
     value = summary.get(key)
     if not isinstance(value, dict):
         return {}
-    return {
-        str(item_key): bool(item_value)
-        for item_key, item_value in value.items()
-        if isinstance(item_key, str)
-    }
+    return {str(item_key): bool(item_value) for item_key, item_value in value.items() if isinstance(item_key, str)}
 
 
 def _retry_counts(summary: dict[str, Any]) -> dict[str, int]:

app/services/domains/scholar/author_rows.py

@@ -79,9 +79,7 @@ class ScholarAuthorSearchParser(HTMLParser):
             return
 
         affiliation = normalize_space("".join(self._candidate["aff_parts"])) or None
-        email_domain = _extract_verified_email_domain(
-            normalize_space("".join(self._candidate["eml_parts"])) or None
-        )
+        email_domain = _extract_verified_email_domain(normalize_space("".join(self._candidate["eml_parts"])) or None)
         cited_by_text = normalize_space("".join(self._candidate["cby_parts"]))
         cited_by_match = re.search(r"\d+", cited_by_text)
         cited_by_count = int(cited_by_match.group(0)) if cited_by_match else None
@@ -97,10 +95,7 @@ class ScholarAuthorSearchParser(HTMLParser):
 
         profile_url = build_absolute_scholar_url(self._candidate["name_href"])
         if not profile_url:
-            profile_url = (
-                "https://scholar.google.com/citations"
-                f"?hl=en&user={scholar_id}"
-            )
+            profile_url = f"https://scholar.google.com/citations?hl=en&user={scholar_id}"
 
         self.candidates.append(
             ScholarSearchCandidate(

app/services/domains/scholar/parser.py

@@ -3,40 +3,29 @@ from __future__ import annotations
 from app.services.domains.scholar.author_rows import (
     ScholarAuthorSearchParser,
     count_author_search_markers,
-    parse_scholar_id_from_href,
 )
 from app.services.domains.scholar.parser_constants import SCRIPT_STYLE_RE
 from app.services.domains.scholar.parser_types import (
-    ParseState,
     ParsedAuthorSearchPage,
     ParsedProfilePage,
     PublicationCandidate,
     ScholarDomInvariantError,
     ScholarMalformedDataError,
-    ScholarParserError,
-    ScholarSearchCandidate,
 )
 from app.services.domains.scholar.parser_utils import (
     strip_tags,
 )
 from app.services.domains.scholar.profile_rows import (
-    ScholarRowParser,
     count_markers,
     extract_articles_range,
     extract_profile_image_url,
     extract_profile_name,
-    extract_rows,
     has_operation_error_banner,
     has_show_more_button,
-    parse_citation_count,
-    parse_cluster_id_from_href,
     parse_publications,
-    parse_year,
 )
 from app.services.domains.scholar.source import FetchResult
 from app.services.domains.scholar.state_detection import (
-    classify_block_or_captcha_reason,
-    classify_network_error_reason,
     detect_author_search_state,
     detect_state,
 )

app/services/domains/scholar/profile_rows.py

@@ -12,7 +12,6 @@ from app.services.domains.scholar.parser_types import PublicationCandidate
 from app.services.domains.scholar.parser_utils import (
     attr_class,
     attr_href,
-    attr_src,
     build_absolute_scholar_url,
     normalize_space,
     strip_tags,
@@ -260,7 +259,7 @@ def has_show_more_button(html: str) -> bool:
 
 def has_operation_error_banner(html: str) -> bool:
     lowered = html.lower()
-    if "id=\"gsc_a_err\"" not in lowered and "id='gsc_a_err'" not in lowered:
+    if 'id="gsc_a_err"' not in lowered and "id='gsc_a_err'" not in lowered:
         return False
     return "can't perform the operation now" in lowered or "cannot perform the operation now" in lowered
 

app/services/domains/scholar/source.py

@@ -17,18 +17,12 @@ SCHOLAR_PROFILE_URL = "https://scholar.google.com/citations"
 DEFAULT_PAGE_SIZE = 100
 
 DEFAULT_USER_AGENTS = [
-    (
-        "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
-        "(KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
-    ),
+    ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"),
     (
         "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7) AppleWebKit/605.1.15 "
         "(KHTML, like Gecko) Version/18.1 Safari/605.1.15"
     ),
-    (
-        "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) "
-        "Gecko/20100101 Firefox/131.0"
-    ),
+    ("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) Gecko/20100101 Firefox/131.0"),
 ]
 
 logger = logging.getLogger(__name__)
@@ -44,8 +38,7 @@ class FetchResult:
 
 
 class ScholarSource(Protocol):
-    async def fetch_profile_html(self, scholar_id: str) -> FetchResult:
-        ...
+    async def fetch_profile_html(self, scholar_id: str) -> FetchResult: ...
 
     async def fetch_profile_page_html(
         self,
@@ -53,16 +46,14 @@ class ScholarSource(Protocol):
         *,
         cstart: int,
         pagesize: int,
-    ) -> FetchResult:
-        ...
+    ) -> FetchResult: ...
 
     async def fetch_author_search_html(
         self,
         query: str,
         *,
         start: int,
-    ) -> FetchResult:
-        ...
+    ) -> FetchResult: ...
 
 
 class LiveScholarSource:
@@ -145,7 +136,9 @@ class LiveScholarSource:
     @staticmethod
     def _network_error_result(requested_url: str, exc: URLError) -> FetchResult:
         structured_log(
-            logger, "warning", "scholar_source.fetch_network_error",
+            logger,
+            "warning",
+            "scholar_source.fetch_network_error",
             requested_url=requested_url,
         )
         return FetchResult(
@@ -159,7 +152,9 @@ class LiveScholarSource:
     @staticmethod
     def _http_error_result(requested_url: str, exc: HTTPError) -> FetchResult:
         structured_log(
-            logger, "warning", "scholar_source.fetch_http_error",
+            logger,
+            "warning",
+            "scholar_source.fetch_http_error",
             requested_url=requested_url,
             status_code=exc.code,
         )
@@ -176,7 +171,9 @@ class LiveScholarSource:
         body = response.read().decode("utf-8", errors="replace")
         status_code = getattr(response, "status", 200)
         structured_log(
-            logger, "debug", "scholar_source.fetch_succeeded",
+            logger,
+            "debug",
+            "scholar_source.fetch_succeeded",
             requested_url=requested_url,
             status_code=status_code,
         )

app/services/domains/scholars/application.py

@@ -1,13 +1,11 @@
 from __future__ import annotations
 
 import asyncio
-from dataclasses import replace
-from datetime import datetime
-from datetime import timedelta
-from datetime import timezone
 import logging
 import os
 import random
+from dataclasses import replace
+from datetime import UTC, datetime, timedelta
 from uuid import uuid4
 
 from sqlalchemy import delete, func, select, text
@@ -15,9 +13,10 @@ from sqlalchemy.exc import IntegrityError
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.models import AuthorSearchCacheEntry, AuthorSearchRuntimeState, ScholarProfile
+from app.logging_utils import structured_log
 from app.services.domains.scholar.parser import (
-    ParseState,
     ParsedAuthorSearchPage,
+    ParseState,
     ScholarParserError,
     parse_author_search_page,
     parse_profile_page,
@@ -34,33 +33,30 @@ from app.services.domains.scholars.constants import (
     DEFAULT_AUTHOR_SEARCH_COOLDOWN_REJECTION_ALERT_THRESHOLD,
     DEFAULT_AUTHOR_SEARCH_COOLDOWN_SECONDS,
     DEFAULT_AUTHOR_SEARCH_INTERVAL_JITTER_SECONDS,
-    MAX_AUTHOR_SEARCH_LIMIT,
     DEFAULT_AUTHOR_SEARCH_MIN_INTERVAL_SECONDS,
     DEFAULT_AUTHOR_SEARCH_RETRY_ALERT_THRESHOLD,
+    MAX_AUTHOR_SEARCH_LIMIT,
     SEARCH_CACHED_BLOCK_REASON,
     SEARCH_COOLDOWN_REASON,
     SEARCH_DISABLED_REASON,
 )
-from app.logging_utils import structured_log
 from app.services.domains.scholars.exceptions import ScholarServiceError
 from app.services.domains.scholars.search_hints import (
     _merge_warnings,
     _policy_blocked_author_search_result,
     _trim_author_search_result,
-    resolve_profile_image,
-    scrape_state_hint,
 )
 from app.services.domains.scholars.uploads import (
     _ensure_upload_root,
     _resolve_upload_path,
     _safe_remove_upload,
-    resolve_upload_file_path,
 )
 from app.services.domains.scholars.validators import (
     normalize_display_name,
     normalize_profile_image_url,
     validate_scholar_id,
 )
+
 logger = logging.getLogger(__name__)
 
 
@@ -247,7 +243,7 @@ async def _cache_get_author_search_result(
         return None
     expires_at = entry.expires_at
     if expires_at.tzinfo is None:
-        expires_at = expires_at.replace(tzinfo=timezone.utc)
+        expires_at = expires_at.replace(tzinfo=UTC)
     if expires_at <= now_utc:
         await db_session.delete(entry)
         return None
@@ -305,27 +301,19 @@ async def _prune_author_search_cache(
     now_utc: datetime,
     max_entries: int,
 ) -> None:
-    await db_session.execute(
-        delete(AuthorSearchCacheEntry).where(AuthorSearchCacheEntry.expires_at <= now_utc)
-    )
+    await db_session.execute(delete(AuthorSearchCacheEntry).where(AuthorSearchCacheEntry.expires_at <= now_utc))
     bounded_max_entries = max(1, int(max_entries))
-    count_result = await db_session.execute(
-        select(func.count()).select_from(AuthorSearchCacheEntry)
-    )
+    count_result = await db_session.execute(select(func.count()).select_from(AuthorSearchCacheEntry))
     entry_count = int(count_result.scalar_one() or 0)
     overflow = max(0, entry_count - bounded_max_entries)
     if overflow <= 0:
         return
     stale_keys_result = await db_session.execute(
-        select(AuthorSearchCacheEntry.query_key)
-        .order_by(AuthorSearchCacheEntry.cached_at.asc())
-        .limit(overflow)
+        select(AuthorSearchCacheEntry.query_key).order_by(AuthorSearchCacheEntry.cached_at.asc()).limit(overflow)
     )
     stale_keys = [str(row[0]) for row in stale_keys_result.all()]
     if stale_keys:
-        await db_session.execute(
-            delete(AuthorSearchCacheEntry).where(AuthorSearchCacheEntry.query_key.in_(stale_keys))
-        )
+        await db_session.execute(delete(AuthorSearchCacheEntry).where(AuthorSearchCacheEntry.query_key.in_(stale_keys)))
 
 
 def _is_author_search_block_state(parsed: ParsedAuthorSearchPage) -> bool:
@@ -340,7 +328,7 @@ def _author_search_cooldown_remaining_seconds(
     if cooldown_until is None:
         return 0
     if cooldown_until.tzinfo is None:
-        cooldown_until = cooldown_until.replace(tzinfo=timezone.utc)
+        cooldown_until = cooldown_until.replace(tzinfo=UTC)
     remaining_seconds = int((cooldown_until - now_utc).total_seconds())
     return max(0, remaining_seconds)
 
@@ -432,7 +420,9 @@ def _normalize_author_search_inputs(query: str, limit: int) -> tuple[str, int, s
 
 def _disabled_search_result(*, normalized_query: str, bounded_limit: int) -> ParsedAuthorSearchPage:
     structured_log(
-        logger, "warning", "scholar_search.disabled_by_configuration",
+        logger,
+        "warning",
+        "scholar_search.disabled_by_configuration",
         query=normalized_query,
     )
     return _policy_blocked_author_search_result(
@@ -452,13 +442,15 @@ def _normalize_runtime_cooldown_state(
     cooldown_until = runtime_state.cooldown_until
     updated = False
     if cooldown_until.tzinfo is None:
-        cooldown_until = cooldown_until.replace(tzinfo=timezone.utc)
+        cooldown_until = cooldown_until.replace(tzinfo=UTC)
         runtime_state.cooldown_until = cooldown_until
         updated = True
     if now_utc < cooldown_until:
         return updated
     structured_log(
-        logger, "info", "scholar_search.cooldown_expired",
+        logger,
+        "info",
+        "scholar_search.cooldown_expired",
         cooldown_until_utc=cooldown_until.isoformat(),
     )
     runtime_state.cooldown_until = None
@@ -494,7 +486,9 @@ def _emit_cooldown_threshold_alert(
     if bool(runtime_state.cooldown_alert_emitted):
         return True
     structured_log(
-        logger, "error", "scholar_search.cooldown_rejection_threshold_exceeded",
+        logger,
+        "error",
+        "scholar_search.cooldown_rejection_threshold_exceeded",
         query=normalized_query,
         cooldown_rejection_count=int(runtime_state.cooldown_rejection_count),
         threshold=threshold,
@@ -518,7 +512,9 @@ def _cooldown_block_result(
         cooldown_rejection_alert_threshold=cooldown_rejection_alert_threshold,
     )
     structured_log(
-        logger, "warning", "scholar_search.cooldown_active",
+        logger,
+        "warning",
+        "scholar_search.cooldown_active",
         query=normalized_query,
         cooldown_remaining_seconds=cooldown_remaining_seconds,
         cooldown_until_utc=runtime_state.cooldown_until.isoformat() if runtime_state.cooldown_until else None,
@@ -549,7 +545,9 @@ async def _cache_hit_result(
     if cached is None:
         return None
     structured_log(
-        logger, "info", "scholar_search.cache_hit",
+        logger,
+        "info",
+        "scholar_search.cache_hit",
         query=normalized_query,
         state=cached.state.value,
         state_reason=cached.state_reason,
@@ -576,7 +574,7 @@ def _throttle_sleep_seconds(
     else:
         last_live_request_at = runtime_state.last_live_request_at
         if last_live_request_at.tzinfo is None:
-            last_live_request_at = last_live_request_at.replace(tzinfo=timezone.utc)
+            last_live_request_at = last_live_request_at.replace(tzinfo=UTC)
             runtime_state.last_live_request_at = last_live_request_at
             updated = True
         enforced_wait_seconds = (
@@ -603,7 +601,9 @@ async def _wait_for_author_search_throttle(
     if sleep_seconds <= 0.0:
         return updated
     structured_log(
-        logger, "info", "scholar_search.throttle_wait",
+        logger,
+        "info",
+        "scholar_search.throttle_wait",
         query=normalized_query,
         sleep_seconds=round(sleep_seconds, 3),
     )
@@ -659,7 +659,9 @@ def _with_retry_warnings(
     if retry_scheduled_count < threshold:
         return merged
     structured_log(
-        logger, "warning", "scholar_search.retry_threshold_exceeded",
+        logger,
+        "warning",
+        "scholar_search.retry_threshold_exceeded",
         query=normalized_query,
         retry_scheduled_count=retry_scheduled_count,
         threshold=threshold,
@@ -688,19 +690,23 @@ def _apply_block_circuit_breaker(
         return merged_parsed
     runtime_state.consecutive_blocked_count = int(runtime_state.consecutive_blocked_count) + 1
     structured_log(
-        logger, "warning", "scholar_search.block_detected",
+        logger,
+        "warning",
+        "scholar_search.block_detected",
         query=normalized_query,
         state_reason=merged_parsed.state_reason,
         consecutive_blocked_count=int(runtime_state.consecutive_blocked_count),
     )
     if int(runtime_state.consecutive_blocked_count) < max(1, int(cooldown_block_threshold)):
         return merged_parsed
-    runtime_state.cooldown_until = datetime.now(timezone.utc) + timedelta(seconds=max(60, int(cooldown_seconds)))
+    runtime_state.cooldown_until = datetime.now(UTC) + timedelta(seconds=max(60, int(cooldown_seconds)))
     runtime_state.consecutive_blocked_count = 0
     runtime_state.cooldown_rejection_count = 0
     runtime_state.cooldown_alert_emitted = False
     structured_log(
-        logger, "error", "scholar_search.cooldown_activated",
+        logger,
+        "error",
+        "scholar_search.cooldown_activated",
         query=normalized_query,
         cooldown_until_utc=runtime_state.cooldown_until.isoformat() if runtime_state.cooldown_until else None,
     )
@@ -739,11 +745,11 @@ async def _cooldown_or_cache_result(
 ) -> tuple[ParsedAuthorSearchPage | None, bool]:
     runtime_state_updated = _normalize_runtime_cooldown_state(
         runtime_state,
-        now_utc=datetime.now(timezone.utc),
+        now_utc=datetime.now(UTC),
     )
     cooldown_remaining_seconds = _author_search_cooldown_remaining_seconds(
         runtime_state,
-        datetime.now(timezone.utc),
+        datetime.now(UTC),
     )
     if cooldown_remaining_seconds > 0:
         return (
@@ -759,18 +765,35 @@ async def _cooldown_or_cache_result(
     cached_result = await _cache_hit_result(
         db_session,
         query_key=query_key,
-        now_utc=datetime.now(timezone.utc),
+        now_utc=datetime.now(UTC),
         normalized_query=normalized_query,
         bounded_limit=bounded_limit,
     )
     return cached_result, runtime_state_updated
 
 
-async def _perform_live_author_search(db_session: AsyncSession, *, source: ScholarSource, runtime_state: AuthorSearchRuntimeState, normalized_query: str, query_key: str, network_error_retries: int, retry_backoff_seconds: float, min_interval_seconds: float, interval_jitter_seconds: float, retry_alert_threshold: int, cooldown_block_threshold: int, cooldown_seconds: int, blocked_cache_ttl_seconds: int, cache_ttl_seconds: int, cache_max_entries: int) -> tuple[ParsedAuthorSearchPage, bool]:
+async def _perform_live_author_search(
+    db_session: AsyncSession,
+    *,
+    source: ScholarSource,
+    runtime_state: AuthorSearchRuntimeState,
+    normalized_query: str,
+    query_key: str,
+    network_error_retries: int,
+    retry_backoff_seconds: float,
+    min_interval_seconds: float,
+    interval_jitter_seconds: float,
+    retry_alert_threshold: int,
+    cooldown_block_threshold: int,
+    cooldown_seconds: int,
+    blocked_cache_ttl_seconds: int,
+    cache_ttl_seconds: int,
+    cache_max_entries: int,
+) -> tuple[ParsedAuthorSearchPage, bool]:
     runtime_state_updated = await _wait_for_author_search_throttle(
         runtime_state=runtime_state,
         normalized_query=normalized_query,
-        now_utc=datetime.now(timezone.utc),
+        now_utc=datetime.now(UTC),
         min_interval_seconds=min_interval_seconds,
         interval_jitter_seconds=interval_jitter_seconds,
     )
@@ -780,7 +803,7 @@ async def _perform_live_author_search(db_session: AsyncSession, *, source: Schol
         network_error_retries=network_error_retries,
         retry_backoff_seconds=retry_backoff_seconds,
     )
-    runtime_state.last_live_request_at = datetime.now(timezone.utc)
+    runtime_state.last_live_request_at = datetime.now(UTC)
     merged = _with_retry_warnings(
         parsed,
         retry_warnings=retry_warnings,
@@ -806,12 +829,30 @@ async def _perform_live_author_search(db_session: AsyncSession, *, source: Schol
         parsed=merged,
         ttl_seconds=float(ttl_seconds),
         max_entries=cache_max_entries,
-        now_utc=datetime.now(timezone.utc),
+        now_utc=datetime.now(UTC),
     )
     return merged, True
 
 
-async def search_author_candidates(*, source: ScholarSource, db_session: AsyncSession, query: str, limit: int, network_error_retries: int = 1, retry_backoff_seconds: float = 1.0, search_enabled: bool = True, cache_ttl_seconds: int = 21_600, blocked_cache_ttl_seconds: int = DEFAULT_AUTHOR_SEARCH_BLOCKED_CACHE_TTL_SECONDS, cache_max_entries: int = DEFAULT_AUTHOR_SEARCH_CACHE_MAX_ENTRIES, min_interval_seconds: float = DEFAULT_AUTHOR_SEARCH_MIN_INTERVAL_SECONDS, interval_jitter_seconds: float = DEFAULT_AUTHOR_SEARCH_INTERVAL_JITTER_SECONDS, cooldown_block_threshold: int = DEFAULT_AUTHOR_SEARCH_COOLDOWN_BLOCK_THRESHOLD, cooldown_seconds: int = DEFAULT_AUTHOR_SEARCH_COOLDOWN_SECONDS, retry_alert_threshold: int = DEFAULT_AUTHOR_SEARCH_RETRY_ALERT_THRESHOLD, cooldown_rejection_alert_threshold: int = DEFAULT_AUTHOR_SEARCH_COOLDOWN_REJECTION_ALERT_THRESHOLD) -> ParsedAuthorSearchPage:
+async def search_author_candidates(
+    *,
+    source: ScholarSource,
+    db_session: AsyncSession,
+    query: str,
+    limit: int,
+    network_error_retries: int = 1,
+    retry_backoff_seconds: float = 1.0,
+    search_enabled: bool = True,
+    cache_ttl_seconds: int = 21_600,
+    blocked_cache_ttl_seconds: int = DEFAULT_AUTHOR_SEARCH_BLOCKED_CACHE_TTL_SECONDS,
+    cache_max_entries: int = DEFAULT_AUTHOR_SEARCH_CACHE_MAX_ENTRIES,
+    min_interval_seconds: float = DEFAULT_AUTHOR_SEARCH_MIN_INTERVAL_SECONDS,
+    interval_jitter_seconds: float = DEFAULT_AUTHOR_SEARCH_INTERVAL_JITTER_SECONDS,
+    cooldown_block_threshold: int = DEFAULT_AUTHOR_SEARCH_COOLDOWN_BLOCK_THRESHOLD,
+    cooldown_seconds: int = DEFAULT_AUTHOR_SEARCH_COOLDOWN_SECONDS,
+    retry_alert_threshold: int = DEFAULT_AUTHOR_SEARCH_RETRY_ALERT_THRESHOLD,
+    cooldown_rejection_alert_threshold: int = DEFAULT_AUTHOR_SEARCH_COOLDOWN_REJECTION_ALERT_THRESHOLD,
+) -> ParsedAuthorSearchPage:
     normalized_query, bounded_limit, query_key = _normalize_author_search_inputs(query, limit)
     if not search_enabled:
         return _disabled_search_result(
@@ -924,17 +965,13 @@ async def set_profile_image_upload(
     normalized_content_type = (content_type or "").strip().lower()
     extension = ALLOWED_IMAGE_UPLOAD_CONTENT_TYPES.get(normalized_content_type)
     if extension is None:
-        raise ScholarServiceError(
-            "Unsupported image type. Use JPEG, PNG, WEBP, or GIF."
-        )
+        raise ScholarServiceError("Unsupported image type. Use JPEG, PNG, WEBP, or GIF.")
 
     if not image_bytes:
         raise ScholarServiceError("Uploaded image file is empty.")
 
     if len(image_bytes) > max_upload_bytes:
-        raise ScholarServiceError(
-            f"Uploaded image exceeds {max_upload_bytes} bytes."
-        )
+        raise ScholarServiceError(f"Uploaded image exceeds {max_upload_bytes} bytes.")
 
     upload_root = _ensure_upload_root(upload_dir, create=True)
     user_dir = upload_root / str(profile.user_id)

app/services/domains/scholars/constants.py

@@ -32,39 +32,25 @@ SEARCH_CACHED_BLOCK_REASON = "search_temporarily_disabled_from_cached_blocked_re
 
 STATE_REASON_HINTS: dict[str, str] = {
     SEARCH_DISABLED_REASON: (
-        "Scholar name search is currently disabled by service policy. "
-        "Add scholars by profile URL or Scholar ID."
+        "Scholar name search is currently disabled by service policy. Add scholars by profile URL or Scholar ID."
     ),
     SEARCH_COOLDOWN_REASON: (
         "Scholar name search is temporarily paused after repeated block responses. "
         "Use Scholar URL/ID adds until cooldown expires."
     ),
     SEARCH_CACHED_BLOCK_REASON: (
-        "A recent blocked response was cached to reduce traffic. "
-        "Retry later or add by Scholar URL/ID."
+        "A recent blocked response was cached to reduce traffic. Retry later or add by Scholar URL/ID."
     ),
     "network_dns_resolution_failed": (
-        "DNS resolution failed while reaching scholar.google.com. "
-        "Verify container DNS/network and retry."
-    ),
-    "network_timeout": (
-        "Request timed out before Google Scholar responded. "
-        "Increase delay/backoff and retry."
-    ),
-    "network_tls_error": (
-        "TLS handshake/certificate validation failed. "
-        "Verify outbound TLS/network configuration."
-    ),
-    "blocked_http_429_rate_limited": (
-        "Google Scholar rate-limited the request. "
-        "Slow request cadence and retry later."
+        "DNS resolution failed while reaching scholar.google.com. Verify container DNS/network and retry."
     ),
+    "network_timeout": ("Request timed out before Google Scholar responded. Increase delay/backoff and retry."),
+    "network_tls_error": ("TLS handshake/certificate validation failed. Verify outbound TLS/network configuration."),
+    "blocked_http_429_rate_limited": ("Google Scholar rate-limited the request. Slow request cadence and retry later."),
     "blocked_unusual_traffic_detected": (
-        "Google Scholar flagged traffic as unusual. "
-        "Increase delay/jitter and reduce concurrent scraping."
+        "Google Scholar flagged traffic as unusual. Increase delay/jitter and reduce concurrent scraping."
     ),
     "blocked_accounts_redirect": (
-        "Request was redirected to Google Account sign-in. "
-        "Treat as access block and retry later."
+        "Request was redirected to Google Account sign-in. Treat as access block and retry later."
     ),
 }

app/services/domains/scholars/search_hints.py

@@ -1,7 +1,7 @@
 from __future__ import annotations
 
 from app.db.models import ScholarProfile
-from app.services.domains.scholar.parser import ParseState, ParsedAuthorSearchPage
+from app.services.domains.scholar.parser import ParsedAuthorSearchPage, ParseState
 from app.services.domains.scholars.constants import (
     MAX_AUTHOR_SEARCH_LIMIT,
     STATE_REASON_HINTS,

app/services/domains/scholars/validators.py

@@ -27,9 +27,7 @@ def normalize_profile_image_url(value: str | None) -> str | None:
         return None
 
     if len(candidate) > MAX_IMAGE_URL_LENGTH:
-        raise ScholarServiceError(
-            f"Image URL must be {MAX_IMAGE_URL_LENGTH} characters or fewer."
-        )
+        raise ScholarServiceError(f"Image URL must be {MAX_IMAGE_URL_LENGTH} characters or fewer.")
 
     parsed = urlparse(candidate)
     if parsed.scheme.lower() not in {"http", "https"} or not parsed.netloc: