justinzeus/scholarr
1
0
Fork 0
Code Issues 1 Pull requests 18 Projects Releases Packages Wiki Activity Actions

docs: rebuild documentation from scratch with clean IA

Browse source 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Justin Visser 2026-02-27 09:39:27 +01:00
parent 6c31b331f7
commit 0c5b199b76
30 changed files with 1647 additions and 649 deletions
Download patch file Download diff file Expand all files Collapse all files

31
docs/reference/api-contract.md
View file

@ -1,31 +0,0 @@
# API Contract
## Envelope Invariant
All API responses under `/api/v1` use one of these envelopes:
- Success: `{"data": ..., "meta": {"request_id": "..."}}`
- Error: `{"error": {"code": "...", "message": "...", "details": ...}, "meta": {"request_id": "..."}}`
`meta.request_id` must be present for success and error responses.
Binary media assets are served outside `/api/v1` (for example, `GET /scholar-images/{scholar_profile_id}/upload`).
## Publications Semantics
- `GET /api/v1/publications` supports `mode=all|unread|latest`.
- `mode=new` is currently accepted as a compatibility alias for `latest`.
- Pagination controls: `page`, `page_size` (with backward-compatible `limit`/`offset` support).
- Pagination fields in response: `page`, `page_size`, `has_prev`, `has_next`, `total_count`.
- `unread` represents read-state (`is_read=false`).
- `latest` represents discovery-state (`first seen in the latest completed run`).
Publication payloads expose:
- `pub_url` (canonical scholar detail URL)
- `doi` (normalized DOI)
- `pdf_url` (resolved OA PDF when available)
## Scholar Portability
- `GET /api/v1/scholars/export` exports tracked scholars and scholar-publication link state.
- `POST /api/v1/scholars/import` imports that payload while preserving global publication deduplication.

177
docs/reference/api.md Normal file
View file

@ -0,0 +1,177 @@
---
title: API Contract
sidebar_position: 2
---
# API Contract
## Envelope Invariant
All API responses under `/api/v1` use one of these envelopes:
### Success
```json
{
"data": "...",
"meta": {
"request_id": "..."
}
}
```
### Error
```json
{
"error": {
"code": "...",
"message": "...",
"details": "..."
},
"meta": {
"request_id": "..."
}
}
```
`meta.request_id` is present on both success and error responses.
### Binary Assets
Binary media assets are served outside `/api/v1`:
```
GET /scholar-images/{scholar_profile_id}/upload
```
## DTO Structure
Scholarr uses strictly typed Pydantic V2 models serialized through OpenAPI v3 via FastAPI.
- `PublicationListItem` exposes a `.display_identifier` property that resolves the highest-confidence identifier regardless of backend origin, rather than a hardcoded `.doi`.
- Frontend TypeScript types are compiled from the OpenAPI spec to ensure type safety across the stack.
## Endpoints
### Auth
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/v1/auth/login` | Login (rate-limited sliding window) |
| `GET` | `/api/v1/auth/me` | Current session user |
| `GET` | `/api/v1/auth/csrf` | Bootstrap CSRF token |
| `POST` | `/api/v1/auth/change-password` | Change password |
| `POST` | `/api/v1/auth/logout` | Logout |
### Scholars
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/scholars` | List tracked scholars |
| `POST` | `/api/v1/scholars` | Create scholar (auto-enqueue, metadata hydration) |
| `GET` | `/api/v1/scholars/search` | Search author candidates by name |
| `PATCH` | `/api/v1/scholars/{id}/toggle` | Toggle enabled status |
| `DELETE` | `/api/v1/scholars/{id}` | Delete scholar |
| `PUT` | `/api/v1/scholars/{id}/image/url` | Update image URL |
| `POST` | `/api/v1/scholars/{id}/image/upload` | Upload image |
| `DELETE` | `/api/v1/scholars/{id}/image` | Clear image customization |
### Scholar Portability
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/scholars/export` | Export tracked scholars and publication link state |
| `POST` | `/api/v1/scholars/import` | Import scholars with global publication deduplication |
The export payload includes scholar metadata, tracked publication data, and link state (read/unread, favorites). Import preserves global deduplication.
### Publications
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/publications` | List publications (filtered, paginated) |
| `POST` | `/api/v1/publications/mark-all-read` | Mark all as read |
| `POST` | `/api/v1/publications/mark-read` | Mark selected as read |
| `POST` | `/api/v1/publications/{id}/retry-pdf` | Retry PDF resolution |
| `POST` | `/api/v1/publications/{id}/favorite` | Toggle favorite |
#### Publication Modes
`GET /api/v1/publications` supports a `mode` parameter:
| Mode | Description |
|------|-------------|
| `all` | All publications |
| `unread` | Publications with `is_read=false` |
| `latest` | Publications first seen in the latest completed run |
`mode=new` is accepted as a compatibility alias for `latest`.
#### Pagination
Query parameters: `page`, `page_size` (with backward-compatible `limit`/`offset` support).
Response pagination fields:
```json
{
"page": 1,
"page_size": 20,
"has_prev": false,
"has_next": true,
"total_count": 142
}
```
#### Publication Payload Fields
| Field | Description |
|-------|-------------|
| `pub_url` | Canonical scholar detail URL |
| `doi` | Normalized DOI |
| `pdf_url` | Resolved open-access PDF URL (when available) |
| `display_identifier` | Highest-confidence identifier regardless of source |
### Runs
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/runs` | List runs with safety state |
| `GET` | `/api/v1/runs/{run_id}` | Run detail with scholar results |
| `POST` | `/api/v1/runs/{run_id}/cancel` | Cancel active run |
| `POST` | `/api/v1/runs/manual` | Trigger manual run (idempotent, safety-checked) |
| `GET` | `/api/v1/runs/queue/items` | List queue items |
| `POST` | `/api/v1/runs/queue/{id}/retry` | Retry queue item |
| `POST` | `/api/v1/runs/queue/{id}/drop` | Drop queue item |
| `DELETE` | `/api/v1/runs/queue/{id}` | Clear queue item |
| `GET` | `/api/v1/runs/{run_id}/stream` | Stream run events (SSE) |
### Settings
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/settings` | Get user settings (with cooldown expiry check) |
| `PUT` | `/api/v1/settings` | Update settings (interval, delay, nav, API keys) |
### Admin - User Management
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/admin/users` | List all users |
| `POST` | `/api/v1/admin/users` | Create user |
| `PATCH` | `/api/v1/admin/users/{id}/active` | Set user active status |
| `POST` | `/api/v1/admin/users/{id}/reset-password` | Reset password |
### Admin - Database Operations
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/admin/db/integrity` | Get integrity report |
| `GET` | `/api/v1/admin/db/repair-jobs` | List repair jobs |
| `GET` | `/api/v1/admin/db/pdf-queue` | List PDF queue |
| `POST` | `/api/v1/admin/db/pdf-queue/{id}/requeue` | Requeue single PDF |
| `POST` | `/api/v1/admin/db/pdf-queue/requeue-all` | Bulk requeue missing PDFs |
| `POST` | `/api/v1/admin/db/repairs/publication-links` | Trigger link repair |
| `POST` | `/api/v1/admin/db/repairs/publication-near-duplicates` | Trigger dedup repair |
| `POST` | `/api/v1/admin/db/drop-all-publications` | Drop all publications (destructive) |

12
docs/reference/changelog.md Normal file
View file

@ -0,0 +1,12 @@
---
title: Changelog
sidebar_position: 4
---
# Changelog
This changelog is auto-generated by [python-semantic-release](https://python-semantic-release.readthedocs.io/).
Release versions follow [Semantic Versioning](https://semver.org/). Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/).
<!-- semantic-release will insert entries here -->

86
docs/reference/environment.md
View file

@ -1,68 +1,32 @@
# Environment Reference and Audit
---
title: Environment Variables
sidebar_position: 3
---
This document is the source-of-truth audit for what remains configurable versus internal defaults.
# Environment Variables Quick Reference
## Decision Summary
All environment variables are documented in detail in [Configuration](../user/configuration.md).
- Keep in `.env.example`: all runtime and compose controls currently exposed by `app/settings.py` plus compose/bootstrap extras.
- Move to internal defaults only: none at this time.
- Internal fallback behavior still exists in code to keep local/test startup resilient when values are omitted.
## Required Variables
## Configurable Variables (By Section)
| Variable | Description |
|----------|-------------|
| `POSTGRES_PASSWORD` | PostgreSQL password |
| `SESSION_SECRET_KEY` | Session cookie signing key (32+ characters) |
### Compose + Database
## Categories
`POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `DATABASE_URL`, `TEST_DATABASE_URL`, `SCHOLARR_IMAGE`
| Category | Key Variables |
|----------|--------------|
| Database | `POSTGRES_DB`, `POSTGRES_USER`, `DATABASE_URL`, `DATABASE_POOL_*` |
| Runtime | `APP_HOST`, `APP_PORT`, `MIGRATE_ON_START`, `FRONTEND_ENABLED` |
| Auth | `SESSION_SECRET_KEY`, `SESSION_COOKIE_SECURE`, `LOGIN_RATE_LIMIT_*` |
| Security | `SECURITY_HEADERS_ENABLED`, `SECURITY_CSP_*`, `SECURITY_STRICT_TRANSPORT_*` |
| Logging | `LOG_LEVEL`, `LOG_FORMAT`, `LOG_REQUESTS` |
| Scheduler | `SCHEDULER_ENABLED`, `SCHEDULER_TICK_SECONDS`, `SCHEDULER_*_BATCH_SIZE` |
| Ingestion | `INGESTION_*` (safety floors, cooldowns, retry policies) |
| Scholar | `SCHOLAR_IMAGE_*`, `SCHOLAR_NAME_SEARCH_*` |
| Enrichment | `UNPAYWALL_*`, `ARXIV_*`, `CROSSREF_*`, `OPENALEX_*`, `PDF_AUTO_RETRY_*` |
| Bootstrap | `BOOTSTRAP_ADMIN_*`, `DB_WAIT_*` |
### App Runtime + Networking
`APP_NAME`, `APP_HOST`, `APP_PORT`, `APP_HOST_PORT`, `APP_RELOAD`, `MIGRATE_ON_START`, `FRONTEND_ENABLED`, `FRONTEND_DIST_DIR`
### Database Pool
`DATABASE_POOL_MODE`, `DATABASE_POOL_SIZE`, `DATABASE_POOL_MAX_OVERFLOW`, `DATABASE_POOL_TIMEOUT_SECONDS`
### Frontend Dev Overrides
`FRONTEND_HOST_PORT`, `CHOKIDAR_USEPOLLING`, `VITE_DEV_API_PROXY_TARGET`
### Auth + Session
`SESSION_SECRET_KEY`, `SESSION_COOKIE_SECURE`, `LOGIN_RATE_LIMIT_ATTEMPTS`, `LOGIN_RATE_LIMIT_WINDOW_SECONDS`
### HTTP Security Headers + CSP
`SECURITY_HEADERS_ENABLED`, `SECURITY_X_CONTENT_TYPE_OPTIONS`, `SECURITY_X_FRAME_OPTIONS`, `SECURITY_REFERRER_POLICY`, `SECURITY_PERMISSIONS_POLICY`, `SECURITY_CROSS_ORIGIN_OPENER_POLICY`, `SECURITY_CROSS_ORIGIN_RESOURCE_POLICY`, `SECURITY_CSP_ENABLED`, `SECURITY_CSP_POLICY`, `SECURITY_CSP_DOCS_POLICY`, `SECURITY_CSP_REPORT_ONLY`, `SECURITY_STRICT_TRANSPORT_SECURITY_ENABLED`, `SECURITY_STRICT_TRANSPORT_SECURITY_MAX_AGE`, `SECURITY_STRICT_TRANSPORT_SECURITY_INCLUDE_SUBDOMAINS`, `SECURITY_STRICT_TRANSPORT_SECURITY_PRELOAD`
### Logging
`LOG_LEVEL`, `LOG_FORMAT`, `LOG_REQUESTS`, `LOG_UVICORN_ACCESS`, `LOG_REQUEST_SKIP_PATHS`, `LOG_REDACT_FIELDS`
### Scheduler + Ingestion Safety
`SCHEDULER_ENABLED`, `SCHEDULER_TICK_SECONDS`, `SCHEDULER_QUEUE_BATCH_SIZE`, `SCHEDULER_PDF_QUEUE_BATCH_SIZE`, `INGESTION_AUTOMATION_ALLOWED`, `INGESTION_MANUAL_RUN_ALLOWED`, `INGESTION_MIN_RUN_INTERVAL_MINUTES`, `INGESTION_MIN_REQUEST_DELAY_SECONDS`, `INGESTION_NETWORK_ERROR_RETRIES`, `INGESTION_RETRY_BACKOFF_SECONDS`, `INGESTION_RATE_LIMIT_RETRIES`, `INGESTION_RATE_LIMIT_BACKOFF_SECONDS`, `INGESTION_MAX_PAGES_PER_SCHOLAR`, `INGESTION_PAGE_SIZE`, `INGESTION_ALERT_BLOCKED_FAILURE_THRESHOLD`, `INGESTION_ALERT_NETWORK_FAILURE_THRESHOLD`, `INGESTION_ALERT_RETRY_SCHEDULED_THRESHOLD`, `INGESTION_SAFETY_COOLDOWN_BLOCKED_SECONDS`, `INGESTION_SAFETY_COOLDOWN_NETWORK_SECONDS`, `INGESTION_CONTINUATION_QUEUE_ENABLED`, `INGESTION_CONTINUATION_BASE_DELAY_SECONDS`, `INGESTION_CONTINUATION_MAX_DELAY_SECONDS`, `INGESTION_CONTINUATION_MAX_ATTEMPTS`
### Scholar Images + Name Search Safety
`SCHOLAR_IMAGE_UPLOAD_DIR`, `SCHOLAR_IMAGE_UPLOAD_MAX_BYTES`, `SCHOLAR_NAME_SEARCH_ENABLED`, `SCHOLAR_NAME_SEARCH_CACHE_TTL_SECONDS`, `SCHOLAR_NAME_SEARCH_BLOCKED_CACHE_TTL_SECONDS`, `SCHOLAR_NAME_SEARCH_CACHE_MAX_ENTRIES`, `SCHOLAR_NAME_SEARCH_MIN_INTERVAL_SECONDS`, `SCHOLAR_NAME_SEARCH_INTERVAL_JITTER_SECONDS`, `SCHOLAR_NAME_SEARCH_COOLDOWN_BLOCK_THRESHOLD`, `SCHOLAR_NAME_SEARCH_COOLDOWN_SECONDS`, `SCHOLAR_NAME_SEARCH_ALERT_RETRY_COUNT_THRESHOLD`, `SCHOLAR_NAME_SEARCH_ALERT_COOLDOWN_REJECTIONS_THRESHOLD`
### OA Enrichment + PDF Resolution
`UNPAYWALL_ENABLED`, `UNPAYWALL_EMAIL`, `UNPAYWALL_TIMEOUT_SECONDS`, `UNPAYWALL_MIN_INTERVAL_SECONDS`, `UNPAYWALL_MAX_ITEMS_PER_REQUEST`, `UNPAYWALL_RETRY_COOLDOWN_SECONDS`, `UNPAYWALL_PDF_DISCOVERY_ENABLED`, `UNPAYWALL_PDF_DISCOVERY_MAX_CANDIDATES`, `UNPAYWALL_PDF_DISCOVERY_MAX_HTML_BYTES`, `ARXIV_ENABLED`, `ARXIV_TIMEOUT_SECONDS`, `ARXIV_MIN_INTERVAL_SECONDS`, `ARXIV_RATE_LIMIT_COOLDOWN_SECONDS`, `ARXIV_DEFAULT_MAX_RESULTS`, `ARXIV_CACHE_TTL_SECONDS`, `ARXIV_CACHE_MAX_ENTRIES`, `ARXIV_MAILTO`, `PDF_AUTO_RETRY_INTERVAL_SECONDS`, `PDF_AUTO_RETRY_FIRST_INTERVAL_SECONDS`, `PDF_AUTO_RETRY_MAX_ATTEMPTS`, `CROSSREF_ENABLED`, `CROSSREF_MAX_ROWS`, `CROSSREF_TIMEOUT_SECONDS`, `CROSSREF_MIN_INTERVAL_SECONDS`, `CROSSREF_MAX_LOOKUPS_PER_REQUEST`, `OPENALEX_API_KEY`, `CROSSREF_API_TOKEN`, `CROSSREF_API_MAILTO`
### Startup Bootstrap + DB Wait
`BOOTSTRAP_ADMIN_ON_START`, `BOOTSTRAP_ADMIN_EMAIL`, `BOOTSTRAP_ADMIN_PASSWORD`, `BOOTSTRAP_ADMIN_FORCE_PASSWORD`, `DB_WAIT_TIMEOUT_SECONDS`, `DB_WAIT_INTERVAL_SECONDS`
## Drift Guard
CI enforces env parity with:
```bash
python3 scripts/check_env_contract.py
```
The check fails when:
- `app/settings.py` references a variable missing from `.env.example`.
- `.env.example` contains an unknown key (outside the approved compose/bootstrap extras).
- `.env.example` contains duplicate keys.
See [Configuration](../user/configuration.md) for the complete table with types, defaults, and descriptions.

16
docs/reference/overview.md
View file

@ -1,6 +1,14 @@
# Reference Documentation
---
title: Reference
sidebar_position: 1
---
Use this section for canonical contracts and configuration references.
# Reference
- [API Contract](./api-contract.md)
- [Environment Reference](./environment.md)
Quick-reference documentation for the scholarr API, environment variables, and release changelog.
| Document | Description |
|----------|-------------|
| [API Contract](api.md) | Envelope spec, endpoints, DTO structure, publication semantics |
| [Environment Variables](environment.md) | Quick-reference table linking to full configuration docs |
| [Changelog](changelog.md) | Auto-generated release history |