docs: rebuild documentation from scratch with clean IA
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
6c31b331f7
commit
0c5b199b76
30 changed files with 1647 additions and 649 deletions
|
|
@ -1,46 +1,80 @@
|
|||
---
|
||||
title: arXiv Runbook
|
||||
sidebar_position: 5
|
||||
---
|
||||
|
||||
# arXiv Operations Runbook
|
||||
|
||||
Use this runbook when arXiv lookups are slow, rate-limited, or behaving unexpectedly.
|
||||
|
||||
## Signals To Check
|
||||
- logs for `arxiv.request_scheduled`, `arxiv.request_completed`, `arxiv.cooldown_activated`, `arxiv.cache_hit`, `arxiv.cache_miss`
|
||||
## Signals to Check
|
||||
|
||||
- Logs for `arxiv.request_scheduled`, `arxiv.request_completed`, `arxiv.cooldown_activated`, `arxiv.cache_hit`, `arxiv.cache_miss`
|
||||
- `arxiv_runtime_state` row for cooldown and next-allowed timestamps
|
||||
- `arxiv_query_cache_entries` size and expiry churn
|
||||
|
||||
## Event Field Guide
|
||||
- `wait_seconds`: enforced pre-request delay from the global limiter.
|
||||
- `status_code`: upstream response code from arXiv.
|
||||
- `cooldown_remaining_seconds`: remaining cooldown when blocked or after 429.
|
||||
- `source_path`: caller path (`search` or `lookup_ids`).
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `wait_seconds` | Enforced pre-request delay from the global limiter |
|
||||
| `status_code` | Upstream response code from arXiv |
|
||||
| `cooldown_remaining_seconds` | Remaining cooldown when blocked or after 429 |
|
||||
| `source_path` | Caller path (`search` or `lookup_ids`) |
|
||||
|
||||
## Quick SQL Checks
|
||||
|
||||
### Runtime State
|
||||
|
||||
```sql
|
||||
SELECT state_key, next_allowed_at, cooldown_until, updated_at
|
||||
FROM arxiv_runtime_state;
|
||||
```
|
||||
|
||||
### Cache Status
|
||||
|
||||
```sql
|
||||
SELECT count(*) AS cache_rows, min(expires_at) AS earliest_expiry, max(expires_at) AS latest_expiry
|
||||
SELECT count(*) AS cache_rows,
|
||||
min(expires_at) AS earliest_expiry,
|
||||
max(expires_at) AS latest_expiry
|
||||
FROM arxiv_query_cache_entries;
|
||||
```
|
||||
|
||||
## Common Scenarios
|
||||
1. Repeated `arxiv.cooldown_activated` events:
|
||||
- Confirm recent `429` statuses in `arxiv.request_completed`.
|
||||
- Reduce caller pressure (check new title-quality/identifier guards are active).
|
||||
|
||||
### 1. Repeated `arxiv.cooldown_activated` Events
|
||||
|
||||
- Confirm recent `429` statuses in `arxiv.request_completed` logs.
|
||||
- Reduce caller pressure (check title-quality/identifier guards are active).
|
||||
- Temporarily raise `ARXIV_RATE_LIMIT_COOLDOWN_SECONDS` if upstream remains strict.
|
||||
|
||||
2. High request latency with few completions:
|
||||
### 2. High Request Latency with Few Completions
|
||||
|
||||
- Inspect `wait_seconds` in `arxiv.request_scheduled`.
|
||||
- Verify only one process path is repeatedly hitting arXiv (`source_path`).
|
||||
- Confirm cache is enabled (`ARXIV_CACHE_TTL_SECONDS > 0`) and effective (`cache_hit` appears).
|
||||
|
||||
3. Low cache effectiveness:
|
||||
### 3. Low Cache Effectiveness
|
||||
|
||||
- Validate normalized query behavior and caller churn.
|
||||
- Increase `ARXIV_CACHE_TTL_SECONDS` for stable workloads.
|
||||
- Increase `ARXIV_CACHE_MAX_ENTRIES` if heavy eviction is observed.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `ARXIV_ENABLED` | `1` | Enable arXiv lookups |
|
||||
| `ARXIV_TIMEOUT_SECONDS` | `3.0` | Request timeout |
|
||||
| `ARXIV_MIN_INTERVAL_SECONDS` | `4.0` | Min interval between requests |
|
||||
| `ARXIV_RATE_LIMIT_COOLDOWN_SECONDS` | `60.0` | Cooldown after 429 |
|
||||
| `ARXIV_DEFAULT_MAX_RESULTS` | `3` | Max results per query |
|
||||
| `ARXIV_CACHE_TTL_SECONDS` | `900` | Cache TTL (15 min) |
|
||||
| `ARXIV_CACHE_MAX_ENTRIES` | `512` | Max cached queries |
|
||||
| `ARXIV_MAILTO` | *(empty)* | Contact email for API headers |
|
||||
|
||||
## Safe Recovery
|
||||
|
||||
1. Pause automated ingestion if rate-limit storms persist.
|
||||
2. Let cooldown expire naturally; avoid manual burst retries.
|
||||
3. Resume and monitor event rates before restoring full load.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue