This commit is contained in:
Justin Visser 2026-02-19 22:50:22 +01:00
parent 710f7675c3
commit ba7976d935
10 changed files with 271 additions and 356 deletions

342
AGENTS.MD
View file

@ -1,314 +1,28 @@
# scholarr Agent Handbook # AI Agent Instructions: Scholarr
This file is the consolidated source of truth for project scope, constraints, scrape contract, and UI implementation flow. Adhere strictly to these constraints.
## 1. Product Objective ## 1. Coding Standards (Strict Enforcement)
* **Function Length:** Maximum 50 lines of code per function. Break down complex logic into small, testable, single-responsibility functions.
- Build a self-hosted scholar tracking system ("scholarr") with reliable, low-cost scraping and clear, actionable UI. * **DRY (Don't Repeat Yourself):** Abstract repetitive logic immediately. No duplicate boilerplate for database queries, API responses, or error handling.
- Keep the MVP scope intentionally small. * **Negative Space Programming:** Utilize explicit assertions and constraints to define invalid states. Fail fast and early. Do not allow silent failures or cascading malformed data, especially in DOM parsing.
- Prioritize reliability and ease of use over advanced/fancy processing. * **Cyclomatic Complexity:** Flatten logic. Use early returns and guard clauses instead of deep nesting.
## 2. Locked Constraints and Preferences ## 2. Domain Architecture & Data Model
* **Data Isolation:** Scholar tracking is **user-scoped**. Validate mapping/join tables; never assume global links between users and Scholar IDs.
- Multi-user system with strict tenant isolation. * **Data Deduplication:** Publications are **global records**. Deduplicate via Scholar cluster ID and normalized fingerprinting prior to database insertion.
- Users are admin-created only. * **State Management:** Visibility and "read/unread" states exist exclusively on the scholar-publication link table, not the global publication table.
- Users can change their own password. * **API Contract:** Exact envelope format required:
- Admin features are shown only to admin users. * Success: `{"data": ..., "meta": {"request_id": "..."}}`
- Same-origin cookie session model with CSRF protection. * Error: `{"error": {"code": "...", "message": "...", "details": ...}, "meta": {"request_id": "..."}}`
- Container-first development workflow (`docker compose`, `uv`).
- Test while developing; avoid shipping untested flows. ## 3. Scrape Safety & Rate Limiting (Immutable)
- Keep backend modular and DRY. These limits prevent IP bans and are not to be optimized away.
- UI is being rebuilt from scratch against API contracts. * **Minimum Delay:** Enforce `INGESTION_MIN_REQUEST_DELAY_SECONDS` (default 2s) between all external requests.
* **Anti-Detection:** Default to direct ID or profile URL ingestion. Name searches trigger CAPTCHAs.
## 3. Current Backend Architecture * **Cooldowns:** Respect `INGESTION_SAFETY_COOLDOWN_BLOCKED_SECONDS` (1800s) and `INGESTION_SAFETY_COOLDOWN_NETWORK_SECONDS` (900s) upon threshold breaches.
- Runtime: Python + FastAPI. ## 4. Current Environment & Stack
- DB: PostgreSQL + SQLAlchemy + Alembic migrations. * **Backend:** Python 3.12+, FastAPI, SQLAlchemy (Async/asyncpg), Alembic.
- Scheduler: background scheduler + continuation queue processing. * **Frontend:** TypeScript, React, Vite.
- Auth: * **Infrastructure:** Multi-stage Docker.
- Argon2 password hashing.
- Session cookie (`HttpOnly`, `SameSite=Lax`, secure flag configurable).
- CSRF required for unsafe methods via `X-CSRF-Token`.
- Logging:
- structured request logging with request IDs.
- redaction controls for sensitive fields.
## 4. Scraping Contract (Probe-Based)
Status: sufficient for MVP implementation with graceful degradation.
Target endpoint:
- `https://scholar.google.com/citations?hl=en&user=<scholar_id>`
Primary selectors:
- Row: `tr.gsc_a_tr`
- Title/link: `a.gsc_a_at`
- Citation count: `a.gsc_a_ac`
- Year: `span.gsc_a_h` (fallback regex on year cell text)
- Metadata lines: first/second `div.gs_gray` -> authors/venue
- Cluster ID: from `citation_for_view=<user>:<cluster_id>` in title URL
Required parser states:
- `ok`
- `no_results`
- `blocked_or_captcha`
- `layout_changed`
- `network_error`
Required page flags:
- `has_show_more_button`
- `articles_range`
Quality assumptions from probe:
- title/cluster_id/citation/authors coverage observed near 100%
- year and venue may be missing and must remain nullable
Guardrails:
- Never crash the full run when one scholar fails.
- Persist structured failure/debug information for diagnosis.
- Handle inaccessible/redirected IDs as states, not exceptions.
## 5. Current API Scope
Base path: `/api/v1`
Envelope model:
- success: `{"data": ..., "meta": {"request_id": "..."}}`
- error: `{"error": {"code": "...", "message": "...", "details": ...}, "meta": {"request_id": "..."}}`
Auth/session:
- `GET /api/v1/auth/csrf`
- `POST /api/v1/auth/login`
- `GET /api/v1/auth/me`
- `POST /api/v1/auth/change-password`
- `POST /api/v1/auth/logout`
Admin users:
- `GET /api/v1/admin/users`
- `POST /api/v1/admin/users`
- `PATCH /api/v1/admin/users/{id}/active`
- `POST /api/v1/admin/users/{id}/reset-password`
Scholars:
- `GET /api/v1/scholars`
- `POST /api/v1/scholars`
- `GET /api/v1/scholars/search`
- `PATCH /api/v1/scholars/{id}/toggle`
- `PUT /api/v1/scholars/{id}/image/url`
- `POST /api/v1/scholars/{id}/image/upload`
- `GET /api/v1/scholars/{id}/image/upload`
- `DELETE /api/v1/scholars/{id}/image`
- `DELETE /api/v1/scholars/{id}`
Settings:
- `GET /api/v1/settings`
- `PUT /api/v1/settings`
Runs/diagnostics/queue:
- `GET /api/v1/runs`
- `GET /api/v1/runs/{id}`
- `POST /api/v1/runs/manual` (`Idempotency-Key` supported)
- `GET /api/v1/runs/queue/items`
- `POST /api/v1/runs/queue/{id}/retry`
- `POST /api/v1/runs/queue/{id}/drop`
- `DELETE /api/v1/runs/queue/{id}`
Publications:
- `GET /api/v1/publications`
- `POST /api/v1/publications/mark-read`
- `POST /api/v1/publications/mark-all-read`
Ops:
- `GET /healthz`
## 5.1 Runtime Semantics (Locked)
Manual run idempotency:
- `Idempotency-Key` is optional on `POST /api/v1/runs/manual`.
- Idempotency scope is `(user_id, idempotency_key)`.
- Replaying the same key for the same user returns the same run payload after completion.
- If the keyed run is still in progress, respond with `409 run_in_progress` and include `run_id`.
- Enforcement is database-backed, not best-effort in-memory logic.
- Frontend triggers generate keys internally; key input is not user-facing.
Continuation queue attempt policy:
- `attempt_count` is a failed-attempt budget, not a total execution counter.
- Increment attempts only when a continuation execution fails to make progress.
- Reset attempts to `0` after successful progress (success or resumable partial progress).
- Drop queue items only after failed-attempt threshold is reached.
Scholar no-change skip policy:
- Persist first-page fingerprint snapshots per scholar.
- If the first-page fingerprint is unchanged for a normal run (`cstart=0`), skip deep pagination/upsert work.
- Record run scholar result `state_reason` as `no_change_initial_page_signature`.
Scholar create naming policy:
- Do not accept custom display names during scholar create.
- Hydrate names from scraped profile metadata.
Scholar name-search safety policy:
- Treat name search as best-effort only.
- Apply single-flight execution, pacing/jitter, cache, and cooldown circuit breaker after repeated blocked responses.
- Prefer Scholar URL/ID adds for reliable ingestion onboarding.
## 6. Required UI Behavior (MVP)
Core UX entities:
- scholars
- publications (`all` and `new`)
- runs + run diagnostics
- queue visibility/actions
- settings/account
- admin users (role-gated)
Critical semantics:
- Keep `is_new_in_latest_run` separate from `is_read`.
- First-time ingestion should not imply user read-state.
- Surface partial runs and warnings clearly.
- Show loading, empty, and error states explicitly.
- Support both light and dark modes with consistent status semantics.
## 7. UI Information Architecture
Public:
- Login
Authenticated:
- Dashboard
- Scholars
- Publications
- Settings
- Admin Users (admin only)
## 8. End-to-End UI Flow Plan
## 8.1 Session bootstrap
1. On load call `GET /api/v1/auth/csrf`.
2. Call `GET /api/v1/auth/me`.
3. Route to login or dashboard based on auth state.
## 8.2 Login/logout
1. Login form submits `POST /api/v1/auth/login` with `X-CSRF-Token`.
2. Store CSRF token from response for future unsafe requests.
3. Logout calls `POST /api/v1/auth/logout`.
## 8.3 Dashboard
1. Fetch latest publications summary from `GET /api/v1/publications?mode=new&limit=20`.
2. Fetch recent runs from `GET /api/v1/runs?limit=5`.
3. Fetch queue status from `GET /api/v1/runs/queue/items?limit=200` (aggregate in UI).
4. Show:
- new publications count (`new_count`)
- total tracked publications (`total_count`)
- latest run status/started time
- queue badges (`queued`, `retrying`, `dropped`)
5. Admin users additionally see a diagnostics shortcut block to full runs/queue page.
6. Every dashboard error panel must display request-id for support.
## 8.4 Scholars
1. List via `GET /api/v1/scholars`.
2. Add/toggle/delete actions mapped to scholar endpoints.
3. Search by name via `GET /api/v1/scholars/search` with candidate add actions.
4. Profile image controls:
- scraped image fallback from Scholar metadata
- custom URL override
- image upload + reset to scraped/default
5. Per-row quick links:
- all publications filtered by scholar
- new publications filtered by scholar
## 8.5 Publications
1. Default view mode is `new` (`GET /api/v1/publications?mode=new`).
2. Support mode toggle:
- `new`: scoped to latest completed run
- `all`: full history
3. Support scholar filter via `scholar_profile_id` query param.
4. Render both badges independently:
- `is_new_in_latest_run`
- `is_read`
5. "Mark all read" action calls `POST /api/v1/publications/mark-all-read` with CSRF.
6. Show explicit empty states:
- no publications tracked
- no new publications in latest run
- filtered scholar has no items
7. Surface run recency context when mode is `new` (latest completed run semantics).
## 8.6 Runs and diagnostics (admin)
1. List runs from `GET /api/v1/runs`.
2. Run details from `GET /api/v1/runs/{id}`:
- scholar result state/reason
- warnings
- page logs + attempt logs
- debug metadata
3. Queue operations from queue endpoints (`retry`, `drop`, `clear`).
## 8.7 Settings + account
1. User ingest settings from `/api/v1/settings`.
2. Password change via `/api/v1/auth/change-password`.
## 8.8 Admin user management
1. Show section only when `current_user.is_admin=true`.
2. Use admin user endpoints for create/activate/deactivate/reset password.
## 9. Development Method
- API-contract-first frontend.
- Vertical slices (complete flow before moving on).
- Shared API client module:
- always send credentials
- inject CSRF header for unsafe methods
- normalize envelope parsing and error handling
- Keep frontend modules separate (`auth`, `scholars`, `publications`, `runs`, `settings`, `admin`).
- Write tests while implementing each slice.
## 10. Definition of Done (UI Readiness)
- All main flows above implemented and mapped to existing API endpoints.
- Role-gated admin experience enforced in UI.
- Distinct new-vs-read publication semantics visible.
- Run/queue diagnostics visible and actionable.
- Reliable loading/empty/error states with request-id surfaced for support.
- No dependence on removed server-rendered UI layer.
## 11. Open Items (Known Future Work)
- API contract freeze/changelog discipline for external UI clients.
- Additional API-first smoke coverage for end-to-end UI critical flows.
- Optional background image enrichment/refresh jobs for already-tracked scholars.
## 12. UI Rebuild Program Artifacts
- Phase 0 through Phase 3 decisions are consolidated into this handbook and `README.md`.
- Local scratch notes, probes, and temporary planning artifacts are intentionally not tracked in git.
## 13. Frontend Scaffold Status (Implemented)
- Frontend scaffold path: `frontend/`
- Implemented core:
- App shell + routing + auth/role route guards
- Shared API client with envelope parsing, credentials, and CSRF header injection
- Pinia stores for auth, theme, and global UI errors
- Tailwind-first design foundation with light/dark themes and pre-paint theme initialization
- Page shells for login, dashboard, scholars, publications, runs, run detail, settings, and admin users
- Initial feature modules under `frontend/src/features/*` mapped to current API endpoints
- Deployment boundary:
- No in-repo reverse-proxy implementation
- Frontend edge/proxy routing is managed externally (for example, Caddy)
## 14. Frontend Quality Gates (Phase 3 Implemented)
- CI now includes frontend quality gates in `.github/workflows/ci.yml`:
- API contract drift check (`scripts/check_frontend_api_contract.py`)
- frontend typecheck (`npm run typecheck`)
- frontend unit tests (`npm run test:run`)
- frontend build (`npm run build`)
- Frontend test harness:
- `vitest` configured in `frontend/vitest.config.ts`
- baseline tests in `frontend/src/lib/api/envelope.test.ts` and `frontend/src/lib/api/errors.test.ts`

View file

@ -266,3 +266,13 @@ Scheduled fixture probes run in GitHub Actions via `.github/workflows/scheduled-
- Base path: `/api/v1` - Base path: `/api/v1`
- Success envelope: `{"data": ..., "meta": {"request_id": "..."}}` - Success envelope: `{"data": ..., "meta": {"request_id": "..."}}`
- Error envelope: `{"error": {"code": "...", "message": "...", "details": ...}, "meta": {"request_id": "..."}}` - Error envelope: `{"error": {"code": "...", "message": "...", "details": ...}, "meta": {"request_id": "..."}}`
### Publications semantics
- `GET /api/v1/publications` supports `mode=all|unread|latest` (plus temporary alias `mode=new`).
- `unread` = actionable read-state (`is_read=false`).
- `latest` = discovery-state (`first seen in the latest completed check`).
- Response counters:
- `unread_count`: unread publications in current scope.
- `latest_count`: publications discovered in latest completed check.
- `new_count`: compatibility alias for `latest_count` (temporary).

View file

@ -31,13 +31,13 @@ router = APIRouter(prefix="/publications", tags=["api-publications"])
) )
async def list_publications( async def list_publications(
request: Request, request: Request,
mode: Literal["all", "new"] | None = Query(default=None), mode: Literal["all", "unread", "latest", "new"] | None = Query(default=None),
scholar_profile_id: int | None = Query(default=None, ge=1), scholar_profile_id: int | None = Query(default=None, ge=1),
limit: int = Query(default=300, ge=1, le=1000), limit: int = Query(default=300, ge=1, le=1000),
db_session: AsyncSession = Depends(get_db_session), db_session: AsyncSession = Depends(get_db_session),
current_user: User = Depends(get_api_current_user), current_user: User = Depends(get_api_current_user),
): ):
resolved_mode = publication_service.resolve_mode(mode) resolved_mode = publication_service.resolve_publication_view_mode(mode)
selected_scholar_id = scholar_profile_id selected_scholar_id = scholar_profile_id
if selected_scholar_id is not None: if selected_scholar_id is not None:
selected_profile = await scholar_service.get_user_scholar_by_id( selected_profile = await scholar_service.get_user_scholar_by_id(
@ -59,10 +59,14 @@ async def list_publications(
scholar_profile_id=selected_scholar_id, scholar_profile_id=selected_scholar_id,
limit=limit, limit=limit,
) )
new_count = await publication_service.count_for_user( unread_count = await publication_service.count_unread_for_user(
db_session,
user_id=current_user.id,
scholar_profile_id=selected_scholar_id,
)
latest_count = await publication_service.count_latest_for_user(
db_session, db_session,
user_id=current_user.id, user_id=current_user.id,
mode=publication_service.MODE_NEW,
scholar_profile_id=selected_scholar_id, scholar_profile_id=selected_scholar_id,
) )
total_count = await publication_service.count_for_user( total_count = await publication_service.count_for_user(
@ -76,7 +80,9 @@ async def list_publications(
data={ data={
"mode": resolved_mode, "mode": resolved_mode,
"selected_scholar_profile_id": selected_scholar_id, "selected_scholar_profile_id": selected_scholar_id,
"new_count": new_count, "unread_count": unread_count,
"latest_count": latest_count,
"new_count": latest_count,
"total_count": total_count, "total_count": total_count,
"publications": [ "publications": [
{ {

View file

@ -515,6 +515,8 @@ class PublicationItemData(BaseModel):
class PublicationsListData(BaseModel): class PublicationsListData(BaseModel):
mode: str mode: str
selected_scholar_profile_id: int | None selected_scholar_profile_id: int | None
unread_count: int
latest_count: int
new_count: int new_count: int
total_count: int total_count: int
publications: list[PublicationItemData] publications: list[PublicationItemData]

View file

@ -14,8 +14,10 @@ from app.db.models import (
ScholarPublication, ScholarPublication,
) )
MODE_NEW = "new"
MODE_ALL = "all" MODE_ALL = "all"
MODE_UNREAD = "unread"
MODE_LATEST = "latest"
MODE_NEW = "new" # compatibility alias for MODE_LATEST
@dataclass(frozen=True) @dataclass(frozen=True)
@ -45,10 +47,16 @@ class UnreadPublicationItem:
pub_url: str | None pub_url: str | None
def resolve_publication_view_mode(value: str | None) -> str:
if value == MODE_UNREAD:
return MODE_UNREAD
if value in {MODE_LATEST, MODE_NEW}:
return MODE_LATEST
return MODE_ALL
def resolve_mode(value: str | None) -> str: def resolve_mode(value: str | None) -> str:
if value == MODE_ALL: return resolve_publication_view_mode(value)
return MODE_ALL
return MODE_NEW
async def get_latest_completed_run_id_for_user( async def get_latest_completed_run_id_for_user(
@ -99,8 +107,10 @@ def publications_query(
) )
if scholar_profile_id is not None: if scholar_profile_id is not None:
stmt = stmt.where(ScholarProfile.id == scholar_profile_id) stmt = stmt.where(ScholarProfile.id == scholar_profile_id)
if mode == MODE_NEW: if mode == MODE_UNREAD:
# "New" means discovered in the latest completed run. stmt = stmt.where(ScholarPublication.is_read.is_(False))
if mode == MODE_LATEST:
# "Latest" means discovered in the latest completed run.
if latest_run_id is None: if latest_run_id is None:
stmt = stmt.where(False) stmt = stmt.where(False)
else: else:
@ -116,7 +126,7 @@ async def list_for_user(
scholar_profile_id: int | None = None, scholar_profile_id: int | None = None,
limit: int = 300, limit: int = 300,
) -> list[PublicationListItem]: ) -> list[PublicationListItem]:
resolved_mode = resolve_mode(mode) resolved_mode = resolve_publication_view_mode(mode)
latest_run_id = await get_latest_completed_run_id_for_user( latest_run_id = await get_latest_completed_run_id_for_user(
db_session, db_session,
user_id=user_id, user_id=user_id,
@ -177,11 +187,11 @@ async def list_unread_for_user(
result = await db_session.execute( result = await db_session.execute(
publications_query( publications_query(
user_id=user_id, user_id=user_id,
mode=MODE_ALL, mode=MODE_UNREAD,
latest_run_id=None, latest_run_id=None,
scholar_profile_id=None, scholar_profile_id=None,
limit=limit, limit=limit,
).where(ScholarPublication.is_read.is_(False)) )
) )
rows = result.all() rows = result.all()
items: list[UnreadPublicationItem] = [] items: list[UnreadPublicationItem] = []
@ -224,7 +234,7 @@ async def list_new_for_latest_run_for_user(
rows = await list_for_user( rows = await list_for_user(
db_session, db_session,
user_id=user_id, user_id=user_id,
mode=MODE_NEW, mode=MODE_LATEST,
scholar_profile_id=None, scholar_profile_id=None,
limit=limit, limit=limit,
) )
@ -250,7 +260,7 @@ async def count_for_user(
mode: str = MODE_ALL, mode: str = MODE_ALL,
scholar_profile_id: int | None = None, scholar_profile_id: int | None = None,
) -> int: ) -> int:
resolved_mode = resolve_mode(mode) resolved_mode = resolve_publication_view_mode(mode)
latest_run_id = await get_latest_completed_run_id_for_user( latest_run_id = await get_latest_completed_run_id_for_user(
db_session, db_session,
user_id=user_id, user_id=user_id,
@ -263,7 +273,9 @@ async def count_for_user(
) )
if scholar_profile_id is not None: if scholar_profile_id is not None:
stmt = stmt.where(ScholarProfile.id == scholar_profile_id) stmt = stmt.where(ScholarProfile.id == scholar_profile_id)
if resolved_mode == MODE_NEW: if resolved_mode == MODE_UNREAD:
stmt = stmt.where(ScholarPublication.is_read.is_(False))
if resolved_mode == MODE_LATEST:
if latest_run_id is None: if latest_run_id is None:
return 0 return 0
stmt = stmt.where(ScholarPublication.first_seen_run_id == latest_run_id) stmt = stmt.where(ScholarPublication.first_seen_run_id == latest_run_id)
@ -271,6 +283,34 @@ async def count_for_user(
return int(result.scalar_one() or 0) return int(result.scalar_one() or 0)
async def count_unread_for_user(
db_session: AsyncSession,
*,
user_id: int,
scholar_profile_id: int | None = None,
) -> int:
return await count_for_user(
db_session,
user_id=user_id,
mode=MODE_UNREAD,
scholar_profile_id=scholar_profile_id,
)
async def count_latest_for_user(
db_session: AsyncSession,
*,
user_id: int,
scholar_profile_id: int | None = None,
) -> int:
return await count_for_user(
db_session,
user_id=user_id,
mode=MODE_LATEST,
scholar_profile_id=scholar_profile_id,
)
async def mark_all_unread_as_read_for_user( async def mark_all_unread_as_read_for_user(
db_session: AsyncSession, db_session: AsyncSession,
*, *,

View file

@ -41,7 +41,7 @@ function countQueueStatuses(statuses: string[]): QueueHealth {
export async function fetchDashboardSnapshot(): Promise<DashboardSnapshot> { export async function fetchDashboardSnapshot(): Promise<DashboardSnapshot> {
const [publications, runsPayload, queueItems] = await Promise.all([ const [publications, runsPayload, queueItems] = await Promise.all([
listPublications({ mode: "new", limit: 20 }), listPublications({ mode: "latest", limit: 20 }),
listRuns({ limit: 20 }), listRuns({ limit: 20 }),
listQueueItems(200), listQueueItems(200),
]); ]);
@ -50,7 +50,7 @@ export async function fetchDashboardSnapshot(): Promise<DashboardSnapshot> {
const queueHealth = countQueueStatuses(queueItems.map((item) => item.status)); const queueHealth = countQueueStatuses(queueItems.map((item) => item.status));
return { return {
newCount: publications.new_count, newCount: publications.latest_count,
totalCount: publications.total_count, totalCount: publications.total_count,
mode: publications.mode, mode: publications.mode,
latestRun: runs[0] ?? null, latestRun: runs[0] ?? null,

View file

@ -1,6 +1,6 @@
import { apiRequest } from "@/lib/api/client"; import { apiRequest } from "@/lib/api/client";
export type PublicationMode = "all" | "new"; export type PublicationMode = "all" | "unread" | "latest";
export interface PublicationItem { export interface PublicationItem {
publication_id: number; publication_id: number;
@ -19,6 +19,9 @@ export interface PublicationItem {
export interface PublicationsResult { export interface PublicationsResult {
mode: PublicationMode; mode: PublicationMode;
selected_scholar_profile_id: number | null; selected_scholar_profile_id: number | null;
unread_count: number;
latest_count: number;
// Compatibility alias for latest_count; retained while API clients migrate.
new_count: number; new_count: number;
total_count: number; total_count: number;
publications: PublicationItem[]; publications: PublicationItem[];

View file

@ -186,7 +186,7 @@ const visibleCount = computed(() => sortedPublications.value.length);
const visibleUnreadCount = computed(() => visibleUnreadKeys.value.size); const visibleUnreadCount = computed(() => visibleUnreadKeys.value.size);
const actionBusy = computed(() => loading.value || publishingAll.value || publishingSelected.value); const actionBusy = computed(() => loading.value || publishingAll.value || publishingSelected.value);
const showingEmptyList = computed(() => Boolean(listState.value) && sortedPublications.value.length === 0); const showingEmptyList = computed(() => Boolean(listState.value) && sortedPublications.value.length === 0);
const modeLabel = computed(() => (mode.value === "new" ? "Needs review" : "All records")); const modeLabel = computed(() => (mode.value === "unread" ? "Unread" : "All records"));
const emptyTitle = computed(() => const emptyTitle = computed(() =>
searchQuery.value.trim().length > 0 ? "No publications match this search" : "No publications found", searchQuery.value.trim().length > 0 ? "No publications match this search" : "No publications found",
@ -196,8 +196,8 @@ const emptyBody = computed(() => {
if (searchQuery.value.trim().length > 0) { if (searchQuery.value.trim().length > 0) {
return "Try another title, scholar, venue, or year."; return "Try another title, scholar, venue, or year.";
} }
if (mode.value === "new") { if (mode.value === "unread") {
return `No publications currently need review for ${selectedScholarName.value}.`; return `No unread publications for ${selectedScholarName.value}.`;
} }
return `No publication records for ${selectedScholarName.value}.`; return `No publication records for ${selectedScholarName.value}.`;
}); });
@ -340,14 +340,14 @@ async function onMarkSelectedRead(): Promise<void> {
try { try {
const response = await markSelectedRead(selections); const response = await markSelectedRead(selections);
successMessage.value = `${response.updated_count} publication${response.updated_count === 1 ? "" : "s"} marked as reviewed.`; successMessage.value = `${response.updated_count} publication${response.updated_count === 1 ? "" : "s"} marked as read.`;
await loadPublications(); await loadPublications();
} catch (error) { } catch (error) {
if (error instanceof ApiRequestError) { if (error instanceof ApiRequestError) {
errorMessage.value = error.message; errorMessage.value = error.message;
errorRequestId.value = error.requestId; errorRequestId.value = error.requestId;
} else { } else {
errorMessage.value = "Unable to mark selected publications as reviewed."; errorMessage.value = "Unable to mark selected publications as read.";
} }
} finally { } finally {
publishingSelected.value = false; publishingSelected.value = false;
@ -362,14 +362,14 @@ async function onMarkAllRead(): Promise<void> {
try { try {
const response = await markAllRead(); const response = await markAllRead();
successMessage.value = `${response.updated_count} publication${response.updated_count === 1 ? "" : "s"} marked as reviewed.`; successMessage.value = `${response.updated_count} publication${response.updated_count === 1 ? "" : "s"} marked as read.`;
await loadPublications(); await loadPublications();
} catch (error) { } catch (error) {
if (error instanceof ApiRequestError) { if (error instanceof ApiRequestError) {
errorMessage.value = error.message; errorMessage.value = error.message;
errorRequestId.value = error.requestId; errorRequestId.value = error.requestId;
} else { } else {
errorMessage.value = "Unable to mark publications as reviewed."; errorMessage.value = "Unable to mark publications as read.";
} }
} finally { } finally {
publishingAll.value = false; publishingAll.value = false;
@ -400,7 +400,7 @@ watch(
<template> <template>
<AppPage <AppPage
title="Publications" title="Publications"
subtitle="Filter and review discovered publications, then mark what you have handled so the next check stays focused." subtitle="Filter discovered publications, then mark what you have read so upcoming checks stay focused."
> >
<AppCard class="space-y-4"> <AppCard class="space-y-4">
<div class="flex flex-wrap items-center justify-between gap-2"> <div class="flex flex-wrap items-center justify-between gap-2">
@ -408,11 +408,11 @@ watch(
<div class="flex items-center gap-1"> <div class="flex items-center gap-1">
<h2 class="text-lg font-semibold text-ink-primary">What you can do here</h2> <h2 class="text-lg font-semibold text-ink-primary">What you can do here</h2>
<AppHelpHint <AppHelpHint
text="Publications are discovered records from tracked scholar profiles. Review mode helps focus only on newly detected items." text="Publications are records discovered from tracked scholar profiles. Unread mode focuses only on items you have not marked as read."
/> />
</div> </div>
<p class="text-sm text-secondary"> <p class="text-sm text-secondary">
Select a scholar or time scope, search within results, and mark items reviewed when you are done. Select a scholar or scope, search within results, and mark items as read when you are done.
</p> </p>
</div> </div>
@ -426,12 +426,12 @@ watch(
<span>View mode</span> <span>View mode</span>
<div class="flex min-h-10 flex-wrap items-center gap-2"> <div class="flex min-h-10 flex-wrap items-center gap-2">
<AppButton <AppButton
:variant="mode === 'new' ? 'primary' : 'secondary'" :variant="mode === 'unread' ? 'primary' : 'secondary'"
class="min-w-16 justify-center" class="min-w-16 justify-center"
:disabled="actionBusy" :disabled="actionBusy"
@click="setMode('new')" @click="setMode('unread')"
> >
Needs review Unread
</AppButton> </AppButton>
<AppButton <AppButton
:variant="mode === 'all' ? 'primary' : 'secondary'" :variant="mode === 'all' ? 'primary' : 'secondary'"
@ -485,7 +485,7 @@ watch(
<div class="flex flex-wrap items-center gap-2"> <div class="flex flex-wrap items-center gap-2">
<AppBadge tone="info">Mode: {{ modeLabel }}</AppBadge> <AppBadge tone="info">Mode: {{ modeLabel }}</AppBadge>
<AppBadge tone="neutral">Visible: {{ visibleCount }}</AppBadge> <AppBadge tone="neutral">Visible: {{ visibleCount }}</AppBadge>
<AppBadge tone="warning">Needs review: {{ visibleUnreadCount }}</AppBadge> <AppBadge tone="warning">Unread: {{ visibleUnreadCount }}</AppBadge>
<AppBadge tone="success">Selected: {{ selectedCount }}</AppBadge> <AppBadge tone="success">Selected: {{ selectedCount }}</AppBadge>
</div> </div>
@ -495,10 +495,10 @@ watch(
:disabled="selectedCount === 0 || actionBusy" :disabled="selectedCount === 0 || actionBusy"
@click="onMarkSelectedRead" @click="onMarkSelectedRead"
> >
{{ publishingSelected ? "Updating..." : `Mark selected reviewed (${selectedCount})` }} {{ publishingSelected ? "Updating..." : `Mark selected as read (${selectedCount})` }}
</AppButton> </AppButton>
<AppButton variant="secondary" :disabled="actionBusy || visibleUnreadCount === 0" @click="onMarkAllRead"> <AppButton variant="secondary" :disabled="actionBusy || visibleUnreadCount === 0" @click="onMarkAllRead">
{{ publishingAll ? "Updating..." : "Mark all as reviewed" }} {{ publishingAll ? "Updating..." : "Mark all unread as read" }}
</AppButton> </AppButton>
</div> </div>
</div> </div>
@ -518,7 +518,7 @@ watch(
<div class="flex items-center gap-1"> <div class="flex items-center gap-1">
<h2 class="text-lg font-semibold text-ink-primary">Publication List</h2> <h2 class="text-lg font-semibold text-ink-primary">Publication List</h2>
<AppHelpHint <AppHelpHint
text="Use sorting, search, and bulk actions here to move discovered records from needs-review into reviewed history." text="Use sorting, search, and bulk actions here to move discovered records from unread into read history."
/> />
</div> </div>
<span class="text-xs text-secondary">Currently showing {{ selectedScholarName }}</span> <span class="text-xs text-secondary">Currently showing {{ selectedScholarName }}</span>
@ -541,7 +541,7 @@ watch(
class="h-4 w-4 rounded border-stroke-interactive bg-surface-input text-brand-600 focus-visible:ring-2 focus-visible:ring-focus-ring focus-visible:ring-offset-2 focus-visible:ring-offset-focus-offset" class="h-4 w-4 rounded border-stroke-interactive bg-surface-input text-brand-600 focus-visible:ring-2 focus-visible:ring-focus-ring focus-visible:ring-offset-2 focus-visible:ring-offset-focus-offset"
:checked="allVisibleUnreadSelected" :checked="allVisibleUnreadSelected"
:disabled="visibleUnreadKeys.size === 0" :disabled="visibleUnreadKeys.size === 0"
aria-label="Select all visible publications that need review" aria-label="Select all visible unread publications"
@change="onToggleAllVisible" @change="onToggleAllVisible"
/> />
</th> </th>
@ -567,7 +567,7 @@ watch(
</th> </th>
<th scope="col"> <th scope="col">
<button type="button" class="table-sort" @click="toggleSort('status')"> <button type="button" class="table-sort" @click="toggleSort('status')">
Review status <span aria-hidden="true">{{ sortMarker("status") }}</span> Read status <span aria-hidden="true">{{ sortMarker("status") }}</span>
</button> </button>
</th> </th>
<th scope="col"> <th scope="col">
@ -607,10 +607,10 @@ watch(
<td> <td>
<div class="flex flex-wrap items-center gap-2"> <div class="flex flex-wrap items-center gap-2">
<AppBadge :tone="item.is_new_in_latest_run ? 'info' : 'neutral'"> <AppBadge :tone="item.is_new_in_latest_run ? 'info' : 'neutral'">
{{ item.is_new_in_latest_run ? "New this check" : "Previously seen" }} {{ item.is_new_in_latest_run ? "New this check" : "Seen before" }}
</AppBadge> </AppBadge>
<AppBadge :tone="item.is_read ? 'success' : 'warning'"> <AppBadge :tone="item.is_read ? 'success' : 'warning'">
{{ item.is_read ? "Reviewed" : "Needs review" }} {{ item.is_read ? "Read" : "Unread" }}
</AppBadge> </AppBadge>
</div> </div>
</td> </td>

View file

@ -277,7 +277,7 @@ function onToggleTheme(): void {
<div class="flex flex-wrap gap-2"> <div class="flex flex-wrap gap-2">
<AppButton>Primary action</AppButton> <AppButton>Primary action</AppButton>
<AppButton variant="secondary">Secondary</AppButton> <AppButton variant="secondary">Secondary</AppButton>
<AppBadge tone="warning">Needs review</AppBadge> <AppBadge tone="warning">Unread</AppBadge>
</div> </div>
<AppTable label="Shell preview table"> <AppTable label="Shell preview table">
<thead> <thead>

View file

@ -1222,9 +1222,32 @@ async def test_api_publications_list_and_mark_read(db_session: AsyncSession) ->
assert list_response.status_code == 200 assert list_response.status_code == 200
data = list_response.json()["data"] data = list_response.json()["data"]
assert data["mode"] == "all" assert data["mode"] == "all"
assert data["total_count"] == 2
assert data["unread_count"] == 2
assert data["latest_count"] == 0
assert data["new_count"] == data["latest_count"]
assert isinstance(data["publications"], list) assert isinstance(data["publications"], list)
assert len(data["publications"]) == 2 assert len(data["publications"]) == 2
latest_response = client.get("/api/v1/publications?mode=latest")
assert latest_response.status_code == 200
latest_data = latest_response.json()["data"]
assert latest_data["mode"] == "latest"
assert latest_data["latest_count"] == 0
unread_response = client.get("/api/v1/publications?mode=unread")
assert unread_response.status_code == 200
unread_data = unread_response.json()["data"]
assert unread_data["mode"] == "unread"
assert unread_data["unread_count"] == 2
alias_response = client.get("/api/v1/publications?mode=new")
assert alias_response.status_code == 200
alias_data = alias_response.json()["data"]
assert alias_data["mode"] == "latest"
assert alias_data["latest_count"] == latest_data["latest_count"]
assert alias_data["publications"] == latest_data["publications"]
mark_selected_response = client.post( mark_selected_response = client.post(
"/api/v1/publications/mark-read", "/api/v1/publications/mark-read",
json={ json={
@ -1259,3 +1282,120 @@ async def test_api_publications_list_and_mark_read(db_session: AsyncSession) ->
mark_response = client.post("/api/v1/publications/mark-all-read", headers=headers) mark_response = client.post("/api/v1/publications/mark-all-read", headers=headers)
assert mark_response.status_code == 200 assert mark_response.status_code == 200
assert mark_response.json()["data"]["updated_count"] == 1 assert mark_response.json()["data"]["updated_count"] == 1
@pytest.mark.integration
@pytest.mark.db
@pytest.mark.asyncio
async def test_api_publications_unread_and_latest_modes_can_diverge(
db_session: AsyncSession,
) -> None:
user_id = await insert_user(
db_session,
email="api-pubs-mismatch@example.com",
password="api-password",
)
scholar_result = await db_session.execute(
text(
"""
INSERT INTO scholar_profiles (user_id, scholar_id, display_name, is_enabled)
VALUES (:user_id, :scholar_id, :display_name, true)
RETURNING id
"""
),
{
"user_id": user_id,
"scholar_id": "mismatchScholar01",
"display_name": "Mismatch Scholar",
},
)
scholar_profile_id = int(scholar_result.scalar_one())
older_run_result = await db_session.execute(
text(
"""
INSERT INTO crawl_runs (user_id, trigger_type, status, scholar_count, new_pub_count)
VALUES (:user_id, 'manual', 'success', 1, 1)
RETURNING id
"""
),
{"user_id": user_id},
)
older_run_id = int(older_run_result.scalar_one())
latest_run_result = await db_session.execute(
text(
"""
INSERT INTO crawl_runs (user_id, trigger_type, status, scholar_count, new_pub_count)
VALUES (:user_id, 'scheduled', 'success', 1, 0)
RETURNING id
"""
),
{"user_id": user_id},
)
latest_run_id = int(latest_run_result.scalar_one())
assert latest_run_id > older_run_id
publication_result = await db_session.execute(
text(
"""
INSERT INTO publications (fingerprint_sha256, title_raw, title_normalized, citation_count)
VALUES (:fingerprint, :title_raw, :title_normalized, 12)
RETURNING id
"""
),
{
"fingerprint": f"{(user_id + 99):064x}",
"title_raw": "Unread But Not Latest",
"title_normalized": "unread but not latest",
},
)
publication_id = int(publication_result.scalar_one())
await db_session.execute(
text(
"""
INSERT INTO scholar_publications (
scholar_profile_id,
publication_id,
is_read,
first_seen_run_id
)
VALUES (:scholar_profile_id, :publication_id, false, :first_seen_run_id)
"""
),
{
"scholar_profile_id": scholar_profile_id,
"publication_id": publication_id,
"first_seen_run_id": older_run_id,
},
)
await db_session.commit()
client = TestClient(app)
login_user(client, email="api-pubs-mismatch@example.com", password="api-password")
unread_response = client.get("/api/v1/publications?mode=unread")
assert unread_response.status_code == 200
unread_data = unread_response.json()["data"]
assert unread_data["mode"] == "unread"
assert unread_data["unread_count"] == 1
assert unread_data["latest_count"] == 0
assert unread_data["new_count"] == 0
assert len(unread_data["publications"]) == 1
assert int(unread_data["publications"][0]["publication_id"]) == publication_id
assert unread_data["publications"][0]["is_new_in_latest_run"] is False
latest_response = client.get("/api/v1/publications?mode=latest")
assert latest_response.status_code == 200
latest_data = latest_response.json()["data"]
assert latest_data["mode"] == "latest"
assert latest_data["latest_count"] == 0
assert len(latest_data["publications"]) == 0
alias_response = client.get("/api/v1/publications?mode=new")
assert alias_response.status_code == 200
alias_data = alias_response.json()["data"]
assert alias_data["mode"] == "latest"
assert alias_data["latest_count"] == latest_data["latest_count"]
assert alias_data["publications"] == latest_data["publications"]