ai_synth/CLAUDE.md

AI Weekly Synth

Overview

AI Weekly Synth is a self-hosted web application that generates AI-powered weekly news syntheses. Users create themes (topics), configure categories and sources, then the app scrapes sources, classifies articles via LLM, and produces structured summaries. Supports scheduled generation with email delivery.

Architecture

• Backend: Rust (Axum) — backend/
• Frontend: SolidJS + Tailwind CSS v4 — frontend/
• Database: PostgreSQL (via sqlx with runtime-checked queries)
• Deployment: Docker (docker-compose.yml, restart: unless-stopped)

Project Structure

ai_synth/
├── backend/                 Rust/Axum backend
│   ├── src/
│   │   ├── main.rs          Entry point, CLI, background tasks (session cleanup, scheduler)
│   │   ├── router.rs        All API routes + middleware stack
│   │   ├── handlers/        HTTP handlers (auth, settings, sources, themes, schedules, syntheses, admin, etc.)
│   │   ├── services/        Business logic (synthesis pipeline, job_store, scheduler, LLM providers, scraper, email, encryption)
│   │   ├── db/              Database queries (sqlx)
│   │   ├── models/          Data types + validation (settings, theme, schedule, source, synthesis, etc.)
│   │   ├── middleware/      Auth session extraction, CSRF check
│   │   └── util/            Token generation, hashing
│   ├── migrations/          SQL migrations (30 files)
│   ├── tests/               Integration tests (17 files, require Postgres)
│   ├── Cargo.toml
│   └── Dockerfile           Multi-stage build
├── frontend/                SolidJS frontend
│   ├── src/
│   │   ├── App.tsx          Router, layouts, route guards
│   │   ├── pages/           Home, Settings, ThemeManager, GenerateSynthesis, SynthesisDetail, ArticleHistory, admin/*
│   │   ├── components/      Navbar, ApiKeyManager, settings/* (BraveSearch, Schedule, RateLimit), ui/*
│   │   ├── api/             API clients (auth, settings, sources, themes, schedules, syntheses, admin, config, apiKeys)
│   │   ├── contexts/        AuthContext (session-based)
│   │   ├── i18n/            French translations
│   │   └── utils/           SSE client, date formatting, URL utils, provider info
│   ├── package.json
│   └── vite.config.ts       SolidJS + Tailwind + dev proxy
├── e2e/                     E2E tests (Playwright)
│   ├── tests/               7 test specs
│   ├── helpers/             Auth helpers, seed.ts
│   └── docker-compose.test.yml
├── scripts/                 Test runner scripts
│   ├── run-integration-tests.sh
│   └── run-e2e-tests.sh
├── docs/                    Consolidated documentation (see below)
├── docker-compose.yml       App + Postgres
├── .env.example             All required env vars documented
└── CLAUDE.md                This file

Documentation

• docs/requirements.md — Product vision, features, user roles, non-functional requirements
• docs/functional_specs.md — User journeys, feature details, pipeline description
• docs/architecture.md — System design, layers, data model, security, concurrency
• docs/technical_specs.md — Tech stack, DB schema, API specs, pipeline flow
• docs/dev_guidelines.md — Coding standards, patterns, how-tos, pitfalls
• docs/qa_guidelines.md — Test inventory, infrastructure, writing tests
• docs/deployment.md — Docker setup, env vars, monitoring, security

Key Features

• Multi-Theme: Users create multiple themes, each with its own categories, sources, and schedule
• LLM Providers: Google Gemini, OpenAI, Anthropic — users bring their own API keys
• Generation Pipeline: Two-phase (personalized sources → web search fallback), windowed extraction, batched scrape+classify
• Brave Search: Optional alternative to LLM web search for Phase 2
• Scheduled Generation: Per-theme day/time schedule with email delivery to up to 3 addresses
• Preferred Sources: Mark sources as priority — processed first during generation
• Stop Generation: Cancel in-progress generation, saves partial results
• Admin Module: Provider/model curation, rate limit config, user management
• Security: AES-256-GCM encryption, SSRF prevention, CSRF, HttpOnly cookies
• Export: Email (Resend), PDF, Markdown
• Real-time: SSE progress streaming with cancellation support
• Article Intelligence: LLM-extracted dates, is_article detection, configurable summary length

Running Locally

Docker (production)

cp .env.example .env   # Fill in values
docker compose up -d

Development

# Backend (requires Postgres running)
cd backend && cargo run -- serve

# Frontend (proxies /api to backend)
cd frontend && npm install && npm run dev

CLI

cd backend && cargo run -- create-admin admin@example.com

Testing

# Backend unit tests (no Postgres needed)
cd backend && cargo test --lib

# Integration tests (uses docker-compose.test.yml Postgres)
./scripts/run-integration-tests.sh

# E2E tests (builds Docker, seeds DB, runs Playwright)
./scripts/run-e2e-tests.sh

# Frontend unit tests
cd frontend && npx vitest run

# Frontend type check
cd frontend && npx tsc --noEmit

Database (30 migrations)

Tables: users, sessions, magic_link_tokens, settings, themes, theme_schedules, sources, syntheses, article_history, llm_call_log, admin_providers, admin_rate_limits, user_api_keys, audit_log

Environment Variables

See .env.example for the complete list. Key ones:

• DATABASE_URL — Postgres connection string
• MASTER_ENCRYPTION_KEY — 64 hex chars for AES-256-GCM
• RESEND_API_KEY — for email sending
• TURNSTILE_SECRET_KEY / TURNSTILE_SITE_KEY — captcha
• APP_URL — public URL (for CORS, magic links, cookies)

Design Decisions

• Idiomatic Rust (learning project) — no unwrap() in production code
• Users bring their own LLM API keys (encrypted at rest)
• Admin curates available providers/models, users select from the list
• Single-tenant self-hosted (one instance per deployment)
• i18n-ready (French only for now)
• Per-theme content settings, global infrastructure settings