260 Commits (48b5e77e7ea22c16a167f4b06311029960a5e8a6)
 

Author SHA1 Message Date
oabrivard a158f14311 fix: don't poll SSE stream in model resolution test
The SSE stream blocks until the generation completes or times out
(15 min). With a fake API key, the LLM call hangs for 120s before
failing. Just verify the 202 trigger succeeded — that confirms
model resolution and provider creation worked.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard eadfbc000b fix: expect 201 Created for source creation in syntheses test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 54c637647b fix: add missing fields to syntheses test settings payload
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 7ec491b6ac fix: update settings test payloads for new required fields + fix unused var warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard a0a8f72caa fix: don't join Drop cleanup thread — prevents deadlock in tokio tests
The Drop impl spawned a thread with a new tokio runtime and called
.join(), which blocked the test thread. The spawned thread's block_on
deadlocked when pool.close() tried to communicate with connections
owned by the outer tokio runtime. Removing .join() makes cleanup
fire-and-forget, avoiding the deadlock.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 243558b950 debug: add full_app_health_check test with logging 3 months ago
oabrivard aee70b37d4 fix: use docker-compose.test.yml for integration test DB
Rewrite run-integration-tests.sh to use the e2e docker-compose config
(Postgres on port 5433). Add --db-check flag for connectivity debugging.
Remove build_test_router (reverted to build_router). Keep minimal_test
for oneshot debugging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 5fa060fadc fix: use invalid session token for admin auth rejection tests
Same fix as other test files — avoids oneshot() hang with no cookie.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard bd4f101d16 fix: use invalid session token instead of no cookie for auth rejection tests
Unauthenticated requests (no Cookie header) hang with oneshot() in tests.
Using an invalid session token achieves the same 401 result without hanging.
3 months ago
oabrivard 53813007c6 fix: use lightweight test router without SPA fallback and TraceLayer
Unauthenticated requests were hanging in integration tests due to
tower middleware layers interacting with oneshot(). Add build_test_router()
that only includes API routes + CSRF middleware.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 7cbafdfb31 fix: create test static dir to prevent ServeDir/ServeFile hang
The SPA fallback uses ServeDir/ServeFile which can hang when the
directory doesn't exist. Create it in TestApp::new() with a minimal
index.html.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard cda9e2fc25 fix: pass all args to cargo test in integration test script
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard bb2209e425 fix: update admin tests for models_scraping/models_websearch split
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard de8e5ab85c feat: add test runner scripts for integration and E2E tests
- scripts/run-integration-tests.sh: spins up Postgres, runs cargo tests
- scripts/run-e2e-tests.sh: builds Docker, starts stack, runs Playwright
- fix: remove SESSION_SECRET from e2e docker-compose.test.yml

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 370e033506 feat: add pipeline integration tests with MockLlmProvider and wiremock
Add three integration tests that exercise the synthesis generation
pipeline end-to-end using MockLlmProvider and wiremock for HTTP mocking:
- phase1_with_llm_link_extraction_classifies_articles
- phase2_search_fills_gaps_when_no_sources
- category_overflow_spills_to_autre

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard ccecaa2d13 refactor: add provider_override for pipeline dependency injection
Adds an optional LlmProvider override to run_generation and
run_generation_inner, allowing tests to inject a mock provider without
touching real credentials or the provider-resolution path. Makes
run_generation_inner pub so integration tests can call it directly.
Production callers pass None and behaviour is unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 17e054c257 feat: add MockLlmProvider for integration testing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard ecf95ffe35 fix: update test config for session_secret removal and Arc master key
Remove session_secret field (no longer in AppConfig), wrap
master_encryption_key in Arc<String>, and pass a generated job_id
to db::syntheses::create which now requires it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 1f8f2ddc9d docs: add implementation plan for integration tests with mock LLM provider
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard f678275f59 docs: add spec for integration tests with mock LLM provider + update algorithm.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 0b71215ddc docs: update algorithm.md to reflect current pipeline state
Documents the Brave Search path in Phase 2, batch_size setting for
parallelism, batched article tracing with build_trace_entry/batch_insert_entries,
15-minute pipeline timeout, panic handling, session cleanup background task,
SSRF checks on source pages, and the max_links change from 10 to 15.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 4bbdd5c4d1 perf: batch article history INSERTs to reduce DB round-trips
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard f37e0b42a0 perf: use Arc for immutable values in pipeline to reduce cloning
Wrap `model_research` (String), `classify_schema` (Value), and
`classification_categories` (Vec<String>) in Arc before the batch
loops so spawned tasks clone a cheap pointer instead of the full
heap data on every iteration. Also removes the redundant intermediate
`mdl`/`class_sys`/`class_user` bindings in both classify loops.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 0aa0541958 refactor: standardize createResource and use Button component
Convert LlmLogs from onMount+signals to createResource for idiomatic
reactive data fetching. Replace duplicated inline save/add button markup
in Settings and Sources with the shared Button component.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 60494aeceb perf: cache CSS selectors with LazyLock to avoid re-parsing
Replace runtime Selector::parse calls on static strings with module-level
LazyLock statics in source_scraper.rs (ANCHOR_SELECTOR) and scraper.rs
(SEL_TITLE, SEL_H1, SEL_BODY), so each selector is compiled once at
first use instead of on every function call.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 69c1688bc7 chore: remove SESSION_SECRET and wrap master_encryption_key in Arc
SESSION_SECRET was loaded and validated but never used anywhere in the
codebase. master_encryption_key is now wrapped in Arc<String> to avoid
cloning the secret string on every AppState clone.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 150090a112 chore: align frontend DEFAULT_SETTINGS with backend defaults
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 2e87ae854a docs: add implementation plan for polish and optimization items
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 825c7212e1 docs: add spec for polish and optimization items
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard ee6d833d70 refactor: decompose Settings.tsx into sub-components
Extract three self-contained sections from Settings.tsx (1025 lines)
into dedicated components under frontend/src/components/settings/:
- SettingsBraveSearch: Brave API key lifecycle + use_brave_search toggle
- SettingsAdvanced: article_history_days, batch_size, use_llm_for_source_links, search_agent_behavior
- SettingsRateLimit: rate limit inputs + effective rate display + reset

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard f44aa44c48 refactor: replace trace_article 11 parameters with ArticleTrace struct
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard f5466a6bd5 refactor: extract shared LLM error mapping to reduce duplication
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 2036c12b24 refactor: eliminate SettingsResponse struct, serialize UserSettings directly
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard e056ef9d3e refactor: extract assign_category and filter_phase2_url helpers from synthesis pipeline
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 030fe6f941 docs: add implementation plan for structural refactoring
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard fd1130fd5a docs: add spec for structural refactoring (audit sub-project B)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 5a66443a94 fix: add onCleanup for setTimeout in frontend pages
Prevents timers from firing after component unmount in Home (delete
confirmation timers), ArticleHistory (clear confirmation timer), and
SynthesisDetail (email success banner timer).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 24d53a01d1 fix: block SSRF via IPv4-mapped IPv6 and add check to source page fetching
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 93003229f1 fix: add periodic expired session cleanup (hourly)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 3da246bef6 fix: remove XSS via innerHTML and add setTimeout cleanup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 347558a278 fix: atomic job creation, 15min timeout, and panic handling
- Replace iterating DashMap check with atomic DashSet insert in create_job to
  eliminate the race condition where double-click could create two concurrent
  jobs for the same user
- Add release_user method called at end of generation task (normal, timeout,
  and panic paths) so the generating slot is always freed
- Wrap run_generation in tokio::time::timeout(900s) to prevent hung LLM calls
  from blocking the generation slot forever
- Spawn a second task to await the JoinHandle and call release_user + send
  error event if the generation task panics, preventing SSE clients from
  hanging indefinitely
- Update cleanup_expired to also remove users from generating_users set
- Update tests to call release_user after completion/error to match new contract

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 59932589cc fix: prevent UTF-8 panic in error message truncation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard 4123ae38f7 docs: add implementation plan for audit bug fixes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 114f2912dd docs: add spec for audit bug fixes (P0 + P1)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard 811c5c87d1 docs: add comprehensive code audit reports from 5-agent team
Architect, Frontend Expert, Rust Expert, Tech Lead, and Devil's Advocate
analyzed the codebase independently. Key findings: UTF-8 panic bug, XSS
risk, synthesis.rs duplication (3x batch loop), Settings.tsx complexity.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard e74a1850bf fix: log source URL in link_extraction LLM call logs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard a968fdc308 fix: allow brave_search as valid API key provider
Split VALID_PROVIDERS (LLM only) from VALID_API_KEY_PROVIDERS (includes
brave_search) so Brave keys can be stored without allowing brave_search
as an admin LLM provider.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 months ago
oabrivard c5d23ecd10 feat: add Brave Search section to Settings page
Adds a dedicated Brave Search section after the Advanced Extraction section,
including inline API key management (save/test/delete) and a use_brave_search
toggle that auto-disables when the key is removed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard f124b056fe feat: add Brave Search Phase 2 pipeline path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago
oabrivard e05c2ae75a feat: handle brave_search in API key test endpoint
Add a branch in test_key to route brave_search provider to
crate::services::brave_search::test_api_key instead of the LLM factory.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 months ago