ai_synth/docs/superpowers/plans/2026-03-22-coverage-assessment-v2.md

Coverage Assessment V2 — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Update docs/tech_lead_assessment_Coverage_Documentation.md in-place to reflect current test counts, closed gaps, and new recommendations.

Architecture: Single file edit — 7 sequential sections of the existing document are modified. No code changes, no tests. Git history preserves the original.

Tech Stack: Markdown only.

Spec: docs/superpowers/specs/2026-03-22-coverage-assessment-v2-design.md

---

Task 1: Update header and add "Changes since last assessment"

Files:

• Modify: docs/tech_lead_assessment_Coverage_Documentation.md:1-6

• Step 1: Update the date and add previous-assessment reference

Replace lines 1-4:

# Tech Lead Assessment: Test Coverage & Documentation

**Date**: 2026-03-22
**Scope**: Full codebase audit of AI Weekly Synth (Rust/SolidJS)

With:

# Tech Lead Assessment: Test Coverage & Documentation

**Date**: 2026-03-22
**Previous assessment**: commit `3a59362` (2026-03-22)
**Scope**: Full codebase audit of AI Weekly Synth (Rust/SolidJS)

• Step 2: Insert "Changes since last assessment" section after the --- on line 6

Insert after line 6:

## Changes since last assessment

- All 3 backend "should fix" gaps closed (auth middleware +5, token.rs +8, schema.rs +6 tests)
- LLM provider modules now tested (anthropic +20, openai +17, gemini +11, factory +5)
- 6 frontend page test files added (39 tests): Home, Settings, Sources, Generate, Login, Register
- JSDoc added to key frontend files (Settings, GenerateSynthesis, Home, api/client, utils/sse)
- Shared typed test fixtures introduced to prevent mock drift from backend contracts
- E2E infrastructure added with 5 Playwright flows (registration, admin providers, settings, sources, settings export)

---

• Step 3: Commit

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: add changes-since-last-assessment section to coverage report"

---

Task 2: Update Overall Confidence Level table

Files:

• Modify: docs/tech_lead_assessment_Coverage_Documentation.md — "Overall Confidence Level" section

• Step 1: Replace the confidence table

Replace:

## Overall Confidence Level

| Component | Tests | Docs | Grade |
|---|---|---|---|
| Backend | 332 unit + 145 integration | Good | **A** |
| Frontend | 103 (utilities/API only) | Weak | **C** |

With:

## Overall Confidence Level

| Component | Tests | Docs | Grade |
|---|---|---|---|
| Backend | 337 unit + 145 integration | Good | **A+** |
| Frontend | 142 (utilities + pages + contexts) | Adequate | **B-** |

• Step 2: Commit

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: update confidence grades — backend A+, frontend B-"

---

Task 3: Update Backend section

Files:

• Modify: docs/tech_lead_assessment_Coverage_Documentation.md — "Backend: Strong" section

• Step 1: Update the "What's well tested" list

Replace:

## Backend: Strong (high confidence)

### What's well tested

- All 25+ API endpoints have integration tests (145 total across 9 test files)
- Models have thorough validation tests (settings: 25 tests, source: 17, api_key: 11, provider: 13)
- Core services tested: encryption (roundtrip + failure cases), scraper (69 tests), rate limiter, CSV, email, prompts, synthesis pipeline
- Security is covered: CSRF, auth flow, ownership isolation, rate limiting, admin RBAC, self-demotion guard

With:

## Backend: Strong (high confidence)

### What's well tested

- All 25+ API endpoints have integration tests (145 total across 9 test files)
- Models have thorough validation tests (settings: 19 tests, source: 12, api_key: 8, provider: 9, synthesis: 16, rate_limit: 6)
- Core services tested: encryption (8), scraper (69), rate limiter (15), CSV (18), email (9), prompts (12), synthesis pipeline (27), export (12)
- LLM providers tested: anthropic (20), openai (17), gemini (11), factory (5), schema (6)
- Security is covered: CSRF (7), auth middleware (5), auth flow, ownership isolation, rate limiting, admin RBAC, self-demotion guard
- Utilities tested: token generation and hashing (8)

• Step 2: Update acceptable gaps — remove LLM providers

Replace:

### What's NOT tested (acceptable gaps)

- LLM providers (Gemini/OpenAI/Anthropic) -- external API calls, can't unit test meaningfully without mocking entire HTTP layer
- DB layer (`db/*.rs`) -- no unit tests, but fully exercised by integration tests
- Pure data models (user.rs, session.rs, audit.rs) -- no logic to test
- `main.rs`, `router.rs`, `cli.rs` -- architectural, tested implicitly

With:

### What's NOT tested (acceptable gaps)

- DB layer (`db/*.rs`) -- no unit tests, but fully exercised by integration tests
- Pure data models (user.rs, session.rs, audit.rs) -- no logic to test
- `main.rs`, `router.rs`, `cli.rs` -- architectural, tested implicitly

• Step 3: Replace "should fix" with "Resolved gaps"

Replace:

### What's NOT tested (should fix)

- `middleware/auth.rs` -- the session extraction logic deserves unit tests for edge cases (malformed cookies, expired sessions)
- `util/token.rs` -- token generation randomness and hash verification should have explicit tests
- `services/llm/schema.rs` -- the dynamic category schema builder has no tests; malformed category names could produce invalid JSON Schema

With:

### Resolved gaps (since last assessment)

- `middleware/auth.rs` -- now has 5 unit tests for session cookie extraction (valid, missing, multiple cookies, whitespace, empty)
- `util/token.rs` -- now has 8 unit tests for token generation (length, uniqueness, URL-safety) and hashing (determinism, hex format, roundtrip)
- `services/llm/schema.rs` -- now has 6 unit tests for category schema building (1/3/5 categories, empty, special characters, required fields)
- LLM providers -- anthropic (20), openai (17), gemini (11) now have response parsing and error handling tests; factory (5) tests provider selection

• Step 4: Update Documentation subsection

Replace:

### Documentation

Backend is well documented. Module-level `//!` comments on all handler and service files. Public functions have `///` doc comments. The synthesis pipeline, encryption, and rate limiter are especially well explained.

**Gaps**: `db/` layer, `middleware/auth.rs`, and LLM service implementations have minimal comments.

With:

### Documentation

Backend is well documented. Module-level `//!` comments on all handler and service files. Public functions have `///` doc comments. The synthesis pipeline, encryption, and rate limiter are especially well explained.

**Gaps**: `db/` layer has minimal comments. (Previously flagged `middleware/auth.rs` and LLM services now have inline test documentation.)

• Step 5: Commit

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: update backend section — all gaps resolved, LLM providers tested"

---

Task 4: Update Frontend section

Files:

• Modify: docs/tech_lead_assessment_Coverage_Documentation.md — "Frontend" section

• Step 1: Update section header and "What IS tested"

Replace:

## Frontend: Weak (low confidence)

### What IS tested (103 tests)

- API client: CSRF headers, credentials, error handling, 401 redirect (9 tests)
- Auth context: loading/authenticated/unauthenticated states (3 tests)
- i18n: translation keys, interpolation (9 tests)
- Utilities: date formatting, SSE parsing, URL normalization, provider info (47 tests)
- API key management, settings validation, admin route guard, export logic

With:

## Frontend: Improved (moderate confidence)

### What IS tested (142 tests)

**Utility & API tests (103 tests):**
- API client: CSRF headers, credentials, error handling, 401 redirect (7 tests)
- Auth context: loading/authenticated/unauthenticated states (3 tests)
- Admin route guard: access control (3 tests)
- i18n: translation keys, interpolation (9 tests)
- Settings validation: defaults, constraints (7 tests)
- Sources utilities: URL normalization (17 tests)
- SSE: event parsing, steps, reconnection (11 tests)
- Synthesis utilities: week extraction, dates (11 tests)
- Synthesis export: file download logic (6 tests)
- API keys: key CRUD, prefix handling (11 tests)
- Config API: provider config (7 tests)
- Provider info: web search capability detection (11 tests)

**Page interaction tests (39 tests):**
- Home: list rendering, empty state, delete confirmation, in-progress banner (7 tests)
- Settings: form rendering, provider switching, rate limits, export/import (10 tests)
- Sources: source CRUD, bulk import, CSV operations (8 tests)
- GenerateSynthesis: launch, SSE progress, completion (6 tests)
- Login: email input, submit, success/error states (4 tests)
- Register: registration flow, confirmation (4 tests)

**Test infrastructure:**
- Shared typed fixtures (`fixtures.ts`) prevent mock drift from backend contracts
- `test-utils.tsx` provides `renderWithProviders()` and `mockFetch()` helpers

• Step 2: Update "What is NOT tested"

Replace:

### What is NOT tested (critical gap)

- **ZERO page component tests** -- all 11 pages (Home, Settings, Sources, GenerateSynthesis, SynthesisDetail, Login, Register, AuthVerify, 3 admin pages) have no rendering or interaction tests
- **ZERO UI component tests** -- Navbar, Layout, AdminLayout, MobileMenu, ApiKeyManager, ErrorBoundary, Turnstile, Button, LoadingSpinner, Toast -- none tested
- **No form interaction tests** -- Settings form (the most complex page with export/import, dual models, rate limits, categories) is entirely untested
- **No SSE integration test** -- the generation progress flow (connect, receive events, update UI) has no component-level test

With:

### What is NOT tested (remaining gaps)

- **5 pages untested** -- SynthesisDetail, AuthVerify, admin/Providers, admin/RateLimits, admin/Users have no rendering or interaction tests
- **ZERO dedicated UI component tests** -- Navbar, Layout, AdminLayout, MobileMenu, ApiKeyManager, ErrorBoundary, Turnstile, Button, LoadingSpinner, Toast -- none have dedicated tests (some exercised indirectly through page tests)
- **No SSE reconnection test** -- GenerateSynthesis page test mocks SSE but does not test reconnection or error recovery

• Step 3: Update Documentation subsection

Replace:

### Documentation

Frontend documentation is weak. Most pages and components have zero JSDoc. Complex logic in `Settings.tsx` (export/import, provider detection, rate limit handling), `GenerateSynthesis.tsx` (SSE state machine), and `Home.tsx` (delete confirmation with timers) is uncommented. The API client's CSRF and credential handling is not explained inline.

With:

### Documentation

Frontend documentation is adequate. JSDoc has been added to the 5 most complex files: `Settings.tsx` (export/import, provider auto-detection, rate limit null handling), `GenerateSynthesis.tsx` (SSE state machine, step progression, reconnection), `Home.tsx` (delete confirmation timer pattern), `api/client.ts` (CSRF strategy, 401 redirect), and `utils/sse.ts` (reconnection backoff, event parsing, cleanup).

**Gaps**: Admin pages (Providers, RateLimits, Users), SynthesisDetail, AuthVerify, and 8 of 10 UI components have no JSDoc.

• Step 4: Commit

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: update frontend section — 142 tests, 6 pages tested, JSDoc adequate"

---

Task 5: Replace Recommendations section

Files:

• Modify: docs/tech_lead_assessment_Coverage_Documentation.md — "Recommendations" section

• Step 1: Replace entire Recommendations section

Replace everything from ## Recommendations (priority order) through the E2E section (lines 70-119) with:

## Recommendations (priority order)

### 1. Frontend: remaining page tests (HIGH)

Add component tests for the 5 untested pages:

- `SynthesisDetail.tsx` -- section rendering, export buttons (markdown/PDF), email trigger
- `AuthVerify.tsx` -- token extraction from URL, verification success/error states
- `admin/Providers.tsx` -- provider CRUD, model list management
- `admin/RateLimits.tsx` -- rate limit display and update per provider
- `admin/Users.tsx` -- user list rendering, role change

This would bring page coverage from 6/11 to 11/11 and frontend grade to B+.

### 2. Frontend: UI component tests (MEDIUM)

Add dedicated tests for at least the 4 most critical components:

- `ErrorBoundary.tsx` -- error capture and fallback rendering
- `Navbar.tsx` -- auth-aware navigation, admin link visibility, mobile menu toggle
- `ApiKeyManager.tsx` -- key CRUD, masking, test-key flow
- `ui/Toast.tsx` -- auto-dismiss timing, multiple toast stacking

Page tests exercise these indirectly, but a broken component would not be caught.

### 3. E2E: CI integration and coverage depth (MEDIUM)

- Wire the 5 existing Playwright flows into CI pipeline
- Expand flows with failure-path scenarios (invalid login, API errors, network timeouts)

### 4. Frontend: contract testing (LOW)

- Extend shared typed fixtures to cover all API response shapes
- Consider Zod for runtime contract validation at the API boundary

• Step 2: Commit

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: replace recommendations with current priorities"

---

Task 6: Update Detailed Test Inventory

Files:

• Modify: docs/tech_lead_assessment_Coverage_Documentation.md — "Detailed Test Inventory" section

• Step 1: Replace Backend Unit Tests table

Replace the existing table (lines 127-153) with:

### Backend Unit Tests by Module

| Module | File | Tests | Status |
|---|---|---|---|
| models | settings.rs | 19 | Thorough |
| models | synthesis.rs | 16 | Thorough |
| models | source.rs | 12 | Good |
| models | api_key.rs | 8 | Good |
| models | provider.rs | 9 | Good |
| models | rate_limit.rs | 6 | Good |
| models | user.rs, session.rs, audit.rs, magic_link.rs | 0 | Pure data, acceptable |
| services | scraper.rs | 69 | Excellent |
| services | synthesis.rs | 27 | Good |
| services | prompts.rs | 12 | Good |
| services | encryption.rs | 8 | Good |
| services | email.rs | 9 | Good |
| services | export.rs | 12 | Good |
| services | csv.rs | 18 | Good |
| services | rate_limiter.rs | 15 | Good |
| services | auth.rs | 0 | Covered by integration |
| services | turnstile.rs | 0 | Covered by integration |
| services | llm/anthropic.rs | 20 | Good |
| services | llm/openai.rs | 17 | Good |
| services | llm/gemini.rs | 11 | Good |
| services | llm/factory.rs | 5 | Good |
| services | llm/schema.rs | 6 | Good |
| handlers | admin.rs | 4 | Minimal inline |
| handlers | all others | 0 | Covered by integration |
| middleware | csrf.rs | 7 | Good |
| middleware | auth.rs | 5 | Good |
| config | config.rs | 7 | Good |
| errors | errors.rs | 7 | Good |
| util | token.rs | 8 | Good |

• Step 2: Keep Backend Integration Tests table unchanged

No changes needed — all counts verified as accurate at 145 total.

• Step 3: Replace Frontend Tests table

Replace the existing frontend tests table (lines 172-186) with:

### Frontend Tests

| Test File | Tests | Coverage |
|---|---|---|
| api-client.test.ts | 7 | CSRF, credentials, errors |
| auth-context.test.tsx | 3 | User state management |
| admin-route-guard.test.tsx | 3 | Admin access control |
| i18n.test.ts | 9 | Translations, interpolation |
| settings-validation.test.ts | 7 | Defaults, validation |
| sources-utils.test.ts | 17 | URL normalization |
| sse.test.ts | 11 | Event parsing, steps, reconnection |
| synthesis-utils.test.ts | 11 | Week extraction, dates |
| synthesis-export.test.ts | 6 | File download logic |
| api-keys.test.ts | 11 | Key CRUD, prefix |
| config-api.test.ts | 7 | Provider config API |
| provider-info.test.ts | 11 | Web search info |
| **Subtotal** | **103** | **Utilities & API** |
| pages/home.test.tsx | 7 | List, empty state, delete flow |
| pages/settings.test.tsx | 10 | Form, providers, export/import |
| pages/sources.test.tsx | 8 | CRUD, bulk import, CSV |
| pages/generate.test.tsx | 6 | Launch, SSE progress, completion |
| pages/login.test.tsx | 4 | Email submit, success/error |
| pages/register.test.tsx | 4 | Registration flow |
| **Subtotal** | **39** | **Page interactions** |
| **Total** | **142** | **Utilities + pages** |

• Step 4: Replace Frontend Untested Files section

Replace:

### Frontend: Untested Files

**Pages (0/11 tested):**
- Home.tsx, Settings.tsx, Sources.tsx, GenerateSynthesis.tsx, SynthesisDetail.tsx
- Login.tsx, Register.tsx, AuthVerify.tsx
- admin/Providers.tsx, admin/RateLimits.tsx, admin/Users.tsx

**Components (0/10 tested):**
- Navbar.tsx, Layout.tsx, AdminLayout.tsx, MobileMenu.tsx
- ApiKeyManager.tsx, ErrorBoundary.tsx, Turnstile.tsx
- ui/Button.tsx, ui/LoadingSpinner.tsx, ui/Toast.tsx

With:

### Frontend: Untested Files

**Pages (5/11 untested):**
- SynthesisDetail.tsx, AuthVerify.tsx
- admin/Providers.tsx, admin/RateLimits.tsx, admin/Users.tsx

**Components (0/10 with dedicated tests):**
- Navbar.tsx, Layout.tsx, AdminLayout.tsx, MobileMenu.tsx
- ApiKeyManager.tsx, ErrorBoundary.tsx, Turnstile.tsx
- ui/Button.tsx, ui/LoadingSpinner.tsx, ui/Toast.tsx

### E2E Tests (Playwright)

| Test File | Flow |
|---|---|
| registration.spec.ts | Register → magic link → verify → home |
| admin-providers.spec.ts | Login as admin → enable provider → add model |
| settings.spec.ts | Login → configure settings → reload → verify |
| sources.spec.ts | Add source → bulk import → delete → CSV export |
| settings-export.spec.ts | Configure → export JSON → change → import → verify |
| **Total** | **5 flows** |

• Step 5: Commit

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: update detailed test inventory with current counts"

---

Task 7: Update Bottom Line

Files:

• Modify: docs/tech_lead_assessment_Coverage_Documentation.md — "Bottom Line" section

• Step 1: Replace Bottom Line

Replace:

## Bottom Line

**Backend: You can be confident.** 477 tests with good coverage of all endpoints, security controls, and business logic. The gaps are in areas that are either architectural or require external services.

**Frontend: You should NOT be confident yet.** The utilities and API layer are tested, but every single page and component -- where the actual user-facing bugs live -- has zero test coverage. A typo in a signal binding, a broken `<Show>` condition, or a missing `onCleanup` would not be caught by any test. This is the single biggest quality risk in the codebase.

With:

## Bottom Line

**Backend: You can be confident.** 482 tests (337 unit + 145 integration) with good coverage of all endpoints, security controls, business logic, and LLM provider integrations. All previously flagged gaps have been addressed. The remaining untested areas are architectural or pure data models.

**Frontend: Confidence is improving.** 142 tests now cover utilities, the API layer, and the 6 most critical pages. Shared typed fixtures prevent mock drift from backend contracts. The remaining risk is in the 5 untested pages (admin, detail, verify) and all 10 UI components which lack dedicated tests — a broken ErrorBoundary, a Toast that doesn't auto-dismiss, or an admin CRUD page that silently fails would not be caught. E2E infrastructure is in place but not yet wired into CI.

• Step 2: Commit

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: update bottom line — 482 backend, 142 frontend, improving confidence"

---

Task 8: Final review

• Step 1: Read the full updated document end-to-end

Run: Read tool on docs/tech_lead_assessment_Coverage_Documentation.md

Verify:

• All numbers are internally consistent (337+145=482 backend, 103+39=142 frontend)

• No stale content remains from the original assessment

• Section ordering is: Changes → Confidence → Backend → Frontend → Recommendations → Inventory → Bottom Line

• Grade table matches bottom line matches section headers

• Step 2: Run a grep sanity check for stale numbers

Run: grep -n "477\|332 " docs/tech_lead_assessment_Coverage_Documentation.md

Expected: No matches. If any found, fix them. (Note: "103" is valid in the updated document as the utility test subtotal.)

• Step 3: Amend last commit if fixes were needed, otherwise done

If fixes were needed:

git add docs/tech_lead_assessment_Coverage_Documentation.md
git commit -m "docs: fix stale numbers in coverage assessment"