Code analysis and migration recommendations
parent
2a10f95b22
commit
e7e2f8fa90
@ -0,0 +1,139 @@
|
||||
# Summary Index: AI Weekly Synth Refactoring Analysis
|
||||
|
||||
**Generated**: 2026-03-21
|
||||
**Source reports**: 4 team analysis files
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Four specialists have independently analyzed the proposed refactoring of AI Weekly Synth from its current React/Firebase/Gemini SPA architecture to a Rust backend, SolidJS frontend, SQLite database, and custom authentication system. Their reports cover UX migration risks and new interaction flows, backend architecture and API design, security vulnerabilities and hardening measures, and a critical challenge of the entire refactoring's scope and technology choices. Together, these analyses reveal a project that is technically ambitious but carries significant risks around scope, timeline, and the gap between a "refactoring" label and what is effectively a complete greenfield rewrite.
|
||||
|
||||
---
|
||||
|
||||
## 2. Reports
|
||||
|
||||
| # | Filename | Focus Area | Summary |
|
||||
|---|---|---|---|
|
||||
| 1 | `01-ux-analysis.md` | UX and user experience | Inventories every screen, interaction pattern, and visual design element of the current app. Maps React-to-SolidJS migration risks at the UI level, designs new auth flows (sign-up, magic link login, session management), proposes admin module layouts, multi-provider settings UX, and SSE-based generation progress feedback. Identifies 8 concrete UX issues to fix during the rewrite (mobile nav, inconsistent deletion, missing validation, accessibility). |
|
||||
| 2 | `02-architecture-analysis.md` | Backend architecture and API design | Recommends Axum over Actix-web, defines a clean 3-layer project structure (handlers/services/db), designs 7 SQL migrations, a full REST API with 20+ endpoints, an LLM provider trait abstraction with per-provider web search strategies, a token-bucket rate limiter, the two-pass generation pipeline in Rust, SolidJS frontend scaffolding with auth context, and a Firebase-to-SQLite data migration plan. |
|
||||
| 3 | `03-security-analysis.md` | Security audit and hardening | Documents 9 security vulnerabilities in the current app (API key exposure, Gmail OAuth handling, prompt injection, CORS proxy data leakage, hardcoded PII). Designs the magic link token lifecycle with SHA-256 hashing, session cookie security, CSRF double-submit pattern, AES-256-GCM encryption for API keys at rest, SSRF prevention for URL scraping, Content Security Policy headers, Docker container hardening, and a 10-item threat model. |
|
||||
| 4 | `04-devils-advocate.md` | Critical challenge and risk assessment | Questions every technology choice (Rust, SolidJS, SQLite), argues this is a complete rewrite not a refactoring, estimates 7-14 months realistic timeline for a single developer, identifies the multi-LLM abstraction as the hardest unsolved problem, warns that hand-rolled auth replaces a battle-tested managed service, and proposes two alternative approaches: a minimal 2-4 week backend proxy, or a phased 5-phase incremental migration. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Consolidated Open Questions
|
||||
|
||||
All questions and ambiguities raised across the four reports, deduplicated and organized by theme. Reference numbers (e.g., Q-AUTH-1) allow the project owner to respond item by item.
|
||||
|
||||
### Authentication
|
||||
|
||||
| # | Question | Raised by |
|
||||
|---|---|---|
|
||||
| Q-AUTH-1 | Will Google SSO be kept alongside the new email+magic link auth, or dropped entirely? If dropped, existing users lose access unless a migration path exists. | UX, Architecture, Devil's Advocate |
|
||||
| Q-AUTH-2 | What is the migration path for existing Google-authenticated users? Options: dual auth during transition (UX recommends), forced migration, or pre-migration email campaign. | UX |
|
||||
| Q-AUTH-3 | How will the first admin account be created? Options: (a) CLI command at deployment, (b) first-user-is-admin, (c) environment variable with seed admin email. Security recommends (a); Architecture assumes env var. | Security, Architecture |
|
||||
| Q-AUTH-4 | Should registration be open (anyone can create an account) or require admin approval / invite-only? | Security |
|
||||
| Q-AUTH-5 | What session duration is appropriate? Architecture proposes 30 days; Security proposes 7 days absolute with 24-hour idle timeout. These differ significantly. | Architecture, Security |
|
||||
|
||||
### Architecture
|
||||
|
||||
| # | Question | Raised by |
|
||||
|---|---|---|
|
||||
| Q-ARCH-1 | How many concurrent users are expected? This determines whether SQLite is viable (fine for <5, problematic at 10-20 concurrent writers) or whether Postgres is needed from day one. | Architecture, Devil's Advocate |
|
||||
| Q-ARCH-2 | Is the Rust choice driven by the project's technical needs or by a learning goal? Both are valid but lead to different decisions. | Devil's Advocate |
|
||||
| Q-ARCH-3 | Is "without Docker" deployment a hard requirement or nice-to-have? Supporting both Docker and bare-metal doubles deployment documentation and testing. | Devil's Advocate |
|
||||
| Q-ARCH-4 | What is the data migration volume? How many existing users and syntheses need migrating? This determines whether a one-shot script suffices or incremental tooling is needed. | Architecture |
|
||||
| Q-ARCH-5 | Should legacy `SynthesisData` fields (majorAnnouncements, financialSector, etc.) be migrated to the `sections[]` format, or dropped entirely? | Architecture |
|
||||
| Q-ARCH-6 | What is the concrete timeline and deadline? Without one, scope expands indefinitely. | Devil's Advocate |
|
||||
| Q-ARCH-7 | What is the target response time for synthesis generation? Should generation be synchronous or asynchronous (start and get notified when done)? | Devil's Advocate |
|
||||
|
||||
### Scope
|
||||
|
||||
| # | Question | Raised by |
|
||||
|---|---|---|
|
||||
| Q-SCOPE-1 | Is this a refactoring or a complete rewrite? Every component is being replaced with zero code reuse. The project should be named and planned accordingly. | Devil's Advocate |
|
||||
| Q-SCOPE-2 | What happens if the project is abandoned at 50% completion? With a total rewrite, all investment is wasted. With phased delivery, each phase provides value independently. | Devil's Advocate |
|
||||
| Q-SCOPE-3 | Which LLM providers are actually needed on day one? If only Gemini, the multi-provider abstraction can be deferred (YAGNI). | Devil's Advocate |
|
||||
| Q-SCOPE-4 | What is the testing strategy? The current app has zero tests and the proposal does not mention testing. Is testing budgeted? | Devil's Advocate |
|
||||
|
||||
### UX
|
||||
|
||||
| # | Question | Raised by |
|
||||
|---|---|---|
|
||||
| Q-UX-1 | Will the new backend provide WebSocket/SSE for real-time updates, or will the app switch to polling? The current Firestore `onSnapshot` provides live updates on Home and Sources. Losing this is a perceived UX regression. | UX |
|
||||
| Q-UX-2 | Is internationalization (i18n) planned? The entire UI is currently in French. This affects how text is structured in the SolidJS rewrite. | UX |
|
||||
| Q-UX-3 | Who is "admin"? Is this a single-tenant self-hosted scenario (deployer = admin) or multi-tenant SaaS with super-admin? | UX |
|
||||
| Q-UX-4 | Can each user choose their preferred LLM provider, or does the admin configure one global provider? If user-level, do users bring their own API keys? | UX |
|
||||
| Q-UX-5 | What replaces the Gmail API email sending feature now that Google auth is being removed? Backend SMTP? Dedicated email service? Or is the feature dropped? | UX |
|
||||
|
||||
### Security
|
||||
|
||||
| # | Question | Raised by |
|
||||
|---|---|---|
|
||||
| Q-SEC-1 | Will there ever be shared syntheses between users (e.g., team workspaces)? This fundamentally changes the authorization model from strict per-user isolation. | Security |
|
||||
| Q-SEC-2 | For the master encryption key (API keys at rest), is an environment variable acceptable, or should a KMS (Vault, AWS KMS) be used? | Security |
|
||||
| Q-SEC-3 | Should LLM API rate limits be global (shared across all users) or per-user with a global ceiling? | Security |
|
||||
| Q-SEC-4 | Will you self-host SMTP (e.g., Postfix in Docker) or use an external transactional email service (Resend, SES, Mailgun)? This affects DNS configuration (SPF/DKIM/DMARC) and deliverability. | Security |
|
||||
| Q-SEC-5 | What is the budget for external services (email, LLM APIs, domain/TLS, VM hosting)? If budget is zero, Firebase's free tier is hard to beat. | Devil's Advocate |
|
||||
| Q-SEC-6 | Who will operate this in production? Self-hosted means self-operated: backups, TLS renewal, security patches, monitoring. Is someone committed to this ongoing work? | Devil's Advocate |
|
||||
| Q-SEC-7 | How will API keys stored in SQLite be protected, given that SQLite stores data in a plain file readable by anyone with file access? | Devil's Advocate |
|
||||
|
||||
### LLM Providers
|
||||
|
||||
| # | Question | Raised by |
|
||||
|---|---|---|
|
||||
| Q-LLM-1 | How will web search grounding be handled for non-Google providers? Gemini's `googleSearch` is deeply integrated; OpenAI and Anthropic have different or no equivalent. | UX, Architecture, Devil's Advocate |
|
||||
| Q-LLM-2 | Is the 2-pass generation pipeline (search + scrape + rewrite) non-negotiable? If it can be simplified for some providers, the multi-provider problem becomes much easier. | Devil's Advocate |
|
||||
| Q-LLM-3 | Are you building one abstraction over fundamentally different capabilities, or three separate pipelines behind a common interface? The first is a leaky abstraction; the second is honest but expensive. | Devil's Advocate |
|
||||
| Q-LLM-4 | Is the Gmail email sending feature (user sends from their own Gmail account) being kept, or replaced with system-sent email? These have different privacy implications. | Devil's Advocate |
|
||||
|
||||
---
|
||||
|
||||
## 4. Key Tensions
|
||||
|
||||
The following areas surface conflicting perspectives or implicit disagreements across the reports.
|
||||
|
||||
### Tension 1: Full Rewrite vs. Incremental Migration
|
||||
|
||||
The **Architect** (02) designs a comprehensive new system assuming the full rewrite is happening. The **Devil's Advocate** (04) argues this is reckless -- none of the current problems require a full rewrite, and a 200-line proxy server solves the two most critical issues (API key exposure and CORS). The Devil's Advocate proposes a 5-phase incremental plan where each phase delivers value independently, versus the all-or-nothing approach the Architect designs for.
|
||||
|
||||
### Tension 2: Rust as the Backend Language
|
||||
|
||||
The **Architect** (02) designs the entire backend around Rust/Axum with detailed code samples, crate selections, and compile-time query checking. The **Devil's Advocate** (04) challenges this directly: the application proxies LLM API calls that take 10-60 seconds each, so Rust's performance advantages are irrelevant. Go or Python/FastAPI would deliver the same user experience with 3-5x faster development velocity and a dramatically larger hiring pool.
|
||||
|
||||
### Tension 3: SolidJS vs. Keeping React
|
||||
|
||||
The **UX Specialist** (01) and **Architect** (02) accept SolidJS as a given and work through the migration risks. The **Devil's Advocate** (04) points out that the current React frontend is only ~900 lines across 7 files, and SolidJS provides zero user-visible benefit for an app with 5 pages. The cost/benefit ratio of the framework switch is questioned: React 19 or Svelte would avoid the rewrite cost or provide a larger ecosystem respectively.
|
||||
|
||||
### Tension 4: Custom Auth vs. Managed/Third-Party Auth
|
||||
|
||||
The **Architect** (02) designs a full custom auth system (magic links, sessions, captcha). The **Security Specialist** (03) provides detailed implementation guidance but acknowledges the complexity. The **Devil's Advocate** (04) warns this is "building a mini identity provider" with hidden work (email deliverability, token management, session storage, account recovery, rate limiting, GDPR compliance) and recommends self-hosted solutions like Authentik or Keycloak that provide all of this out of the box.
|
||||
|
||||
### Tension 5: SQLite vs. Postgres
|
||||
|
||||
The **Architect** (02) designs for SQLite with a claimed "Postgres upgrade path" via sqlx feature flags. The **Devil's Advocate** (04) calls this an upgrade path "myth" -- the SQL dialects differ enough that you either write to the lowest common denominator of both (losing benefits of each) or maintain separate query sets. The Devil's Advocate recommends starting with Postgres from the outset since Docker makes it trivial.
|
||||
|
||||
### Tension 6: Session Duration
|
||||
|
||||
The **Architect** (02) proposes 30-day sessions with extension on each request. The **Security Specialist** (03) recommends 7-day absolute expiration with a 24-hour idle timeout. For a weekly-use app, the **UX Specialist** (01) notes that session duration must be at least 7 days to avoid forcing weekly re-login. These three positions need reconciliation.
|
||||
|
||||
### Tension 7: CSRF Strategy
|
||||
|
||||
The **Architect** (02) recommends a custom `X-Requested-With` header as sufficient CSRF protection when combined with `SameSite=Lax`. The **Security Specialist** (03) recommends a full Synchronizer Token pattern (dedicated CSRF token stored server-side, sent via `X-CSRF-Token` header) plus Origin header validation for defense-in-depth. These are different approaches at different security/complexity tradeoffs.
|
||||
|
||||
### Tension 8: Captcha Provider
|
||||
|
||||
The **Architect** (02) recommends Cloudflare Turnstile (external SaaS). The **Security Specialist** (03) recommends mCaptcha (self-hosted, open-source) to align with the "no external dependencies" philosophy. The **UX Specialist** (01) recommends against Google reCAPTCHA but suggests hCaptcha or Turnstile for better UX. These need a decision based on the self-hosting requirement's strictness.
|
||||
|
||||
---
|
||||
|
||||
## 5. Recommended Reading Order
|
||||
|
||||
1. **`04-devils-advocate.md`** -- Read first. This report challenges every assumption and raises the most fundamental questions (should this rewrite happen at all? is the scope realistic?). Answering its 15 hard questions will shape every subsequent decision.
|
||||
|
||||
2. **`01-ux-analysis.md`** -- Read second. This provides a complete inventory of the current app's UX, identifies what must be preserved, and designs the new auth and admin flows. Understanding the user-facing surface area grounds the technical decisions.
|
||||
|
||||
3. **`02-architecture-analysis.md`** -- Read third. Once scope and UX are decided, this report provides the detailed technical blueprint: API design, database schema, project structure, LLM abstraction, and deployment configuration.
|
||||
|
||||
4. **`03-security-analysis.md`** -- Read last. This report audits the current vulnerabilities, then provides security requirements for every component designed by the Architect. It adds constraints and refinements (encryption, SSRF protection, threat model) that should be layered on top of the architecture.
|
||||
@ -0,0 +1,583 @@
|
||||
# UX Analysis: AI Weekly Synth Refactoring
|
||||
|
||||
**Author**: UX Specialist
|
||||
**Date**: 2026-03-21
|
||||
**Scope**: Complete UX audit of the current application and analysis of the planned refactoring impact on user experience.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions and Clarification Requests
|
||||
|
||||
Before diving into the analysis, the following ambiguities could lead to UX problems if not resolved early:
|
||||
|
||||
1. **Existing user migration path**: The current app uses Google SSO exclusively. Requirement #11 introduces email+captcha and magic link auth, but does not mention keeping Google SSO. Will existing Google-authenticated users be able to continue using Google login, or will they be forced to create an email-based account? This is a critical UX decision -- forcing migration risks losing users.
|
||||
|
||||
2. **Admin role definition**: Requirements #5 and #6 introduce admin modules (API keys, rate limiter). Who is "admin"? Is it a self-hosted single-tenant scenario where the person deploying is the admin? Or is there a multi-tenant SaaS model with a super-admin? The answer fundamentally changes the admin UX.
|
||||
|
||||
3. **Multi-provider selection granularity**: Requirement #4 says "support multiple LLM providers." Does the admin configure one global provider, or can each user choose their preferred provider? If user-level, do users bring their own API keys, or does the admin provision all keys? This affects both the Settings page and the Admin module.
|
||||
|
||||
4. **Email sending feature**: The current app sends synthesis by email via the Gmail API (OAuth popup for access token). With Google auth being removed, what replaces this? Backend SMTP? A dedicated email service? Or is this feature dropped?
|
||||
|
||||
5. **Real-time updates**: The current app uses Firestore `onSnapshot` for real-time data in Home and Sources pages. Will the new backend provide WebSocket/SSE equivalents, or will we switch to polling? This significantly impacts perceived responsiveness.
|
||||
|
||||
6. **Grounding / Web search capability**: The current pipeline uses Google Gemini's `googleSearch` tool for real-time web grounding. OpenAI and Anthropic do not have equivalent built-in web search. How will web search be handled for non-Google providers? This impacts what the user can expect from the generation.
|
||||
|
||||
7. **Locale/Language**: The entire UI is in French. Is internationalization planned? This affects how we structure text in the SolidJS rewrite.
|
||||
|
||||
---
|
||||
|
||||
## 1. Current UX Inventory
|
||||
|
||||
### 1.1 Screens and Routes
|
||||
|
||||
| Route | Screen | Purpose |
|
||||
|---|---|---|
|
||||
| `/login` | Login | Single-button Google SSO authentication |
|
||||
| `/` | Home (Dashboard) | List of all user syntheses as cards, with inline delete (double-click confirm) |
|
||||
| `/generate` | Generate Synthesis | One-click synthesis generation with loading/error/success feedback |
|
||||
| `/synthesis/:id` | Synthesis Detail | Full reading view of a synthesis, email sending, deletion with modal confirm |
|
||||
| `/sources` | Custom Sources | CRUD for custom URLs: single add, CSV import/export, bulk text import |
|
||||
| `/settings` | Settings | Generation parameters: theme, age window, categories (dynamic list), AI model, search agent behavior, JSON import/export |
|
||||
|
||||
### 1.2 Navigation Architecture
|
||||
|
||||
- **Top navigation bar** (persistent across all authenticated pages):
|
||||
- Left: Logo (BrainCircuit icon) + app name link to Home, then two nav links ("Syntheses", "Sources personnalisees")
|
||||
- Right: User email display, Settings gear icon link, Logout button
|
||||
- **No mobile hamburger menu**: The `sm:` prefix hides the nav links on small screens. The current app has a responsive gap on mobile -- the "Syntheses" and "Sources personnalisees" links disappear on narrow viewports with no alternative navigation.
|
||||
- **No breadcrumbs or active route indicator**: Nav links do not highlight the active page.
|
||||
|
||||
### 1.3 Interaction Patterns Catalog
|
||||
|
||||
#### Authentication
|
||||
- **Login**: Single button "Se connecter avec Google" triggers Firebase `signInWithPopup`. No form fields, no password.
|
||||
- **Logout**: Button in navbar triggers `signOut`, returns to `/login` via `ProtectedRoute` redirect.
|
||||
- **Session persistence**: Firebase `onAuthStateChanged` auto-restores session on reload.
|
||||
|
||||
#### Loading States
|
||||
- **Full-page spinner**: Used on initial auth check (`ProtectedRoute`) -- centered spinner, full viewport height.
|
||||
- **Section spinner**: Used on Home, Sources, Settings, SynthesisDetail -- centered spinner in a 256px-height container.
|
||||
- **Inline spinner**: Used on GenerateSynthesis button, Save Settings button, Send Email button, Delete buttons -- small spinner inside the button alongside text.
|
||||
- **No skeleton screens**: The app shows spinners, never content placeholders.
|
||||
|
||||
#### Data Feedback
|
||||
- **Success banners**: Green background with left border (GenerateSynthesis), green toast (SynthesisDetail email sent), green inline message (Settings save).
|
||||
- **Error banners**: Red background with left border (GenerateSynthesis), red inline message (Sources import, Settings save, SynthesisDetail email error).
|
||||
- **Auto-redirect after success**: GenerateSynthesis redirects to the new synthesis detail after 1.5 seconds.
|
||||
- **Auto-dismiss**: Delete confirmation on Home cards auto-cancels after 3 seconds if not confirmed.
|
||||
|
||||
#### Deletion Patterns
|
||||
- **Home (card list)**: Two-click inline pattern. First click shows "Confirmer" label + red background on the trash icon. Second click deletes. Auto-resets after 3 seconds.
|
||||
- **SynthesisDetail**: Separate delete button in header, opens a persistent confirmation banner with "Annuler" and "Confirmer la suppression" buttons. Does not auto-dismiss.
|
||||
- **Sources**: Single-click delete, no confirmation.
|
||||
|
||||
These three different deletion patterns are inconsistent. The refactoring is an opportunity to standardize.
|
||||
|
||||
#### Forms
|
||||
- **Settings**: All fields are inline-editable. Categories use a dynamic list with add/remove. Save button at bottom. No auto-save. Import/export JSON buttons in header.
|
||||
- **Sources**: Add form uses title + URL side-by-side fields with submit button. Bulk import uses a textarea. CSV import uses a hidden file input.
|
||||
- **No form validation UI**: Fields use HTML5 `required` attributes but no visible validation messages or field-level error states.
|
||||
|
||||
#### Content Display
|
||||
- **Home cards**: Grid layout (1/2/3 columns responsive), showing week number badge, creation date, preview of first section items (bullet points, line-clamped to 2 lines). Footer with "Lire la synthese" link and delete button.
|
||||
- **SynthesisDetail sections**: Vertical list of categorized sections. Each section has a title with bottom border, then cards for each news item (title as external link, summary paragraph).
|
||||
- **Legacy data handling**: Both Home and SynthesisDetail gracefully degrade to show old-format fields (`majorAnnouncements`, `financialSector`, etc.) when `sections[]` is absent.
|
||||
|
||||
### 1.4 Visual Design System
|
||||
|
||||
- **Color palette**: Indigo as primary (buttons, links, badges, focus rings), gray scale for text/backgrounds/borders, red for destructive actions, green for success states.
|
||||
- **Typography**: System defaults via Tailwind (no custom font import). Font sizes range from `text-xs` to `text-3xl`. Bold headings (`font-extrabold`, `font-bold`, `font-semibold`).
|
||||
- **Spacing**: Consistent use of Tailwind spacing scale. Max content width varies: `max-w-5xl` (Home), `max-w-4xl` (Sources, SynthesisDetail), `max-w-3xl` (Settings, GenerateSynthesis), `max-w-md` (Login).
|
||||
- **Components**: No formal component library. UI is built with raw Tailwind utility classes. Buttons, cards, form inputs are styled inline.
|
||||
- **Icons**: `lucide-react` throughout (BrainCircuit, LogOut, Settings, Plus, Trash2, FileText, Loader2, AlertCircle, CheckCircle2, ArrowLeft, ExternalLink, Mail, Send, AlertTriangle, Download, Upload, Link).
|
||||
- **Shadows and borders**: Cards use `shadow-sm` with `border-gray-200`, hover state adds `shadow-md` and `border-indigo-300`.
|
||||
- **Animations**: Only `animate-spin` on loader spinners. No page transitions, no entrance animations.
|
||||
|
||||
---
|
||||
|
||||
## 2. Migration Risks: React to SolidJS
|
||||
|
||||
### 2.1 Framework-Level Risks
|
||||
|
||||
| Risk | Severity | Detail |
|
||||
|---|---|---|
|
||||
| **Reactivity model change** | Medium | React uses re-rendering on state change; SolidJS uses fine-grained reactivity with signals. Components that rely on `useState` + re-render cycles need careful translation to `createSignal`/`createStore`. Incorrect translation can cause stale UI. |
|
||||
| **Context API differences** | High | `AuthContext` uses React Context + `useContext`. SolidJS has its own Context API, but reactive propagation works differently. The `useAuth()` hook pattern must be recreated as a SolidJS context with signals. |
|
||||
| **Lifecycle hooks** | Medium | `useEffect` with dependency arrays has no direct SolidJS equivalent. SolidJS uses `createEffect` (auto-tracking), `onMount`, and `onCleanup`. The Firestore `onSnapshot` subscriptions in Home, Sources, and SynthesisDetail rely heavily on `useEffect` cleanup -- these must be carefully ported to `onCleanup`. |
|
||||
| **Conditional rendering** | Low | React uses ternary/`&&` operators; SolidJS uses `<Show>` and `<For>` components. Straightforward but needs systematic conversion. |
|
||||
| **Router** | Medium | Moving from `react-router-dom` v7 to SolidJS Router (`@solidjs/router`). API is similar but different: `useParams`, `useNavigate`, `<A>` instead of `<Link>`, `<Route>` structure. `ProtectedRoute` must be reimplemented as a route guard or layout route. |
|
||||
| **Event handling** | Low | Mostly identical (`onClick`, `onChange`, etc.). SolidJS uses `on:click` in some patterns but supports standard JSX event handlers. |
|
||||
|
||||
### 2.2 UI/UX-Specific Risks
|
||||
|
||||
| Risk | Impact on UX |
|
||||
|---|---|
|
||||
| **Loss of real-time updates** | Home and Sources currently use Firestore `onSnapshot` for live updates. If the new backend uses REST API instead, users will not see new/deleted items without refreshing. This is a perceived UX regression. Mitigation: implement WebSocket/SSE for list subscriptions, or at minimum, optimistic UI updates + periodic polling. |
|
||||
| **Loading state timing changes** | Currently, Firestore queries resolve in ~100-500ms. REST API calls to a Rust backend may be faster, but they also introduce HTTP overhead and network latency that Firestore's SDK (with local cache) does not have. Loading spinners may appear more frequently or for different durations. |
|
||||
| **Icon library change** | `lucide-react` must be replaced by `lucide-solid` or a generic SVG approach. The icon set is identical, but the import pattern changes. Risk of missing icons or inconsistent sizing. |
|
||||
| **Tailwind CSS compatibility** | Tailwind is framework-agnostic. The utility classes can be copied verbatim to SolidJS JSX. The main risk is `className` vs `class` -- SolidJS uses `class` (standard HTML attribute). This is a systematic find-and-replace, but if missed, styles silently break with no visual output. |
|
||||
| **Date formatting** | `date-fns` is framework-agnostic and can be reused. But `Timestamp.toDate()` from Firestore must be replaced by standard ISO date string parsing from the API. |
|
||||
| **Gmail email popup flow** | Currently relies on Firebase `signInWithPopup` to get a Gmail OAuth token. This entire flow disappears. Email sending must be redesigned (see Section 4). |
|
||||
|
||||
### 2.3 Recommendations for Safe Migration
|
||||
|
||||
1. **Build a component-by-component mapping** before starting. Every React component should have a SolidJS counterpart spec'd with identical props and behavior.
|
||||
2. **Create a SolidJS design system** with extracted, reusable primitives (Button, Card, Spinner, Badge, FormField, ConfirmDialog) to replace the inline Tailwind patterns. This reduces duplication risk.
|
||||
3. **Visual regression testing**: Capture screenshots of every screen/state in the current app. Compare against the SolidJS implementation pixel-by-pixel.
|
||||
4. **Preserve all Tailwind classes verbatim** (modulo `className` -> `class`).
|
||||
5. **Keep `lucide` icons**: Use `lucide-solid` or direct SVG imports to maintain visual consistency.
|
||||
|
||||
---
|
||||
|
||||
## 3. New UX Flows: Authentication
|
||||
|
||||
### 3.1 Current State
|
||||
|
||||
- Single button: "Se connecter avec Google"
|
||||
- No sign-up flow (Google creates the account implicitly)
|
||||
- No password management, no email verification
|
||||
- Session persisted via Firebase auth token (auto-refresh)
|
||||
|
||||
### 3.2 Proposed New Flows
|
||||
|
||||
#### 3.2.1 Sign-Up (Email + Captcha)
|
||||
|
||||
**Screen**: `/signup`
|
||||
|
||||
**Flow**:
|
||||
1. User lands on Login page, sees "Creer un compte" link below the login form.
|
||||
2. Sign-up form with fields:
|
||||
- Email address (required, validated format)
|
||||
- Display name (optional, for navbar greeting)
|
||||
- CAPTCHA widget (e.g., hCaptcha or Cloudflare Turnstile -- avoid Google reCAPTCHA to stay provider-agnostic)
|
||||
3. User submits. Backend creates account, sends a magic link to the email.
|
||||
4. User sees a confirmation screen: "Un lien de connexion vous a ete envoye a [email]. Verifiez votre boite de reception."
|
||||
5. User clicks the magic link in their email, which authenticates them and redirects to `/`.
|
||||
|
||||
**UX considerations**:
|
||||
- No password field. The magic link serves as both email verification and initial authentication. This is simpler, more secure, and avoids the need for password complexity rules, forgot-password flows, and password storage.
|
||||
- CAPTCHA should be invisible or minimally intrusive (Turnstile "managed" mode). A full visual puzzle challenge degrades conversion.
|
||||
- The sign-up confirmation screen must handle edge cases: email not received (resend button with cooldown), email in spam, typo in email (back button to re-enter).
|
||||
|
||||
#### 3.2.2 Login (Magic Link / Passwordless)
|
||||
|
||||
**Screen**: `/login`
|
||||
|
||||
**Flow**:
|
||||
1. User enters their email address.
|
||||
2. User completes CAPTCHA (if not previously solved in session).
|
||||
3. Backend sends a magic link to the email.
|
||||
4. User sees: "Un lien de connexion vous a ete envoye a [email]."
|
||||
5. User clicks the link, which sets a session cookie and redirects to `/`.
|
||||
|
||||
**UX considerations**:
|
||||
- The magic link should expire (e.g., 15 minutes) and be single-use.
|
||||
- If the user tries to log in with an unregistered email, the system should respond with the same message ("link sent") to prevent email enumeration attacks. Alternatively, offer to create the account.
|
||||
- Provide a "Renvoyer le lien" button with a cooldown timer (e.g., 60 seconds) on the confirmation screen.
|
||||
- Consider deep-link support: if the user clicks the magic link on a different device/browser, the new session should be created there instead.
|
||||
|
||||
#### 3.2.3 Session Management
|
||||
|
||||
**Current**: Firebase manages JWT tokens client-side, auto-refreshes.
|
||||
**Proposed**: Session-based auth with secure cookies (HttpOnly, Secure, SameSite=Strict).
|
||||
|
||||
**UX implications**:
|
||||
- Session expiry must be handled gracefully. If a session expires while the user is on a page, the next API call will return 401. The frontend must catch this and redirect to `/login` with a message: "Votre session a expire. Veuillez vous reconnecter."
|
||||
- No more `onAuthStateChanged` listener. The SolidJS auth context must check session validity on app load (e.g., `GET /api/auth/me` endpoint).
|
||||
- Consider a session duration of at least 7 days (with sliding window) to avoid forcing weekly login on a weekly-use app.
|
||||
- Logout must call a backend endpoint to invalidate the server-side session, not just clear client state.
|
||||
|
||||
#### 3.2.4 Login Page Redesign
|
||||
|
||||
The new login page should accommodate both login and sign-up while preserving the visual identity:
|
||||
|
||||
```
|
||||
+----------------------------------------------+
|
||||
| [BrainCircuit icon] |
|
||||
| AI Weekly Synth |
|
||||
| Votre synthese hebdomadaire des actualites |
|
||||
| |
|
||||
| [Email address field ] |
|
||||
| [ Turnstile CAPTCHA widget ] |
|
||||
| [ Recevoir un lien de connexion ] |
|
||||
| |
|
||||
| Pas encore de compte ? Creer un compte |
|
||||
+----------------------------------------------+
|
||||
```
|
||||
|
||||
For sign-up, swap to:
|
||||
```
|
||||
| [Email address field ] |
|
||||
| [Display name (optionnel) ] |
|
||||
| [ Turnstile CAPTCHA widget ] |
|
||||
| [ Creer mon compte ] |
|
||||
| |
|
||||
| Deja un compte ? Se connecter |
|
||||
```
|
||||
|
||||
### 3.3 Migration Path for Existing Google SSO Users
|
||||
|
||||
This is the most sensitive UX challenge. Options:
|
||||
|
||||
**Option A (Recommended): Dual auth during transition period**
|
||||
- Keep Google SSO as an option alongside the new email-based auth for 3-6 months.
|
||||
- When a Google-authenticated user logs in, prompt them to add an email address to their account: "Pour assurer la continuite de votre compte, veuillez associer une adresse email."
|
||||
- Once the email is associated, both login methods work.
|
||||
- After the transition period, remove Google SSO.
|
||||
- Pro: Zero friction for existing users. Con: Maintains two auth systems temporarily.
|
||||
|
||||
**Option B: Forced migration on first visit**
|
||||
- On first visit to the new app, the old Google session is invalid.
|
||||
- Show a migration screen: "Nous avons mis a jour notre systeme d'authentification. Entrez l'adresse email associee a votre compte Google pour recevoir un lien de connexion."
|
||||
- Backend matches the email from the old Google auth record to migrate the account.
|
||||
- Pro: Clean cut. Con: Users with different Google/personal emails may struggle.
|
||||
|
||||
**Option C: Pre-migration email campaign**
|
||||
- Before deploying the new auth system, send an email to all existing users explaining the change and providing a pre-migration link.
|
||||
- Pro: Users are prepared. Con: Some users will ignore the email and be confused.
|
||||
|
||||
**Recommendation**: Option A provides the smoothest UX. Existing users notice no disruption while new users get the new flow.
|
||||
|
||||
---
|
||||
|
||||
## 4. Admin Module UX
|
||||
|
||||
### 4.1 Access Model
|
||||
|
||||
**Recommendation**: The first user to register becomes the admin (self-hosted model). Alternatively, admin is designated via environment variable or CLI command at deployment time. There should be no in-app admin registration -- it is a deployment-time decision.
|
||||
|
||||
Admin pages should be:
|
||||
- Hidden from navigation for non-admin users (not just disabled -- completely absent from the DOM).
|
||||
- Protected by both frontend route guards and backend authorization.
|
||||
- Accessible via a gear/shield icon in the navbar, only visible to admins.
|
||||
|
||||
### 4.2 Admin Navigation
|
||||
|
||||
Add an admin section to the navbar (visible only to admin users):
|
||||
|
||||
```
|
||||
[Logo] AI Weekly Synth | Syntheses | Sources | [Admin dropdown: Configuration LLM | Limites d'usage] | [user@email] [Settings] [Logout]
|
||||
```
|
||||
|
||||
Or a dedicated `/admin` layout with sidebar:
|
||||
```
|
||||
/admin/llm-providers -- API key management
|
||||
/admin/rate-limits -- Rate limiter configuration
|
||||
/admin/users -- (future) User management
|
||||
```
|
||||
|
||||
**Recommendation**: Use a separate `/admin` route prefix with its own layout. Keeps admin concerns cleanly separated and makes authorization simpler.
|
||||
|
||||
### 4.3 API Key Configuration Page (`/admin/llm-providers`)
|
||||
|
||||
**Layout**:
|
||||
```
|
||||
+------------------------------------------------------+
|
||||
| Configuration des fournisseurs LLM |
|
||||
| |
|
||||
| [Google Gemini] [OpenAI] [Anthropic] |
|
||||
| (card/tab per provider) |
|
||||
| |
|
||||
| --- Google Gemini --- |
|
||||
| Cle API: [********************] [Show/Hide] [Test] |
|
||||
| Statut: Connecte (dernier test: il y a 2 min) |
|
||||
| Modeles disponibles: |
|
||||
| [x] gemini-3.1-pro-preview |
|
||||
| [x] gemini-3-flash-preview |
|
||||
| [ ] gemini-3.1-flash-lite-preview |
|
||||
| Modele par defaut: [dropdown] |
|
||||
| |
|
||||
| [Enregistrer] |
|
||||
+------------------------------------------------------+
|
||||
```
|
||||
|
||||
**Key UX decisions**:
|
||||
- API keys must be masked by default (show dots), with a toggle to reveal.
|
||||
- A "Test connection" button per provider validates the key and lists available models.
|
||||
- Clear status indicators: green dot = connected/valid, red dot = invalid/expired, gray dot = not configured.
|
||||
- The list of enabled models feeds the dropdown in user Settings.
|
||||
|
||||
### 4.4 Rate Limiter Configuration Page (`/admin/rate-limits`)
|
||||
|
||||
**Layout**:
|
||||
```
|
||||
+------------------------------------------------------+
|
||||
| Configuration des limites d'usage |
|
||||
| |
|
||||
| --- Limites globales --- |
|
||||
| Requetes max par minute (tous utilisateurs): [30] |
|
||||
| Requetes max par utilisateur par jour: [10] |
|
||||
| |
|
||||
| --- Par fournisseur --- |
|
||||
| Google Gemini: |
|
||||
| Requetes max / minute: [29] |
|
||||
| OpenAI: |
|
||||
| Requetes max / minute: [60] |
|
||||
| Anthropic: |
|
||||
| Requetes max / minute: [50] |
|
||||
| |
|
||||
| [Enregistrer] |
|
||||
+------------------------------------------------------+
|
||||
```
|
||||
|
||||
**Key UX decisions**:
|
||||
- Show current usage stats if available (requests today, current rate).
|
||||
- Explain what happens when a limit is reached: "L'utilisateur verra un message d'erreur lui demandant de reessayer plus tard."
|
||||
- Provide sensible defaults that match free-tier quotas of each provider.
|
||||
|
||||
---
|
||||
|
||||
## 5. Multi-Provider UX: Impact on Settings Page
|
||||
|
||||
### 5.1 Current Settings Page Structure
|
||||
|
||||
The Settings page currently has one "Modele d'IA" dropdown with 4 Gemini-only options, hardcoded in the JSX.
|
||||
|
||||
### 5.2 Proposed Redesign
|
||||
|
||||
Replace the single dropdown with a two-level selection:
|
||||
|
||||
```
|
||||
Fournisseur d'IA:
|
||||
[Google Gemini v] (dropdown, only shows providers configured by admin)
|
||||
|
||||
Modele:
|
||||
[gemini-3.1-pro-preview (conseille) v] (dropdown, populated dynamically based on selected provider)
|
||||
```
|
||||
|
||||
**UX considerations**:
|
||||
- If the admin has only configured one provider, hide the provider dropdown entirely and show just the model dropdown. This avoids unnecessary complexity for simple setups.
|
||||
- If a previously selected provider is removed by the admin, show a warning on the Settings page: "Le fournisseur que vous utilisiez n'est plus disponible. Veuillez en selectionner un autre."
|
||||
- Display a brief description of each model (speed vs. quality tradeoff) to help users choose.
|
||||
- The "Comportement de l'agent de recherche" textarea should include a note about web search availability: "Note : la recherche web en temps reel est disponible uniquement avec Google Gemini."
|
||||
|
||||
### 5.3 Impact on GenerateSynthesis Page
|
||||
|
||||
The generation confirmation text should show the selected provider and model:
|
||||
> "Cette action va lancer l'analyse des actualites des **7 derniers jours** sur le theme **"Intelligence Artificielle"** via **Google Gemini (gemini-3.1-pro-preview)**."
|
||||
|
||||
If the selected provider does not support web search grounding, add a warning:
|
||||
> "Note : Le fournisseur selectionne ne dispose pas de la recherche web integree. Les resultats seront bases sur les connaissances du modele uniquement."
|
||||
|
||||
---
|
||||
|
||||
## 6. Loading States and Feedback with Backend AI Calls
|
||||
|
||||
### 6.1 Current Behavior
|
||||
|
||||
- GenerateSynthesis shows a single spinner with "Generation en cours..." for the entire process.
|
||||
- The process runs in the browser (Gemini API calls + CORS proxy scraping), typically taking 30-90 seconds.
|
||||
- No progress indication, no intermediate feedback.
|
||||
- If the user navigates away, the generation is lost.
|
||||
|
||||
### 6.2 New Backend Processing: UX Implications
|
||||
|
||||
Moving AI calls to the backend changes the timing and feedback model:
|
||||
|
||||
**Advantages**:
|
||||
- The generation survives page navigation (backend keeps processing).
|
||||
- No CORS proxy issues (backend fetches URLs directly).
|
||||
- API keys are secure.
|
||||
|
||||
**New challenges**:
|
||||
- Longer perceived wait if no intermediate feedback.
|
||||
- HTTP request timeout risk (if generation takes > 60s, reverse proxies may cut the connection).
|
||||
- User may think the app is frozen.
|
||||
|
||||
### 6.3 Recommended Approach: Server-Sent Events (SSE) for Progress
|
||||
|
||||
**Flow**:
|
||||
1. User clicks "Lancer la generation."
|
||||
2. Frontend sends `POST /api/syntheses/generate` which returns immediately with a `task_id`.
|
||||
3. Frontend opens an SSE connection: `GET /api/syntheses/generate/{task_id}/progress`.
|
||||
4. Backend sends progress events:
|
||||
```
|
||||
event: progress
|
||||
data: {"step": "search", "message": "Recherche d'actualites en cours...", "percent": 10}
|
||||
|
||||
event: progress
|
||||
data: {"step": "scraping", "message": "Verification des sources (3/12)...", "percent": 40}
|
||||
|
||||
event: progress
|
||||
data: {"step": "rewrite", "message": "Redaction des resumes...", "percent": 75}
|
||||
|
||||
event: progress
|
||||
data: {"step": "saving", "message": "Sauvegarde...", "percent": 95}
|
||||
|
||||
event: complete
|
||||
data: {"synthesis_id": "abc123"}
|
||||
|
||||
event: error
|
||||
data: {"message": "Quota depasse. Reessayez plus tard."}
|
||||
```
|
||||
5. Frontend displays a progress bar with step descriptions.
|
||||
|
||||
**Proposed UI for generation page**:
|
||||
```
|
||||
+----------------------------------------------+
|
||||
| Generer la Synthese Hebdomadaire |
|
||||
| |
|
||||
| [========================------] 75% |
|
||||
| Redaction des resumes... |
|
||||
| |
|
||||
| Etapes: |
|
||||
| [done] Recherche d'actualites |
|
||||
| [done] Verification des 12 sources |
|
||||
| [>> ] Redaction des resumes |
|
||||
| [ ] Sauvegarde |
|
||||
| |
|
||||
| Note : Vous pouvez quitter cette page. |
|
||||
| La generation continuera en arriere-plan. |
|
||||
+----------------------------------------------+
|
||||
```
|
||||
|
||||
**If the user navigates away and comes back**:
|
||||
- The Home page should show a banner: "Une generation est en cours... [Voir la progression]"
|
||||
- Or the new synthesis appears in the list with a "En cours..." badge that updates in real-time.
|
||||
|
||||
### 6.4 Fallback: Polling
|
||||
|
||||
If SSE is not feasible in the first iteration:
|
||||
1. `POST /api/syntheses/generate` returns `{ "task_id": "abc123" }`.
|
||||
2. Frontend polls `GET /api/syntheses/generate/{task_id}/status` every 3 seconds.
|
||||
3. UI shows an indeterminate spinner with rotating messages:
|
||||
- "Recherche en cours..."
|
||||
- "Analyse des sources..."
|
||||
- "Redaction des resumes..."
|
||||
4. On completion, auto-redirect to synthesis detail page (preserving current behavior).
|
||||
|
||||
### 6.5 Streaming the Synthesis Content
|
||||
|
||||
A more ambitious approach: stream the synthesis content as it is generated (section by section). The user sees sections appear progressively, similar to a chat interface. This provides the richest feedback but requires significant backend work (streaming JSON fragments or markdown).
|
||||
|
||||
**Recommendation for MVP**: Implement the SSE progress approach. Reserve streaming content for a later iteration.
|
||||
|
||||
---
|
||||
|
||||
## 7. Email Feature Redesign
|
||||
|
||||
### 7.1 Current Implementation
|
||||
|
||||
- Hardcoded default email (`olivier.abrivard@desjardins.com` -- this should be removed/replaced).
|
||||
- Uses Gmail API directly from the frontend via OAuth popup.
|
||||
- Tied to Google authentication.
|
||||
|
||||
### 7.2 Proposed Replacement
|
||||
|
||||
**Backend SMTP approach**:
|
||||
- Move email sending to `POST /api/syntheses/{id}/send-email` on the backend.
|
||||
- Admin configures SMTP settings in the admin panel (or via environment variables): SMTP host, port, username, password, sender address.
|
||||
- User enters recipient email on the SynthesisDetail page (same UI as today, minus the Gmail-specific parts).
|
||||
- Default recipient should be the user's own email address (from their account).
|
||||
|
||||
**UX changes**:
|
||||
- Remove the Gmail OAuth popup entirely.
|
||||
- The "Envoyer par email" button triggers a backend call.
|
||||
- Add a small loading spinner on the button (already exists in current UI).
|
||||
- Success/error feedback remains the same.
|
||||
- Consider adding a "S'envoyer a soi-meme" quick button that pre-fills with the user's account email.
|
||||
|
||||
### 7.3 Admin SMTP Configuration Page
|
||||
|
||||
If email functionality requires admin SMTP setup, add a section to the admin area:
|
||||
```
|
||||
--- Configuration email ---
|
||||
Serveur SMTP: [smtp.example.com]
|
||||
Port: [587]
|
||||
Utilisateur: [user@example.com]
|
||||
Mot de passe: [********]
|
||||
Adresse d'expedition: [noreply@example.com]
|
||||
Chiffrement: [STARTTLS v]
|
||||
[Tester l'envoi] [Enregistrer]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Additional UX Improvements to Include in the Refactoring
|
||||
|
||||
The refactoring is an opportunity to fix existing UX issues. The following are not new requirements but improvements that should be made while the UI is being rewritten.
|
||||
|
||||
### 8.1 Mobile Navigation
|
||||
|
||||
**Problem**: The current nav links ("Syntheses", "Sources personnalisees") are hidden on mobile (`hidden sm:ml-6 sm:flex`) with no alternative.
|
||||
**Fix**: Add a hamburger menu for mobile viewports, or a bottom navigation bar.
|
||||
|
||||
### 8.2 Consistent Deletion Pattern
|
||||
|
||||
**Problem**: Three different deletion patterns across the app (inline double-click, modal confirm, single-click no confirm).
|
||||
**Fix**: Standardize on a confirmation dialog (modal or inline banner) for all destructive actions.
|
||||
|
||||
### 8.3 Active Route Indicator
|
||||
|
||||
**Problem**: Nav links do not indicate which page is active.
|
||||
**Fix**: Add `border-indigo-500 text-gray-900` classes to the active link (the border-b-2 is already in the HTML, just always set to `border-transparent`).
|
||||
|
||||
### 8.4 Empty States
|
||||
|
||||
**Problem**: Sources and Home have empty states, but they could be richer.
|
||||
**Fix**: Add contextual onboarding hints. E.g., on first visit to Sources: "L'ajout de sources permet a l'IA de consulter vos sites preferes en priorite."
|
||||
|
||||
### 8.5 Error Boundary
|
||||
|
||||
**Problem**: No error boundary exists. A crash in any component brings down the entire app.
|
||||
**Fix**: Add a top-level error boundary in SolidJS (`ErrorBoundary` component) with a user-friendly fallback page.
|
||||
|
||||
### 8.6 Form Validation
|
||||
|
||||
**Problem**: No visible validation feedback on form fields.
|
||||
**Fix**: Add field-level error messages (red text below fields) for invalid inputs (e.g., malformed URL, empty required fields, invalid email format on login).
|
||||
|
||||
### 8.7 Accessibility
|
||||
|
||||
**Problem**: Screen reader labels are minimal (`sr-only` labels on some inputs). No `aria-live` regions for dynamic content updates. No keyboard navigation testing.
|
||||
**Fix**: Audit all interactive elements for ARIA attributes, ensure focus management on route changes, add `aria-live="polite"` on success/error messages.
|
||||
|
||||
### 8.8 Hardcoded Email
|
||||
|
||||
**Problem**: `SynthesisDetail.tsx` line 41 hardcodes `olivier.abrivard@desjardins.com` as the default email.
|
||||
**Fix**: Default to the authenticated user's email address, or empty.
|
||||
|
||||
---
|
||||
|
||||
## 9. Summary of UX Impact by Requirement
|
||||
|
||||
| Req # | Requirement | UX Impact | Risk Level |
|
||||
|---|---|---|---|
|
||||
| 1 | Rust + SQLite backend | Invisible to user if API contracts match | Low |
|
||||
| 2 | Convert Firestore rules | Invisible to user | None |
|
||||
| 3 | Move AI calls to backend | Major: loading/feedback redesign, generation survives navigation | High |
|
||||
| 4 | Multiple LLM providers | Medium: Settings page redesign, provider-dependent feature availability | Medium |
|
||||
| 5 | Admin API key config | New screens for admin only | Medium |
|
||||
| 6 | Admin rate limiter config | New screens for admin only | Low |
|
||||
| 7 | SolidJS frontend | Full rewrite risk, every screen must be verified | High |
|
||||
| 8 | Remove legacy data/code | Positive: cleaner data model, simpler rendering | Low |
|
||||
| 9 | Improve CORS proxy | Invisible to user (backend handles it now) | None |
|
||||
| 10 | Docker deployment | Invisible to user | None |
|
||||
| 11 | New auth system | Major: new login/signup flows, migration of existing users | Critical |
|
||||
|
||||
### Priority Order for UX Work
|
||||
|
||||
1. **Auth system design** (Req #11) -- must be finalized first as it gates all other flows.
|
||||
2. **SolidJS component mapping** (Req #7) -- defines the implementation structure for everything.
|
||||
3. **Generation feedback/progress** (Req #3) -- highest-impact user-facing change.
|
||||
4. **Multi-provider Settings** (Req #4) -- affects core user workflow.
|
||||
5. **Admin module design** (Req #5, #6) -- separate from main user flows, can be parallelized.
|
||||
6. **Legacy cleanup** (Req #8) -- simplifies rendering code during rewrite.
|
||||
|
||||
---
|
||||
|
||||
## 10. File Reference
|
||||
|
||||
All analysis is based on the following source files:
|
||||
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/CLAUDE.md` -- project documentation
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/App.tsx` -- router, layout, login, protected routes
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/pages/Home.tsx` -- dashboard with synthesis list
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/pages/GenerateSynthesis.tsx` -- generation trigger page
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/pages/SynthesisDetail.tsx` -- synthesis reading view + email
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/pages/Sources.tsx` -- custom sources CRUD
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/pages/Settings.tsx` -- user settings
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/components/AuthContext.tsx` -- auth context provider
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/types.ts` -- TypeScript interfaces and defaults
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/services/geminiService.ts` -- AI generation pipeline
|
||||
- `/Users/oabrivard/Projects/rust/ai_synth/src/firebase.ts` -- Firebase init, auth helpers, error handling
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,786 @@
|
||||
# Security Analysis: AI Weekly Synth Refactoring
|
||||
|
||||
**Role**: Security Specialist
|
||||
**Date**: 2026-03-21
|
||||
**Scope**: Full security audit of the current application and security architecture for the Rust/SolidJS refactoring
|
||||
|
||||
---
|
||||
|
||||
## Questions Requiring User Decision
|
||||
|
||||
Before implementation begins, the following security-sensitive questions need answers:
|
||||
|
||||
1. **Admin bootstrapping**: How will the first admin account be created? Options: (a) CLI command during deployment, (b) first-user-is-admin, (c) environment variable with seed admin email. Option (a) is recommended -- (b) is dangerous in production, (c) leaks info in env vars.
|
||||
|
||||
2. **Multi-tenancy scope**: Will there ever be shared syntheses between users (e.g., team workspaces)? This fundamentally affects the authorization model. The current analysis assumes strict per-user isolation.
|
||||
|
||||
3. **Self-registration**: Should anyone be able to create an account, or should there be an admin-approval flow or invite-only mechanism? Open registration with captcha is assumed below.
|
||||
|
||||
4. **Email provider for magic links**: Will you self-host SMTP (e.g., via Postfix in the Docker stack) or use an external transactional email service (Resend, AWS SES, Mailgun)? This affects DNS configuration (SPF/DKIM/DMARC) and deliverability. External service is recommended.
|
||||
|
||||
5. **Master encryption key management**: For encrypting LLM API keys at rest, are you comfortable storing the master key in an environment variable, or do you want to integrate with a KMS (e.g., HashiCorp Vault, AWS KMS)? Environment variable is assumed below for single-VM simplicity.
|
||||
|
||||
6. **Rate limiting granularity**: Should LLM API rate limits be global (shared across all users) or per-user? Per-user is recommended, with a global ceiling.
|
||||
|
||||
---
|
||||
|
||||
## 1. Current Security Issues
|
||||
|
||||
### 1.1 CRITICAL: Gemini API Key Exposed in Frontend Bundle
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/vite.config.ts` (line 11)
|
||||
```typescript
|
||||
'process.env.GEMINI_API_KEY': JSON.stringify(env.GEMINI_API_KEY),
|
||||
```
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/services/geminiService.ts` (line 4)
|
||||
```typescript
|
||||
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
|
||||
```
|
||||
|
||||
The Gemini API key is injected at build time via Vite's `define` and embedded as a string literal in the client-side JavaScript bundle. Anyone loading the page can extract the key from browser DevTools (Sources tab or Network tab) and use it to make arbitrary Gemini API calls at the owner's expense.
|
||||
|
||||
**Impact**: Financial abuse (API billing), quota exhaustion, potential data exfiltration if the key grants access to other Google Cloud resources.
|
||||
|
||||
**Mitigation in refactoring**: All LLM API calls move to the Rust backend. API keys never leave the server process.
|
||||
|
||||
### 1.2 HIGH: Gmail OAuth Token Handling
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/firebase.ts` (lines 19-30)
|
||||
|
||||
```typescript
|
||||
export const getGmailAccessToken = async (): Promise<string | null> => {
|
||||
const provider = new GoogleAuthProvider();
|
||||
provider.addScope('https://www.googleapis.com/auth/gmail.send');
|
||||
// ...
|
||||
return credential?.accessToken || null;
|
||||
};
|
||||
```
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/pages/SynthesisDetail.tsx` (lines 112-171)
|
||||
|
||||
Issues:
|
||||
- The `gmail.send` OAuth scope grants the ability to send emails as the user. The access token is obtained client-side and used to call the Gmail API directly from the browser.
|
||||
- Each email send triggers a full `signInWithPopup` flow, re-requesting the `gmail.send` scope. This is disruptive UX but also means the token is short-lived (good). However, the token is held in JavaScript memory and could be intercepted by XSS.
|
||||
- The email recipient field (line 41) is hardcoded to a specific email: `olivier.abrivard@desjardins.com`. This is PII committed to the repository.
|
||||
|
||||
**Mitigation in refactoring**: Email sending should move to the backend. The backend sends emails using its own SMTP credentials, never exposing OAuth tokens to the client.
|
||||
|
||||
### 1.3 HIGH: Prompt Injection via User-Controlled Input
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/services/geminiService.ts` (lines 85-100)
|
||||
|
||||
User-controlled fields are interpolated directly into LLM prompts without sanitization:
|
||||
- `settings.theme` (line 87): User-defined string injected into `"Tu es un expert en analyse de l'actualite sur le theme : "${settings.theme}""`
|
||||
- `settings.searchAgentBehavior` (line 92): Free-text prompt injected verbatim -- this is literally a prompt injection vector by design
|
||||
- `settings.categories` (line 83): Array of user strings injected as numbered list items
|
||||
- `customSources[].title` and `customSources[].url` (line 62): Injected as source list
|
||||
|
||||
A malicious user could craft `theme`, `searchAgentBehavior`, or category names that override the system prompt behavior, potentially causing the LLM to:
|
||||
- Ignore safety guidelines
|
||||
- Generate harmful or misleading content
|
||||
- Exfiltrate data via crafted URLs in grounding results
|
||||
|
||||
**Mitigation in refactoring**: While users inherently need to customize prompts, the backend should:
|
||||
- Enforce maximum lengths for all user-provided prompt fragments
|
||||
- Apply a sanitization layer that strips common injection patterns (e.g., "ignore previous instructions", "system:", role-switching patterns)
|
||||
- Log and monitor unusual prompt patterns
|
||||
- Use structured prompt templates where user input is clearly delimited as data, not instructions
|
||||
|
||||
### 1.4 HIGH: CORS Proxy Data Leakage
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/services/geminiService.ts` (lines 174-213)
|
||||
|
||||
Three third-party CORS proxies are used in cascade:
|
||||
1. `api.allorigins.win`
|
||||
2. `api.codetabs.com`
|
||||
3. `corsproxy.io`
|
||||
|
||||
Issues:
|
||||
- **Data exfiltration**: Every URL the user scrapes (their custom sources, AI-generated article URLs) is sent to these third-party services. They can log, modify, or block content.
|
||||
- **Man-in-the-middle**: The proxied HTML content could be tampered with. If an attacker controls one of these services, they could inject malicious content into scraped pages.
|
||||
- **Availability**: These are free, community-run services with no SLA. They can disappear at any time.
|
||||
- **No timeout or size limits**: The `fetch` calls have no explicit timeout or response size limit, potentially causing the browser to hang or consume excessive memory.
|
||||
|
||||
**Mitigation in refactoring**: The Rust backend performs HTTP requests directly (no CORS restriction server-side). This eliminates the need for proxies entirely. Add SSRF protections (see Section 5.4).
|
||||
|
||||
### 1.5 MEDIUM: Firebase Config Committed to Repository
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/firebase-applet-config.json`
|
||||
|
||||
Firebase configuration (API key, project ID, app ID) is committed to the repository. While Firebase API keys are designed to be public (they are restricted by Firebase Security Rules and authorized domains), committing them creates a false sense of security and makes key rotation harder.
|
||||
|
||||
### 1.6 MEDIUM: Hardcoded PII in Source Code
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/pages/SynthesisDetail.tsx` (line 41)
|
||||
```typescript
|
||||
const [email, setEmail] = useState('olivier.abrivard@desjardins.com');
|
||||
```
|
||||
|
||||
A personal corporate email address is hardcoded as the default email recipient. This should not be in source code.
|
||||
|
||||
### 1.7 MEDIUM: Client-Side Rate Limiter is Ineffective
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/services/geminiService.ts` (lines 6-33)
|
||||
|
||||
The `RateLimiter` class runs in browser memory. It does not protect against:
|
||||
- Multiple browser tabs
|
||||
- Multiple users (each has their own in-memory limiter)
|
||||
- A malicious user who bypasses the frontend entirely and calls the Gemini API directly with the exposed key
|
||||
|
||||
**Mitigation in refactoring**: Server-side rate limiting with shared state (see Section 5.3).
|
||||
|
||||
### 1.8 LOW: No Content Security Policy
|
||||
|
||||
The application serves no CSP headers. Combined with the fact that user-generated content (article titles, summaries, URLs) is rendered in the DOM, this creates potential for XSS if React's default escaping is ever bypassed.
|
||||
|
||||
### 1.9 LOW: Error Messages Leak Internal Details
|
||||
|
||||
**File**: `/Users/oabrivard/Projects/rust/ai_synth/src/firebase.ts` (lines 68-89)
|
||||
|
||||
The `handleFirestoreError` function logs and throws detailed error objects containing `userId`, `email`, `emailVerified`, `tenantId`, and provider information. While this is useful for debugging, these details should not be exposed to the client in production.
|
||||
|
||||
---
|
||||
|
||||
## 2. Authentication and Session Security
|
||||
|
||||
### 2.1 Magic Link Implementation
|
||||
|
||||
#### Token Generation
|
||||
- Use `rand::rngs::OsRng` (Rust's CSPRNG) to generate tokens: 32 bytes of cryptographic randomness, encoded as URL-safe base64 (43 characters).
|
||||
- Do NOT use UUIDs -- while v4 UUIDs use random bytes, the format is predictable and shorter effective entropy.
|
||||
- Store a SHA-256 hash of the token in the database, never the token itself. This way, a database breach does not compromise pending magic links.
|
||||
|
||||
#### Token Lifecycle
|
||||
```
|
||||
User enters email -> Backend generates token -> Stores SHA-256(token) + email + expires_at + used=false
|
||||
-> Sends email with link: https://app.example.com/auth/verify?token=<raw_token>
|
||||
User clicks link -> Backend computes SHA-256(submitted_token) -> Looks up in DB
|
||||
-> Validates: not expired, not used, email matches
|
||||
-> Marks token as used=true
|
||||
-> Creates session (see 2.2)
|
||||
```
|
||||
|
||||
#### Token Expiration
|
||||
- Magic link tokens expire after **15 minutes** (not longer -- the user has their email open).
|
||||
- Implement a cleanup job (background task or on-request pruning) to delete expired tokens.
|
||||
|
||||
#### Single-Use Enforcement
|
||||
- The `used` boolean column prevents replay attacks.
|
||||
- Use a database transaction: `UPDATE magic_tokens SET used = true WHERE token_hash = ? AND used = false AND expires_at > NOW()`. If `rows_affected == 0`, the token is invalid.
|
||||
- This is atomic and race-condition-safe.
|
||||
|
||||
#### Email Enumeration Prevention
|
||||
- The `/auth/magic-link` endpoint MUST return the same response (HTTP 200, same message) regardless of whether the email exists in the database.
|
||||
- Message: "If an account with this email exists, a login link has been sent."
|
||||
- If the email is not registered, silently do nothing (no email sent, no error).
|
||||
- Apply the same timing: if sending an email takes 200ms, add a random delay (100-300ms) when no email is sent, so timing attacks cannot distinguish the two cases.
|
||||
|
||||
### 2.2 Session Management
|
||||
|
||||
#### Session Cookie Attributes
|
||||
```
|
||||
Set-Cookie: session_id=<value>;
|
||||
HttpOnly; # Prevents JavaScript access (XSS mitigation)
|
||||
Secure; # Only sent over HTTPS
|
||||
SameSite=Lax; # Prevents CSRF on cross-origin POST (allows top-level navigation)
|
||||
Path=/; # Available to all paths
|
||||
Max-Age=604800; # 7 days (server-side expiration is authoritative)
|
||||
```
|
||||
|
||||
Why `SameSite=Lax` and not `Strict`: `Strict` would prevent the session cookie from being sent when the user clicks a magic link from their email client (which is a cross-site navigation). Since magic links are the primary auth mechanism, `Lax` is necessary.
|
||||
|
||||
#### Session ID Generation
|
||||
- 32 bytes from `OsRng`, hex-encoded (64 characters) or base64url-encoded (43 characters).
|
||||
- Store SHA-256(session_id) in the database. The raw session_id is only in the cookie.
|
||||
- Schema:
|
||||
```sql
|
||||
CREATE TABLE sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_hash TEXT NOT NULL UNIQUE, -- SHA-256(session_id)
|
||||
user_id INTEGER NOT NULL REFERENCES users(id),
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
expires_at TIMESTAMP NOT NULL,
|
||||
last_active_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
ip_address TEXT,
|
||||
user_agent TEXT
|
||||
);
|
||||
CREATE INDEX idx_sessions_hash ON sessions(session_hash);
|
||||
CREATE INDEX idx_sessions_user ON sessions(user_id);
|
||||
CREATE INDEX idx_sessions_expires ON sessions(expires_at);
|
||||
```
|
||||
|
||||
#### Session Expiration and Rotation
|
||||
- **Absolute expiration**: 7 days from creation. After this, the user must re-authenticate.
|
||||
- **Idle timeout**: If `last_active_at` is more than 24 hours ago, invalidate the session.
|
||||
- **Session rotation**: After successful authentication (magic link click), issue a new session ID and invalidate the old one. This prevents session fixation attacks.
|
||||
- **Sliding window**: Update `last_active_at` on each request, but only write to DB at most once per 5 minutes to avoid excessive writes.
|
||||
|
||||
#### Logout and Revocation
|
||||
- On logout: DELETE the session row from the database and clear the cookie (set `Max-Age=0`).
|
||||
- Provide "Log out all sessions" functionality: `DELETE FROM sessions WHERE user_id = ?`.
|
||||
- Admin capability: revoke all sessions for a specific user (for account compromise response).
|
||||
|
||||
### 2.3 Captcha for Self-Hosted Deployment
|
||||
|
||||
**Recommended**: [mCaptcha](https://mcaptcha.org/) -- fully open-source, self-hostable, proof-of-work based (no third-party dependency). It can run as a sidecar container in the Docker stack.
|
||||
|
||||
**Alternative**: [hCaptcha](https://www.hcaptcha.com/) -- privacy-focused, free tier available, but requires an external service call.
|
||||
|
||||
**NOT recommended**: Google reCAPTCHA -- contradicts the "remove Google hosting dependencies" requirement.
|
||||
|
||||
Captcha should be applied to:
|
||||
- Account registration (`POST /auth/register`)
|
||||
- Magic link request (`POST /auth/magic-link`)
|
||||
- NOT to every login -- rate limiting handles brute-force on session endpoints
|
||||
|
||||
#### Complementary Rate Limiting on Auth Endpoints
|
||||
| Endpoint | Rate Limit | Window | Scope |
|
||||
|---|---|---|---|
|
||||
| `POST /auth/register` | 3 requests | 1 hour | Per IP |
|
||||
| `POST /auth/magic-link` | 5 requests | 15 minutes | Per IP |
|
||||
| `POST /auth/magic-link` | 3 requests | 1 hour | Per email |
|
||||
| `POST /auth/verify` | 10 requests | 15 minutes | Per IP |
|
||||
| `POST /auth/verify` (failed) | 5 failures | 15 minutes | Per IP, then block |
|
||||
|
||||
### 2.4 CSRF Protection Strategy
|
||||
|
||||
Since the frontend (SolidJS SPA) and backend (Rust API) may be on different origins during development, a robust CSRF strategy is needed.
|
||||
|
||||
**Recommended approach: Double-Submit Cookie pattern with SameSite**
|
||||
|
||||
1. `SameSite=Lax` on the session cookie provides baseline CSRF protection for non-GET requests from cross-origin sites.
|
||||
2. For defense-in-depth, implement the Synchronizer Token pattern:
|
||||
- On session creation, generate a CSRF token (32 random bytes, hex-encoded).
|
||||
- Store it in the session (server-side).
|
||||
- Send it to the frontend via a dedicated endpoint (`GET /auth/csrf-token`) or as a response header.
|
||||
- The SolidJS app includes it as an `X-CSRF-Token` header on every state-changing request.
|
||||
- The backend middleware validates `X-CSRF-Token` header matches the session's CSRF token for all POST/PUT/DELETE requests.
|
||||
|
||||
3. Additionally, validate the `Origin` header on state-changing requests. Reject requests where `Origin` does not match the configured `APP_URL`.
|
||||
|
||||
### 2.5 Account Enumeration Protection
|
||||
|
||||
Beyond magic link (covered in 2.1):
|
||||
- **Registration**: If an email is already registered, do NOT return "Email already exists." Instead, send an email to the existing address saying "Someone tried to register with your email. If this was you, use the login link instead." Return the same success message to the client.
|
||||
- **Login (magic link request)**: Same as 2.1 -- identical response regardless of email existence.
|
||||
- **Error messages**: Never distinguish between "user not found" and "wrong password" (not applicable here since there are no passwords, but important if password auth is ever added).
|
||||
|
||||
---
|
||||
|
||||
## 3. API Key Storage Security
|
||||
|
||||
### 3.1 Encryption at Rest
|
||||
|
||||
LLM API keys (Gemini, OpenAI, Anthropic) stored in the database must be encrypted. These are high-value secrets -- a database leak would expose them.
|
||||
|
||||
#### Encryption Scheme
|
||||
- **Algorithm**: AES-256-GCM (authenticated encryption -- provides both confidentiality and integrity).
|
||||
- **Implementation**: Use the `aes-gcm` crate in Rust.
|
||||
- **Per-key nonce**: Generate a unique 96-bit (12-byte) nonce for each encryption operation using `OsRng`. Store the nonce alongside the ciphertext.
|
||||
- **Schema**:
|
||||
```sql
|
||||
CREATE TABLE llm_api_keys (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
provider TEXT NOT NULL, -- 'google', 'openai', 'anthropic'
|
||||
label TEXT NOT NULL, -- Human-readable label
|
||||
encrypted_key BLOB NOT NULL, -- AES-256-GCM ciphertext
|
||||
nonce BLOB NOT NULL, -- 12-byte GCM nonce
|
||||
key_prefix TEXT NOT NULL, -- First 4 chars of the key (for UI display: "sk-pr...")
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
created_by INTEGER NOT NULL REFERENCES users(id),
|
||||
is_active BOOLEAN NOT NULL DEFAULT true
|
||||
);
|
||||
```
|
||||
|
||||
#### Key Derivation
|
||||
- **Master key**: A 256-bit (32-byte) key derived from a passphrase/secret using Argon2id (via the `argon2` crate).
|
||||
- **Input**: `MASTER_KEY_SECRET` environment variable (a high-entropy string, minimum 32 characters).
|
||||
- **Salt**: A fixed, application-specific salt stored in the config (not secret, but must not change). Alternatively, derive the key once and use the raw 32-byte key directly from the environment variable.
|
||||
- **Simpler alternative**: If `MASTER_KEY_SECRET` is already a 64-character hex string (32 bytes), skip KDF and use it directly. This is acceptable for a single-VM deployment where the env var is properly protected.
|
||||
|
||||
#### Master Key Storage
|
||||
- Store `MASTER_KEY_SECRET` as an environment variable, injected via Docker Compose `env_file` or Docker secrets.
|
||||
- The `.env` file containing it must have permissions `600` (owner read/write only).
|
||||
- **NEVER** commit the master key to version control.
|
||||
- **NEVER** log the master key or the decrypted API keys.
|
||||
- For key rotation: implement a re-encryption command that reads all keys with the old master key, encrypts with the new one, and writes them back in a transaction.
|
||||
|
||||
### 3.2 Access Control
|
||||
|
||||
- **View API keys**: Admin-only. The UI displays only the `key_prefix` (e.g., "sk-pr...") and the `label`. The full key is NEVER sent to the frontend.
|
||||
- **Create/Update API keys**: Admin-only. The key is sent from the admin UI to the backend via HTTPS, encrypted in transit. On the backend, it is immediately encrypted at rest before being stored.
|
||||
- **Delete API keys**: Admin-only, with confirmation.
|
||||
- **Use API keys**: The backend decrypts keys in memory only when making LLM API calls. The decrypted key is held in memory for the duration of the API call, then dropped (Rust's ownership model helps here -- the `String` holding the key is dropped when it goes out of scope).
|
||||
- **Test API keys**: Provide an admin endpoint that attempts a minimal API call (e.g., a simple completion with a tiny prompt) to validate the key works, without exposing the key itself.
|
||||
|
||||
### 3.3 Audit Logging
|
||||
|
||||
```sql
|
||||
CREATE TABLE audit_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
user_id INTEGER NOT NULL REFERENCES users(id),
|
||||
action TEXT NOT NULL, -- 'api_key.create', 'api_key.update', 'api_key.delete', 'api_key.view_list', 'api_key.test'
|
||||
target_type TEXT NOT NULL, -- 'llm_api_key'
|
||||
target_id INTEGER,
|
||||
details TEXT, -- JSON with non-sensitive context (provider, label, NOT the key)
|
||||
ip_address TEXT,
|
||||
user_agent TEXT
|
||||
);
|
||||
CREATE INDEX idx_audit_timestamp ON audit_log(timestamp);
|
||||
CREATE INDEX idx_audit_user ON audit_log(user_id);
|
||||
```
|
||||
|
||||
Log every access to the API keys admin module:
|
||||
- Creating a key: log provider + label
|
||||
- Updating a key: log which fields changed (but not the old or new key value)
|
||||
- Deleting a key: log provider + label
|
||||
- Listing keys: log that the admin viewed the key list
|
||||
- Testing a key: log provider + result (success/failure)
|
||||
|
||||
Audit logs should be append-only. Even admins should not be able to delete audit entries.
|
||||
|
||||
---
|
||||
|
||||
## 4. Authorization and Data Isolation
|
||||
|
||||
### 4.1 Converting Firestore Rules to Rust Middleware
|
||||
|
||||
The current Firestore rules implement three core patterns that must be translated.
|
||||
|
||||
#### Current Rules (from `/Users/oabrivard/Projects/rust/ai_synth/firestore.rules`):
|
||||
|
||||
| Firestore Rule | Rust Equivalent |
|
||||
|---|---|
|
||||
| `isAuthenticated()` | Middleware: extract and validate session cookie, reject 401 if invalid |
|
||||
| `isDocOwner()` / `isOwner(userId)` | Query filter: always include `WHERE user_id = ?` using the authenticated user's ID from the session |
|
||||
| `uidUnchanged()` / `uidNotModified()` | Business logic: set `user_id` from the session on create; reject requests that attempt to change `user_id` on update |
|
||||
| `isValidSynthesis()`, `isValidSettings()`, `isValidSource()` | Request body validation using `serde` deserialization with `#[validate]` derive (via the `validator` crate) |
|
||||
| Field size/type constraints | `validator` attributes: `#[validate(length(min = 1, max = 200))]`, `#[validate(range(min = 1, max = 365))]`, etc. |
|
||||
|
||||
#### Recommended Middleware Stack (using Axum)
|
||||
|
||||
```
|
||||
Request
|
||||
-> CORS middleware (tower-http)
|
||||
-> Rate limiting middleware (tower::limit or custom)
|
||||
-> Session extraction middleware (reads cookie, validates session, injects AuthUser into request extensions)
|
||||
-> CSRF validation middleware (for POST/PUT/DELETE)
|
||||
-> Route handler
|
||||
-> Request body validation (serde + validator)
|
||||
-> Business logic (always scopes queries to authenticated user)
|
||||
-> Response
|
||||
```
|
||||
|
||||
#### Authentication Extractor Pattern
|
||||
Define an `AuthUser` extractor that:
|
||||
1. Reads the `session_id` cookie.
|
||||
2. Looks up `SHA-256(session_id)` in the `sessions` table.
|
||||
3. Validates expiration.
|
||||
4. Returns the user record or rejects with 401.
|
||||
|
||||
```rust
|
||||
// Pseudo-code for the extractor
|
||||
struct AuthUser {
|
||||
id: i64,
|
||||
email: String,
|
||||
is_admin: bool,
|
||||
}
|
||||
```
|
||||
|
||||
All route handlers that require authentication take `AuthUser` as a parameter. If the session is invalid, Axum automatically returns 401 before the handler runs.
|
||||
|
||||
### 4.2 Multi-Tenant Data Isolation
|
||||
|
||||
**Principle**: The user ID from the session is the ONLY source of truth for data ownership. Never trust a `user_id` from the request body or URL parameters for ownership decisions.
|
||||
|
||||
**Implementation**:
|
||||
- Every data table (`syntheses`, `sources`, `settings`) has a `user_id` column with a foreign key to `users(id)`.
|
||||
- Every SELECT query includes `WHERE user_id = $1` using the authenticated user's ID.
|
||||
- Every INSERT sets `user_id` from the session, ignoring any `user_id` in the request body.
|
||||
- Every UPDATE/DELETE query includes `WHERE id = $1 AND user_id = $2` -- if 0 rows affected, return 404 (not 403, to avoid revealing that the resource exists for another user).
|
||||
- Create a database index on `(user_id, created_at)` for every table to ensure efficient queries.
|
||||
|
||||
### 4.3 Admin Role
|
||||
|
||||
#### Definition
|
||||
- A boolean `is_admin` column on the `users` table.
|
||||
- The first admin is bootstrapped via a CLI command or migration (see Questions section).
|
||||
|
||||
#### Protection
|
||||
- Admin-only endpoints use an `AdminUser` extractor that extends `AuthUser` with an additional `is_admin == true` check. Returns 403 if not admin.
|
||||
- Admin endpoints:
|
||||
- `GET/POST/PUT/DELETE /admin/api-keys` -- LLM API key management
|
||||
- `GET/PUT /admin/rate-limits` -- Rate limiter configuration
|
||||
- `GET /admin/audit-log` -- View audit logs
|
||||
- `POST /admin/users/:id/revoke-sessions` -- Revoke all sessions for a user
|
||||
- Admin actions are always logged to the audit log.
|
||||
- Consider requiring re-authentication (e.g., a fresh magic link) for sensitive admin operations like API key changes.
|
||||
|
||||
### 4.4 Input Validation
|
||||
|
||||
#### Request Body Validation
|
||||
Use `serde` for deserialization and the `validator` crate for constraint validation. Example:
|
||||
|
||||
```rust
|
||||
#[derive(Deserialize, Validate)]
|
||||
struct CreateSource {
|
||||
#[validate(length(min = 1, max = 200))]
|
||||
title: String,
|
||||
|
||||
#[validate(url, length(max = 1000))]
|
||||
url: String,
|
||||
}
|
||||
```
|
||||
|
||||
Reject invalid requests with 400 and a generic error message (do not echo back the invalid input to prevent reflected XSS in error responses).
|
||||
|
||||
#### SQL Injection Prevention
|
||||
- **sqlx with parameterized queries**: sqlx compiles queries at build time (with `query!` / `query_as!` macros) and uses prepared statements. This eliminates SQL injection by design.
|
||||
- NEVER use string formatting/interpolation to build SQL queries.
|
||||
- For dynamic queries (e.g., sorting, filtering), use an allowlist of valid column names, not user input.
|
||||
|
||||
#### XSS Prevention
|
||||
- The SolidJS frontend handles escaping by default (like React, it escapes strings rendered in JSX).
|
||||
- NEVER use `innerHTML` or SolidJS's equivalent (`innerHTML` prop) with user-generated content.
|
||||
- Article titles, summaries, and URLs from the LLM should be treated as untrusted user input -- the LLM could generate malicious content.
|
||||
- URLs rendered as `<a href>` must be validated: only allow `http://` and `https://` schemes. Block `javascript:`, `data:`, `vbscript:` schemes.
|
||||
- Set `Content-Type: application/json` on all API responses (never `text/html` for API endpoints).
|
||||
|
||||
---
|
||||
|
||||
## 5. Backend Security
|
||||
|
||||
### 5.1 Rust-Specific Security Considerations
|
||||
|
||||
Rust provides significant security advantages:
|
||||
- **Memory safety**: No buffer overflows, use-after-free, or data races (without `unsafe`).
|
||||
- **No null pointer dereferences**: `Option<T>` forces explicit handling.
|
||||
- **Ownership model**: Secrets (API keys, session tokens) are dropped when they go out of scope, reducing the window of exposure.
|
||||
|
||||
Recommendations:
|
||||
- **Minimize `unsafe` blocks**: Audit any `unsafe` code carefully. Prefer safe abstractions.
|
||||
- **Dependency auditing**: Run `cargo audit` in CI to check for known vulnerabilities in dependencies.
|
||||
- **Use `secrecy` crate**: Wrap sensitive values (API keys, session tokens) in `Secret<String>` to prevent accidental logging via `Debug` or `Display` trait implementations.
|
||||
- **Zeroize secrets**: Use the `zeroize` crate to overwrite sensitive memory on drop (the `secrecy` crate integrates with this).
|
||||
|
||||
### 5.2 Recommended Crate Ecosystem
|
||||
|
||||
| Concern | Crate | Notes |
|
||||
|---|---|---|
|
||||
| Web framework | `axum` | Tower-based, async, well-maintained |
|
||||
| Database | `sqlx` | Compile-time checked queries, async |
|
||||
| Password/KDF | `argon2` | For master key derivation |
|
||||
| Encryption | `aes-gcm` | AES-256-GCM authenticated encryption |
|
||||
| Random | `rand` | `OsRng` for cryptographic randomness |
|
||||
| Hashing | `sha2` | SHA-256 for token hashing |
|
||||
| Secrets | `secrecy` + `zeroize` | Prevent accidental exposure |
|
||||
| Validation | `validator` | Derive-based request validation |
|
||||
| Rate limiting | `tower` + `governor` | Token-bucket rate limiting as middleware |
|
||||
| CORS | `tower-http` | `CorsLayer` |
|
||||
| HTTP client | `reqwest` | For LLM API calls and URL scraping |
|
||||
| Serialization | `serde` + `serde_json` | Request/response serialization |
|
||||
| Logging | `tracing` + `tracing-subscriber` | Structured logging (filter sensitive fields) |
|
||||
| Email | `lettre` | SMTP client for magic links |
|
||||
|
||||
### 5.3 Rate Limiting Implementation
|
||||
|
||||
Use `tower::ServiceBuilder` with the `governor` crate for token-bucket rate limiting.
|
||||
|
||||
#### Rate Limiting Layers
|
||||
|
||||
1. **Global layer** (outermost): Protects the server from DDoS. Example: 1000 requests/minute total.
|
||||
2. **Per-IP layer**: Prevents abuse from a single source. Example: 100 requests/minute per IP.
|
||||
3. **Per-user layer** (after authentication): Prevents abuse by authenticated users. Example: 60 requests/minute per user.
|
||||
4. **Per-endpoint layer**: Specific limits for expensive operations.
|
||||
|
||||
| Endpoint | Limit | Window | Note |
|
||||
|---|---|---|---|
|
||||
| `POST /api/syntheses/generate` | 3 | 1 hour per user | LLM calls are expensive |
|
||||
| `POST /auth/*` | See Section 2.3 | | Auth endpoints |
|
||||
| `GET /api/*` | 120 | 1 minute per user | General API |
|
||||
| `POST/PUT/DELETE /api/*` | 30 | 1 minute per user | Write operations |
|
||||
|
||||
#### Admin-Configurable Rate Limits
|
||||
Store rate limit configuration in the database:
|
||||
```sql
|
||||
CREATE TABLE rate_limit_config (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
endpoint_pattern TEXT NOT NULL UNIQUE, -- e.g., 'generate', 'auth', 'api_read', 'api_write'
|
||||
max_requests INTEGER NOT NULL,
|
||||
window_seconds INTEGER NOT NULL,
|
||||
scope TEXT NOT NULL DEFAULT 'per_user', -- 'global', 'per_ip', 'per_user'
|
||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_by INTEGER REFERENCES users(id)
|
||||
);
|
||||
```
|
||||
|
||||
The middleware reads this config on startup and reloads periodically (e.g., every 60 seconds) or via an admin trigger.
|
||||
|
||||
### 5.4 SSRF Prevention for URL Scraping
|
||||
|
||||
When the backend scrapes URLs (to validate and extract content from news articles), it becomes a potential SSRF vector. A malicious user could add a source URL pointing to internal services.
|
||||
|
||||
#### Protections
|
||||
|
||||
1. **DNS resolution check**: Before connecting, resolve the hostname and reject if the IP is:
|
||||
- Private ranges: `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`
|
||||
- Loopback: `127.0.0.0/8`, `::1`
|
||||
- Link-local: `169.254.0.0/16`, `fe80::/10`
|
||||
- Cloud metadata: `169.254.169.254` (AWS/GCP/Azure metadata endpoint)
|
||||
- Localhost variants: `0.0.0.0`, `[::0]`
|
||||
|
||||
2. **Protocol restriction**: Only allow `http://` and `https://` schemes. Block `file://`, `ftp://`, `gopher://`, `dict://`, etc.
|
||||
|
||||
3. **Timeouts**: Set aggressive timeouts on the `reqwest` client:
|
||||
- Connection timeout: 5 seconds
|
||||
- Response timeout: 15 seconds
|
||||
- Total request timeout: 30 seconds
|
||||
|
||||
4. **Response size limit**: Maximum 5 MB response body. Use `reqwest`'s `.bytes()` with a streaming check, or set `content-length` limits.
|
||||
|
||||
5. **Redirect limit**: Maximum 3 redirects. Validate each redirect destination against the same IP blocklist.
|
||||
|
||||
6. **User-Agent**: Set a custom `User-Agent` header identifying the application (e.g., `AI-Weekly-Synth/1.0 (URL Validator)`). This is courteous and allows target sites to identify the bot.
|
||||
|
||||
#### Implementation Pattern (Rust pseudo-code)
|
||||
```rust
|
||||
fn is_safe_url(url: &Url) -> Result<(), SsrfError> {
|
||||
// 1. Check scheme
|
||||
if url.scheme() != "http" && url.scheme() != "https" {
|
||||
return Err(SsrfError::UnsafeScheme);
|
||||
}
|
||||
// 2. Resolve DNS and check IP
|
||||
let addrs = url.socket_addrs(|| Some(443))?;
|
||||
for addr in &addrs {
|
||||
if is_private_ip(addr.ip()) {
|
||||
return Err(SsrfError::PrivateIp);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### 5.5 Content Security Policy
|
||||
|
||||
Set the following CSP headers on the HTML response that serves the SPA:
|
||||
|
||||
```
|
||||
Content-Security-Policy:
|
||||
default-src 'none';
|
||||
script-src 'self';
|
||||
style-src 'self' 'unsafe-inline'; # Tailwind may need inline styles
|
||||
img-src 'self' data: https:; # Allow images from HTTPS sources
|
||||
font-src 'self';
|
||||
connect-src 'self'; # API calls only to same origin
|
||||
frame-src 'none'; # No iframes
|
||||
base-uri 'self';
|
||||
form-action 'self';
|
||||
frame-ancestors 'none'; # Prevent clickjacking
|
||||
```
|
||||
|
||||
Additional headers:
|
||||
```
|
||||
X-Content-Type-Options: nosniff
|
||||
X-Frame-Options: DENY
|
||||
Referrer-Policy: strict-origin-when-cross-origin
|
||||
Permissions-Policy: camera=(), microphone=(), geolocation=()
|
||||
Strict-Transport-Security: max-age=31536000; includeSubDomains # Only if HTTPS
|
||||
```
|
||||
|
||||
### 5.6 CORS Configuration
|
||||
|
||||
```rust
|
||||
let cors = CorsLayer::new()
|
||||
.allow_origin(AllowOrigin::exact(app_url.parse().unwrap())) // Only the frontend origin
|
||||
.allow_methods([Method::GET, Method::POST, Method::PUT, Method::DELETE])
|
||||
.allow_headers([
|
||||
header::CONTENT_TYPE,
|
||||
header::AUTHORIZATION,
|
||||
HeaderName::from_static("x-csrf-token"),
|
||||
])
|
||||
.allow_credentials(true) // Required for cookies
|
||||
.max_age(Duration::from_secs(3600));
|
||||
```
|
||||
|
||||
Key points:
|
||||
- **Never** use `AllowOrigin::any()` with `allow_credentials(true)` -- browsers reject this combination.
|
||||
- The allowed origin must match exactly (including scheme and port).
|
||||
- In development, allow `http://localhost:3000`; in production, only the deployment URL.
|
||||
- Read `APP_URL` from environment to configure this dynamically.
|
||||
|
||||
---
|
||||
|
||||
## 6. Deployment Security
|
||||
|
||||
### 6.1 Docker Security
|
||||
|
||||
#### Dockerfile Best Practices
|
||||
```dockerfile
|
||||
# Multi-stage build
|
||||
FROM rust:1.78-slim AS builder
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN cargo build --release
|
||||
|
||||
FROM debian:bookworm-slim
|
||||
# Create non-root user
|
||||
RUN groupadd -r appuser && useradd -r -g appuser -d /app -s /sbin/nologin appuser
|
||||
# Install only required runtime dependencies
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*
|
||||
WORKDIR /app
|
||||
COPY --from=builder /app/target/release/ai-weekly-synth .
|
||||
COPY --from=builder /app/static ./static
|
||||
# Own the data directory
|
||||
RUN mkdir -p /app/data && chown -R appuser:appuser /app
|
||||
USER appuser
|
||||
EXPOSE 8080
|
||||
CMD ["./ai-weekly-synth"]
|
||||
```
|
||||
|
||||
Checklist:
|
||||
- [x] Non-root user (`USER appuser`)
|
||||
- [x] Minimal base image (`debian:bookworm-slim`, not `ubuntu` or full `debian`)
|
||||
- [x] Multi-stage build (no compiler, source code, or build artifacts in final image)
|
||||
- [x] No secrets in the image (API keys, master key are injected via env vars at runtime)
|
||||
- [x] `.dockerignore` excludes `.env`, `.git`, `target/`, `node_modules/`
|
||||
- [x] Pin base image versions for reproducibility
|
||||
- [x] `ca-certificates` installed for HTTPS requests to LLM APIs
|
||||
|
||||
#### Docker Compose Security
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
# ...
|
||||
env_file: .env # Contains MASTER_KEY_SECRET, DATABASE_URL, etc.
|
||||
read_only: true # Read-only root filesystem
|
||||
tmpfs:
|
||||
- /tmp # Writable temp directory
|
||||
volumes:
|
||||
- ./data:/app/data # SQLite database (persistent)
|
||||
security_opt:
|
||||
- no-new-privileges:true # Prevent privilege escalation
|
||||
cap_drop:
|
||||
- ALL # Drop all Linux capabilities
|
||||
```
|
||||
|
||||
### 6.2 SQLite File Permissions
|
||||
|
||||
- The SQLite database file should be owned by the `appuser` and have permissions `600` (owner read/write only).
|
||||
- The directory containing the SQLite file should have permissions `700`.
|
||||
- Enable WAL mode for concurrent reads: `PRAGMA journal_mode=WAL;`
|
||||
- Set `PRAGMA foreign_keys = ON;` at connection startup.
|
||||
- Consider `PRAGMA secure_delete = ON;` to overwrite deleted data (relevant for API keys).
|
||||
|
||||
### 6.3 HTTPS/TLS Termination
|
||||
|
||||
**Do NOT terminate TLS in the Rust application**. Use a reverse proxy:
|
||||
|
||||
**Recommended: Caddy** (automatic HTTPS with Let's Encrypt, zero-config)
|
||||
|
||||
```
|
||||
# Caddyfile
|
||||
app.example.com {
|
||||
reverse_proxy app:8080
|
||||
encode gzip
|
||||
header {
|
||||
Strict-Transport-Security "max-age=31536000; includeSubDomains"
|
||||
X-Content-Type-Options "nosniff"
|
||||
X-Frame-Options "DENY"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Alternative: nginx with certbot for Let's Encrypt.
|
||||
|
||||
Add Caddy as a service in Docker Compose:
|
||||
```yaml
|
||||
services:
|
||||
caddy:
|
||||
image: caddy:2-alpine
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
volumes:
|
||||
- ./Caddyfile:/etc/caddy/Caddyfile
|
||||
- caddy_data:/data
|
||||
- caddy_config:/config
|
||||
depends_on:
|
||||
- app
|
||||
```
|
||||
|
||||
The Rust app only listens on `0.0.0.0:8080` (internal Docker network, not exposed to the host).
|
||||
|
||||
### 6.4 Environment Variable Management
|
||||
|
||||
Required environment variables:
|
||||
| Variable | Description | Example |
|
||||
|---|---|---|
|
||||
| `MASTER_KEY_SECRET` | 256-bit key for encrypting LLM API keys | 64-char hex string |
|
||||
| `DATABASE_URL` | SQLite path | `sqlite:///app/data/ai_synth.db` |
|
||||
| `APP_URL` | Public URL of the application | `https://app.example.com` |
|
||||
| `SMTP_HOST` | SMTP server for magic link emails | `smtp.resend.com` |
|
||||
| `SMTP_PORT` | SMTP port | `465` |
|
||||
| `SMTP_USERNAME` | SMTP username | `resend` |
|
||||
| `SMTP_PASSWORD` | SMTP password/API key | `re_...` |
|
||||
| `SMTP_FROM` | Sender email address | `noreply@example.com` |
|
||||
| `MCAPTCHA_URL` | mCaptcha service URL (if used) | `http://mcaptcha:7000` |
|
||||
| `MCAPTCHA_SECRET` | mCaptcha site key | `...` |
|
||||
| `RUST_LOG` | Log level | `info,ai_weekly_synth=debug` |
|
||||
|
||||
Storage:
|
||||
- `.env` file with permissions `600`, excluded from version control.
|
||||
- For Docker: use `env_file` directive, or Docker secrets for Swarm deployments.
|
||||
- NEVER pass secrets as command-line arguments (visible in `ps` output).
|
||||
- NEVER use `docker run -e SECRET=value` (visible in `docker inspect`).
|
||||
|
||||
### 6.5 Backup Strategy
|
||||
|
||||
#### SQLite Backup
|
||||
- Use SQLite's `.backup` command or the `sqlite3_backup_*` API for consistent hot backups.
|
||||
- **Do NOT** simply copy the SQLite file while the application is running -- this can result in a corrupted backup.
|
||||
- Schedule backups via a cron job or a background task in the application:
|
||||
```
|
||||
sqlite3 /app/data/ai_synth.db ".backup /app/data/backups/ai_synth_$(date +%Y%m%d_%H%M%S).db"
|
||||
```
|
||||
- Retain backups for 30 days, with daily rotation.
|
||||
- Encrypt backups before storing them off-site (if applicable).
|
||||
- Test restore procedure periodically.
|
||||
|
||||
#### Postgres Upgrade Path
|
||||
When migrating to Postgres:
|
||||
- Use `pg_dump` for logical backups.
|
||||
- Consider point-in-time recovery with WAL archiving for production.
|
||||
- Connection string should use SSL (`sslmode=require`).
|
||||
|
||||
---
|
||||
|
||||
## 7. Threat Model Summary
|
||||
|
||||
| # | Threat | Likelihood | Impact | Mitigation |
|
||||
|---|---|---|---|---|
|
||||
| T1 | **LLM API key theft from database** -- An attacker gains read access to the SQLite database (file access, SQL injection, backup leak) and extracts LLM API keys | Medium | High (financial abuse, quota exhaustion) | Encrypt API keys at rest with AES-256-GCM (Section 3.1). Use parameterized queries to prevent SQL injection. Restrict SQLite file permissions. Encrypt backups. |
|
||||
| T2 | **Session hijacking** -- An attacker steals a session cookie via XSS, network sniffing, or physical access | Medium | High (full account takeover) | HttpOnly + Secure + SameSite cookies (Section 2.2). Enforce HTTPS via Caddy. Implement CSP to mitigate XSS (Section 5.5). Session expiration and idle timeout. |
|
||||
| T3 | **Prompt injection via user settings** -- A user crafts malicious theme/category/behavior text to manipulate the LLM into producing harmful output or leaking system prompt details | High (easy to attempt) | Medium (misleading content, potential data leak via grounding) | Validate and sanitize user prompt inputs (Section 1.3). Enforce length limits. Log unusual patterns. Keep system instructions and user input structurally separated in the prompt. |
|
||||
| T4 | **SSRF via custom source URLs** -- A user adds a source URL pointing to internal infrastructure (`http://169.254.169.254/`, `http://localhost:8080/admin/api-keys`) | Medium | High (internal network access, credential theft from cloud metadata) | IP blocklist, scheme restriction, DNS validation before connecting (Section 5.4). Timeouts and response size limits. |
|
||||
| T5 | **Account takeover via magic link interception** -- An attacker intercepts a magic link email (compromised email account, network sniffing, email forwarding rules) | Low-Medium | High (full account takeover) | Short token expiration (15 min), single-use tokens, session rotation on login (Section 2.1). Users should secure their email accounts (out of scope but documentable). |
|
||||
| T6 | **Brute-force on authentication endpoints** -- An attacker attempts to guess magic link tokens or flood registration/login endpoints | Medium | Medium (denial of service, account enumeration) | Rate limiting on auth endpoints (Section 2.3). Captcha on registration and magic link requests. Cryptographically random 256-bit tokens make guessing infeasible. |
|
||||
| T7 | **Cross-Site Request Forgery (CSRF)** -- An attacker tricks an authenticated user into making unintended API calls (e.g., delete all syntheses, change settings) | Low (mitigated by SameSite) | Medium (data loss, settings manipulation) | SameSite=Lax cookies + CSRF token header + Origin validation (Section 2.4). |
|
||||
| T8 | **Admin privilege escalation** -- A regular user finds a way to access admin endpoints (direct URL access, manipulated request) | Low | Critical (LLM API key exposure, rate limit removal, user session revocation) | Server-side admin check via `AdminUser` extractor (Section 4.3). No client-side-only admin checks. Audit logging of all admin actions. |
|
||||
| T9 | **Denial of service via expensive LLM operations** -- A user triggers many concurrent synthesis generations, exhausting LLM API quota or server resources | Medium | Medium (service degradation for all users, financial impact) | Per-user rate limiting on generation endpoint (Section 5.3). Queue-based generation with concurrency limits. Admin-configurable rate limits. |
|
||||
| T10 | **XSS via LLM-generated content** -- The LLM produces article titles or summaries containing HTML/JavaScript that gets rendered unsafely in the SolidJS frontend | Low (frameworks escape by default) | High (session theft, data exfiltration) | SolidJS default escaping. Never use `innerHTML` with LLM output. CSP headers as defense-in-depth. Validate URLs (scheme allowlist). Sanitize HTML if rich text is ever needed. |
|
||||
|
||||
---
|
||||
|
||||
## Summary of Key Architectural Decisions
|
||||
|
||||
1. **All LLM API calls on the backend**: Eliminates the most critical current vulnerability (exposed API key).
|
||||
2. **Session-based auth with secure cookies**: Replaces Firebase Auth. Simpler, no third-party dependency, full control.
|
||||
3. **AES-256-GCM encryption for API keys at rest**: Protects against database leaks.
|
||||
4. **SSRF-safe URL scraping**: Backend replaces CORS proxies with direct HTTP requests + IP blocklist.
|
||||
5. **Defense-in-depth**: Multiple layers (CSP, CSRF tokens, SameSite cookies, rate limiting, input validation) rather than relying on any single control.
|
||||
6. **Audit logging**: All admin and security-relevant actions are logged for incident response.
|
||||
7. **Self-hosted captcha (mCaptcha)**: Aligns with the "no external dependencies" philosophy.
|
||||
8. **Caddy for TLS termination**: Automatic HTTPS, minimal configuration, runs in Docker.
|
||||
@ -0,0 +1,358 @@
|
||||
# Devil's Advocate Analysis: AI Weekly Synth Refactoring
|
||||
|
||||
## Preamble
|
||||
|
||||
This document exists to stress-test every assumption behind the proposed refactoring. The current application -- a React/Firebase/Gemini SPA -- is functional, deployed, and serving users. The proposal is to rewrite **everything**: new backend language (Rust), new database (SQLite), new frontend framework (SolidJS), new auth system, new LLM abstraction layer, and new deployment model. That is not a refactoring. That is a greenfield rewrite wearing a refactoring's name, and it deserves brutal scrutiny.
|
||||
|
||||
---
|
||||
|
||||
## 1. Questioning the Technology Choices
|
||||
|
||||
### 1.1 Rust for the Backend
|
||||
|
||||
**The case against:**
|
||||
|
||||
The current application has zero backend code. It is a thin SPA that talks to Firebase and Gemini directly from the browser. The "backend" being introduced serves three purposes: (a) proxy LLM API calls, (b) manage auth sessions, and (c) CRUD operations on a database. None of these require Rust.
|
||||
|
||||
- **Development velocity**: Rust's compile times, borrow checker friction, and verbose error handling will slow feature iteration by 3-5x compared to Go, Python/FastAPI, or Node.js/Express. This is a CRUD app with LLM API calls, not a systems programming project.
|
||||
- **Ecosystem maturity for web**: The Rust web ecosystem (actix-web, axum, tower) is maturing but remains significantly less documented and less battle-tested for standard web applications than Express, FastAPI, or Go's net/http. Libraries for session management, email sending, CSRF protection, and OAuth flows are less polished.
|
||||
- **Hiring and maintenance**: If the original developer leaves, finding a Rust web developer is substantially harder and more expensive than finding someone who knows Python, Go, or Node.js. The codebase becomes a bus-factor-1 liability.
|
||||
- **What Rust gives you that you do not need**: Memory safety without a GC, zero-cost abstractions, and bare-metal performance. This application proxies LLM API calls that take 10-60 seconds each. The bottleneck is external API latency, not backend compute. You could run this on Python with FastAPI and it would perform identically from the user's perspective.
|
||||
|
||||
**The question to ask**: Is Rust chosen because it is the right tool, or because someone wants to learn Rust?
|
||||
|
||||
**Alternative**: Go gives you compiled binary deployment, excellent concurrency, trivial cross-compilation, a mature web ecosystem (Gin, Echo, Chi), and a learning curve measured in weeks rather than months. Python/FastAPI gives you the fastest development speed, native async, and the richest LLM SDK ecosystem (every LLM provider ships a Python SDK first).
|
||||
|
||||
### 1.2 SolidJS for the Frontend
|
||||
|
||||
**The case against:**
|
||||
|
||||
The current React frontend is approximately 900 lines of code across 7 files. It is straightforward, uses standard patterns (Context, useEffect, useState), and works. The proposal is to rewrite all of it in SolidJS.
|
||||
|
||||
- **Cost/benefit ratio**: SolidJS's reactive model is technically superior to React's virtual DOM diffing. But for an app with 5 pages, no complex animations, no heavy lists, and no real-time data visualization, the performance difference is imperceptible. You are paying the full cost of a rewrite for zero user-visible benefit.
|
||||
- **Ecosystem**: SolidJS has roughly 1/100th of React's ecosystem. `lucide-react` does not have a SolidJS version (you would need `lucide-solid` or raw SVGs). `react-router-dom` becomes `@solidjs/router` with different APIs. `date-fns` works, but any React-specific utility does not.
|
||||
- **Tailwind CSS**: Works with SolidJS, but the JSX-like syntax (JSX vs TSX) is slightly different (e.g., `class` vs `className`, `for` vs `htmlFor`). Every component needs manual translation.
|
||||
- **Real-time listeners**: The current app uses Firestore's `onSnapshot` for real-time updates on the Home and Sources pages. SolidJS's reactive primitives (`createSignal`, `createEffect`) differ fundamentally from React's `useState`/`useEffect`. The subscription cleanup patterns are different. This is not a find-and-replace operation.
|
||||
- **Developer pool**: If you hire someone to maintain this, they probably know React. They probably do not know SolidJS.
|
||||
|
||||
**The question to ask**: What is SolidJS giving you that React 19 with its compiler cannot? Name one user-facing feature that is impossible or unreasonably difficult in React but easy in SolidJS.
|
||||
|
||||
**Alternative**: Keep React. The frontend works. If you want to modernize, adopt React Server Components or Next.js/Remix. If you want a different framework for legitimate reasons, consider Svelte (larger ecosystem than SolidJS, similar reactivity model, larger community).
|
||||
|
||||
### 1.3 SQLite with "Postgres Upgrade Path"
|
||||
|
||||
**The case against:**
|
||||
|
||||
- **Write concurrency**: SQLite uses a single-writer model. When user A is generating a synthesis (which involves multiple writes during scraping/validation), user B's writes will block. With WAL mode, reads are concurrent, but writes serialize. For a "multi-user" application, this is a real limitation that manifests at surprisingly low user counts (10-20 concurrent writers).
|
||||
- **The "upgrade path" is a myth**: `sqlx` supports both SQLite and Postgres, but the SQL dialects differ. SQLite lacks: `JSONB` operators, `ARRAY` types, `LATERAL JOIN`, `INSERT ... ON CONFLICT DO UPDATE` (supported since 3.24 but with different syntax), `ENUM` types, advisory locks, and `LISTEN/NOTIFY`. If you write queries that use any Postgres-specific feature, you cannot "downgrade" to SQLite. If you restrict yourself to the SQLite subset, you are not getting the benefits of Postgres. You end up writing to the lowest common denominator of both databases.
|
||||
- **Migration tooling**: `sqlx` migrations work for both, but you will need separate migration files if the dialects diverge. Testing requires running against both databases. CI doubles.
|
||||
- **Operational burden**: SQLite is a file on disk. Backups are `cp`. But that means backup-during-write requires `VACUUM INTO` or filesystem snapshots. There is no `pg_dump`-equivalent streaming backup. There is no replication. There is no point-in-time recovery.
|
||||
|
||||
**The question to ask**: How many concurrent users are expected? If more than 10-20, start with Postgres. If fewer than 5, SQLite is fine, but then why plan for a "Postgres upgrade path" you will never need?
|
||||
|
||||
**Alternative**: Just use Postgres from the start. Docker makes this trivial. A `docker-compose.yml` with Postgres + the app is no more complex than SQLite, and you avoid the "upgrade path" fiction entirely.
|
||||
|
||||
---
|
||||
|
||||
## 2. Challenging the Scope
|
||||
|
||||
### 2.1 This Is a Complete Rewrite, Not a Refactoring
|
||||
|
||||
Let me enumerate what is being changed:
|
||||
|
||||
| Component | Current | Proposed | Effort |
|
||||
|---|---|---|---|
|
||||
| Backend | None (serverless) | Rust + axum/actix | Build from scratch |
|
||||
| Database | Firestore (managed) | SQLite/Postgres (self-managed) | Build from scratch |
|
||||
| Frontend framework | React 19 | SolidJS | Complete rewrite |
|
||||
| Auth | Firebase Auth (Google SSO) | Email+captcha, magic link, sessions | Build from scratch |
|
||||
| LLM integration | Gemini only (frontend) | Multi-provider (backend) | Major redesign |
|
||||
| Deployment | Firebase Hosting | Docker/VM | Build from scratch |
|
||||
| Admin module | None | New feature | Build from scratch |
|
||||
| Email system | Gmail API (frontend) | Backend email service | Build from scratch |
|
||||
|
||||
**Every single component is being replaced.** There is no code that survives this refactoring. Zero reuse. This is the textbook definition of a "Second System Effect" (Brooks, 1975).
|
||||
|
||||
### 2.2 What Is Actually Broken?
|
||||
|
||||
Before rewriting everything, let us be honest about what problems exist today:
|
||||
|
||||
1. **API key exposure**: The Gemini API key is bundled in the frontend JavaScript. This is a real security issue, but it can be fixed with a single Cloud Function or a tiny proxy server -- no full rewrite needed.
|
||||
2. **CORS proxy reliance**: The scraping uses third-party CORS proxies that can go down. A backend proxy fixes this, but again, a single endpoint on any lightweight server suffices.
|
||||
3. **Vendor lock-in**: Firebase Authentication + Firestore. This is a legitimate concern for a self-hosted deployment, but the migration path is incremental: swap Firestore for Supabase (Postgres + real-time), keep Firebase Auth temporarily.
|
||||
4. **Single LLM provider**: Only Gemini is supported. Adding providers is valuable, but can be done incrementally behind the existing frontend.
|
||||
|
||||
None of these problems require rewriting the frontend. None require Rust. None require SolidJS.
|
||||
|
||||
### 2.3 What Happens If the Project Is Abandoned Halfway?
|
||||
|
||||
With the current approach (total rewrite), abandonment at 50% means:
|
||||
|
||||
- The old app still runs on Firebase, but receives no updates.
|
||||
- The new app has a partially built Rust backend, a half-ported SolidJS frontend, and no working deployment.
|
||||
- **Total investment: wasted.**
|
||||
|
||||
With an incremental approach, each phase delivers value independently:
|
||||
|
||||
- Phase 1 (add backend proxy): Old app gets secure API key handling + server-side scraping. Value delivered even if Phase 2 never happens.
|
||||
- Phase 2 (multi-LLM): Backend supports multiple providers. Value delivered even if Phase 3 never happens.
|
||||
- Phase 3 (replace Firebase): Database migration to self-hosted. Value delivered even if Phase 4 never happens.
|
||||
- Phase 4 (frontend modernization): If still needed.
|
||||
|
||||
### 2.4 Realistic Timeline Estimate
|
||||
|
||||
For a single developer working part-time on this:
|
||||
|
||||
| Task | Optimistic | Realistic | Pessimistic |
|
||||
|---|---|---|---|
|
||||
| Rust backend skeleton (auth, CRUD, LLM proxy) | 3 weeks | 6 weeks | 12 weeks |
|
||||
| Multi-provider LLM abstraction | 2 weeks | 4 weeks | 8 weeks |
|
||||
| Auth system (email, magic link, sessions, captcha) | 2 weeks | 5 weeks | 10 weeks |
|
||||
| SolidJS frontend rewrite (5 pages + components) | 2 weeks | 4 weeks | 8 weeks |
|
||||
| Admin module | 1 week | 2 weeks | 4 weeks |
|
||||
| Data migration tooling | 1 week | 2 weeks | 3 weeks |
|
||||
| Docker/deployment | 1 week | 2 weeks | 3 weeks |
|
||||
| Testing and debugging | 2 weeks | 4 weeks | 8 weeks |
|
||||
| **Total** | **14 weeks** | **29 weeks** | **56 weeks** |
|
||||
|
||||
That is 7 months realistically, over a year pessimistically. For an application that is currently approximately 1,200 lines of TypeScript.
|
||||
|
||||
---
|
||||
|
||||
## 3. Hidden Complexity
|
||||
|
||||
### 3.1 Multi-Provider LLM Abstraction: The Hardest Problem Nobody Is Talking About
|
||||
|
||||
The current pipeline in `geminiService.ts` is deeply coupled to Gemini's specific capabilities:
|
||||
|
||||
1. **Pass 1** uses `googleSearch` grounding -- Gemini searches the web in real-time and returns grounded results with citations. This is not a generic "LLM call." It is a tool-use pattern specific to Gemini's architecture.
|
||||
2. **Structured output** uses `responseSchema` with `Type` enums from `@google/genai`. This forces JSON output conforming to a schema. OpenAI has `response_format: { type: "json_schema" }`. Anthropic has tool-use-based structured output. They are similar but not identical.
|
||||
3. **Pass 2** is a standard completion call. This is the easy part to abstract.
|
||||
|
||||
**The fundamental problem**: Gemini's `googleSearch` grounding is not just "web search." It is integrated into the model's reasoning process. The model sees search results inline and can refine its queries. To replicate this with OpenAI, you would need:
|
||||
- OpenAI with browsing (ChatGPT-only, not available via API for all models).
|
||||
- Or: A separate web search API (Serper, Brave, Google Custom Search) + a tool-use loop where the model calls search, reads results, decides to search again, etc.
|
||||
|
||||
With Anthropic, you would need:
|
||||
- Claude with the `web_search` tool (available but with different result formatting).
|
||||
- Or: Same external search + tool-use loop.
|
||||
|
||||
**What gets lost in abstraction**: Each provider returns search results differently. Gemini's grounded responses include inline citations with confidence scores. OpenAI's browsing returns rendered page content. Anthropic's web search returns snippets. The quality and structure of Pass 1 output will vary dramatically by provider, and Pass 2's rewriting quality depends on Pass 1's quality.
|
||||
|
||||
**The question to ask**: Are you building an abstraction over fundamentally different capabilities, or are you building three separate pipelines that happen to share a common interface? The latter is honest but expensive. The former is a leaky abstraction waiting to break.
|
||||
|
||||
### 3.2 The Scraping Pipeline
|
||||
|
||||
The current scraping in `filterValidNewsItems` runs in the browser using `DOMParser`. It:
|
||||
- Fetches HTML via CORS proxies.
|
||||
- Parses the DOM for meta tags, JSON-LD, `<time>` elements.
|
||||
- Detects soft 404s by scanning title/h1 content.
|
||||
- Extracts body text (up to 4,000 chars) after removing scripts/nav/footer.
|
||||
|
||||
Moving this server-side is straightforward (use `reqwest` + `scraper` or `select.rs` in Rust), but consider:
|
||||
- **Rate limiting against target sites**: Server-side scraping from a single IP is more likely to be blocked than distributed browser requests via CORS proxies.
|
||||
- **JavaScript-rendered content**: Many news sites render content via JavaScript. The current approach works because CORS proxies often return server-rendered HTML. A Rust backend would need a headless browser (Chromium via `headless_chrome` or `fantoccini`) for JS-heavy sites, adding significant complexity and resource usage.
|
||||
- **Legal considerations**: Server-side scraping at scale may trigger ToS violations. The current approach is effectively each user's browser making requests through proxies, which is more defensible.
|
||||
|
||||
### 3.3 Authentication: You Are Building a Small Identity Provider
|
||||
|
||||
The proposed auth system includes:
|
||||
- Email + captcha account creation
|
||||
- Magic link / passwordless auth
|
||||
- Session-based auth with secure cookies
|
||||
|
||||
This is not a feature. This is building a mini identity provider. Let me enumerate the hidden work:
|
||||
|
||||
1. **Email delivery infrastructure**: Magic links require sending emails. Self-hosted SMTP is a minefield of deliverability issues (SPF, DKIM, DMARC, IP reputation, bounce handling). You will use a transactional email service (SendGrid, Postmark, SES). That is a new dependency, a new account, a new API integration, and a new failure mode.
|
||||
2. **Token management**: Magic link tokens must be cryptographically secure, single-use, time-limited, and stored server-side. This is a custom token store with cleanup jobs.
|
||||
3. **Session management**: Secure cookies require HTTPS (certificate management), proper `SameSite`/`Secure`/`HttpOnly` flags, session storage (in-memory? database? Redis?), session rotation on privilege changes, and idle/absolute timeout handling.
|
||||
4. **Captcha integration**: reCAPTCHA or hCaptcha requires a frontend widget + server-side verification. Another API key to manage.
|
||||
5. **Account recovery**: What happens when a user loses access to their email? Password reset? Account deletion? GDPR compliance?
|
||||
6. **Rate limiting on auth endpoints**: Brute-force protection, account enumeration prevention, IP-based throttling.
|
||||
7. **Email verification**: Is the email valid? Bounce handling? Typo detection?
|
||||
|
||||
**Comparison**: Firebase Auth gives you all of this for free, plus Google SSO, Apple SSO, email/password auth, phone auth, anonymous auth, multi-factor auth, and is SOC2/ISO27001 compliant. You are proposing to replace a battle-tested managed service with a hand-rolled implementation.
|
||||
|
||||
**Alternative**: Use a self-hosted auth service like Authentik, Keycloak, or Authelia. These are deployable via Docker, support email+password, magic link, OAuth, and session management out of the box. Zero custom auth code needed.
|
||||
|
||||
### 3.4 Admin Module
|
||||
|
||||
The admin module for "LLM API key configuration" and "rate limiter configuration" sounds simple, but implies:
|
||||
- Role-based access control (who is an admin?)
|
||||
- A secure key storage mechanism (encrypting API keys at rest)
|
||||
- A UI for managing per-provider rate limits
|
||||
- Runtime configuration changes (changing rate limits without restart)
|
||||
- Audit logging (who changed what, when?)
|
||||
|
||||
This is not a settings page. This is an operational control plane.
|
||||
|
||||
---
|
||||
|
||||
## 4. Spotting the Contradictions
|
||||
|
||||
### 4.1 "Single VM deployment" + "with or without Docker"
|
||||
|
||||
- **With Docker**: Straightforward. `docker build` produces a container, `docker-compose` adds Postgres. Works anywhere Docker runs.
|
||||
- **Without Docker**: You need to compile Rust for the target architecture. If developing on macOS (darwin/arm64) and deploying to Linux (amd64), you need cross-compilation via `cross` or a CI/CD pipeline that builds on Linux. Rust cross-compilation is possible but non-trivial (especially with OpenSSL/TLS dependencies). You also need to install and configure SQLite/Postgres natively, manage the SolidJS build output, set up a reverse proxy (nginx/caddy), and handle process management (systemd).
|
||||
|
||||
**The question to ask**: Is "without Docker" a hard requirement or a nice-to-have? If the latter, drop it and save significant deployment complexity.
|
||||
|
||||
### 4.2 "Preserve UX and look & feel" + "switch to SolidJS"
|
||||
|
||||
The current app uses:
|
||||
- React Context for auth state propagation.
|
||||
- `useEffect` + `onSnapshot` for real-time Firestore listeners with cleanup.
|
||||
- `react-router-dom` v7 for routing with `<Link>`, `useParams`, `useNavigate`.
|
||||
- Controlled form inputs with `useState`.
|
||||
- Conditional rendering patterns (`{condition && <Component />}`).
|
||||
|
||||
SolidJS equivalents exist, but they work differently:
|
||||
- `createContext` exists but components do not re-render -- signals propagate granularly. This changes how you structure state.
|
||||
- `createEffect` does not take a dependency array. It auto-tracks. Cleanup uses `onCleanup`. The mental model is different.
|
||||
- `@solidjs/router` has `<A>` instead of `<Link>`, `useParams()` returns a reactive proxy, not a plain object.
|
||||
- Signals replace `useState`. `createSignal` returns `[getter, setter]` where the getter is a function, not a value. Every template reference becomes `count()` instead of `count`.
|
||||
- Conditional rendering uses `<Show>` and `<For>` components, not inline JSX ternaries (which would defeat fine-grained reactivity).
|
||||
|
||||
"Preserving UX" means the user sees the same screens. "Preserving look and feel" means Tailwind classes copy over. But the underlying component logic must be substantially rewritten. This is not a port; it is a rewrite guided by screenshots.
|
||||
|
||||
### 4.3 "Improve CORS proxy handling"
|
||||
|
||||
The current app uses CORS proxies because it scrapes from the browser. A backend can scrape directly. So the "improvement" is not improving CORS handling -- it is eliminating the need for CORS proxies entirely. This is a side effect of adding a backend, not a feature to list separately. Listing it as a requirement inflates the apparent scope and obscures the real motivation.
|
||||
|
||||
### 4.4 "Remove Google hosting dependencies" + Email-based Auth Still Needs an External Service
|
||||
|
||||
You are removing dependency on Firebase (Google) but adding dependency on:
|
||||
- A transactional email provider (SendGrid/Postmark/SES) for magic links.
|
||||
- A captcha provider (Google reCAPTCHA or hCaptcha).
|
||||
- Potentially a web search API (Serper/Brave/Google CSE) for non-Gemini providers.
|
||||
|
||||
You are trading one set of dependencies for another. The net dependency count may actually increase.
|
||||
|
||||
---
|
||||
|
||||
## 5. Risk Assessment
|
||||
|
||||
### Top 5 Risks That Could Derail This Project
|
||||
|
||||
1. **Scope creep leading to abandonment**: The project is already at maximum scope (full rewrite + new features). Any scope creep (even small -- "oh, we also need password auth" or "can we add dark mode?") pushes the timeline past the point where motivation sustains the effort. For a part-time/side project, this is the #1 killer.
|
||||
|
||||
2. **Multi-LLM abstraction turns into a quagmire**: The web search grounding capabilities differ so fundamentally across providers that the abstraction layer either (a) drops to lowest common denominator (losing Gemini's best feature), (b) becomes provider-specific spaghetti code behind a thin interface, or (c) consumes weeks of engineering time trying to build equivalence that does not exist.
|
||||
|
||||
3. **Auth system vulnerabilities**: Hand-rolled authentication is the most common source of security vulnerabilities in web applications. Missing CSRF protection, insecure token storage, session fixation, timing attacks on token comparison, missing rate limiting -- any of these creates a vulnerability that Firebase Auth handled for free.
|
||||
|
||||
4. **Data migration failures**: Existing users have data in Firestore (syntheses, sources, settings). Migrating this to SQLite/Postgres requires: (a) exporting from Firestore, (b) transforming the schema (NoSQL to relational), (c) handling the legacy `SynthesisData` format (with both old field-based and new `sections[]`-based structures), and (d) validating that nothing was lost. This is tedious, error-prone work that is easy to underestimate.
|
||||
|
||||
5. **Deployment complexity kills adoption**: The current app deploys to Firebase Hosting with `firebase deploy`. The new app requires: (a) compiling Rust, (b) building SolidJS, (c) configuring a reverse proxy, (d) managing a database, (e) setting up email delivery, (f) managing TLS certificates, and (g) process supervision. Even with Docker, this is an order of magnitude more operational burden. For a personal/small-team tool, operational burden directly reduces the likelihood of the app surviving.
|
||||
|
||||
### Rollback Plan
|
||||
|
||||
There is no rollback plan because there is nothing to roll back to. The old app continues to run on Firebase until it is deliberately shut down. The risk is not "rolling back" but rather "having spent months on a new version that never reaches feature parity with the original."
|
||||
|
||||
### Impact on Existing Users and Data
|
||||
|
||||
- Existing users must create new accounts (no Firebase Auth = no Google SSO migration path unless you also implement Google OAuth in the new system).
|
||||
- Data must be explicitly migrated from Firestore to the new database.
|
||||
- The email feature currently uses Gmail API via OAuth popup. The new system would need its own email sending capability.
|
||||
- If the migration is botched, user data is lost. Firestore has no "undo."
|
||||
|
||||
---
|
||||
|
||||
## 6. Alternative Approaches
|
||||
|
||||
### 6.1 Minimal Viable Refactoring (2-4 weeks)
|
||||
|
||||
Keep React. Keep Firestore. Add a tiny backend (Node.js/Express or Python/FastAPI) that:
|
||||
1. Proxies Gemini API calls (solves the API key exposure problem).
|
||||
2. Handles scraping server-side (eliminates CORS proxy dependency).
|
||||
3. Exposes an endpoint for email sending (replaces Gmail API popup).
|
||||
|
||||
**What this gives you**: The two biggest security/reliability issues solved, with zero frontend changes, zero database migration, and zero user impact. Deploy the proxy on a small VM or as a Cloud Function.
|
||||
|
||||
### 6.2 Incremental Backend Migration (3-6 months, phased)
|
||||
|
||||
**Phase 1 -- API Gateway (4 weeks)**:
|
||||
- Build a lightweight backend (Go or Python) that proxies LLM calls and handles scraping.
|
||||
- Frontend calls the new backend instead of Gemini directly.
|
||||
- Firebase Auth + Firestore remain.
|
||||
- Deliverable: Secure API key handling, reliable scraping.
|
||||
|
||||
**Phase 2 -- Multi-Provider LLM (4 weeks)**:
|
||||
- Backend supports Gemini, OpenAI, Anthropic.
|
||||
- Settings page adds provider selection.
|
||||
- Provider-specific adapters behind a common interface.
|
||||
- Deliverable: LLM provider choice.
|
||||
|
||||
**Phase 3 -- Database Migration (4 weeks)**:
|
||||
- Replace Firestore with Postgres (via Supabase for managed, or raw Postgres for self-hosted).
|
||||
- Write migration scripts.
|
||||
- Update frontend Firestore calls to REST API calls.
|
||||
- Deliverable: No more Firebase dependency for data.
|
||||
|
||||
**Phase 4 -- Auth Migration (4 weeks)**:
|
||||
- Replace Firebase Auth with self-hosted auth (Authentik/Keycloak) or add email auth alongside Google OAuth.
|
||||
- Deliverable: Self-hosted auth.
|
||||
|
||||
**Phase 5 -- Frontend Modernization (optional, 4 weeks)**:
|
||||
- If React is genuinely insufficient, consider Svelte or keep React.
|
||||
- Deliverable: Modernized frontend (if needed).
|
||||
|
||||
Each phase ships independently. Each phase delivers value. Abandonment at any phase leaves the system in a better state than it started.
|
||||
|
||||
### 6.3 For Each Major Decision: Alternatives and Trade-offs
|
||||
|
||||
| Decision | Proposed | Alternative 1 | Alternative 2 |
|
||||
|---|---|---|---|
|
||||
| Backend language | Rust | Go: faster dev, easy cross-compile, good web ecosystem | Python/FastAPI: fastest dev, best LLM SDK ecosystem, async |
|
||||
| Frontend framework | SolidJS | Keep React: zero rewrite cost, huge ecosystem | Svelte: similar reactivity to SolidJS, larger community |
|
||||
| Database | SQLite + "Postgres path" | Postgres from start: no migration debt | Supabase: managed Postgres + real-time + auth |
|
||||
| Auth | Custom email+magic link | Authentik/Keycloak: self-hosted, full-featured, zero custom code | Keep Firebase Auth: works, managed, free tier |
|
||||
| Email delivery | Custom implementation | Postmark/SendGrid: managed, deliverable, $0-20/month | Keep Gmail API: already works, zero infrastructure |
|
||||
| Deployment | Docker on VM | Fly.io/Railway: managed Docker, free tier, simpler ops | Keep Firebase Hosting + add Cloud Functions |
|
||||
|
||||
---
|
||||
|
||||
## 7. Hard Questions for the Project Owner
|
||||
|
||||
These are questions where the answer significantly changes the implementation. They must be answered before writing a single line of code.
|
||||
|
||||
1. **How many users will this serve?** If < 5 (personal/team tool), SQLite is fine, self-hosted auth is overkill, and operational simplicity should dominate every decision. If > 50, you need Postgres, proper auth, and monitoring from day one.
|
||||
|
||||
2. **Is the Rust choice driven by the project's needs or by a learning goal?** Both answers are valid, but they lead to different architectures. A learning project should embrace complexity. A product should minimize it.
|
||||
|
||||
3. **What is the budget for external services?** Transactional email ($0-20/month), LLM APIs ($10-100/month), domain + TLS ($10/year), VM ($5-50/month). If budget is zero, Firebase's free tier is hard to beat.
|
||||
|
||||
4. **What happens to existing Firestore data?** Is there a migration plan? How many syntheses exist? Are the legacy format syntheses (with `majorAnnouncements`, `financialSector`, etc.) being migrated or abandoned?
|
||||
|
||||
5. **Is Google SSO being kept alongside the new auth methods, or replaced entirely?** If replaced, existing users lose access to their accounts. If kept, you need OAuth2 client implementation in the new backend.
|
||||
|
||||
6. **Which LLM providers are actually needed on day one?** If the answer is "just Gemini for now, but we want the option for others," build the abstraction later. YAGNI applies. Adding a second provider later is cheaper than building a three-provider abstraction upfront that may never be used.
|
||||
|
||||
7. **Who will operate this in production?** Self-hosted means self-operated. Database backups, TLS certificate renewal, security patches, log monitoring, uptime monitoring. Is someone committed to this ongoing work?
|
||||
|
||||
8. **Is "without Docker" a hard requirement?** Supporting both Docker and non-Docker deployment doubles the deployment documentation and testing surface. If Docker is available on the target VM, mandate it and simplify.
|
||||
|
||||
9. **What is the desired SLA?** If the app can be down for hours without consequence (personal tool), the architecture can be simpler. If uptime matters, you need health checks, restart policies, and monitoring.
|
||||
|
||||
10. **How will the admin module's API keys be secured at rest?** SQLite stores data in a plain file. API keys in SQLite are readable by anyone with file access. Do you need encryption at rest? If so, how? (SQLCipher? Application-level encryption? OS-level disk encryption?)
|
||||
|
||||
11. **What is the testing strategy?** The current app has zero tests. The proposed rewrite has zero mention of testing. A Rust backend should have integration tests for every endpoint, unit tests for the LLM abstraction, and end-to-end tests for the auth flow. Is this budgeted?
|
||||
|
||||
12. **Is the 2-pass generation pipeline (search + scrape + rewrite) non-negotiable?** Some LLM providers may not support grounded web search at all. If the pipeline is mandatory, this constrains which providers are viable. If Pass 1 can be simplified (e.g., just use a search API + LLM summary without scraping), the multi-provider problem becomes much simpler.
|
||||
|
||||
13. **What is the target response time for synthesis generation?** Currently, generation takes 30-60+ seconds (two Gemini calls + parallel scraping). Will users accept the same latency? Should generation be asynchronous (start generation, get notified when done)?
|
||||
|
||||
14. **Is the Gmail email sending feature being kept?** The current implementation uses a Gmail OAuth popup to send via the user's own Gmail account. The new system would presumably send from a system email address. These are fundamentally different features with different privacy implications.
|
||||
|
||||
15. **What is the concrete timeline and deadline?** Without a deadline, scope expands indefinitely. With a deadline, scope must be ruthlessly cut. Which features are must-have for v1, and which are nice-to-have for v2?
|
||||
|
||||
---
|
||||
|
||||
## 8. Summary Verdict
|
||||
|
||||
The proposed refactoring is ambitious to the point of being reckless. It simultaneously changes every layer of the stack, introduces multiple new features, and removes all existing infrastructure -- with no phased delivery plan and no fallback position.
|
||||
|
||||
The current application has real problems (API key exposure, CORS proxy fragility, vendor lock-in), but none of them require a full rewrite. A 200-line proxy server solves the two most critical issues. An incremental migration plan addresses vendor lock-in without risking the entire project.
|
||||
|
||||
**My recommendation**: Before writing any code, answer the 15 questions above. Then build the smallest possible backend that solves the API key and scraping problems. Ship that. Use it for a month. Then decide if the rest of the rewrite is still justified, with the benefit of real experience operating a backend for this application.
|
||||
|
||||
The best refactoring is the one that delivers value at every intermediate step, not the one that promises everything at the end.
|
||||
Loading…
Reference in New Issue