You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
359 lines
29 KiB
Markdown
359 lines
29 KiB
Markdown
# Devil's Advocate Analysis: AI Weekly Synth Refactoring
|
|
|
|
## Preamble
|
|
|
|
This document exists to stress-test every assumption behind the proposed refactoring. The current application -- a React/Firebase/Gemini SPA -- is functional, deployed, and serving users. The proposal is to rewrite **everything**: new backend language (Rust), new database (SQLite), new frontend framework (SolidJS), new auth system, new LLM abstraction layer, and new deployment model. That is not a refactoring. That is a greenfield rewrite wearing a refactoring's name, and it deserves brutal scrutiny.
|
|
|
|
---
|
|
|
|
## 1. Questioning the Technology Choices
|
|
|
|
### 1.1 Rust for the Backend
|
|
|
|
**The case against:**
|
|
|
|
The current application has zero backend code. It is a thin SPA that talks to Firebase and Gemini directly from the browser. The "backend" being introduced serves three purposes: (a) proxy LLM API calls, (b) manage auth sessions, and (c) CRUD operations on a database. None of these require Rust.
|
|
|
|
- **Development velocity**: Rust's compile times, borrow checker friction, and verbose error handling will slow feature iteration by 3-5x compared to Go, Python/FastAPI, or Node.js/Express. This is a CRUD app with LLM API calls, not a systems programming project.
|
|
- **Ecosystem maturity for web**: The Rust web ecosystem (actix-web, axum, tower) is maturing but remains significantly less documented and less battle-tested for standard web applications than Express, FastAPI, or Go's net/http. Libraries for session management, email sending, CSRF protection, and OAuth flows are less polished.
|
|
- **Hiring and maintenance**: If the original developer leaves, finding a Rust web developer is substantially harder and more expensive than finding someone who knows Python, Go, or Node.js. The codebase becomes a bus-factor-1 liability.
|
|
- **What Rust gives you that you do not need**: Memory safety without a GC, zero-cost abstractions, and bare-metal performance. This application proxies LLM API calls that take 10-60 seconds each. The bottleneck is external API latency, not backend compute. You could run this on Python with FastAPI and it would perform identically from the user's perspective.
|
|
|
|
**The question to ask**: Is Rust chosen because it is the right tool, or because someone wants to learn Rust?
|
|
|
|
**Alternative**: Go gives you compiled binary deployment, excellent concurrency, trivial cross-compilation, a mature web ecosystem (Gin, Echo, Chi), and a learning curve measured in weeks rather than months. Python/FastAPI gives you the fastest development speed, native async, and the richest LLM SDK ecosystem (every LLM provider ships a Python SDK first).
|
|
|
|
### 1.2 SolidJS for the Frontend
|
|
|
|
**The case against:**
|
|
|
|
The current React frontend is approximately 900 lines of code across 7 files. It is straightforward, uses standard patterns (Context, useEffect, useState), and works. The proposal is to rewrite all of it in SolidJS.
|
|
|
|
- **Cost/benefit ratio**: SolidJS's reactive model is technically superior to React's virtual DOM diffing. But for an app with 5 pages, no complex animations, no heavy lists, and no real-time data visualization, the performance difference is imperceptible. You are paying the full cost of a rewrite for zero user-visible benefit.
|
|
- **Ecosystem**: SolidJS has roughly 1/100th of React's ecosystem. `lucide-react` does not have a SolidJS version (you would need `lucide-solid` or raw SVGs). `react-router-dom` becomes `@solidjs/router` with different APIs. `date-fns` works, but any React-specific utility does not.
|
|
- **Tailwind CSS**: Works with SolidJS, but the JSX-like syntax (JSX vs TSX) is slightly different (e.g., `class` vs `className`, `for` vs `htmlFor`). Every component needs manual translation.
|
|
- **Real-time listeners**: The current app uses Firestore's `onSnapshot` for real-time updates on the Home and Sources pages. SolidJS's reactive primitives (`createSignal`, `createEffect`) differ fundamentally from React's `useState`/`useEffect`. The subscription cleanup patterns are different. This is not a find-and-replace operation.
|
|
- **Developer pool**: If you hire someone to maintain this, they probably know React. They probably do not know SolidJS.
|
|
|
|
**The question to ask**: What is SolidJS giving you that React 19 with its compiler cannot? Name one user-facing feature that is impossible or unreasonably difficult in React but easy in SolidJS.
|
|
|
|
**Alternative**: Keep React. The frontend works. If you want to modernize, adopt React Server Components or Next.js/Remix. If you want a different framework for legitimate reasons, consider Svelte (larger ecosystem than SolidJS, similar reactivity model, larger community).
|
|
|
|
### 1.3 SQLite with "Postgres Upgrade Path"
|
|
|
|
**The case against:**
|
|
|
|
- **Write concurrency**: SQLite uses a single-writer model. When user A is generating a synthesis (which involves multiple writes during scraping/validation), user B's writes will block. With WAL mode, reads are concurrent, but writes serialize. For a "multi-user" application, this is a real limitation that manifests at surprisingly low user counts (10-20 concurrent writers).
|
|
- **The "upgrade path" is a myth**: `sqlx` supports both SQLite and Postgres, but the SQL dialects differ. SQLite lacks: `JSONB` operators, `ARRAY` types, `LATERAL JOIN`, `INSERT ... ON CONFLICT DO UPDATE` (supported since 3.24 but with different syntax), `ENUM` types, advisory locks, and `LISTEN/NOTIFY`. If you write queries that use any Postgres-specific feature, you cannot "downgrade" to SQLite. If you restrict yourself to the SQLite subset, you are not getting the benefits of Postgres. You end up writing to the lowest common denominator of both databases.
|
|
- **Migration tooling**: `sqlx` migrations work for both, but you will need separate migration files if the dialects diverge. Testing requires running against both databases. CI doubles.
|
|
- **Operational burden**: SQLite is a file on disk. Backups are `cp`. But that means backup-during-write requires `VACUUM INTO` or filesystem snapshots. There is no `pg_dump`-equivalent streaming backup. There is no replication. There is no point-in-time recovery.
|
|
|
|
**The question to ask**: How many concurrent users are expected? If more than 10-20, start with Postgres. If fewer than 5, SQLite is fine, but then why plan for a "Postgres upgrade path" you will never need?
|
|
|
|
**Alternative**: Just use Postgres from the start. Docker makes this trivial. A `docker-compose.yml` with Postgres + the app is no more complex than SQLite, and you avoid the "upgrade path" fiction entirely.
|
|
|
|
---
|
|
|
|
## 2. Challenging the Scope
|
|
|
|
### 2.1 This Is a Complete Rewrite, Not a Refactoring
|
|
|
|
Let me enumerate what is being changed:
|
|
|
|
| Component | Current | Proposed | Effort |
|
|
|---|---|---|---|
|
|
| Backend | None (serverless) | Rust + axum/actix | Build from scratch |
|
|
| Database | Firestore (managed) | SQLite/Postgres (self-managed) | Build from scratch |
|
|
| Frontend framework | React 19 | SolidJS | Complete rewrite |
|
|
| Auth | Firebase Auth (Google SSO) | Email+captcha, magic link, sessions | Build from scratch |
|
|
| LLM integration | Gemini only (frontend) | Multi-provider (backend) | Major redesign |
|
|
| Deployment | Firebase Hosting | Docker/VM | Build from scratch |
|
|
| Admin module | None | New feature | Build from scratch |
|
|
| Email system | Gmail API (frontend) | Backend email service | Build from scratch |
|
|
|
|
**Every single component is being replaced.** There is no code that survives this refactoring. Zero reuse. This is the textbook definition of a "Second System Effect" (Brooks, 1975).
|
|
|
|
### 2.2 What Is Actually Broken?
|
|
|
|
Before rewriting everything, let us be honest about what problems exist today:
|
|
|
|
1. **API key exposure**: The Gemini API key is bundled in the frontend JavaScript. This is a real security issue, but it can be fixed with a single Cloud Function or a tiny proxy server -- no full rewrite needed.
|
|
2. **CORS proxy reliance**: The scraping uses third-party CORS proxies that can go down. A backend proxy fixes this, but again, a single endpoint on any lightweight server suffices.
|
|
3. **Vendor lock-in**: Firebase Authentication + Firestore. This is a legitimate concern for a self-hosted deployment, but the migration path is incremental: swap Firestore for Supabase (Postgres + real-time), keep Firebase Auth temporarily.
|
|
4. **Single LLM provider**: Only Gemini is supported. Adding providers is valuable, but can be done incrementally behind the existing frontend.
|
|
|
|
None of these problems require rewriting the frontend. None require Rust. None require SolidJS.
|
|
|
|
### 2.3 What Happens If the Project Is Abandoned Halfway?
|
|
|
|
With the current approach (total rewrite), abandonment at 50% means:
|
|
|
|
- The old app still runs on Firebase, but receives no updates.
|
|
- The new app has a partially built Rust backend, a half-ported SolidJS frontend, and no working deployment.
|
|
- **Total investment: wasted.**
|
|
|
|
With an incremental approach, each phase delivers value independently:
|
|
|
|
- Phase 1 (add backend proxy): Old app gets secure API key handling + server-side scraping. Value delivered even if Phase 2 never happens.
|
|
- Phase 2 (multi-LLM): Backend supports multiple providers. Value delivered even if Phase 3 never happens.
|
|
- Phase 3 (replace Firebase): Database migration to self-hosted. Value delivered even if Phase 4 never happens.
|
|
- Phase 4 (frontend modernization): If still needed.
|
|
|
|
### 2.4 Realistic Timeline Estimate
|
|
|
|
For a single developer working part-time on this:
|
|
|
|
| Task | Optimistic | Realistic | Pessimistic |
|
|
|---|---|---|---|
|
|
| Rust backend skeleton (auth, CRUD, LLM proxy) | 3 weeks | 6 weeks | 12 weeks |
|
|
| Multi-provider LLM abstraction | 2 weeks | 4 weeks | 8 weeks |
|
|
| Auth system (email, magic link, sessions, captcha) | 2 weeks | 5 weeks | 10 weeks |
|
|
| SolidJS frontend rewrite (5 pages + components) | 2 weeks | 4 weeks | 8 weeks |
|
|
| Admin module | 1 week | 2 weeks | 4 weeks |
|
|
| Data migration tooling | 1 week | 2 weeks | 3 weeks |
|
|
| Docker/deployment | 1 week | 2 weeks | 3 weeks |
|
|
| Testing and debugging | 2 weeks | 4 weeks | 8 weeks |
|
|
| **Total** | **14 weeks** | **29 weeks** | **56 weeks** |
|
|
|
|
That is 7 months realistically, over a year pessimistically. For an application that is currently approximately 1,200 lines of TypeScript.
|
|
|
|
---
|
|
|
|
## 3. Hidden Complexity
|
|
|
|
### 3.1 Multi-Provider LLM Abstraction: The Hardest Problem Nobody Is Talking About
|
|
|
|
The current pipeline in `geminiService.ts` is deeply coupled to Gemini's specific capabilities:
|
|
|
|
1. **Pass 1** uses `googleSearch` grounding -- Gemini searches the web in real-time and returns grounded results with citations. This is not a generic "LLM call." It is a tool-use pattern specific to Gemini's architecture.
|
|
2. **Structured output** uses `responseSchema` with `Type` enums from `@google/genai`. This forces JSON output conforming to a schema. OpenAI has `response_format: { type: "json_schema" }`. Anthropic has tool-use-based structured output. They are similar but not identical.
|
|
3. **Pass 2** is a standard completion call. This is the easy part to abstract.
|
|
|
|
**The fundamental problem**: Gemini's `googleSearch` grounding is not just "web search." It is integrated into the model's reasoning process. The model sees search results inline and can refine its queries. To replicate this with OpenAI, you would need:
|
|
- OpenAI with browsing (ChatGPT-only, not available via API for all models).
|
|
- Or: A separate web search API (Serper, Brave, Google Custom Search) + a tool-use loop where the model calls search, reads results, decides to search again, etc.
|
|
|
|
With Anthropic, you would need:
|
|
- Claude with the `web_search` tool (available but with different result formatting).
|
|
- Or: Same external search + tool-use loop.
|
|
|
|
**What gets lost in abstraction**: Each provider returns search results differently. Gemini's grounded responses include inline citations with confidence scores. OpenAI's browsing returns rendered page content. Anthropic's web search returns snippets. The quality and structure of Pass 1 output will vary dramatically by provider, and Pass 2's rewriting quality depends on Pass 1's quality.
|
|
|
|
**The question to ask**: Are you building an abstraction over fundamentally different capabilities, or are you building three separate pipelines that happen to share a common interface? The latter is honest but expensive. The former is a leaky abstraction waiting to break.
|
|
|
|
### 3.2 The Scraping Pipeline
|
|
|
|
The current scraping in `filterValidNewsItems` runs in the browser using `DOMParser`. It:
|
|
- Fetches HTML via CORS proxies.
|
|
- Parses the DOM for meta tags, JSON-LD, `<time>` elements.
|
|
- Detects soft 404s by scanning title/h1 content.
|
|
- Extracts body text (up to 4,000 chars) after removing scripts/nav/footer.
|
|
|
|
Moving this server-side is straightforward (use `reqwest` + `scraper` or `select.rs` in Rust), but consider:
|
|
- **Rate limiting against target sites**: Server-side scraping from a single IP is more likely to be blocked than distributed browser requests via CORS proxies.
|
|
- **JavaScript-rendered content**: Many news sites render content via JavaScript. The current approach works because CORS proxies often return server-rendered HTML. A Rust backend would need a headless browser (Chromium via `headless_chrome` or `fantoccini`) for JS-heavy sites, adding significant complexity and resource usage.
|
|
- **Legal considerations**: Server-side scraping at scale may trigger ToS violations. The current approach is effectively each user's browser making requests through proxies, which is more defensible.
|
|
|
|
### 3.3 Authentication: You Are Building a Small Identity Provider
|
|
|
|
The proposed auth system includes:
|
|
- Email + captcha account creation
|
|
- Magic link / passwordless auth
|
|
- Session-based auth with secure cookies
|
|
|
|
This is not a feature. This is building a mini identity provider. Let me enumerate the hidden work:
|
|
|
|
1. **Email delivery infrastructure**: Magic links require sending emails. Self-hosted SMTP is a minefield of deliverability issues (SPF, DKIM, DMARC, IP reputation, bounce handling). You will use a transactional email service (SendGrid, Postmark, SES). That is a new dependency, a new account, a new API integration, and a new failure mode.
|
|
2. **Token management**: Magic link tokens must be cryptographically secure, single-use, time-limited, and stored server-side. This is a custom token store with cleanup jobs.
|
|
3. **Session management**: Secure cookies require HTTPS (certificate management), proper `SameSite`/`Secure`/`HttpOnly` flags, session storage (in-memory? database? Redis?), session rotation on privilege changes, and idle/absolute timeout handling.
|
|
4. **Captcha integration**: reCAPTCHA or hCaptcha requires a frontend widget + server-side verification. Another API key to manage.
|
|
5. **Account recovery**: What happens when a user loses access to their email? Password reset? Account deletion? GDPR compliance?
|
|
6. **Rate limiting on auth endpoints**: Brute-force protection, account enumeration prevention, IP-based throttling.
|
|
7. **Email verification**: Is the email valid? Bounce handling? Typo detection?
|
|
|
|
**Comparison**: Firebase Auth gives you all of this for free, plus Google SSO, Apple SSO, email/password auth, phone auth, anonymous auth, multi-factor auth, and is SOC2/ISO27001 compliant. You are proposing to replace a battle-tested managed service with a hand-rolled implementation.
|
|
|
|
**Alternative**: Use a self-hosted auth service like Authentik, Keycloak, or Authelia. These are deployable via Docker, support email+password, magic link, OAuth, and session management out of the box. Zero custom auth code needed.
|
|
|
|
### 3.4 Admin Module
|
|
|
|
The admin module for "LLM API key configuration" and "rate limiter configuration" sounds simple, but implies:
|
|
- Role-based access control (who is an admin?)
|
|
- A secure key storage mechanism (encrypting API keys at rest)
|
|
- A UI for managing per-provider rate limits
|
|
- Runtime configuration changes (changing rate limits without restart)
|
|
- Audit logging (who changed what, when?)
|
|
|
|
This is not a settings page. This is an operational control plane.
|
|
|
|
---
|
|
|
|
## 4. Spotting the Contradictions
|
|
|
|
### 4.1 "Single VM deployment" + "with or without Docker"
|
|
|
|
- **With Docker**: Straightforward. `docker build` produces a container, `docker-compose` adds Postgres. Works anywhere Docker runs.
|
|
- **Without Docker**: You need to compile Rust for the target architecture. If developing on macOS (darwin/arm64) and deploying to Linux (amd64), you need cross-compilation via `cross` or a CI/CD pipeline that builds on Linux. Rust cross-compilation is possible but non-trivial (especially with OpenSSL/TLS dependencies). You also need to install and configure SQLite/Postgres natively, manage the SolidJS build output, set up a reverse proxy (nginx/caddy), and handle process management (systemd).
|
|
|
|
**The question to ask**: Is "without Docker" a hard requirement or a nice-to-have? If the latter, drop it and save significant deployment complexity.
|
|
|
|
### 4.2 "Preserve UX and look & feel" + "switch to SolidJS"
|
|
|
|
The current app uses:
|
|
- React Context for auth state propagation.
|
|
- `useEffect` + `onSnapshot` for real-time Firestore listeners with cleanup.
|
|
- `react-router-dom` v7 for routing with `<Link>`, `useParams`, `useNavigate`.
|
|
- Controlled form inputs with `useState`.
|
|
- Conditional rendering patterns (`{condition && <Component />}`).
|
|
|
|
SolidJS equivalents exist, but they work differently:
|
|
- `createContext` exists but components do not re-render -- signals propagate granularly. This changes how you structure state.
|
|
- `createEffect` does not take a dependency array. It auto-tracks. Cleanup uses `onCleanup`. The mental model is different.
|
|
- `@solidjs/router` has `<A>` instead of `<Link>`, `useParams()` returns a reactive proxy, not a plain object.
|
|
- Signals replace `useState`. `createSignal` returns `[getter, setter]` where the getter is a function, not a value. Every template reference becomes `count()` instead of `count`.
|
|
- Conditional rendering uses `<Show>` and `<For>` components, not inline JSX ternaries (which would defeat fine-grained reactivity).
|
|
|
|
"Preserving UX" means the user sees the same screens. "Preserving look and feel" means Tailwind classes copy over. But the underlying component logic must be substantially rewritten. This is not a port; it is a rewrite guided by screenshots.
|
|
|
|
### 4.3 "Improve CORS proxy handling"
|
|
|
|
The current app uses CORS proxies because it scrapes from the browser. A backend can scrape directly. So the "improvement" is not improving CORS handling -- it is eliminating the need for CORS proxies entirely. This is a side effect of adding a backend, not a feature to list separately. Listing it as a requirement inflates the apparent scope and obscures the real motivation.
|
|
|
|
### 4.4 "Remove Google hosting dependencies" + Email-based Auth Still Needs an External Service
|
|
|
|
You are removing dependency on Firebase (Google) but adding dependency on:
|
|
- A transactional email provider (SendGrid/Postmark/SES) for magic links.
|
|
- A captcha provider (Google reCAPTCHA or hCaptcha).
|
|
- Potentially a web search API (Serper/Brave/Google CSE) for non-Gemini providers.
|
|
|
|
You are trading one set of dependencies for another. The net dependency count may actually increase.
|
|
|
|
---
|
|
|
|
## 5. Risk Assessment
|
|
|
|
### Top 5 Risks That Could Derail This Project
|
|
|
|
1. **Scope creep leading to abandonment**: The project is already at maximum scope (full rewrite + new features). Any scope creep (even small -- "oh, we also need password auth" or "can we add dark mode?") pushes the timeline past the point where motivation sustains the effort. For a part-time/side project, this is the #1 killer.
|
|
|
|
2. **Multi-LLM abstraction turns into a quagmire**: The web search grounding capabilities differ so fundamentally across providers that the abstraction layer either (a) drops to lowest common denominator (losing Gemini's best feature), (b) becomes provider-specific spaghetti code behind a thin interface, or (c) consumes weeks of engineering time trying to build equivalence that does not exist.
|
|
|
|
3. **Auth system vulnerabilities**: Hand-rolled authentication is the most common source of security vulnerabilities in web applications. Missing CSRF protection, insecure token storage, session fixation, timing attacks on token comparison, missing rate limiting -- any of these creates a vulnerability that Firebase Auth handled for free.
|
|
|
|
4. **Data migration failures**: Existing users have data in Firestore (syntheses, sources, settings). Migrating this to SQLite/Postgres requires: (a) exporting from Firestore, (b) transforming the schema (NoSQL to relational), (c) handling the legacy `SynthesisData` format (with both old field-based and new `sections[]`-based structures), and (d) validating that nothing was lost. This is tedious, error-prone work that is easy to underestimate.
|
|
|
|
5. **Deployment complexity kills adoption**: The current app deploys to Firebase Hosting with `firebase deploy`. The new app requires: (a) compiling Rust, (b) building SolidJS, (c) configuring a reverse proxy, (d) managing a database, (e) setting up email delivery, (f) managing TLS certificates, and (g) process supervision. Even with Docker, this is an order of magnitude more operational burden. For a personal/small-team tool, operational burden directly reduces the likelihood of the app surviving.
|
|
|
|
### Rollback Plan
|
|
|
|
There is no rollback plan because there is nothing to roll back to. The old app continues to run on Firebase until it is deliberately shut down. The risk is not "rolling back" but rather "having spent months on a new version that never reaches feature parity with the original."
|
|
|
|
### Impact on Existing Users and Data
|
|
|
|
- Existing users must create new accounts (no Firebase Auth = no Google SSO migration path unless you also implement Google OAuth in the new system).
|
|
- Data must be explicitly migrated from Firestore to the new database.
|
|
- The email feature currently uses Gmail API via OAuth popup. The new system would need its own email sending capability.
|
|
- If the migration is botched, user data is lost. Firestore has no "undo."
|
|
|
|
---
|
|
|
|
## 6. Alternative Approaches
|
|
|
|
### 6.1 Minimal Viable Refactoring (2-4 weeks)
|
|
|
|
Keep React. Keep Firestore. Add a tiny backend (Node.js/Express or Python/FastAPI) that:
|
|
1. Proxies Gemini API calls (solves the API key exposure problem).
|
|
2. Handles scraping server-side (eliminates CORS proxy dependency).
|
|
3. Exposes an endpoint for email sending (replaces Gmail API popup).
|
|
|
|
**What this gives you**: The two biggest security/reliability issues solved, with zero frontend changes, zero database migration, and zero user impact. Deploy the proxy on a small VM or as a Cloud Function.
|
|
|
|
### 6.2 Incremental Backend Migration (3-6 months, phased)
|
|
|
|
**Phase 1 -- API Gateway (4 weeks)**:
|
|
- Build a lightweight backend (Go or Python) that proxies LLM calls and handles scraping.
|
|
- Frontend calls the new backend instead of Gemini directly.
|
|
- Firebase Auth + Firestore remain.
|
|
- Deliverable: Secure API key handling, reliable scraping.
|
|
|
|
**Phase 2 -- Multi-Provider LLM (4 weeks)**:
|
|
- Backend supports Gemini, OpenAI, Anthropic.
|
|
- Settings page adds provider selection.
|
|
- Provider-specific adapters behind a common interface.
|
|
- Deliverable: LLM provider choice.
|
|
|
|
**Phase 3 -- Database Migration (4 weeks)**:
|
|
- Replace Firestore with Postgres (via Supabase for managed, or raw Postgres for self-hosted).
|
|
- Write migration scripts.
|
|
- Update frontend Firestore calls to REST API calls.
|
|
- Deliverable: No more Firebase dependency for data.
|
|
|
|
**Phase 4 -- Auth Migration (4 weeks)**:
|
|
- Replace Firebase Auth with self-hosted auth (Authentik/Keycloak) or add email auth alongside Google OAuth.
|
|
- Deliverable: Self-hosted auth.
|
|
|
|
**Phase 5 -- Frontend Modernization (optional, 4 weeks)**:
|
|
- If React is genuinely insufficient, consider Svelte or keep React.
|
|
- Deliverable: Modernized frontend (if needed).
|
|
|
|
Each phase ships independently. Each phase delivers value. Abandonment at any phase leaves the system in a better state than it started.
|
|
|
|
### 6.3 For Each Major Decision: Alternatives and Trade-offs
|
|
|
|
| Decision | Proposed | Alternative 1 | Alternative 2 |
|
|
|---|---|---|---|
|
|
| Backend language | Rust | Go: faster dev, easy cross-compile, good web ecosystem | Python/FastAPI: fastest dev, best LLM SDK ecosystem, async |
|
|
| Frontend framework | SolidJS | Keep React: zero rewrite cost, huge ecosystem | Svelte: similar reactivity to SolidJS, larger community |
|
|
| Database | SQLite + "Postgres path" | Postgres from start: no migration debt | Supabase: managed Postgres + real-time + auth |
|
|
| Auth | Custom email+magic link | Authentik/Keycloak: self-hosted, full-featured, zero custom code | Keep Firebase Auth: works, managed, free tier |
|
|
| Email delivery | Custom implementation | Postmark/SendGrid: managed, deliverable, $0-20/month | Keep Gmail API: already works, zero infrastructure |
|
|
| Deployment | Docker on VM | Fly.io/Railway: managed Docker, free tier, simpler ops | Keep Firebase Hosting + add Cloud Functions |
|
|
|
|
---
|
|
|
|
## 7. Hard Questions for the Project Owner
|
|
|
|
These are questions where the answer significantly changes the implementation. They must be answered before writing a single line of code.
|
|
|
|
1. **How many users will this serve?** If < 5 (personal/team tool), SQLite is fine, self-hosted auth is overkill, and operational simplicity should dominate every decision. If > 50, you need Postgres, proper auth, and monitoring from day one.
|
|
|
|
2. **Is the Rust choice driven by the project's needs or by a learning goal?** Both answers are valid, but they lead to different architectures. A learning project should embrace complexity. A product should minimize it.
|
|
|
|
3. **What is the budget for external services?** Transactional email ($0-20/month), LLM APIs ($10-100/month), domain + TLS ($10/year), VM ($5-50/month). If budget is zero, Firebase's free tier is hard to beat.
|
|
|
|
4. **What happens to existing Firestore data?** Is there a migration plan? How many syntheses exist? Are the legacy format syntheses (with `majorAnnouncements`, `financialSector`, etc.) being migrated or abandoned?
|
|
|
|
5. **Is Google SSO being kept alongside the new auth methods, or replaced entirely?** If replaced, existing users lose access to their accounts. If kept, you need OAuth2 client implementation in the new backend.
|
|
|
|
6. **Which LLM providers are actually needed on day one?** If the answer is "just Gemini for now, but we want the option for others," build the abstraction later. YAGNI applies. Adding a second provider later is cheaper than building a three-provider abstraction upfront that may never be used.
|
|
|
|
7. **Who will operate this in production?** Self-hosted means self-operated. Database backups, TLS certificate renewal, security patches, log monitoring, uptime monitoring. Is someone committed to this ongoing work?
|
|
|
|
8. **Is "without Docker" a hard requirement?** Supporting both Docker and non-Docker deployment doubles the deployment documentation and testing surface. If Docker is available on the target VM, mandate it and simplify.
|
|
|
|
9. **What is the desired SLA?** If the app can be down for hours without consequence (personal tool), the architecture can be simpler. If uptime matters, you need health checks, restart policies, and monitoring.
|
|
|
|
10. **How will the admin module's API keys be secured at rest?** SQLite stores data in a plain file. API keys in SQLite are readable by anyone with file access. Do you need encryption at rest? If so, how? (SQLCipher? Application-level encryption? OS-level disk encryption?)
|
|
|
|
11. **What is the testing strategy?** The current app has zero tests. The proposed rewrite has zero mention of testing. A Rust backend should have integration tests for every endpoint, unit tests for the LLM abstraction, and end-to-end tests for the auth flow. Is this budgeted?
|
|
|
|
12. **Is the 2-pass generation pipeline (search + scrape + rewrite) non-negotiable?** Some LLM providers may not support grounded web search at all. If the pipeline is mandatory, this constrains which providers are viable. If Pass 1 can be simplified (e.g., just use a search API + LLM summary without scraping), the multi-provider problem becomes much simpler.
|
|
|
|
13. **What is the target response time for synthesis generation?** Currently, generation takes 30-60+ seconds (two Gemini calls + parallel scraping). Will users accept the same latency? Should generation be asynchronous (start generation, get notified when done)?
|
|
|
|
14. **Is the Gmail email sending feature being kept?** The current implementation uses a Gmail OAuth popup to send via the user's own Gmail account. The new system would presumably send from a system email address. These are fundamentally different features with different privacy implications.
|
|
|
|
15. **What is the concrete timeline and deadline?** Without a deadline, scope expands indefinitely. With a deadline, scope must be ruthlessly cut. Which features are must-have for v1, and which are nice-to-have for v2?
|
|
|
|
---
|
|
|
|
## 8. Summary Verdict
|
|
|
|
The proposed refactoring is ambitious to the point of being reckless. It simultaneously changes every layer of the stack, introduces multiple new features, and removes all existing infrastructure -- with no phased delivery plan and no fallback position.
|
|
|
|
The current application has real problems (API key exposure, CORS proxy fragility, vendor lock-in), but none of them require a full rewrite. A 200-line proxy server solves the two most critical issues. An incremental migration plan addresses vendor lock-in without risking the entire project.
|
|
|
|
**My recommendation**: Before writing any code, answer the 15 questions above. Then build the smallest possible backend that solves the API key and scraping problems. Ship that. Use it for a month. Then decide if the rest of the rewrite is still justified, with the benefit of real experience operating a backend for this application.
|
|
|
|
The best refactoring is the one that delivers value at every intermediate step, not the one that promises everything at the end.
|