You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
343 lines
10 KiB
Markdown
343 lines
10 KiB
Markdown
# Source Diversity via Recent History — Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Inject recently-used domains into the LLM search prompt to encourage source diversity across syntheses.
|
|
|
|
**Architecture:** New `source_diversity_window` setting (default 3, 0=disabled). At generation time, load recent syntheses, extract domains from JSONB sections, pass to prompt builder which appends a soft avoidance instruction.
|
|
|
|
**Tech Stack:** Rust (sqlx, serde_json, url crate), SolidJS, PostgreSQL
|
|
|
|
**Spec:** `docs/superpowers/specs/2026-03-23-source-diversity-history-design.md`
|
|
|
|
---
|
|
|
|
### Task 1: Migration + backend model
|
|
|
|
**Files:**
|
|
- Create: `backend/migrations/20260323000013_add_source_diversity_window.sql`
|
|
- Modify: `backend/src/models/settings.rs`
|
|
- Modify: `backend/src/db/settings.rs`
|
|
- Modify: `CLAUDE.md`
|
|
|
|
- [ ] **Step 1: Create migration**
|
|
|
|
```sql
|
|
ALTER TABLE settings ADD COLUMN source_diversity_window INTEGER NOT NULL DEFAULT 3;
|
|
```
|
|
|
|
- [ ] **Step 2: Add field to all structs in `models/settings.rs`**
|
|
|
|
Add `pub source_diversity_window: i32` to `UserSettings`, `SettingsResponse`, `UpdateSettingsRequest` (after `max_articles_per_source`).
|
|
|
|
Add to `From<UserSettings> for SettingsResponse`:
|
|
```rust
|
|
source_diversity_window: s.source_diversity_window,
|
|
```
|
|
|
|
Add validation in `UpdateSettingsRequest::validate()`:
|
|
```rust
|
|
if !(0..=10).contains(&self.source_diversity_window) {
|
|
return Err("source_diversity_window must be between 0 and 10".into());
|
|
}
|
|
```
|
|
|
|
Add to `impl Default for UserSettings`:
|
|
```rust
|
|
source_diversity_window: 3,
|
|
```
|
|
|
|
- [ ] **Step 3: Add column to DB queries in `db/settings.rs`**
|
|
|
|
Add `source_diversity_window: i32` to `SettingsRow`. Add to `TryFrom<SettingsRow>`:
|
|
```rust
|
|
source_diversity_window: row.source_diversity_window,
|
|
```
|
|
|
|
Add to both SQL queries (`get_or_create_default` and `upsert`): INSERT column list, VALUES placeholder, RETURNING clause, `.bind()` call, and ON CONFLICT SET (upsert only). The new column goes after `max_articles_per_source`.
|
|
|
|
- [ ] **Step 4: Update CLAUDE.md migration count to 13**
|
|
|
|
- [ ] **Step 5: Add validation tests in `models/settings.rs`**
|
|
|
|
Add `source_diversity_window: 3` to the `valid_request()` test helper. Then add tests:
|
|
|
|
```rust
|
|
#[test]
|
|
fn test_source_diversity_window_zero_is_valid() {
|
|
let mut req = valid_request();
|
|
req.source_diversity_window = 0;
|
|
assert!(req.validate().is_ok());
|
|
}
|
|
|
|
#[test]
|
|
fn test_source_diversity_window_ten_is_valid() {
|
|
let mut req = valid_request();
|
|
req.source_diversity_window = 10;
|
|
assert!(req.validate().is_ok());
|
|
}
|
|
|
|
#[test]
|
|
fn test_source_diversity_window_below_range() {
|
|
let mut req = valid_request();
|
|
req.source_diversity_window = -1;
|
|
assert!(req.validate().is_err());
|
|
}
|
|
|
|
#[test]
|
|
fn test_source_diversity_window_above_range() {
|
|
let mut req = valid_request();
|
|
req.source_diversity_window = 11;
|
|
assert!(req.validate().is_err());
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 6: Run tests**
|
|
|
|
Run: `cd backend && cargo test --lib`
|
|
Expected: all tests pass
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add backend/migrations/20260323000013_add_source_diversity_window.sql backend/src/models/settings.rs backend/src/db/settings.rs CLAUDE.md
|
|
git commit -m "feat: add source_diversity_window setting (migration + model + DB)"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Prompt modification + tests
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/services/prompts.rs`
|
|
|
|
- [ ] **Step 1: Add `recent_domains` parameter to `build_search_prompt`**
|
|
|
|
Change signature from:
|
|
```rust
|
|
pub fn build_search_prompt(
|
|
settings: &UserSettings,
|
|
sources: &[Source],
|
|
current_date: &str,
|
|
) -> (String, String) {
|
|
```
|
|
|
|
To:
|
|
```rust
|
|
pub fn build_search_prompt(
|
|
settings: &UserSettings,
|
|
sources: &[Source],
|
|
current_date: &str,
|
|
recent_domains: &[String],
|
|
) -> (String, String) {
|
|
```
|
|
|
|
- [ ] **Step 2: Append avoidance instruction when domains are non-empty**
|
|
|
|
At the end of the `user_prompt` format string (after the JSON instruction line, before the closing `"`), add a conditional block. After the `format!()` call that builds `user_prompt`, append:
|
|
|
|
```rust
|
|
let user_prompt = if recent_domains.is_empty() {
|
|
user_prompt
|
|
} else {
|
|
let domains_list = recent_domains.join(", ");
|
|
format!(
|
|
"{}\n\nEvite si possible les sources deja utilisees dans les syntheses precedentes : {}.",
|
|
user_prompt, domains_list
|
|
)
|
|
};
|
|
```
|
|
|
|
- [ ] **Step 3: Update test fixture**
|
|
|
|
In the `test_settings()` function (~line 137), add:
|
|
```rust
|
|
source_diversity_window: 3,
|
|
```
|
|
|
|
- [ ] **Step 4: Update existing test calls**
|
|
|
|
All existing tests that call `build_search_prompt` need the 4th argument. Add `&[]` (empty slice) to each existing call. Search for `build_search_prompt(` in the test module and add `, &[]` before the closing `)`.
|
|
|
|
- [ ] **Step 5: Add new tests**
|
|
|
|
```rust
|
|
#[test]
|
|
fn search_prompt_includes_recent_domains_avoidance() {
|
|
let settings = test_settings();
|
|
let sources = vec![];
|
|
let date = "lundi 17 mars 2026";
|
|
let domains = vec!["techcrunch.com".to_string(), "theverge.com".to_string()];
|
|
let (_, user_prompt) = build_search_prompt(&settings, &sources, date, &domains);
|
|
assert!(user_prompt.contains("Evite si possible"));
|
|
assert!(user_prompt.contains("techcrunch.com"));
|
|
assert!(user_prompt.contains("theverge.com"));
|
|
}
|
|
|
|
#[test]
|
|
fn search_prompt_no_avoidance_when_domains_empty() {
|
|
let settings = test_settings();
|
|
let sources = vec![];
|
|
let date = "lundi 17 mars 2026";
|
|
let (_, user_prompt) = build_search_prompt(&settings, &sources, date, &[]);
|
|
assert!(!user_prompt.contains("Evite si possible"));
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 6: Run tests**
|
|
|
|
Run: `cd backend && cargo test --lib`
|
|
Expected: all tests pass
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add backend/src/services/prompts.rs
|
|
git commit -m "feat: build_search_prompt accepts recent_domains for source diversity"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Pipeline integration — extract domains + wire prompt
|
|
|
|
**Files:**
|
|
- Modify: `backend/src/services/synthesis.rs`
|
|
|
|
- [ ] **Step 1: Add domain extraction from recent syntheses**
|
|
|
|
Before the `build_search_prompt` call (~line 303), add a new step that loads recent syntheses and extracts domains. Insert between the rate limit check (step 5) and the search pass (step 6):
|
|
|
|
```rust
|
|
// Step 5b: Load recently-used domains for source diversity
|
|
let recent_domains = if settings.source_diversity_window > 0 {
|
|
let recent = db::syntheses::list_for_user(
|
|
&state.pool,
|
|
user_id,
|
|
settings.source_diversity_window as i64,
|
|
0,
|
|
)
|
|
.await
|
|
.unwrap_or_default();
|
|
|
|
let mut domains: Vec<String> = recent
|
|
.iter()
|
|
.filter_map(|s| {
|
|
serde_json::from_value::<Vec<crate::models::synthesis::NewsSection>>(
|
|
s.sections.clone(),
|
|
)
|
|
.ok()
|
|
})
|
|
.flat_map(|sections| {
|
|
sections
|
|
.into_iter()
|
|
.flat_map(|sec| sec.items.into_iter())
|
|
.filter_map(|item| extract_domain(&item.url))
|
|
})
|
|
.collect();
|
|
|
|
domains.sort();
|
|
domains.dedup();
|
|
domains
|
|
} else {
|
|
Vec::new()
|
|
};
|
|
```
|
|
|
|
- [ ] **Step 2: Update the `build_search_prompt` call**
|
|
|
|
Change line ~304 from:
|
|
```rust
|
|
let (system_prompt, user_prompt) =
|
|
prompts::build_search_prompt(&settings, &sources, ¤t_date);
|
|
```
|
|
|
|
To:
|
|
```rust
|
|
let (system_prompt, user_prompt) =
|
|
prompts::build_search_prompt(&settings, &sources, ¤t_date, &recent_domains);
|
|
```
|
|
|
|
- [ ] **Step 3: Run tests**
|
|
|
|
Run: `cd backend && cargo test --lib`
|
|
Expected: all tests pass
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add backend/src/services/synthesis.rs
|
|
git commit -m "feat: extract recent domains and pass to search prompt for diversity"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Frontend setting
|
|
|
|
**Files:**
|
|
- Modify: `frontend/src/types.ts`
|
|
- Modify: `frontend/src/i18n/fr.ts`
|
|
- Modify: `frontend/src/pages/Settings.tsx`
|
|
|
|
- [ ] **Step 1: Add field to frontend types**
|
|
|
|
In `frontend/src/types.ts`, add to `UserSettings` interface (after `max_articles_per_source`):
|
|
```typescript
|
|
source_diversity_window: number;
|
|
```
|
|
|
|
Add to `DEFAULT_SETTINGS`:
|
|
```typescript
|
|
source_diversity_window: 3,
|
|
```
|
|
|
|
- [ ] **Step 2: Add i18n label**
|
|
|
|
In `frontend/src/i18n/fr.ts`, add after `settings.maxArticlesPerSource`:
|
|
```typescript
|
|
'settings.diversityWindow': 'Syntheses a examiner pour diversite',
|
|
```
|
|
|
|
- [ ] **Step 3: Add number input to Settings page**
|
|
|
|
In `frontend/src/pages/Settings.tsx`, inside the generation settings grid (after `maxArticlesPerSource`), add:
|
|
|
|
```tsx
|
|
<div>
|
|
<label
|
|
for="diversityWindow"
|
|
class="block text-sm font-medium text-gray-700"
|
|
>
|
|
{t('settings.diversityWindow')}
|
|
</label>
|
|
<div class="mt-1">
|
|
<input
|
|
type="number"
|
|
id="diversityWindow"
|
|
min="0"
|
|
max="10"
|
|
class="shadow-sm focus:ring-indigo-500 focus:border-indigo-500 block w-full sm:text-sm border-gray-300 rounded-md py-2 px-3 border"
|
|
value={settings().source_diversity_window}
|
|
onInput={(e) =>
|
|
setSettings((prev) => ({
|
|
...prev,
|
|
source_diversity_window:
|
|
parseInt(e.currentTarget.value) || 3,
|
|
}))
|
|
}
|
|
/>
|
|
</div>
|
|
</div>
|
|
```
|
|
|
|
- [ ] **Step 4: Run frontend tests**
|
|
|
|
Run: `cd frontend && npx tsc --noEmit && npx vitest run`
|
|
Expected: type check passes, all tests pass
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add frontend/src/types.ts frontend/src/i18n/fr.ts frontend/src/pages/Settings.tsx
|
|
git commit -m "feat: add source_diversity_window setting to frontend"
|
|
```
|