# Design: "Autre" Fill-Up to Reach 75% Synthesis Target

**Date**: 2026-03-24
**Scope**: When total article count is low, expand "Autre" category to bring synthesis to 75% of maximum capacity

---

## Context

After the two-phase pipeline (personalized sources + web search), article drops (history filtering, validation failures, scrape errors) can leave syntheses with few articles. The "Autre" category currently caps at `max_items_per_category` like other categories, meaning overflow articles are silently dropped even when the synthesis is under-filled.

## Approach

After both phases complete, check if the total article count is below 75% of the maximum. If so, expand "Autre" capacity and fill from overflow articles that were dropped during classification.

## Target Calculation

- Maximum articles = `categories.len() × max_items_per_category` (user categories only)
- Target = `(0.75 × maximum).ceil()` as `usize`
- Shortfall = `target.saturating_sub(current_total)` (saturating to avoid panic if total exceeds target)
- If shortfall > 0, add overflow articles to "Autre" up to the shortfall

**Why exclude "Autre" from the maximum:** The goal is to ensure user-defined categories are adequately filled. "Autre" is the overflow bucket — it should not inflate the target. If user categories are well-filled, no fill-up is needed even if "Autre" is empty.

Example: 4 categories × 4 items = 16 max. Target = 12. If user categories have 8 articles and "Autre" has 2 (total 10), shortfall = 2. "Autre" accepts 2 more overflow articles.

## Mechanism

### 1. Collect overflow during classification

Modify `parse_classification_response` to return a second value: `Vec<ScrapedNewsItem>` of overflow articles — articles that were dropped because both their target category AND "Autre" were full.

Current signature:
```rust
fn parse_classification_response(...) -> HashMap<String, Vec<ScrapedNewsItem>>
```

New signature:
```rust
fn parse_classification_response(...) -> (HashMap<String, Vec<ScrapedNewsItem>>, Vec<ScrapedNewsItem>)
```

### 2. Accumulate overflow across phases

In `run_generation_inner`, collect overflow from both Phase 1 and Phase 2 classification calls into a single `all_overflow: Vec<ScrapedNewsItem>`.

### 3. Post-classification fill-up

After both phases, before the rewrite pass:

```
const SYNTHESIS_MIN_FILL_RATIO: f64 = 0.75;

total = sum of all articles in all_scraped
max_articles = categories.len() * max_items_per_category
target = (SYNTHESIS_MIN_FILL_RATIO * max_articles as f64).ceil() as usize
shortfall = target.saturating_sub(total)

if shortfall > 0 and all_overflow is non-empty:
    // Filter overflow against max_articles_per_source (source diversity)
    for each overflow article:
        count domain occurrences in all_scraped
        skip if domain already at max_articles_per_source
    take up to shortfall valid overflow articles
    add them to all_scraped["category_autre"]
```

**Source diversity enforcement:** Overflow articles added back to "Autre" must respect the `max_articles_per_source` limit. Count existing domain occurrences across all categories in `all_scraped`, and only add an overflow article if its domain is still under the limit.

## Hardcoded 75%

The 75% target is hardcoded as a constant `SYNTHESIS_MIN_FILL_RATIO: f64 = 0.75` in `synthesis.rs`. No user setting needed — this is an internal quality threshold.

## Best-effort

If there aren't enough overflow articles to reach 75%, the synthesis proceeds with whatever it has. No error is raised.

## Files to Modify

- **Modify:** `backend/src/services/synthesis.rs`:
  - Add `SYNTHESIS_MIN_FILL_RATIO` constant
  - Modify `parse_classification_response` signature and body to collect and return overflow
  - Update 2 production call sites to destructure the tuple
  - Update 5 existing classification unit tests for new return type
  - Add fill-up logic in `run_generation_inner` between Phase 2 and rewrite pass
  - Add unit tests for overflow collection and fill-up calculation

## What Does NOT Change

- `max_items_per_category` — still the cap for user categories
- Classification prompt — unchanged
- Rewrite pass — sees the final article set including expanded "Autre"
- Frontend — no changes
- Database — no changes
- No new settings