Bench v2 Ablation Study

The v1 benchmark measured throughput only (runs/min). It did not capture retrieval quality, routing convergence speed, or memory compaction ratio — making it impossible to evaluate improvements to SmartRetrieval or Thompson Sampling.

Bench v2 adds four dimensions: (A) retrieval quality (MRR@5, Recall@10 on held-out queries), (B) routing convergence (runs until p(win) ≥0.7 for optimal agent), (C) compaction ratio (bytes/trajectory after archival), (D) token efficiency (output tokens / task complexity score). Ablation sweeps test each component in isolation.

Full regression detection on retrieval and routing changes. Suite adds 45s to CI on standard hardware. The ADR explicitly cautions against selling memory/retrieval gains as a single universal percentage — case studies show 0% measurable difference when project conventions are already standard.