Released Qwen3-8B and Qwen2.5-7B fuser
Exploratory · run September 2026 · evidence files (zip, 157 KB)
Fusion scored below the receiver working alone, and below passing the sharer's text.
| Arm | Correct | Our parser | C2C's parser |
|---|---|---|---|
| R, receiver alone | 6,372 | 77.60% | 77.62% |
| S, sharer alone | 6,161 | 75.03% | 75.06% |
| P, background text | 6,441 | 78.44% | 78.46% |
| T, worked text, reasoning | 6,484 | 78.97% | 81.18% |
| F, C2C fusion | 6,214 | 75.68% | 71.72% |
| T0, worked text, short answer | 6,604 | 80.43% | 80.49% |
Out of 8,211 questions in every arm. Correct counts use our parser.
On the same questions, fusion was 1.9 points below the receiver alone (95% interval −2.6 to −1.3). Fusion alone got 299 questions right that the receiver missed, and missed 457 the receiver got right. Fusion was 4.8 points below T0 (95% interval −5.6 to −3.9). Intervals are paired bootstrap over questions.
What was claimed
Fu et al. (2025) let one model read another model's key-value cache through a small trained network, the fuser, instead of passing text. They report 6.4 to 14.2% higher average accuracy than either model alone, and 3.1 to 5.4% higher than text communication. Their main table uses a Qwen3-0.6B receiver.
The authors also released a fuser for a larger pair: Qwen3-8B receiving from Qwen2.5-7B-Instruct. The paper's text and tables give no accuracy for that pair. Its size-scaling figure covers Qwen3 receivers with Qwen2.5 sharers, using fusers trained on MMLU. We did not read values from that figure.
What we ran
The released 8B fuser, unchanged, on 8,211 multiple-choice questions: OpenBookQA (468), ARC-Challenge (831), MMLU-Redux (5,599) and C-Eval (1,313). Every question went through six arms. Decoding was greedy. One answer parser, fixed before the run, scored every arm.
| Arm | What the receiver answers from |
|---|---|
| R | The question alone. |
| S | The sharer answers alone. |
| P | One sentence of background written by the sharer, without the options and without solving. |
| T | The sharer's worked answer, with the receiver asked to reason. |
| F | The sharer's cache, through the released fuser. |
| T0 | The same sharer text as T, with the same short answer instruction as R and F. |
By benchmark
| Benchmark | Questions | R | F | T0 |
|---|---|---|---|---|
| OpenBookQA | 468 | 85.26% | 80.98% | 89.53% |
| ARC-Challenge | 831 | 91.70% | 90.49% | 92.54% |
| MMLU-Redux | 5,599 | 75.60% | 73.76% | 78.67% |
| C-Eval | 1,313 | 74.49% | 72.58% | 77.00% |
Fusion was below the receiver on all four benchmarks. On ARC-Challenge the gap, 1.2 points, is within noise.
Limits
- This is not the paper's main table. It is a larger pair the paper reports no accuracy for.
- The paper caps answers at 64 tokens. We allowed the full context, with a stop for exact repetition added during recovery. This difference could affect the result.
- The training data of this checkpoint is not documented, so we cannot rule out overlap with the test questions.
- The parser matters. Under C2C's own parser fusion scores lower still, 71.72%. The ordering of fusion against R and T0 is the same under both.
- The original run stopped partway. Recovery runs finished the remaining questions with the same models, prompts and parser, plus the repetition stop above. The evidence files do not mark which questions each run scored.
- 24 answers failed on length or repetition, 20 in T and 4 in T0, and count as wrong. Dropping those 20 questions leaves the gaps unchanged at one decimal (−1.93 and −4.80 points).
- One run. The intervals cover question sampling, not run-to-run variation.
- Before any outcome was seen, we removed questions used in earlier development, questions outside two to four options, and a 32-question qualification sample per benchmark. C-Eval uses its public validation split.
Checks
A separate pass checked all 49,266 question and arm records, including the 24 failed cells, and reparsed the 49,242 outputs that did not fail with both parsers. It matched every stored score and question identity. The evidence files hold per-question scores for every arm, the answer parser, model and checkpoint revisions, and a script that recomputes every number on this page. Raw model outputs and prompt text are not included, so the setup statements above (greedy decoding, the parser freeze, the exclusions) rest on our own records.