A 'Cold Fusion' 27B vs my production MoE: DFlash2, quality suites, and what actually won

The setup#

My home inference server is a single AMD RX 7900 XTX (24 GB) running llama.cpp compiled for ROCm/HIP. It serves a WhatsApp bot around the clock: a 35B-parameter Mixture-of-Experts model (~3.5B active per token, the "BigBang" finetune of a Qwen3.5 MoE base) with block-diffusion speculative decoding — a DFlash v1 drafter that predicts a whole block of future tokens in one parallel pass and gets verified by the target model. Lossless by construction: greedy output is identical to running the big model alone, you just get there roughly 30% faster on mixed workloads.

The dense alternative is a Qwen3.8-27B finetune, which also has a DFlash2 drafter available — the second generation of the same idea, with a candidate-path selector that lifted published acceptance lengths above the official MTP drafters.

The contender#

DavidAU released Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 — a "COLD FUSION" training (their GAIN method plus Unsloth's trainers) on the same Qwen3.8-27B base, shipped as NEO-MAX imatrix quants (the output tensor stays at 16-bit) with an embedded MTP head. The card claims 99% of BF16 quality at 4-bit, and an ARC-C of 0.655 against 0.591 for the stock model.

Claims are claims. The only way to know what a model does on your hardware, your runtime, and your workloads is to measure — so I pulled the exact Q4_K_M MTP file (18.5 GB), verified the GGUF metadata (same qwen35 architecture as my dense model, 256k context, MTP tensors present), and put it through the same harness I use for everything else.

Round 1: decode speed#

Same binary, same flags, same benchmark API, same context settings as every baseline on this box — the only fair way to compare. All numbers are tokens/second at 64k context, q4_0 KV cache:

| Test | Cold-Fusion | + MTP n3 | + DFlash2 n7 | Qwen3.8-U + DFlash2 | BigBang MoE (prod) | |---|---:|---:|---:|---:|---:| | decode 256 | 28.6 | 46.8 | 64.9 | 57.2 | 99.1 | | decode 512 | 28.5 | 48.6 | 68.3 | 51.1 | 125.6 | | mixed 2k-in/256-out | 27.8 | 39.2 | 37.5 | 39.0 | 164.6 | | prefill 16k | 873 | 819 | 743 | 749 | 1827 |

Three findings worth keeping:

1. 65–68 t/s is the fastest dense-27B decode I have ever measured on this GPU. The interesting part is why: speculative decoding amplifies cheap verification. The Cold-Fusion finetune barely hurt drafter agreement — DFlash2 acceptance stayed at 4.6–5.2 mean tokens against a matching-target baseline of 4.6–6.7. Finetune drift, which cost me ~10% throughput when pairing a drafter with a different finetune of the same base, is essentially absent here. And unlike the stock model, whose decode decays over long generations (57.2 → 51.1 t/s), this one holds and even improves (64.9 → 68.3).

2. Speculative decoding is provably lossless here, not just theoretically. I ran identical greedy prompts through plain, MTP, and DFlash2 configs: byte-identical outputs, same token counts. The drafter never changes what the model says — only how fast it says it.

3. Physics still wins. Even at 68 t/s, a 27B dense model is ~2x slower than the 3.5B-active MoE on the same 24 GB card. No drafter closes an 8x active-parameter gap.

Round 2: quality — and what you can even measure on GGUF#

Before scoring models, I had to score the suites. On a pure llama.cpp/GGUF stack there is one hard wall: the server does not return prompt-token logprobs, which every multiple-choice benchmark (MMLU, ARC, HellaSwag, GPQA — the classic leaderboard battery) needs for loglikelihood scoring. What does work, cleanly, through the OpenAI-compatible API:

• lm-evaluation-harness with generative tasks: IFEval, GSM8K-CoT, BBH, TruthfulQA-gen
• LiveBench (objective scoring, no judge model, contamination-resistant)
• EvalPlus (HumanEval+/MBPP+ with sandboxed execution)

I ran both models through the same battery — identical conditions: greedy, serial requests, direct-answer mode (reasoning off), same limits. Here's the result:

| Benchmark (n) | Metric | BigBang MoE | Cold-Fusion + DFlash2 | Δ | |---|---|---:|---:|---:| | GSM8K-CoT 8-shot (150) | exact match | 0.960 | 0.933 | −2.7 | | GSM8K-CoT | strict match | 0.940 | 0.847 | −9.3 | | IFEval (150) | prompt-level strict | 0.773 | 0.673 | −10.0 | | IFEval | instruction-level strict | 0.853 | 0.752 | −10.1 | | BBH boolean expressions (100) | exact match | 0.95 | 0.93 | −2 | | BBH date understanding (100) | exact match | 0.76 | 0.81 | +5 | | BBH web of lies (100) | exact match | 1.00 | 1.00 | 0 |

The production MoE wins five of seven, ties one, loses one. The GSM8K strict-mode gap is mostly format compliance (the new model often skips the #### answer marker — its flexible-to-strict drop is 8.7 points versus 2 for the MoE), but the IFEval gap is real instruction-following. And one vendor claim inverted under test: the "fewer reasoning tokens" efficiency didn't survive direct-answer mode, where the new model actually wrote 15–32% more than the MoE on identical prompts.

Verdict#

Production stays on the MoE. This round it won on both axes — roughly 2x the throughput and higher scores on every generative suite I can run locally. The Cold-Fusion file earned a spot on disk anyway: it is now the best payload for the rare occasions I swap the server into dense-27B mode, where its DFlash2 pairing is genuinely the state of the art I've measured on consumer AMD hardware.

Two operational lessons worth more than the benchmark numbers:

1. Never benchmark against a live service. My first attempt ran the eval against production; concurrent eval requests wedged a request slot, the health watchdog did exactly its job and restarted the server mid-eval, and I got to rediscover why every serious measurement happens in an isolated window with the watchdog parked.
2. Verify the plumbing before trusting a score. The same model scored 0.45 and 0.77 on IFEval in the same session — the difference was thinking-mode truncation, not intelligence. A low score from a good model usually means the harness is lying to you.

Everything here runs on open-source tooling end to end: llama.cpp for serving, z-lab's DFlash drafters, EleutherAI's harness for quality. That's the part I like most — no API bills, no vendor dashboards, just a GPU, a filesystem, and numbers you can reproduce.