A Better lm_head for Qwen3.8-27B: How We Tested and Shipped It

Aug 25, 2026

A new BF16-head checkpoint promised better accuracy. We matched it against production, measured 7.5% faster decode with no regression in our agent corpus, then rolled it across eight GPUs one at a time.

We had just finished tuning Qwen3.8-27B when another promising change appeared: a new NVFP4 checkpoint that keeps the model's final lm_head in BF16.

The lm_head turns the model's hidden state into token probabilities. Quantizing it saves memory, but it is also the last numerical step before choosing the next token. SGLang highlighted the new RadixArk checkpoint, and the MiaAI-Lab RTX PRO 6000 recipe switched to it. SGLang's newest release, v0.5.18, also adds official Qwen3.8 cookbook recipes.

The upstream claim was better accuracy. We wanted to know something narrower and more useful: is it at least as good on our agent workload, and is it safe to put into production?

A matched one-GPU test

We fenced one of our eight SGLang engines out of the load balancer, leaving the other seven serving production. On that isolated GPU we ran the old Inferact checkpoint and the new RadixArk checkpoint with the same SGLang commit, DFlash2 draft model, prompts, sampling settings and cache salts. Only the target weights changed.

So: no observed regression in our corpus, 7.5% faster decode, one extra running slot, and much more cache headroom. The new checkpoint is about 2.6 GB smaller than our previous production target, even with the BF16 head.

That is not proof of a general accuracy improvement. The checkpoint's own GSM8K comparison is close enough to be sampling noise, and our corpus is deliberately small and operational. What we proved is the thing we needed before rollout: the model still handled our objective prompts and tool protocol, while the server's request and token counters agreed exactly with the client.

Rolling eight GPUs without an outage

We downloaded an immutable Hugging Face revision into a revision-named directory, pinned the exact model revision and SGLang source commit in the deployment, and kept the previous checkpoint ready for rollback.

Then we rolled one engine at a time. Seven replicas stayed healthy while each replacement spent 688–823 seconds loading weights and compiling CUDA graphs. After every engine became ready, we checked its revision label, mount, restart count and a real generation before moving to the next GPU.

One rollout step did exactly what safety machinery is for: a conservative log check mistook a non-fatal Triton autotuning traceback for a failed start. The controller restored the previous checkpoint, the other seven engines kept serving, and we corrected the check before retrying. No production outage, no ambiguous half-upgraded state.

The final fleet came back 8/8 healthy, all on the same immutable revision, with zero container restarts. A guarded request through the production load balancer completed successfully.

The useful lesson

A model checkpoint upgrade is a software deployment, not a file copy. Pin the bytes. Compare it against the workload you actually serve. Reconcile engine-side metrics instead of trusting client timing alone. Roll one replica, prove rollback, then continue.

The complete Compose deployment, test harness and experiment journal are public in helixml/ramjet.

Measured on an 8× RTX PRO 6000 Blackwell Server Edition server. Target: RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead, revision 009632fef96dd349150baa780c984e62e70e91fe, with the z-lab/Qwen3.8-27B-DFlash2 block-8 draft. Eight TP=1 SGLang engines sit behind the ramjet prefix-aware load balancer.