What's Actually in the Sovereign Server

Aug 14, 2026

We ship a 4U rack server with 8× NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs and 768 GB of VRAM. It's the same silicon we benchmarked at 946 tok/s fresh and later pushed to 1.82–1.89K output tok/s peak decode. An honest evaluation of what's in the box and what it can really do.

Last week we published the engine benchmarks: DwarfStar → SGLang → vLLM+DSpark on 8× RTX PRO 6000 cards, and the 7.8× throughput jump that came from the software, not the silicon. A lot of people read that and asked the same question: "so what exactly is in the Sovereign Server — and can it run that?"

Short answer: the Sovereign Server is that hardware, in a rack. Same class of GPU, same 768 GB of VRAM, Helix preloaded, shipped to your data centre as a 4U appliance. This post is the honest evaluation: what's in the box, what it can run, and where the numbers from the fleet post carry over — and where they don't.

The box

The Sovereign Server is built on the Gigabyte G494-SB4, a 4U GPU-optimised rack server. The base configuration:

One naming note, because it confuses everyone: the GPU in the Sovereign Server is the NVIDIA RTX PRO 6000 Blackwell Server Edition — a data-centre GPU, not a workstation card. It's the same Blackwell silicon our fleet benchmarks describe (GB202, SM120, 24,064 CUDA cores, 96 GB of GDDR7 on a 512-bit bus), in the server form factor: air-cooled dual-slot FHFL or liquid-cooled single-slot FHXL, up to 600 W configurable, with MIG (up to four isolated instances per GPU) and 9th-gen NVENC for hardware video encoding. NVIDIA's official spec rates it at 1,597 GB/s (~1.6 TB/s) of memory bandwidth and 2 PFLOPS of FP8 tensor performance. The 768 GB total is the same 768 GB.

Same GPUs, same benchmarks?

Mostly yes — and this is the part worth being precise about. The benchmarks in the DeepSeek 4 post ran on exactly this hardware: an 8× RTX PRO 6000 Blackwell Server Edition rack server — the same platform the Sovereign Server ships. For GPU-bound inference — which is what serving a large MoE model is, ~99% of the time — the platform differences don't move the needle. The GPUs do the work; the CPU feeds them tokens.

What we measured on that class of hardware, with DeepSeek-V4-Flash-0731:

Since then our perf work has pushed the box further. These are the latest numbers on a typical Helix coding turn of 18.5K input tokens and 256 generated tokens, using the current K5 setup with two TP4 engines across 8× RTX PRO 6000:

The practical headline: with 95–99% cache reuse, expect approximately 1.1K–1.7K generated tokens/sec across the node, while an individual user sees approximately 165–205 generated tokens/sec when alone or 46–71 tok/s averaged across a fully loaded 24-user node. The currently measured absolute decode ceiling is ~1.82K–1.89K output tok/s.

What does that mean in people? Agentic work is bursty — an agent alternates between generating tokens and running tools, tests, and sandboxes. Assuming one agent per person and a realistic sustained demand, a single Sovereign Server comfortably runs 50+ developers in parallel — the current K5 setup delivers ~46–79 tok/s per user across a fully loaded 24-user node at 95%+ cache, so even a 50+ developer team has headroom — with hundreds of concurrent agent desktops on top (the GPU-accelerated desktops stream via NVENC on the same cards).

What it can actually run

768 GB of GDDR7 sounds abstract until you lay out real checkpoints:

• DeepSeek-V4-Flash — Q4-imatrix at ~164 GB fits on two cards; the FP8 checkpoint (~156 GB, DSpark draft head bundled) fits on two as well. Eight cards give you the full 2× TP=4 fleet topology from the benchmark post, with room to spare. And it's not just fast — it's frontier: 89.0% on ARC-AGI-1 and 61.4% on ARC-AGI-2 (verified on the ARC Prize leaderboard).

  

  Source: ARC Prize. Frontier reasoning at $0.02-0.04 per task — on your own hardware.

• Llama, Qwen, Kimi, Mistral — the current open-weight frontier models, and the next few generations, all fit comfortably. New models drop every few weeks; if it runs on this class of hardware, Helix can run it.

• Multiple models simultaneously — 768 GB is enough to serve several models at once, so different teams on the same box can use different models without stepping on each other.

• Agent desktops, not just tokens — every Helix agent runs in its own GPU-accelerated streaming desktop: browser, terminal, filesystem, GUI apps, rendered and streamed at 60 fps. The server edition's 9th-gen NVENC hardware video encoders handle the streaming, and with this VRAM a single Sovereign Server runs hundreds of those concurrently.

Cost comparison: server vs cloud tokens

If you've read the benchmarks above and are now doing the mental maths against your cloud bill — here's the interactive version. The Sovereign Server is a one-time $175,000 purchase with no token metering. To keep the comparison realistic, the chart below anchors token volume to a cloud-token budget per developer (default $800/month) instead of assuming the server runs at max throughput forever, and compares that recurring bill against the one-time server cost month over month.

That default sits at the lower end of what teams actually spend: "Monthly API costs per engineer ranged from $500 to $2,000 as adoption skyrocketed" — The AI token pricing crisis behind OpenAI and Anthropic's revenue race, Investing.com.

One thing the naive numbers miss: real agent workloads burn a huge amount of tokens, and most of it is repeated context served from the KV cache at a steep discount. At ~98% cache hits, $800/month corresponds to ~895M tokens per developer served from the cache-heavy mix. Anthropic Opus 5 charges more per token than OpenAI GPT-Sol (effective ~$0.89/1M vs ~$0.59/1M after cache discounts), so the same workload costs more on Anthropic — that's why the two lines separate in the chart. The server itself can sustain ~1.1–1.7K generated output tok/s at 95–99% cache reuse (peak decode ceiling ~1.82–1.89K tok/s), which translates to roughly ~1.4B tokens per developer per month at the defaults — so a realistic workload uses about two-thirds of its capacity, with headroom to scale into.

That's not hypothetical: a real heavy user on cloud processed 22.3B tokens across 514 sessions (~745M/active day) at a 98.5% cache-hit rate, spending ~$15.5K on Anthropic/OpenAI while cache savings reached ~$102K (6.6× the raw token cost). On the Sovereign Server those tokens are served locally with no metering. The chart includes a cache-hit-rate slider so you can model that:

Drag the sliders to model your own team size, cloud budget per developer, KV-cache hit rate, cloud prices, and horizon. At the defaults (50 developers, $800/developer/month, ~98% cache hits) the same token workload would cost $40K/month on Anthropic Opus 5 and ~$27K/month on OpenAI GPT-Sol — the $175K server breaks even in its fifth month (≈4.4 months) versus Anthropic and ~month 7 versus OpenAI, and every month after that is money the cloud would have charged you. The hardware keeps working for a decade, and you've used ~64% of its capacity — with headroom to scale.

The software that comes with it

It's not a bare GPU box. The full Helix stack ships pre-installed and configured: private inference with an OpenAI-compatible API, RAG over internal documents (text and vision), autonomous agents with MCP integrations, agent desktops, fleet orchestration, RBAC/SSO/SCIM, audit logging, and observability. Turn it on in your data centre and the team has a private agent fleet on first boot — no Kubernetes expertise required.

Multi-replica inference is handled by our load balancer: the DeepSeek serving stack uses a high-performance Go load balancer across two independent vLLM/DSpark inference replicas. It provides conversation-aware sticky routing for better prompt-cache performance, continuous health checks, and automatic failover when an instance becomes unavailable. SSE token streams are delivered immediately without buffering, while compatibility shims normalize common OpenAI client requests. Prometheus and Grafana track latency, time to first token, throughput, cache efficiency, errors, client disconnects, and per-replica health — all in the box, ready on first boot.

Where the fleet post doesn't carry over

To be fair about it, a few things differ between our benchmark box and a shipped Sovereign Server:

• Driver and recipe versions drift. The fleet post nailed numbers on driver 595.84 with a specific SparkInfer/B12X build. The engine ecosystem is moving fast — the recipe that unlocked DSpark today is not the one from last month. Absolute numbers will move; the relative story (engine choice dwarfs the card) will not.
• No NVLink. These cards talk over PCIe (P2P-capable pairs), not NVLink. This is exactly what the fleet benchmarks ran on — two TP=4 instances rather than one TP=8 — so the numbers already reflect it.

None of this changes the conclusion of the fleet post: on this hardware, the engine determines what you get out of it, and vLLM + DSpark is currently the way to run DeepSeek V4-class models.

Who this is for

The same buyers the fleet post implicitly speaks to, plus the ones for whom sovereignty is the whole point:

• Teams already spending serious money on cloud AI — at $800/developer/month the hardware pays for itself in about five months (faster at $3,000/month), and it keeps running for a decade after.
• Regulated industries — finance, healthcare, legal, defence, government. GDPR, NIS2, DORA, EU AI Act: running on your own hardware is compliance by architecture rather than by contract.
• Air-gapped environments — no outbound network required after setup. No telemetry, no licence heartbeat, no kill switch in another jurisdiction.

The bottom line

The Sovereign Server is the same eight GPUs we benchmarked at 946.6 tok/s fresh decode — and later pushed to 1.82–1.89K output tok/s peak decode with KV cache — in a rack, with Helix preloaded. If you've been reading the fleet post and wondering whether the product can actually do that work — it's the same silicon. The box is just the part you can buy and plug into your own data centre.

Want the full spec sheet? Sovereign Server docs → · Read the fleet benchmark post → · Talk to sales →