The Subtraction Dividend: Helix Now Runs Inference on AWS Trainium

Jun 30, 2026

Serve models on AWS Trainium and Inferentia with Helix: cheaper per token, real capacity when GPUs are scarce, and a drop-in OpenAI-compatible endpoint.

Helix now runs inference on AWS Trainium.

If you have Trainium or Inferentia capacity in your AWS account, you can point Helix at it today and serve models on it the same way you serve them on NVIDIA or AMD. Same profiles, same OpenAI-compatible endpoint, same routing. The silicon underneath is the only thing that changed.

This matters more than another logo on the compatibility list. Trainium is where AWS is putting its weight, it's cost-effective, and it's available when NVIDIA GPUs aren't. If GPU scarcity is squeezing you, it's a real way out, but the tooling has kept most teams away. Helix removes that friction. Here's the case for it.

Why You'd Actually Want Trainium

Start with the cheap end. AWS puts Inferentia2 (inf2) at up to 40% better price-performance than comparable EC2 GPU instances, with an inf2.xlarge starting around $0.99/hour, and that per-token gap compounds fast under a coding agent. It's also silicon you can actually get: NVIDIA capacity is a fight, with H100s quota-gated, region-locked, and gone the moment they free up, while the Neuron accelerators sit in a separate pool that's often available when your GPU request is still queued.

But inf2 is just the on-ramp. The real momentum is Trainium. AWS puts Trainium2 (trn2) at 30-40% better price-performance than H100 and H200 instances, Anthropic trains and serves Claude on more than a million Trainium2 chips, and Trainium3 is ramping through 2026 with up to another 4x on top. This is where a serious slice of the industry's inference is now landing. And because Helix treats every Neuron chip as one neuron vendor, Trainium came free with Inferentia2: the same profiles and routing serve a model on inf2 or trn2 with nothing new to write. Prove it out on a cheap inf2 instance, then scale onto Trainium when it grows.

None of it asks anything of your application. What comes out the front is a standard OpenAI-compatible API, and your agents talk to it exactly as they talk to any other Helix endpoint.

The Dividend

Here's the part we're proud of. Adding Neuron support took a few days of engineering and 163 lines of code.

We added a new silicon vendor, taught Helix to pass /dev/neuron* devices through to containers the same way it already does for NVIDIA and AMD, wired up profile matching so Neuron workloads only land on Neuron hosts, and shipped curated model profiles that are pre-configured and ready to run. That's it. A new hardware platform, in days, in a diff you could read over coffee.

It was 163 lines because two months earlier we deleted 43,000.

The Thing We Killed

For over a year, Helix had a scheduler. Not a simple one. A proper one, the kind with bin-packing, tensor-parallel placement, slot lifecycle management, prewarming, eviction policies, a global allocator, a priority queue, and a dozen heuristics for deciding which GPU should serve which model. It had 486 commits. It was rewritten at least twice. It had its own test suite, its own API surface, its own frontend dashboard with real-time slot visualisation and memory estimation widgets.

The problem it solved no longer existed.

The scheduler assumed a world where Helix needed to own the entire inference stack, deciding in real time which runner process on which GPU should receive which request, managing long-lived runner lifecycles, health checks, slot allocation, and graceful eviction. That world ended when many of our customers showed up with on-premise inference providers already set up. They weren't asking us to schedule GPUs. They were asking us to plug into what they already had. Helix became a thin routing layer. Point it at an existing inference endpoint, and requests flow. No central scheduler needed.

But the scheduler code stuck around. For months. Because someone might need it. Because it works.

In April 2026, we finally pulled the trigger. One commit removed 31,608 lines across 87 files, the entire scheduler package, the standalone runner binary, every runtime handler (vLLM, Ollama, Axolotl, Diffusers), the slot CRUD, the memory estimation engine, and many other components. Roughly 43,000 lines gone. The tests passed, and nothing asked for the scheduler back.

Why the Deletion Is the Feature

Adding Neuron to the old architecture would have been a slog. Neuron works nothing like an NVIDIA GPU. No CUDA, no container toolkit auto-discovering devices, just raw /dev/neuron* nodes. Models pre-compile to a chip-specific artifact instead of loading at runtime. NeuronCores sit in groups with their own local memory and explicit tensor-parallel sharding instead of one flat VRAM pool. And the silicon has no display path at all.

Under the old design, every one of those differences was a fight with the scheduler. Extend the placement logic for a new silicon type. Teach the slot allocator about NeuronCore topologies. Add a Neuron arm to the memory estimation engine. Write new runtime handlers. Update the slot health checks. Thread vendor=neuron through every API path that assumed NVIDIA-or-nothing. We'd have been modifying the exact system we just deleted, and everything that touched it.

The new architecture doesn't care. Declare a profile, apply it, route to it. It asks one question: is there a container running the model you asked for? If yes, route. If no, surface a 503. It has no opinion about whether the chip underneath is NVIDIA, AMD, or Neuron, because we deleted all the code that used to have opinions.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. — Antoine de Saint-Exupéry

Code is a liability, not an asset. The asset is the behaviour your users rely on; the code is the cost you pay to get it. A dead system you keep "just in case" is pure liability with the asset stripped out. It costs you every time someone reads it, every time a new feature has to respect invariants that nothing real depends on anymore, every time it shows up in a security alert. Most expensively, it costs you the features you can't build because they don't fit.

We didn't delete the scheduler to add Trainium. We didn't know any of this was coming. We deleted it because it was dead, and when a customer asked for inf2 eight days later, the support just fell out. Trainium, AWS's flagship AI accelerator, came with it, because we'd already simplified the system past the point where new silicon had to fight to fit.

That's the subtraction dividend. You rarely know in advance what it'll unlock. You always know it's there, on the other side of the delete key. This time it was a whole new class of silicon.

Helix runs AI coding agents on any accelerator, NVIDIA, AMD, or AWS Trainium and Inferentia. If you want to serve models on trn2 or inf2, get in touch.