DeepSeek V4.1 Flash: Why Its Encoder, Engram and KV Cache Matter

Sep 10, 2026

DeepSeek V4.1 Flash splits prompt reading from generation, adds 196B Engram memory and compresses key-value cache. Here is what that means for serving.

DeepSeek-V4.1-Flash is built for a workload that now dominates serious AI agents: a large prompt goes in, tools and files add more context over several turns, and the model generates a much smaller amount of new text.

The model changes the transformer around that workload. Its 40 layers are split into a 20-layer causal encoder and a 20-layer decoder. The encoder processes the prompt with 8 billion active parameters per token. The decoder generates the answer with 16 billion active parameters per token. A separate 196-billion-parameter Engram memory stores token-pattern embeddings, while a compressed global key-value (KV) cache retains long-range context at 890 bytes per token.

Those are architectural claims from DeepSeek's model card and technical report, not our measurements. We have inspected the release and started planning a deployment, but we have not run V4.1 on our hardware yet. The current checkpoint is too large for the two-engine layout we want.

Conceptual data flow based on the pinned V4.1 configuration and technical report. It omits tensor-level detail.

The causal encoder changes prompt processing

Most large language models in production use one decoder-only stack. The same layers process every prompt token and then generate each output token. Prefix caching can skip some repeated prompt work, but the model architecture still treats reading and generation as one path.

V4.1 divides the work. Its first 20 layers form a causal encoder, meaning each prompt position can attend only to earlier positions. This is not a bidirectional encoder like BERT. It preserves the left-to-right structure needed for generation while creating a dedicated representation of the prompt.

The next 20 layers form the decoder. Instead of building separate global KV state from every decoder layer's hidden states, V4.1 projects the decoder's global keys and values from the final encoder states. During prompt processing, the decoder only has to reconstruct the short sliding-window state it needs to begin generation.

DeepSeek reports 8B active backbone parameters per prompt token and 16B per generated token. Active parameter count is not a latency measurement, and it does not prove that generation costs exactly twice as much. It does show the intended asymmetry. Long prompts should use less model compute than they would if all 40 layers performed equivalent work for every input token.

That matters for agents. Repository maps, tool definitions, screenshots, terminal history and retrieved documents can make the input much larger than the answer. Causal Encoder-Decoder (CED) moves more of the model's capacity toward the answer-producing phase without asking the full decoder to reread the entire prompt in the usual way.

The serving runtime now has more responsibilities. It needs distinct vision-encoding, prompt-processing and generation paths, plus the state transfer between them. DeepSeek describes an Encoder-Prefill-Decode deployment that can scale those phases independently. A conventional OpenAI-compatible server cannot support V4.1 by renaming an older DeepSeek architecture.

Engram adds a large, sparse memory

The 552B backbone is only part of the checkpoint. V4.1 also contains 196B Engram parameters, divided evenly between two modules inserted at layers 1 and 14.

Engram starts with local token patterns. For each position, it forms two-, three- and four-token n-grams, compresses the token IDs, and hashes them across eight lookup heads. Those addresses select rows from large FP8 embedding tables. A learned gate controls how much retrieved memory enters the transformer.

This is parameter memory, not the KV cache and not retrieval-augmented generation (RAG). It does not search a document collection or call an external database. The lookup is deterministic for a token sequence, and its contents were learned during training.

The useful property is sparse access. V4.1 can hold a large bank of learned token-pattern embeddings without multiplying every token through all 196B parameters. DeepSeek says deterministic addresses also allow an inference system to prefetch the required rows from host memory over background RDMA transfers while GPU computation continues.

That last detail is important for self-hosting. Engram reduces arithmetic compared with putting the same capacity into dense layers, but the tables still have to live somewhere. The released checkpoint contains 48 weight shards totaling 475.25 GiB. Host prefetch also assumes enough memory and interconnect bandwidth to keep the sparse lookups fed.

CSA2 makes long context cheaper to retain

V4.1 uses two forms of attention at once. Sliding-window attention (SWA) covers the latest 128 tokens in every layer. Compressed Sparse Attention 2 (CSA2) supplies global context for the rest of the sequence.

CSA2 layers have three roles:

• Full layers create global KV state and search the full sparse index.
• Reindex layers reuse earlier global KV but run another index lookup.
• Reuse layers share both global KV and selected sparse positions from an earlier layer.

Sharing across layers removes duplicate cache state. A hierarchical index also narrows later searches to a candidate pool selected earlier in the network. V4.1 then stores the main global KV in FP4, with one scale for every 16 channels.

DeepSeek reports 890 bytes of global KV per token, about one quarter of DeepSeek-V4-Flash. At the declared 1,048,576-token context limit, that is about 0.87 GiB of logical global KV for one completely full sequence. The figure excludes sliding-window state, allocator overhead, activations and any runtime duplication.

The persistent cache is smaller again. DeepSeek keeps compressed global KV on host memory or SSD for at least 72 hours in its deployment. Short-lived SWA state goes into a distributed pool sized at 10% of host memory with a lifetime measured in minutes.

When that short-lived state has gone, SWA Bounded Replay rebuilds an approximate version by replaying only the latest 128 tokens. Exact reconstruction would require replaying the 128-token window through many layers. The bounded method turns a cache miss into a small amount of prompt work, although it also means a global-cache hit is never completely free.

The report says the approximation caused negligible degradation in its tests. It also names cache-resumption boundaries and sparse retrieval over extreme contexts as areas needing more study. A production qualification should compare cold prompts, complete cache hits and global-only hits on the same long-context and multi-turn tasks.

DSpark adapts speculative decoding to load

V4.1 replaces the older multi-token prediction module with DSpark. Three small transformer blocks produce logits for five future positions in one pass. A confidence head predicts how much of that draft is likely to survive verification.

The scheduler combines those probabilities with measured engine throughput curves and current load. It can verify fewer positions when a long draft would waste work, or use more when the expected throughput is better. This policy belongs in the model server because it depends on live batch shape and draft confidence. A load balancer can observe acceptance and effective tokens per step, but it should not try to reproduce the scheduler from outside the engine.

The model also accepts an integer reasoning-effort setting from 1 to 100. DeepSeek reports that higher settings produce longer answers and tend to improve reasoning results, with diminishing returns toward the top of the range. For an agent platform, that control should remain explicit: routine tool turns can request less compute, while difficult planning or recovery can request more.

Why the released checkpoint does not fit our TP4 plan

Our server has eight RTX PRO 6000 Blackwell Server Edition GPUs with about 95.59 GiB of usable memory each. We run production models as two independent copies, with each copy spread across four GPUs. That TP4 layout gives each model copy one non-uniform memory access (NUMA) local GPU group, its own scheduler and its own prompt cache. It also lets us test or restart one engine while its peer keeps serving.

The released weight files total 475.25 GiB. Four GPUs provide 382.37 GiB before the runtime allocates CUDA graphs, activations, communication buffers or KV cache. The checkpoint misses the physical TP4 ceiling by 92.88 GiB before serving starts.

A single eight-GPU copy fits in aggregate memory on paper, but it would occupy the whole box and remove the safe one-engine canary. It would also give us one scheduler and one cache domain instead of two. Our target remains two TP4 engines.

We are waiting for a broader quantized checkpoint or a different validated Engram placement. Quantizing Engram alone from FP8 to FP4 would save roughly 98 GB in decimal units. That still leaves the checkpoint near the full physical capacity of four GPUs, with no practical runtime headroom. Our initial planning band is 300 to 320 GiB of loaded weights per model copy, followed by a real load test that measures the remaining KV capacity.

The engine is the other blocker. It must implement CED, CSA2, FP4 global KV, FP8 SWA state, Engram prefetch, DSpark scheduling, vision input and the new prompt protocol on SM120 GPUs. The readable reference implementation is useful for correctness work, but it is not a continuously batched production server.

What this architecture changes for Ramjet

Ramjet currently routes related requests toward the engine that already holds their reusable prompt, while accounting for current load. V4.1 makes that decision more specific.

First, global KV and SWA state have different lifetimes. A future engine integration should expose long-lived global-cache residency separately from recent session state. Ramjet could keep durable prefix affinity for the global cache and a minute-scale session preference for SWA, both bounded by load.

Second, a warm prefix has a replay cost. Ramjet's capacity estimate should retain an engine-profiled floor for the 128-token reconstruction path instead of treating a complete global hit as zero prompt work.

Third, cache location matters. An HBM hit, a host-memory or SSD restoration, and a cold prompt have different costs. Tier-aware scoring only becomes safe when the engine publishes authenticated cache location and replay metrics. We will start that work in observation mode before it can affect placement.

V4.1 treats prompt processing, generation, learned memory and reusable runtime state as separate systems. That creates more work for inference engines, but it also gives a router better signals than a single count of busy requests. The model will be most interesting when those signals make it through the server API.

Our deployment investigation, upstream blockers and two-TP4 qualification plan are tracked in Ramjet issue #264.

Sources and current status: DeepSeek's pinned model repository, technical report, and configuration, inspected 10 September 2026. Helix's deployment has not run or benchmarked this model. Follow the public qualification work or read how we serve the current Qwen3.8-Flash-Next stack.