DeepSeek V4.1 Flash: what 890 bytes per token actually buys

An assistant that reads a long document pays for two different jobs: processing the document and producing an answer. DeepSeek V4.1 Flash changes how those jobs share information, with particular attention to the memory retained for a long context. The useful question is not whether the cache is smaller. It is which memory became smaller, and what that means for an actual request.

This topic came from AI Search’s architecture video. The technical evidence below comes from the original model card and DeepSeek’s technical report, rather than the video’s headline.

First separate prefill from decoding

Prefill processes the tokens already in a request. Decoding generates new tokens, repeatedly using the previous context. An attention cache avoids rebuilding every earlier key and value at each decoding step. A longer document therefore creates a larger retained state, even when the answer is short.

Imagine a request with 100,000 input tokens and a 500-token answer. The input is 200 times longer than the output. That ratio identifies an input-heavy task; it does not establish a 200-fold compute or price ratio. Prefill can process positions in parallel, while decoding has a sequential dependency on earlier output. A system can improve one phase while leaving the other as the bottleneck.

The model card describes a causal encoder followed by a decoder, with 8 billion activated parameters per token during prefill and 16 billion during decoding. Those are active parameters, not the total weights that a deployment must store. Treating an 8-billion active path as an ordinary 8-billion-parameter download would give a misleading hardware estimate.

The memory arithmetic

The report gives a global KV cache footprint of 890 bytes per token. For that component, the simple estimate is:

M = N × b

Here M is memory in bytes, N is the number of cached tokens, and b is bytes per token. At one million tokens:

1,000,000 × 890 = 890,000,000 bytes = 0.89 GB ≈ 0.829 GiB.

GB uses one billion bytes; GiB uses 1,073,741,824 bytes. The distinction matters when comparing a calculated number with a memory monitor. At 100,000 tokens, the same estimate gives 89 MB. Ten independently stored million-token contexts would require 8.9 GB for this component if no cache sharing or eviction reduced the total.

The graph below evaluates that equation at several context lengths. It is a calculated memory curve using a publisher-reported coefficient. It is not a measured GPU-memory curve and contains no assumption about model weight placement, batch scheduling, temporary activations, or allocator overhead.

Worked calculation

Calculate retained context memory

Y Global KV cache (GB)X Context (thousand tokens)

Focus the graph and use arrow keys, or tap a point, to inspect values. The complete data table is available below.

View data table

Graph data table

Context (thousand tokens)	Global KV cache (GB)
100	0.089
250	0.2225
500	0.445
1000	0.89

Why reuse and compression address different costs

DeepSeek describes cross-layer cache reuse, sparse indexing, and lower-precision global KV storage. Reuse reduces duplicated state; sparse indexing reduces which context positions require attention work; lower precision reduces the representation size of retained values. These are related optimizations, but their savings cannot simply be multiplied as if they affected independent copies of the entire model.

The technical report also distinguishes global KV memory in accelerator memory from persistent KV storage in host memory or SSD. Its sliding-window bounded replay reconstructs recent local states when needed. A reduction in persistent storage is therefore a different claim from the 890-byte global coefficient. Neither number is the total memory required to load and serve the model.

Turn the claim into a useful experiment

For a document assistant, choose one stable question and several lengths of the same document collection. Put the relevant evidence at the beginning, middle, and end in separate trials. This tests whether increasing length changes retrieval reliability, rather than only whether a request fits.

Record time to first token, output tokens per second, peak device memory, retained cache memory, and answer correctness separately. Keep the serving implementation, precision, batching, prompt, and effort setting fixed. Compare cold requests with warmed-cache requests explicitly. A cached prefix can make a second run faster without demonstrating a general improvement on unseen documents.

Use a results table that retains failures. A fast answer with the wrong citation is not a successful document task. If a timeout prevents an answer, keep that observation instead of averaging only completed requests. This is a proposed test design; Qentrah has not run this model on a GPU.

What to read before deployment

Start with the release’s exact inference and model requirements. Read the report’s cache-management and evaluation sections together: the architecture describes a mechanism, while the benchmark protocol describes a particular implemented system. The deployment you choose may expose different cache accounting or support a different subset of features.

The defensible conclusion is narrow but useful: reducing retained context state can make input-heavy work less expensive to store and move. Whether it improves your application’s latency, throughput, and completed-task cost remains an implementation and workload question. Keep the memory estimate as a planning tool, then measure the full system.