It's midnight. DeepSeek V4.1 Flash just dropped on Hugging Face with an MIT licence and a benchmark score that beats the closed frontier. And that little voice — the one you already know — says: this is the one. We're running it locally, at home. No API bills. No cloud tax.
That voice has been wrong before. More than once.
This time it was almost wrong in a new and expensive way, and the arithmetic is worth walking through, because DeepSeek V4.1 Flash is genuinely remarkable and genuinely unfit for the machine you were about to put it on. Both things are true at once, and nobody leads with the second one.
The part that is real
DeepSeek V4.1 Flash is not hype. On DeepSWE v1.1 — an agentic coding benchmark that measures multi-step software work rather than toy puzzles — it scores 74.2. Claude Opus 5 scores 74.0. GPT 5.6 Soul sits at 73.0.
Read that again. An open-weights model that anyone can download sits at the top of one of the most contested benchmarks in the field, ahead of the closed flagships, under a real MIT licence — not "research only", not "free unless you make money, in which case call our lawyers". You can build products on it and sell them.
Then DeepSeek did something that made the point better than any benchmark: they retired V4 Pro, their 1.6-trillion-parameter flagship. Since 14 September, every API call to it routes to V4.1 Flash instead. The lab itself is telling you the small one won.
That is a genuinely good story. It is also the story that got you — and me — excited about running it locally. Here is where the excitement curdles.
Twenty-three seconds per token
Within a day of the weights dropping, people had it running everywhere: a Mac Mini with 16GB of system memory, multi-node clusters, gaming rigs, and at least one home server held together with zip ties and prayer.
Every one of those posts says the same word: it works.
What they don't lead with is the speed. One developer ran it on a 16GB Mac Mini, original weights, no compression, and got it to generate text correctly at 23 seconds per token.
That is not 23 tokens per second. That is 23 seconds for one token. The first word appeared 108 seconds after the prompt — a minute and forty-eight seconds of staring at a cursor.
A 150-word paragraph would take over an hour. A short email draft outlasts your lunch break. Debugging a 200-line script is something you start, leave, go shopping for, cook dinner, and come back to.
The setup isn't broken. It is doing precisely what the physics of the model demand.
Why "16B parameters" is the most misleading number in AI hardware
DeepSeek reports 8B active parameters while the model reads your prompt and 16B while it writes the answer. Sixteen billion sounds like a size that fits comfortably on a gaming GPU with headroom to spare. That is the number people size their machines against.
It is accurate and it is useless.
V4.1 Flash is a mixture-of-experts model. Every layer holds 384 routed expert networks plus one shared expert, and a router picks six of them for each token. So only ~16B parameters are active at any instant — but the other experts still have to be in memory, because the next token picks a different six, and the choice is dynamic. You cannot know in advance which ones you'll need, so they all stay resident.
Selective computation is not selective storage. Tape that to the inside of your skull before your next MoE download.
Then there is the genuinely new piece: Engram, a conditional memory table in two of the model's 40 layers. 196 billion parameters of lookup — sequences of up to four tokens matched against a 16-million-entry vocabulary. It isn't a compute layer that processes anything. It's an index-card drawer the model pulls answers from.
The download is 48 files and roughly 510 GB. If you sized the machine from the active parameter count, you were undersized by about 494 GB. Your OS will report it as ~475 GiB, which has already confused a lot of people who sized drives in the wrong unit and then couldn't work out why the download wouldn't fit.
That Mac Mini wasn't malfunctioning. It was streaming weights off an SSD, seeking and reading about 9.5 GB of scattered data to produce a single token. That is the tax on being 494 GB away from the number on the model card.
"Just quantize it" — the door that closed
When V4 Flash landed in July, everyone said quantize it, and they were right. It was 284B parameters, the community repacked the 4-bit experts into GGUF, and within a week there was a 128 GB model running across four RTX 3060s — cards you can still pick up used for a couple of hundred each. That was genuinely accessible local AI.
That path is gone, and the reason is subtle: DeepSeek already quantized it for you. The expert weights in V4.1 Flash ship at FP4 — four bits per parameter with a power-of-two scale factor per block of 32 values. There is no 8-bit release sitting underneath to compress down from. It was never published.
Push below four bits and you are in territory where the people who build compressed releases professionally say plainly: not suitable for agentic work. The model stops being the thing the benchmark measured.
One lever remains. The Engram table ships at 8 bits, and converting it to 4 bits saves roughly 100 GB — taking the total from about 475 GB down to about 385 GB.
Real progress. Also nowhere near 128 GB, and still a machine-sizing problem that almost nobody's current setup was built for.
What a working setup actually costs
Someone published a configuration that works, which is the most useful thing in the entire discourse because it lets you do the arithmetic before spending money.
Four Nvidia DGX Spark units. Each is a GB10 Grace Blackwell with 128 GB of unified memory, at $4,699 apiece — up from a $3,999 launch price, because apparently the machines needed to run the model also got more expensive while the model was dropping. Four of them: $18,796 before tax, before the network fabric connecting them, before storage.
They run vLLM at tensor-parallel four, with the Engram table offloaded to SSD and split across the boxes. Measured decode speeds, the first real published numbers for this model:
- ~70 tokens/sec for code and HTML
- ~45 tokens/sec for JSON
- ~30 tokens/sec for prose and planning
DeepSeek's own API runs the same model at over 300 tokens per second.
And here is the detail that stings anyone who bought in July: V4 Flash fit on two Sparks. Six weeks later its replacement needs four. The hardware didn't change. The floor moved. A perfectly capable two-Spark setup was one generation behind on the day it was superseded — and the replacement is both better on the benchmark and heavier to host, a trade nobody announced.
The math, written out
Off-peak API pricing is $0.15 per million input tokens and $0.60 per million output. Take a meaningful chunk of work — a million in, 100,000 out:
- Input: 15 cents
- Output: 6 cents
- Total: 21 cents per job
At 21 cents a job, the $18,796 cluster needs about 89,500 jobs before it pays for itself — and that's before electricity. At 70 tokens/sec, 100,000 output tokens takes 24 minutes, so 89,500 jobs is roughly 35,500 hours of pure generation. Run it 24/7 and you break even in just over four years.
Four boxes pulling close to a kilowatt for that long is about 35,000 kWh, which at $0.15/kWh is another $5,300.
So you don't break even in four years. You break even later — on a model that will be four or five generations old, producing tokens at a quarter of the speed the API offered on the day you bought the hardware.
The same job that finishes in about five minutes on the API takes about 24 minutes on the cluster. You paid $19,000 to wait an extra 19 minutes, every time.
Where all of that stops mattering
Every number above is irrelevant for a specific group of people, and it's worth being fair about it.
If your work cannot leave your building — medical records, defence contracts, data-residency regulation, a client agreement that forbids third-party processing — then the API isn't an option you're allowed to take. The comparison was never local versus hosted. It was local versus not doing the work at all. Against a failed audit or a compliance breach, $18,796 is not an expensive number. It's a reasonable one.
There's a subtler point too: open weights are part of why the API costs 21 cents in the first place. When anyone can download and serve a model, there's a ceiling on what any provider can charge. That ceiling benefits everyone, including people who never download the weights. The open release is what keeps the hosted price honest.
And if you already own hardware that can hold it — a machine or cluster sitting there with the memory — then the experiment costs you electricity and an afternoon. That is a completely different decision from ordering four new boxes.
Sizing the machine: the one rule that matters
DeepSeek built V4.1 Flash for the shape of problem a data centre has: thousands of simultaneous conversations, where the KV cache was eating the memory budget. They fixed that brilliantly — the cache is now 890 bytes per token, roughly a quarter of what the previous Flash needed.
Alone in a room, you inherit that trade backwards. You get the discount on the gigabyte you were never going to miss, and you pay full price for the 510 you can't avoid.
So the rule is simple, and it's the whole article in one line: size for what stays resident, not for what activates per token.
| Question | The wrong answer | The right answer |
|---|---|---|
| How much memory does it need? | 16B params ≈ 32GB | 510GB resident, ~385GB at best after 4-bit Engram |
| Will it fit on my GPU? | "It's only 16B active" | No single consumer card; it's a multi-node problem |
| Can I quantize it down? | "4-bit GGUF, like in July" | It already ships FP4; there's no 8-bit to compress from |
| Is it fast enough? | Same model as the API | 70 tok/s local vs 300+ tok/s API for code |
What to do instead
If you need frontier-quality agentic coding and your data can leave the building: use the API. Twenty-one cents for a heavy job is not a number worth beating with $19,000 of hardware and four years of waiting.
If you already own a machine with the memory: run it, measure your own tokens per second, and trust that number over anyone's — mine included.
If you want local AI that actually fits on hardware you can buy today: that's a different, much more enjoyable problem. A 32GB RTX 5090 runs quantized 30B-class models at speeds that feel instant. A 24GB RTX 4090 does the same for smaller builds. And the unified-memory route — a GMKtec EVO-X2 with 128GB — is exactly the class of machine that the July V4 Flash compressed into: two of them give you 256GB, which is a real ceiling for large quantized models, and about a quarter of the price of the DGX cluster.
What none of them do is hold 510GB. That's the honest boundary, and it's the boundary the marketing around active parameter counts keeps hiding.
Before you download anything
- Check the file size, not the parameter count. 48 files, ~510 GB, ~475 GiB on disk.
- Size memory for resident weights. Every expert stays in RAM even though six run per token.
- Check whether a quantized release exists before assuming you can shrink it — V4.1 ships pre-quantized at FP4.
- Measure, don't assume. Published tokens/sec figures are for someone else's interconnect, storage and batch size.
- Run the break-even math with your own electricity price. At 21 cents a job, the number of jobs required is usually larger than the number you will ever run.
The short version
DeepSeek V4.1 Flash is real: 74.2 on DeepSWE, MIT licence, and it retired DeepSeek's own flagship. It is also 510GB, 23 seconds per token on a Mac Mini, and $18,796 plus electricity — over four years — for a four-machine cluster that produces tokens at a quarter of the speed of the API serving the same model.
None of that makes it a bad model. It makes it a model built for the shape of a data centre, which is exactly what it says on the tin.
Go in with your eyes open, and if the pain arrives, at least it arrives by choice.
Sources: the figures here follow the analysis in What It Actually Costs to Run DeepSeek V4.1 Flash Locally? by Kai — benchmark scores, file sizes, the four-DGX-Spark configuration and its measured decode speeds, and the off-peak API pricing used for the break-even arithmetic. Confirm model specifications against the DeepSeek model card before buying hardware; weights and quantisation can change between releases.