On Monday sudoingX posted that DeepSeek v4.1 Flash does 103 tokens per second on code, on two DGX Sparks, with the build and the benchmark script published alongside it.
I have two Sparks. I put local agents on them and I run GLM-5.3-Flash on them every day. So I did what anyone with two Sparks and no self-control would do.
I tried to reproduce it.
102.3 tokens per second. Same script, measured the same way, inside 1 percent of the published number.
Cool. Confirmed. Then I spent a day and a night benchmarking both models on the same boxes with one harness, and that number turned out to be the least useful thing I measured.
Count to 200 is not a workload
The big structured numbers in public Spark posts, anywhere from 114 to 141 tokens per second, mostly come from one prompt: count from 1 to 200.
Both serving stacks use a small draft model that guesses the next few tokens while the big model checks them. On a count to 200 the guesses are almost always right. So the number measures how fast the server confirms a pattern it already expects. Not how fast it writes.
On DeepSeek, counting runs at 136 tokens per second. A real 1,024-token essay runs at 54. Same box. Same harness. Honestly, that gap is the whole post.
On real work DeepSeek still won most rows: a real code task at 98.2 against GLM's 72.7, real JSON at 98.8 against 82.6, an essay at 54.4 against 47.5.
The row that went the other way
With four requests at once, GLM did 101.8 tokens per second in total and DeepSeek did 93.1. At two at once DeepSeek wins again, 75.4 to 70.2.
That one is not the model. DeepSeek's serving recipe decodes two requests at a time and queues the rest. GLM's decodes four. Under four steady requests, the median wait for a first token on DeepSeek was about 5.5 seconds.
I tuned GLM first. One change stuck.
Before blaming the model I made sure GLM's setup was current. The recipe I run went from 96 patches to 107. That bought 2 to 2.5 percent, and the output was byte-for-byte identical. Free speed stays.
Then I doubled GLM's parallel slots from four to eight. About 30 percent more total throughput, but only once more than four requests overlap.
Then I checked my own gateway. It admits four requests at a time. The extra slots could never fill, and they cut the prompt cache in half. Reverted.
The engine's number is not your number. Your number is whatever survives the path your requests actually take.
Reading is slower than everyone says. Writing isn't.
Prefill is the model reading the prompt before it writes anything. GLM read prompts 11 to 13 percent faster than DeepSeek at every size both of them ran: 1,381 against 1,202 tokens per second around 32k, and 1,349 against 1,194 around 128k.
And both of them read slower than the published numbers. GLM about 1,380 against a published 1,950. DeepSeek about 1,300 against a published 2,090. That is 30 to 40 percent short, while writing landed within 1 to 8 percent.
I ruled out what I could: warm-up, clocks, thermals, prompt content, recipe version, settings, and the 200 Gb/s link between the two Sparks. Then the strange part. While reading a prompt, each GPU drew about 53 W. While writing, about 70 W. Reading should be the heavy part. The GPUs are waiting on something.
My best remaining suspect is the OS and driver image, which dates from March. I have not proven it, so I am not going to pretend I have.
Close on code. Not close on instructions.
Coding is a wash. DeepSeek takes HumanEval+ base, 94.5 to 92.7. GLM takes MBPP+ base, 94.2 to 93.4, and edges both of the stricter plus scores.
Instruction following is not close. On IFEval DeepSeek scored 88.7 strict and 91.3 loose. GLM scored 81.9 and 87.1. If you need a model to do exactly what the prompt says, that is the row that matters.
What DeepSeek did overnight
While I slept I gave DeepSeek a few more jobs. GLM has not run these yet, so they are DeepSeek facts, not a comparison.
- Tool calling: 15 of 15 right, including two parallel-call cases and four where the right answer was no tool at all.
- An agent-style chat growing from 22k to 33.5k tokens: 18 seconds to the first token on turn one, about 1.6 seconds on every turn after. The same prompt sent cold took 27.4 seconds.
- 30 of the hardest MATH problems: 26 right with thinking off, 26 with thinking on. Thinking added two seconds per problem and bought nothing.
- GSM8K, 250 problems: 98.4 percent on the answer, 82.4 percent when the exact answer format also has to match.
- Power: 1.25 to 1.6 joules per token across both GPUs. That is GPU draw, not the wall.
So which one?
Context is the real split. GLM's recipe serves a 1M-token window, and it found all three hidden passphrases at 523k. DeepSeek's recipe serves 262k.
So I checked two weeks of my own traffic. The biggest prompt was 252k. A couple went past 200k. Almost everything else was far smaller. For what I run, 262k is fine.
Everything else leans DeepSeek. Faster on real work, better at following instructions, tied on code. GLM reads faster and goes further.
So DeepSeek is in production. I built and benchmarked it overnight, then pointed my gateway at it. Every agent that talked to GLM now talks to DeepSeek, and none of them had to change. GLM is still installed, one config file away.
The first draft of this post came from one of those agents, running on DeepSeek. Then came the edits. A faster model still does not know my tone, apparently.
A watchdog checks DeepSeek every minute. If it stops answering, my agents fall back to GLM on their own. So far it hasn't.