Skip to content
Andrew Voirol
WorkLogAboutContact
HomeWorkLog

AboutContact

✦ Just one prompt away from figuring it all out.

Thursday, April 9, 2026gemma-4-benchmarks

F16 is the ceiling. Q8 already passed it.

Gemma 4F16Quantization

**Ran the E2B at full BF16 precision — 10.**3 GB, 149 minutes, 1.72 tok/s. The result? 91.2%. The Q8 quantized version? 91.8% in 36 minutes at 6.95 tok/s. Read that again: the quantized model scored higher than full precision while running 4× faster. This isn't supposed to happen. Quantization is lossy compression — it should always lose something. But on this hardware, the Q8 model actually benefits from its smaller memory footprint: fewer cache misses, more consistent throughput, and the 'lost' precision apparently doesn't matter for these tasks. The ceiling is a floor. Stop chasing precision. Chase speed.

← Previous

The inverted ladder. Q4 > Q8 > F16.

Next →

The quantization showdown: Unsloth wins on efficiency, E4B is dead weight


Andrew Voirol

Builder, hacker, shipper. Currently leaving localhost.

Navigate

WorkBuilder’s LogAboutContactRSS Feed

Connect

X / TwitterGitHubLinkedIn

© 2026 Andrew Voirol✦Just one prompt away from figuring it all out.