Skip to content
Andrew Voirol
WorkLogAboutContact
HomeWorkLog

AboutContact

✦ Just one prompt away from figuring it all out.

Sunday, June 28, 2026edge-ai-lab-development

GPU memory, latency histograms, and thermal state — the instrumentation nobody asked for

BenchmarksApple SiliconInfrastructure

Added GPU memory tracking via MTLDevice.currentAllocatedSize, per-token latency histograms, thermal state monitoring, memory bandwidth estimates, and model load timing — all in one push. Nobody asked for latency histograms. The benchmarks already showed decode speed and TTFT. But "63 tok/s" hides the distribution — are all tokens fast, or do you have a long tail of 200ms spikes from thermal throttling? Now you can see the P95. Thermal state tracking means the benchmark card tells you whether the machine was cool or cooking when the numbers were recorded. GPU memory via Metal's allocated size API shows exactly how much VRAM the model actually consumes, not what the spec sheet claims. Also built a Canvas side panel with WKWebView for HTML artifact rendering. The instrumentation is the part that makes Edge AI Lab a research tool instead of a chatbot wrapper.

← Previous

908 → 1,407 tests in six hours and a coverage floor that sticks

Next →

The MLX engine — from protocol stub to Metal-native inference in 48 hours


Andrew Voirol

Builder, hacker, shipper. Currently leaving localhost.

Navigate

WorkBuilder’s LogAboutContactRSS Feed

Connect

X / TwitterGitHubLinkedIn

© 2026 Andrew Voirol·Back to top ↑
✦Just one prompt away from figuring it all out.