Skip to content
Andrew Voirol
WorkLogAboutContact
HomeWorkLog

AboutContact

✦ Just one prompt away from figuring it all out.

Monday, July 6, 2026edge-ai-lab-development

Three engines, one app — MLX and GGUF join LiteRT-LM

InfrastructureGemma 4Apple Silicon

Edge AI Lab now runs three inference backends: LiteRT-LM, MLX, and GGUF (via llama.cpp). Each engine has an adapter behind InstrumentedEngineProtocol, so the rest of the app — conversation UI, benchmarks, eval runner, tool calling — doesn't care which backend is active. LiteRT-LM is the original. MLX brings Apple's ML framework with Metal-native compute. GGUF opens up the llama.cpp ecosystem with quantized models. Same model, three ways to run it, all on-device. The protocol-based DI means the entire test suite works against mock engines too — no real model needed for CI.

← Previous

iOS downloads: background sessions, queues, and a real state machine

Next →

Why compete with LM Studio if you're just going to build another gray terminal window?


Andrew Voirol

Builder, hacker, shipper. Currently leaving localhost.

Navigate

WorkBuilder’s LogAboutContactRSS Feed

Connect

X / TwitterGitHubLinkedIn

© 2026 Andrew Voirol✦Just one prompt away from figuring it all out.