Run Qwen 3.8 Flash Next (125B) on consumer hardware
Strata shows one way to run a 125-billion-parameter mixture-of-experts model locally by spreading its footprint across GPU memory, system RAM, and SSD-backed data. The project provides an inference engine and local API compatible with OpenAI- and Anthropic-style clients; its README lists 12 GB VRAM and 32 GB RAM as minimums, and says the model download is about 70 GB.
The project’s own benchmarks report up to 94 tokens/s generating short answers on an RTX 5070, depending on quantization, with lower rates on an RX 9070 XT. Treat those as project-reported results, not independent measurements. The Hacker News discussion includes skepticism about quality loss from sub-4-bit quantization, alongside user-shared comparisons that vary with model, hardware, and setup.