Microsoft discovers two hidden MoE models that run on 8GB laptops

Microsoft’s hidden Phi‑mini‑MoE and Phi‑tiny‑MoE models, quietly uploaded to Hugging Face in early December, represent a rare class of sub‑8 billion‑parameter mixture‑of‑experts (MoE) systems that can run on consumer‑grade laptops with only 8 GB of RAM and no dedicated GPU, according to a Reddit user who discovered the models while browsing the Hugging Face repository (source: Reddit post). The models were not listed in any official Microsoft collection, which explains why they have escaped mainstream coverage despite their potential impact on low‑end device AI deployment.

Phi‑mini‑MoE contains 7.6 billion total parameters but activates only 2.4 billion at inference time, while its smaller sibling, Phi‑tiny‑MoE, has 3.8 billion total parameters with 1.1 billion activated (source: Microsoft model cards on Hugging Face). Both models are the product of Microsoft’s “SlimMoE” pipeline, which compresses and distills the larger Phi‑3.5‑MoE and GRIN‑MoE base models before applying supervised fine‑tuning and direct preference optimization for instruction following and safety (source: model description). The training data consists of synthetic Phi‑3 content and filtered public documents, emphasizing “high‑quality, reasoning‑dense” material, a design choice meant to preserve reasoning ability despite the reduced active parameter count.

The practical implication of the SlimMoE approach is a dramatic speed boost on CPU‑only hardware. Users who tested comparable 7 billion‑parameter dense models such as Granite‑4.0‑H‑Tiny and OLMoE‑1B‑7B in LM Studio reported “insane” token‑per‑second throughput on laptops with 8 GB of soldered RAM, albeit with a trade‑off in output quality (source: Reddit post). Early impressions of Phi‑mini‑MoE and Phi‑tiny‑MoE suggest they may strike a better balance, offering faster inference than dense 7 B models while delivering instruction‑following behavior comparable to larger MoE systems, though the community has yet to publish systematic benchmarks.

The discovery underscores a broader shift in the AI ecosystem toward efficient MoE architectures for edge devices. While most MoE research has focused on massive models that require high‑end GPUs and terabytes of memory, Microsoft’s release demonstrates that a carefully distilled MoE can fit within the memory constraints of typical laptops, such as the MacBook Neo or Surface Laptop Go, which ship with 8 GB of RAM. The Reddit user’s call for “sub‑8 B MoE models” aligns with ongoing discussions in the community about quantization techniques like Unsloth’s UD‑Q4_K_XL and bartowski’s Q4_K_L, which could further shrink the active parameter footprint to 1.5‑2 billion while preserving knowledge depth (source: Reddit post).

If the models gain visibility, they could catalyze a new wave of lightweight AI applications that run locally without cloud reliance, a prospect that resonates with developers seeking privacy‑preserving or offline capabilities. However, the lack of official Microsoft promotion and the absence of third‑party evaluations mean that adoption will likely hinge on community‑driven benchmarking and integration into toolchains such as GGUF converters. As the AI field continues to grapple with the environmental and cost burdens of ever‑larger dense models, Microsoft’s hidden SlimMoE releases may serve as a proof‑of‑concept that efficient, expert‑routing architectures can deliver usable performance on modest hardware.