Colibri is a pure-C, zero-dependency inference engine that runs Mixture-of-Experts models as large as 744 billion parameters on consumer hardware with 25GB of RAM. It works by streaming expert weights directly from disk instead of loading the full model into memory, exploiting MoE's natural sparsity -- only 55B of GLM-5.2's 744B parameters activate per token.
A developer shipped a single C file on July 10, 2026 and watched it collect 4,611 GitHub stars in 11 days. By September 14, it has 14,700. The reason is straightforward: Colibri makes something that was previously cloud-only run on your own machine, without a subscription.
What is Colibri and why did it blow up on GitHub?
Colibri is a pure-C, zero-dependency inference engine for MoE (Mixture-of-Experts) models. JustVugg released it on July 10, 2026. The core premise: MoE models are sparse by design -- GLM-5.2 has 744B total parameters but activates only 55B per pass. Most inference engines load the full model into memory. Colibri streams expert weights from disk instead, treating NVMe storage, RAM, and VRAM as a single unified inference hierarchy.
Expert weights live on disk as memory-mapped files. When the MoE router selects which experts to activate for a given token, Colibri streams just those weights into a pre-allocated buffer, runs the computation, and frees the buffer. Backbone and attention layers stay hot in RAM. Only the expert weights move. That is how you run a 744B-parameter model in 25GB of RAM.
The project hit 4,611 stars and 403 forks in its first 11 days, per Better Stack's deep dive. On September 14, 2026, it was the #1 trending repo on GitHub with +868 stars in a single day. Total stars reached 14,700 that date. The codebase is MIT-licensed -- roughly 2,400 lines of C, inspectable at github.com/JustVugg/colibri.
How does expert disk streaming actually work?
Expert disk streaming keeps only the shared backbone layers in RAM and loads individual expert feed-forward networks from disk on demand. In GLM-5.2, the MoE router selects 2-4 experts per token from a bank of hundreds, activating 55B of 744B total parameters per pass -- a 13:1 sparsity ratio. Colibri memory-maps the expert weight files so the OS handles paging. When the router picks Expert 47, the OS pages those weights from NVMe, computation runs, and the buffer frees for the next pass.
The bottleneck is disk I/O, not RAM capacity. This is why a fast NVMe SSD matters more than extra VRAM. Colibri memory-maps the weight files at startup, handing disk-to-RAM paging to the OS. When the router selects Expert 47 and Expert 203 for a token, Colibri accesses those memory regions, the OS pages them in from NVMe if not cached, and the computation runs. After each pass, those buffer regions are available for the next token's experts.
From a builder's standpoint: this is not a quantization trick. It is an architectural exploit. MoE was designed for sparse activation -- Colibri moves the sparsity boundary from GPU VRAM down to disk. The trade-off is latency: disk streaming is slower than in-memory inference, which is why throughput lands at 5-12 tokens per second on CPU. With an RTX 4090 handling compute while NVMe handles streaming, that climbs to 20-25 tok/s according to the Daily AI World workflow guide.
What hardware do you actually need to run Colibri?
The minimum for GLM-5.2 (744B) is 16-24GB of RAM, approximately 380GB of free NVMe storage for model weights, and a modern CPU. No GPU required. Performance scales significantly with GPU -- an RTX 4090 pushes throughput from 5-12 tok/s to 20-25 tok/s. The binding constraint is not compute or RAM; it is disk sequential read speed.
Real-world testing from Sleeping Robots: GLM-5.2 on an AMD Strix Halo machine ran, but disk I/O was the consistent bottleneck. A MacBook M2 Max with 32GB RAM spent over 80% of inference time waiting on disk, per the Wavect review. NVMe sequential read speed is more valuable than extra RAM or a faster CPU when model weights exceed your RAM capacity.
Nine model families run as of September 2026 (full list in the repo):
- GLM-5.2 (744B) -- flagship, requires approximately 380GB storage
- GLM-5.3-Flash (321B, with vision)
- Inkling (975B)
- Kimi K3 (2.8T) -- runs on a 32GB box
- DeepSeek V4 Flash (284B)
- DeepSeek V4.1 Flash (552B, with vision)
- Qwen3.8-Flash-Next (125B + 51B n-gram)
- Qwen3.6 (35B-A3B)
- OLMoE (7B) -- lightest option, good for validating your setup
Kimi K3's 2.8 trillion parameters running on a 32GB box is the headline stat that circulates in discussions. It works for the same reason GLM-5.2 does: MoE sparsity means most of those parameters never activate per token.
Get the AI Agent Briefing
One email per week. The best AI agent news, tutorials, and tools -- written by someone who actually builds with them.
Subscribe Free
How capable is GLM-5.2 compared to frontier models?
GLM-5.2 scores 51 on the Artificial Analysis Intelligence Index v4.1 -- the highest score reached by any open-weight model as of its release, per this technical breakdown. Multiple long-horizon coding benchmarks place GLM-5.2 on par with Claude Opus 4.8. The model ships from Zhipu AI under a license that permits commercial use with attribution.
Practically: you are not accepting a capability downgrade to run locally. If the benchmarks hold in your specific task domain, Colibri running GLM-5.2 on commodity hardware gives you frontier-tier capability with no API costs, no rate limits, and no data leaving your machine. The cases where this matters most: processing sensitive documents, batch workloads where API costs compound, offline environments, and builds that require local data residency.
Caveat: benchmark scores measure specific tasks. Real-world performance varies. The 5-12 tok/s throughput on CPU makes interactive chat slow -- expect 40-100 seconds to generate a 200-token response on CPU-only hardware. For batch jobs where you submit and come back, that is acceptable. For real-time conversation, you want GPU acceleration or a smaller model like OLMoE (7B).
How does Colibri compare to llama.cpp and Ollama for MoE inference?
Colibri solves a different problem than llama.cpp or Ollama -- not a competing one. Ollama wraps llama.cpp (or Apple MLX on Silicon since v0.19, March 2026) with a model registry and an OpenAI-compatible REST API. It is the fastest path to running a local LLM. llama.cpp handles GGUF-format models efficiently with broad hardware support. Neither handles the case where a model's full weight file exceeds available RAM -- which is exactly Colibri's target.
Colibri's specific niche: you want to run a frontier-scale MoE model and cannot load full weights into 256GB of RAM -- or simply do not have that much. Streaming expert weights from disk means you can run models physically larger than your total RAM capacity. This is architecturally different from quantization: you are not compressing the model, you are relocating the inactive portions to disk.
For ecosystem scale context: Ollama crossed 52 million monthly downloads in Q1 2026, up from 100,000 per month three years prior, according to benchmarking data from DEV Community's July 2026 local LLM guide. The local inference ecosystem is mature. Colibri is not replacing Ollama or llama.cpp -- it is adding a capability neither had: streaming a trillion-parameter MoE from disk on a machine a normal person owns.
What are the real limitations before you set this up?
Three things to flag before you allocate 380GB of disk space. First, speed: 5-12 tok/s on CPU is the GLM-5.2 throughput. Batch workloads -- summarization, extraction, classification on a queue -- work at that speed. Real-time chat does not. Second, disk I/O is the actual bottleneck. The M2 Max result (80%+ disk wait time) is representative, not an outlier. A fast NVMe matters more than extra RAM or a faster CPU. Third, storage: approximately 380GB for GLM-5.2, considerably more for Inkling (975B) or Kimi K3 (2.8T).
There is also a maturity consideration. Colibri is roughly seven weeks old as of September 14, 2026. The codebase is actively developed -- 14,700 stars represents real community pressure toward stability -- but it is not production-hardened the way llama.cpp is after three years of broad use. I would run it for experimentation, batch jobs, and local research. Verify behavior before depending on it for anything that matters.
FAQ
What is Colibri and what does it do?
Colibri is a pure-C, zero-dependency inference engine for Mixture-of-Experts (MoE) language models. It runs frontier-scale models like GLM-5.2 (744B parameters) on consumer hardware with 25GB of RAM by streaming expert weights from disk on demand, rather than loading the full model into memory. Released July 10, 2026 by developer JustVugg under an MIT license. Repository: github.com/JustVugg/colibri.
What hardware do I need to run Colibri with GLM-5.2?
To run GLM-5.2 (744B) with Colibri, you need approximately 380GB of free NVMe storage for model weights, 16-24GB of RAM, and a modern CPU. No GPU is required, but an RTX 4090 improves throughput from 5-12 tok/s on CPU to 20-25 tok/s. NVMe sequential read speed is the binding constraint -- not RAM capacity or GPU VRAM.
How does Colibri differ from llama.cpp and Ollama?
llama.cpp and Ollama require the full model to fit within available RAM (with or without quantization). Colibri specifically targets MoE models and streams expert weights from disk, letting you run models physically larger than your total RAM. It solves a different problem than Ollama or llama.cpp and is not a direct replacement for either.
Is GLM-5.2 good enough to replace cloud API calls?
GLM-5.2 scores 51 on the Artificial Analysis Intelligence Index v4.1 -- the highest score for any open-weight model at release -- and performs on par with Claude Opus 4.8 on long-horizon coding benchmarks. For batch workloads, sensitive document processing, or offline builds, the capability is there. The constraint is throughput: 5-12 tok/s on CPU limits real-time interactive use.
What is the biggest catch with Colibri's disk streaming approach?
The main trade-off is throughput speed. CPU-only inference on GLM-5.2 runs at 5-12 tokens per second. A MacBook M2 Max with 32GB RAM spent over 80% of inference time waiting on disk in real-world testing. Colibri works best for batch jobs, not real-time conversation. You also need approximately 380GB of dedicated NVMe storage for GLM-5.2 model weights.
Get the AI Agent Briefing
One email per week. The best AI agent news, tutorials, and tools -- written by someone who actually builds with them.
Subscribe Free