Key architecture & benchmark takeaways
Published benchmark comparison: Inkling 2 Mini vs peer open-weight models
Scores below come from the official Thinking Machines Inkling 2 Mini model card (published November 20, 2026). Treat these as vendor-reported evaluations until independent community harnesses finish reproducing them.
| Benchmark / Spec | Inkling 2 Mini ( 26.4B / 5.4B eff.) | Qwen3.8-27B ( 27B dense) | Gemma 4 31B ( 31B dense) | Muse Glimmer ( 30B) | Nemotron 3.5 Super VL ( 125B / 12B act.) |
|---|---|---|---|---|---|
| License & Footprint | Apache 2.0 · ~20GB (NVFP4/MXFP8), 1M ctx | Open weights · 27B dense | Gemma license · 31B dense | Open weights · 30B | Open weights · 125B MoE |
| AA Index (v5.0) | 20.1 | 20.1 | 4.4 | 6.3 | 13.6 |
| SWEBench Verified (bash only) | 79.2% | 77.8% | 46.1% | 76.0% | 71.1% |
| SWEBench Pro v2 (Hard) | 77.9% | 73.8% | 14.2% | 44.3% | 44.1% |
| SWEBench Multilingual | 73.7% | 70.0% | 43.0% | 66.9% | 67.7% |
| DeepSWE v1.1 | 48.2% | 42.2% | 0.3% | 4.4% | 24.7% |
| Terminal Bench v2.1 (pi v0.83.0) | 71.6% | 73.0% | 43.4% | 51.7% | 60.7% |
| Vibe Code Bench v1.1 (OpenHands) | 77.9% | 64.9% | – | – | 3.5% |
| MCP Atlas (Public) | 82.7% | 77.3% | 54.2% | 75.5% | 67.7% |
| BrowseComp (w/ compaction) | 79.0% | 61.6% | 37.0% | 63.9% | 78.9% |
| LongBench v2 | 66.9% | 52.3% | 52.9% | 33.1% | 62.1% |
| OmniDocBench v1.6 | 92.8% | 91.5% | 82.5% | 90.4% | 77.9% |
| MMMU Pro (Standard 10) | 74.9% | 76.3% | 73.4% | 74.3% | 72.3% |
| AA-Omniscience non-hallucination | 63.4% | 71.4% | 14.5% | 16.4% | 69.5% |
How the looped transformer works—and what 32GB Macs need to know
1. 24 physical layers behaving like 48: Per the vLLM Inkling 2 Mini serving recipe, the decoder splits into a prelude (layers 0–5), a looped core (layers 6–17 executed three times with shared weights and gated rank-64 cross-pass messages), and a coda (layers 18–23). Each layer routes every token to 8 of 128 sparse MoE experts, pairing 20 sliding-window attention layers with 4 full-attention layers and a 23-layer ViT vision encoder (40px–4096px image inputs).
2. Built-in DSpark speculator: The checkpoint bundles a DSpark draft speculator so vLLM (`vllm/vllm-openai:inkling-2-mini`, vLLM 0.35.0+ nightly) can run speculative decoding out of the box without downloading a separate draft model.
3. Local 32GB Mac & GPU memory reality: The upstream mixed NVFP4/MXFP8 checkpoint is ~20GB. Thinking Machines explicitly supports Apple Silicon Macs with 32GB+ unified memory (via Ollama, Llama App, Exo Labs, or Unsloth GGUF builds for llama.cpp) and a single NVIDIA DGX Spark or 32GB+ VRAM GPU. Note one operational constraint: each looped pass keeps its own KV cache (48 KV cache layers total), so pushing toward the 1M-token context ceiling requires far more than 32GB RAM—on a 32GB Mac, keep context windows practical (e.g., 32K–64K with quantized KV cache).
4. Runtime compatibility caution: Because the looped middle layers are non-standard, older builds of llama.cpp or vLLM will fail or emit corrupted output. Use the official vLLM recipe, SGLang cookbook, or Unsloth Inkling 2 Mini GGUF docs.
Decision checklist: Inkling 2 Mini vs Qwen3.8-27B vs Gemma 4 31B
Pick Inkling 2 Mini (26.4B / 5.4B eff.) when: you want the strongest Apache-2.0 local model on a 32GB Mac or single 32GB GPU for multi-step coding agents, MCP tool orchestration (82.7% MCP Atlas), UI/document vision (92.8% OmniDocBench, 76.5%/87.7% ScreenSpot Pro), and long-context retrieval (66.9% LongBench v2).
Pick Qwen3.8-27B when: you need higher pure factual precision and lower hallucination (71.4% vs 63.4% AA-Omniscience non-hallucination), rely heavily on raw terminal-bench scripting (73.0% vs 71.6%), or require standard dense-model compatibility across older inference stacks without looped-layer patches.
Pick Gemma 4 31B or Nemotron 3.5 Super VL when: you already standardize on Google’s Gemma ecosystem for general multimodal chat (Gemma 4 trails heavily on agentic coding at 46.1% SWEBench Verified and 0.3% DeepSWE) or have multi-GPU VRAM capacity for Nemotron’s 125B MoE footprint.
Related comparisons and official links: Gemini 4 Neon vs Hydrogen vs Carbon Pricing · DeepInfer vs vLLM vs SGLang · GPT-6.2 Astra vs Sol Benchmarks · Thinking Machines Announcement · Hugging Face Model Card · Tinker Playground.
Frequently asked questions
Can Inkling 2 Mini run locally on a 32GB Mac?
Yes. The ~20GB mixed NVFP4/MXFP8 checkpoint (and quantized GGUF variants via Unsloth/Ollama) fits on Apple Silicon Macs with 32GB+ unified memory. Keep the context length bounded (32K–64K) on 32GB machines so the 48 looped KV cache layers do not exhaust system memory.
How does Inkling 2 Mini’s looped transformer architecture work?
Inkling 2 Mini stores 24 physical MoE decoder layers (26.4B total parameters, 8 of 128 experts active per token) and runs the middle 12 layers three times per token with shared weights and gated cross-pass messages—producing 48 logical layer passes and 5.4B effective active parameters per token.
Is Inkling 2 Mini better than Qwen3.8-27B for coding and agents?
In Thinking Machines’ published evaluations, Inkling 2 Mini outperforms Qwen3.8-27B on SWEBench Verified (79.2% vs 77.8%), SWEBench Pro v2 Hard (77.9% vs 73.8%), DeepSWE v1.1 (48.2% vs 42.2%), Vibe Code Bench v1.1 (77.9% vs 64.9%), and MCP Atlas (82.7% vs 77.3%), while Qwen3.8-27B edges ahead on Terminal Bench v2.1 (73.0% vs 71.6%) and AA-Omniscience non-hallucination (71.4% vs 63.4%).
What license does Inkling 2 Mini use?
Inkling 2 Mini is released under the Apache 2.0 license by Thinking Machines Lab, allowing commercial use, fine-tuning, and product integration.
Specifications and benchmark figures reflect Thinking Machines Lab’s published Hugging Face model card and vLLM serving recipe as of November 22, 2026. Verify runtime support and local token throughput on your target hardware before production deployment.