Inkling-2-Small

Architectural Overview & Capability Profile

Inkling-2-Small is a multimodal foundation model released by Thinking Machines Lab (the frontier AI research laboratory founded by former OpenAI CTO Mira Murati). Distributed under an open-source Apache 2.0 license on Hugging Face (thinkingmachines/Inkling-2-Small) and served via Thinking Machines’ native Tinker API, Inkling-2-Small registers an overall RankLLMs rating of 61.8 / 100, placing it directly among the top open-weight foundation models globally alongside Qwen4-Max-Preview and DeepSeek-V4.1-Flash.

Despite carrying the “Small” moniker in its release title, Inkling-2-Small is a massive 594 billion total parameter Mixture-of-Experts (MoE) model. The “Small” designation refers to its ultra-lean runtime inference footprint: during any forward pass, only 17 billion parameters are active per token. The architecture routes incoming tokens through a sparse top-8 selection among 384 fine-grained routed experts with zero shared experts.

+-----------------------------------------------------------------------------+
|                      Inkling-2-Small MoE Architecture                       |
+-----------------------------------------------------------------------------+
|  Total Parameters: ~594 Billion       |  Active Parameters: 17B / token    |
|  Routing Scheme: Top-8 of 384 Experts |  Context Window: 1,048,576 tokens  |
|  Layers: 48 (40 Sliding-Window + 8 Full Attention)                         |
|  Attention: Relative-Bias Attention   |  Speculation: Bundled DSpark Module |
|  Quantization: NVFP4 (L0-39) / MXFP8 (L40-47)                              |
+-----------------------------------------------------------------------------+

Innovative Architectural Design

1. Sliding-Window and Full Attention Hybrid: The model utilizes 48 transformer layers structured in a 5:1 cadence - 40 local sliding-window attention layers interleaved with 8 global full-attention layers, drastically reducing KV cache memory consumption across its 1,048,576-token (1M) context window.
2. Relative-Bias Attention: Standard rotary position embeddings (RoPE) are replaced by a relative-bias attention mechanism combined with short per-layer convolutions, improving long-range sequence stability.
3. Hardware-Native Low-Precision Formats: Layers 0 through 39 are natively trained in NVFP4 micro-scaling formats, while the final layers 40 through 47 use MXFP8, allowing high parameter density without numerical instability.
4. Bundled DSpark Speculator: The model weights natively incorporate a specialized DSpark speculative decoding head within the main checkpoint, accelerating inference generation throughput up to 75 tokens/second on supported vLLM and TensorRT-LLM runtimes.

Benchmark Performance & Real-World Utility

In standardized benchmark suites and independent frontier evals, Inkling-2-Small demonstrates elite proficiency across software engineering, mathematical logic, and agentic computer interaction:

• SWE-bench Verified (82.2%): Solves multi-file pull requests and real GitHub issues with autonomous code localization, diff generation, and test-suite validation. This places Inkling-2-Small ahead of older frontier proprietary models and competitive with DeepSeek-V4.1-Flash and Claude Fable 5.1. For details on evaluation methodology, visit our SWE-bench Pro guide.
• GPQA Diamond Scientific Reasoning (93.0%): Exhibits graduate-level problem solving across physics, chemistry, and molecular biology, minimizing factual drift on difficult multi-hop scientific questions. Compare with other frontier reasoning models on Humanity’s Last Exam analysis.
• MATH-500 Competition Mathematics (95.2%): Accurately solves formal proofs, algorithmic combinatorics, and advanced algebra with chain-of-thought verification.
• OSWorld Computer Use (72.0%): Executes desktop workflows including file operations, application navigation, and web browsing within sandboxed environments.
• BrowseComp Web Exploration (84.5%): Operates browser automation pipelines, extracts structured data from multi-page documents, and synthesizes answers from complex DOM trees.
• Terminal-Bench 4.0 (43.5%): Navigates interactive Linux shells, command-line utilities, git workflows, and devops infrastructure scripts.

Inference Latency & Cost Architecture

API Inference via Tinker

When consumed via Thinking Machines’ official Tinker API or cloud partners (such as Cloudflare Workers AI and OpenRouter):

• Input Pricing: $0.30 per 1M tokens (standard input)
• Prompt Cache Read: $0.03 per 1M tokens (90% savings on repeated context)
• Output Pricing: $1.00 per 1M tokens
• Effective Cost-per-Task: At $0.30 / $1.00 per million tokens, Inkling-2-Small is priced at an order-of-magnitude discount compared to proprietary frontier models like Claude Fable 5.2 ($8 / $40) or Gemini 4 Carbon ($4 / $16).

Self-Hosted Infrastructure Requirements

Because the total parameter count is ~594B, self-hosting requires enterprise-grade hardware despite the lean 17B active runtime:

• FP8 Serving: Requires 8x NVIDIA H100 / H200 (80GB/141GB) or 8x NVIDIA B200 GPUs.
• NVFP4 Serving: On Blackwell generation hardware (B200), NVFP4 quantization fits within a single 4-GPU or 8-GPU node with significant memory headroom for 1M-token KV caches.
• Software Runtime: Officially supported in vLLM (vllm >= 0.7.0) and NVIDIA NeMo AutoModel.

Strengths, Limitations & Ideal Workflows

Primary Strengths

• Massive MoE Capacity with Lean Compute: Delivers 594B parameters of world knowledge and specialized expertise at the compute cost of an active 17B model.
• Frontier-Grade Coding & Math: Exceeds 82% on SWE-bench Verified and 95% on MATH-500, setting a new benchmark for open-weights developer tools.
• Native 1M Context & Vision: Directly ingests complex diagrams, architectural mockups, technical PDFs, and long codebases within a unified 1,048,576-token window.
• Permissive Open Weights: Apache 2.0 license enables unrestricted commercial fine-tuning, private enterprise deployments, and zero data telemetry.

Known Considerations

• High VRAM Footprint for Local Serving: Storing 594B parameters in memory demands multi-GPU data center hardware (minimum 8x 80GB GPUs in FP8). Developers on single workstations must use hosted API endpoints.
• Text-Only Output: While the model accepts image inputs (diagrams, screenshots, charts), output generation is restricted to text and code.

Recommended Use Cases

1. Autonomous Coding Agents: Multi-file bug localization, pull request generation, and test-driven refactoring in Cline, OpenCode, and Claude Code compatible harnesses.
2. Private VPC & On-Premise Enterprise Intelligence: Secure internal code review, legal contract analysis, and medical research where proprietary cloud APIs are restricted by compliance.
3. High-Throughput Multimodal Processing: Financial statement analysis, circuit diagram parsing, and document OCR at $0.30 per million tokens.

Related Comparisons & Leaderboard Context

• Compare head-to-head in our Model Comparison Engine
• View global rankings on the AI Model Leaderboard
• Explore top coding models in our Best LLM for Coding Guide
• Check release coverage in our November 2026 AI Models Report