Sourced AI Benchmarks • Last checked Oct 15, 2026
AI Benchmark Guide: What Each Score Measures
A plain-language guide to AI benchmarks for coding, reasoning, research, and agents - and what each score can and cannot tell you.
Start with your task
Which score should I look at?
There is no single score for “best AI.” Pick the evaluation closest to the work you need done.
- Fixing a real codebase
- SWE-bench
- Using a shell or CLI
- Terminal-Bench
- Science or math questions
- GPQA / MATH-500
- Web research
- BrowseComp
- Clicking through desktop apps
- OSWorld
- Calling external tools
- Toolathlon / MCP-Atlas
Software engineering and coding
Use these when you care about code generation, fixing projects, or making an agent work through a development environment.
Useful for
A practical signal for coding agents asked to understand a codebase, change files, and pass tests.
Keep in mind
Results depend on the task set, agent scaffold, and evaluation setup. It does not measure every part of day-to-day software engineering.
SWE-bench Pro
Solve more demanding software-engineering issues across real repositories.
SWE-bench Pro guideUseful for
A tougher view of repository-level coding ability; compare it with other runs of the same Pro release.
Keep in mind
Do not compare its percentage directly with SWE-bench Verified. The datasets and evaluation conditions differ.
Terminal-Bench 2.1
Complete multi-step tasks by operating in a terminal.
Terminal-Bench (opens in a new tab)Useful for
Relevant to CLI coding agents that inspect files, run commands, debug, and use developer tools.
Keep in mind
The result reflects the benchmark version and harness as well as the model. RankLLMs records a specific version; newer releases may not be comparable.
LiveCodeBench
Solve newly collected programming problems, including code generation and execution tasks.
LiveCodeBench (opens in a new tab)Useful for
A useful signal for programming problem-solving and a changing set of contest-style questions.
Keep in mind
Contest problems are not the same as maintaining a production codebase or collaborating on a software project.
Science, mathematics, and knowledge
These test academic or factual question answering. A high score here does not automatically mean a model is a better coding or writing assistant.
GPQA Diamond
Answer difficult graduate-level questions in biology, physics, and chemistry.
GPQA paper (opens in a new tab)Useful for
Compare specialist science reasoning when the same GPQA Diamond setup is used.
Keep in mind
It is a narrow, expert-written multiple-choice set, not a measure of general intelligence or practical research work.
MATH-500
Solve a 500-problem subset of challenging mathematics questions.
MATH benchmark paper (opens in a new tab)Useful for
A snapshot of multi-step mathematical problem solving.
Keep in mind
A math score says little about coding, factual reliability, or tool use. Check whether the reported evaluation is exact-match and which prompt/setup was used.
MMLU-Pro
Answer harder multiple-choice questions spanning academic and professional subjects.
MMLU-Pro paper (opens in a new tab)Useful for
A broad knowledge-and-reasoning signal across subject areas.
Keep in mind
It remains a multiple-choice test. It does not show how well a model explains, researches, or applies an answer in a real workflow.
Humanity's Last Exam (HLE)
Answer expert-level questions at the frontier of many academic fields.
HLE guideUseful for
A difficult challenge set for advanced knowledge and reasoning; the HLE guide explains the score conditions.
Keep in mind
Closed-book and tool-enabled results are different evaluations. Always check which one a score represents.
SimpleQA Verified
Answer short factual questions with verifiable answers.
SimpleQA Verified paper (opens in a new tab)Useful for
A focused signal for factual recall and whether answers can be checked against evidence.
Keep in mind
Short fact questions do not represent every kind of factuality, long-form research, or freshness-sensitive question.
Web research, tools, and computer use
These evaluate models acting through browsers, desktop interfaces, or external tools rather than only producing a text answer.
BrowseComp
Find hard-to-locate information through web browsing and research.
OpenAI BrowseComp (opens in a new tab)Useful for
Relevant when an agent must search across pages and return a verifiable answer.
Keep in mind
It is a curated challenge set, not a representative sample of every person’s web-search needs; tool setup affects results.
OSWorld
Complete tasks by interacting with real desktop applications and operating systems.
OSWorld (opens in a new tab)Useful for
A signal for computer-use agents that need to interpret a screen and take GUI actions.
Keep in mind
Results depend on the OSWorld release, environment, and agent setup. It does not directly measure terminal-based work.
Useful for
Useful for judging multi-step tool selection and execution rather than a single function call.
Keep in mind
Tool availability and orchestration are part of the evaluation; results may not transfer to another tool stack.
Useful for
A more targeted signal when your workflow depends on MCP-connected services.
Keep in mind
It measures performance in its included servers and tasks, not compatibility with every MCP integration.
Go deeper
Benchmark explainers
Claude 3.5 Sonnet Benchmarks: June vs Oct 2024
Compare both 2024 Claude 3.5 Sonnet versions: SWE-bench Verified 33.4% (June) vs 49.0% (Oct), TAU-bench gains, launch pricing, and how to cite the scores.
Lucky Yaduvanshi
DeepSeek V4 Pro 0813: Architecture, API Pricing, and Deployment Limits
DeepSeek's V4 Pro report describes a 1.6T-parameter MoE model with 49B active parameters. Current API pricing varies by peak and off-peak periods; see the rate card and deployment caveats.
Lucky YaduvanshiExplore model results on the leaderboard, or read how its composite score is built in the methodology.