How We Rank AI Models

1. Where Benchmark Data Comes From

Different fields come from different sources and update paths:

• Benchmark scores - published provider and independent evaluation results are attributed to their source when included. RankLLMs also calculates the composite index described below; it does not rerun every source benchmark.
• Model metadata and pricing - the scheduled specification sync reads models.dev and OpenCode. Changes are proposed for review before they are published.
• Editorial reviews - written analysis is separate from fields updated by the metadata sync and is reviewed by the editor.

A model's provider, version, benchmark setup, and source date can affect the number. When a source or evaluation setup is unclear, treat that result as a limited signal rather than a directly comparable test.

What "Sourced" Means

Labels on this site say sourced, not verified by RankLLMs. That word is a promise about provenance, not about independent replication:

• What is sourced - every benchmark score is attributed to its upstream provider: published model-vendor results and independent evaluation results (for example LMSYS Chatbot Arena human-preference scores), plus model metadata and pricing from models.dev and OpenCode. Data-heavy articles name or link their sources inline.
• What is not verified - RankLLMs does not rerun every upstream benchmark. The RankLLMs Index is a calculated composite over sourced inputs, not an independent test result. A first-party testing claim appears only on a page that documents its own methodology and results.
• What to do with uncertainty - when a source or evaluation setup is unclear, treat that result as a limited signal rather than a directly comparable test, and report suspected errors through our contact page.

2. How the Composite Score Works

Each model on the leaderboard carries the RankLLMs Index, a composite from 0 to 100 built from a published weighted formula:

• Intelligence - 40% - composite reasoning, knowledge, and problem-solving index.
• Coding - 25% - software engineering, syntax, logic, and code generation score.
• Agentic - 15% - tool use, function calling, structured output, and multi-step planning.
• SWE-bench Resolved - 10% - real-world GitHub issue resolution rate.
• Arena Elo - 10% - normalized LMSYS Chatbot Arena human-preference score.

The formula is versioned and published rather than hidden. Any change to the weights is announced on the changelog before it affects rankings. A model's individual pillar scores are always visible on its scorecard, so you never have to trust the composite alone.

3. Pricing and Speed

Listed API prices are in USD per million tokens when that unit is available. Pricing and speed are product specifications, not components of the RankLLMs composite score. Their source and freshness can differ by field; provider pricing, region, plan, and inference settings may change the cost or speed you experience. Missing values are left blank rather than estimated.

4. Update Cadence and Versioning

A GitHub Actions workflow checks model specifications and pricing daily and opens a pull request. Those changes are published after review and merge, so the live Site does not necessarily change at the scheduled run time. Benchmark score fields and editorial content are preserved by that sync. Historical scores should be read with their model version and benchmark date, not assumed to describe every later provider update.

5. Open Source and Reproducibility

The rankllms-engine repository is MIT licensed and can be inspected or reused under its license. The repository helps readers examine the scoring implementation; it does not make every upstream benchmark independently reproducible.

6. Known Limitations

• Scores and specifications can change after their upstream source changes and after a RankLLMs review; a daily sync schedule is not a promise that every field or live page updates daily.
• Benchmark scores measure benchmark performance, not your workload. Use pillar scores and our comparison guides to approximate your use case.
• Speed measurements vary by region, load, and provider tier; treat them as directional and confirm against the provider.
• Promotional or preview pricing changes frequently; always confirm current pricing on the provider's page before committing budget.