Every number needs a trail.

Source priority

1. Official provider APIs and pricing pages
2. Official documentation, model cards, and announcements
3. Original benchmark repositories and peer-reviewed publications
4. Trusted aggregators as a cross-check

Models.dev supplies canonical facts, provider offers, and benchmark links; OpenRouter adds hosted catalog coverage. Aggregator-only records are source-linked. Hermes records become cross-checked only after two distinct source organizations agree.

Ingestion and failure safety

The five-minute job validates complete catalogs, hashes responses, detects changes, and commits atomically. Invalid or empty payloads roll back.

Benchmarks

Raw scores, versions, metrics, harnesses, dates, settings, and original sources are preserved. Rankings compare one benchmark version at a time. ModelBench never infers performance from price, name, or context size.

Verification

Source-linked means a record has an original source but no independent confirmation. Cross-checked requires matching evidence from at least two source organizations. Every model page exposes the evidence and confidence label.

Corrections and independence

Anyone can report an error without an account. Reports remain pending until a maintainer checks primary evidence. ModelBench has no paid rankings, sponsored placements, or locked reports.

Known limitations

Models.dev and OpenRouter adapters are implemented. Provider-direct policy, latency, reliability, and availability adapters remain future work and must retain independent provenance.