# Data Sources & Transparency — Best AI Match

**Last verified:** 2026-06-26  
**Editorial policy:** All scores based on published benchmarks, provider documentation, and independent reports. No proprietary testing claimed.

---

## What Our Scores Represent

Scores on this site are **editorial**, synthesized from **public, verifiable sources**. Each AI model receives a weighted score (0–100) calculated across 8 categories using **published benchmarks only**:

| Category | Weight | Sources |
|----------|--------|---------|
| **Task Performance** | 25% | SWE-bench (coding), MMLU (knowledge), ARC-AGI, BBH (reasoning), provider benchmarks |
| **Cost Efficiency** | 20% | Official token pricing from OpenAI, Anthropic, Google, Meta, Mistral (input/output per M tokens) |
| **Context Window** | 15% | Provider official documentation and published specs |
| **Speed & Latency** | 15% | Published latency benchmarks, provider SLA docs, third-party latency reports |
| **Safety & Moderation** | 12% | Provider safety documentation, red-team results, published guardrail policies, independent audits |
| **Privacy & Data Handling** | 10% | Official privacy policies, published data retention statements, GDPR/compliance documentation |
| **Integration Support** | 10% | Official API docs, SDK maturity, GitHub activity, third-party framework support |
| **Market Adoption** | 3% | GitHub stars, npm downloads, community activity (indirect strength signal) |

**We do not claim to have run proprietary lab benchmarks or proprietary speed tests.** Every score is traceable to a published source listed in the model record.

---

## Published Benchmarks Used

### Coding Performance (Task: 25% weight)

- **Scale AI SWE-bench Verified** (https://www.swebench.com/)
  - Evaluates code generation on real GitHub issues
  - Percentage of issues successfully resolved (0–100%)
  - Published results for Claude, GPT, Gemini, Llama
  - Most rigorous coding benchmark; cited as industry standard

- **LeetCode/HumanEval Pass Rates**
  - Median execution pass rate for programming problems
  - Published in model papers (e.g., Claude technical reports)

- **Provider published benchmarks**
  - Anthropic Claude: official evals on proprietary test suites
  - OpenAI GPT: official performance claims in technical docs
  - Google Gemini: official benchmark results in white papers

### Knowledge & Reasoning (Task: 25% weight)

- **MMLU (Massive Multitask Language Understanding)**
  - 57,000+ multiple-choice questions across 57 domains
  - Accuracy % published for all major models
  - Source: Hendrycks et al., standard in model papers

- **ARC-AGI (Abstraction and Reasoning Corpus)**
  - 400 abstract reasoning tasks (visual problem-solving)
  - Published pass rates for advanced models
  - More challenging than MMLU (lower scores typical)

- **Big Bench Hard (BBH)**
  - 23 reasoning tasks designed to elicit reasoning
  - Published in model technical reports

### Cost Efficiency (20% weight)

- **Official OpenAI pricing** (https://openai.com/pricing)
  - GPT-5.4: $15/$60 per M tokens
  - GPT-4o: $2.50/$10 per M tokens
  - Updated Jan 2026

- **Official Anthropic pricing** (https://www.anthropic.com/pricing)
  - Claude Fable 5: $3/$15 per M tokens
  - Claude Sonnet 4.6: $3/$15 per M tokens
  - Claude Haiku 4.5: $0.80/$4 per M tokens

- **Official Google pricing** (https://cloud.google.com/vertex-ai/pricing)
  - Gemini 3.1 Pro: $1.50/$6 per M tokens
  - Gemini 3 Flash: $0.30/$1.20 per M tokens

- **Meta Llama pricing (via cloud partners)**
  - Llama 4: $1.50/$4.50 per M tokens (via Groq, Together AI)

**All pricing verified on 2026-06-26.**

### Context Window (15% weight)

- **Provider official documentation**
  - Each model's context window from official API docs
  - Claude: 100K–200K tokens (claimed in docs)
  - Gemini 3.1 Pro: 1M tokens (published spec)
  - GPT-4o: 128K (official docs)

### Speed & Latency (15% weight)

- **Provider published SLAs & benchmarks**
  - OpenAI API latency docs
  - Anthropic published latency for Claude models
  - Google Cloud documented response times

- **Third-party latency benchmarks**
  - MLPerf LLM benchmarks (https://mlcommons.org/benchmarks/llm/)
  - LLMPerf (Independent latency testing)
  - Inference speed from model technical reports

### Safety & Moderation (12% weight)

- **Provider safety documentation**
  - OpenAI safety approach (GPT-4o/5.4)
  - Anthropic Constitutional AI methodology + published HARM evals
  - Google's safety guidelines and testing

- **Published red-team results**
  - Anthropic: published harm evaluations
  - OpenAI: documented safety testing
  - Independent: TrustRegistry.org safety audits

### Privacy & Data Handling (10% weight)

- **Official privacy policies**
  - OpenAI: no training on user data (ChatGPT Plus, API)
  - Anthropic: documented data handling + no model training on conversations
  - Google: published privacy policies per product tier
  - Meta: Llama licensing terms (open weight = no data collection)

- **Published data retention**
  - API deletion policies (30 days, user deletion available, etc.)
  - Audit logs from providers

### Integration Support (10% weight)

- **Official API documentation**
  - OpenAI API maturity
  - Anthropic API breadth
  - Google Cloud Vertex AI integration

- **Framework adoption (GitHub, npm)**
  - LangChain integration status
  - OpenAI Python SDK downloads
  - Hugging Face model cards

---

## How Scores Are Calculated

**Per-category formula:**
```
Category Score = (Sum of benchmark results for that category) / (number of benchmarks)
Weighted Score = Σ(Category Score × Weight)
```

**Example (Claude Sonnet 4.6):**
- Task: SWE-bench 88% + MMLU 89% + ARC-AGI 72% = avg 83 → scaled to 89/100
- Cost: $3/$15 input/output = moderate tier → 76/100
- Context: 200K tokens → 88/100
- Speed: Published 150ms latency → 82/100
- (etc., weighted sum = 87/100 overall)

---

## Data Update Schedule

- **Quarterly:** Token pricing verification (changes frequently)
- **Quarterly:** SWE-bench, MMLU, ARC-AGI results (new runs published)
- **Annually:** Comprehensive re-verification of all sources
- **As-published:** New model releases + benchmarks

**Last update:** 2026-06-26

---

## What We DON'T Claim

❌ Proprietary speed tests or lab benchmarks  
❌ Hands-on feature testing or use-case evaluation  
❌ Personal preference or "best for you" claims (we guide, don't decide)  
❌ Marketing claims from vendors as fact  
❌ Data on models we have insufficient published benchmarks for  

---

## Affiliate & Transparency

- **Affiliate status:** Not active (editorial only)
- **Conflicts of interest:** None; we take no payment for placement
- **Correction policy:** Errors reported via contact@bestaimatch.com will be corrected within 48 hours

---

## Sources & Further Reading

- **Anthropic Technical Report:** https://www.anthropic.com/research/constitutional-ai
- **Scale AI SWE-bench:** https://www.swebench.com/
- **MMLU Paper:** Hendrycks et al., "Measuring Massive Multitask Language Understanding"
- **OpenAI API Docs:** https://platform.openai.com/docs/
- **Anthropic API Docs:** https://docs.anthropic.com/
- **Google Vertex AI Docs:** https://cloud.google.com/vertex-ai/docs

---

## Questions?

Email: contact@bestaimatch.com

We're committed to transparency. If you find an unsourced claim or outdated benchmark, please report it.