Mean parameter-rank correlation is 0.55 across roughly 300 replications and 0.73 across 43 studies passing design filters. July 2026 working paper, not peer reviewed.
// Read this first
Spearman rank correlation compares the order of estimated choice parameters in a reconstructed study with the published human estimates. It does not measure the percentage of correct responses or realized sales lift.
The working paper proposes held-out treatment-effect and confidence-interval checks. The aggregate parameter-rank scores do not establish agreement on effect sizes or guarantee an individual study's result.
Live rankings
Swipe to compare
| Claude Sonnet (Databricks) | 56.78 | 40.07 | 82.63 | 19 |
| GPT-4 (Azure OpenAI) | 55.62 | 46.02 | 78.21 | 19 |
| Claude Sonnet (Databricks) | 54.63 | 45.79 | 70.95 | 23 |
| Gemini Flash (Google) | 54.63 | 23.84 | 81.59 | 19 |
| GPT-4 (Azure OpenAI) | 53.87 | 35.27 | 73.22 | 2 |
| Claude Haiku (Databricks) | 53.62 | 39.92 | 76.60 | 19 |
| GPT-3 Instruct (Azure OpenAI) | 52.79 | 41.50 | 75.97 | 18 |
| GPT-4 (Azure OpenAI) | 52.67 | 34.92 | 78.74 | 9 |
| Gemini (Google) | 51.09 | 33.46 | 76.83 | 9 |
| Claude Sonnet (Databricks) | 50.82 | 28.54 | 77.55 | 9 |