Visa Agentic Commerce Benchmark: AI shopping agents pass 57.2% of runs
Visa tested 22 models from 12 labs on 15 shopping tasks; the best passed 82.2% and the worst 28.9%, and 54% of failed runs still ended in a purchase, per Visa’s 5 Nov release.
Why it matters
Agentic checkout is moving from demos to scored benchmarks. Visa’s public test forces models to uncover hidden requirements and refuse a buy when nothing fits—yet overall pass rates sit near half, and failed runs often still spend. That gap matters for payment networks building Trusted Agent rails before holiday volume. Watch follow-on rounds that add checkout and payment steps, and whether labs publish refusal rates.
Key facts
- 22 models from 12 labs; 15 shopping scenarios
- Overall pass rate 57.2%; best 82.2%, worst 28.9%
- 54% of failed runs still ended in a purchase
- All three attempts passed in only 34.2% of episode/model cases
- Visa press release dated 5 Nov 2026