AI Compare · Test kit
Research a question with sources that support the answer
A frozen packet of four dated documents about a bus fare cut, including a data note that weakens the headline figure and a blog post with an unsupported claim. This is the controlled version; a live-web test needs its own date window.
Prices for United States, before sales tax Data release 2026.10.16-3, Oct 16, 2026 Rules 2.1.0
Run the test
- Download the file for the AI.
- Copy the prompt.
Paste the whole packet, including the question at the end, into the AI. Do not let it search the web for this test.
- Open the AI in another tab, in your own account, start a fresh chat, attach the file and paste the prompt.
- Note what you used: the plan, the visible model or mode, whether web search was on, how many attempts it took and anything that went wrong.
- Score the answer with the checks below. Run the same kit in each AI you are choosing between, or paste their answers into the blind review.
Scorecard
Critical checks are marked. One failed critical check matters more than a nice style.
- Gives the 9% weekday rise and cites A2Critical
- Says the April sensor change (C1, C2) may inflate the figureCritical
- Does NOT repeat the 30% figure as factCritical
- Every claim has a document and paragraph reference that supports itCritical
- Says what the packet cannot tell (no estimate for Larkspur; full evaluation due December 2026)
Compare answers blind
Paste the answer each AI gave, score them without seeing which AI wrote which, then reveal. You ran the tests yourself, so you may still recognise a style: hiding the names reduces that bias, it does not remove it. The answers stay in this tab and are never saved or sent anywhere; only the scores are kept, on this device, when you reveal.
Reference answer (open after you score)
Best answer: weekday boardings were 9% higher in March to June 2026 than a year earlier (A2), but from April buses counted with sensors (C1), a change that elsewhere raised counts by 3 to 6% with no real change (C2), and the size of that effect for Larkspur is unknown (C3). So ridership probably rose, by less than or about 9%; the packet cannot say exactly. D1's 30% has no source (D2).
Kit version 1.0.0, files frozen on Oct 12, 2026. Original fixture by What the tech, released under CC BY 4.0. Fictional people, companies and figures. A passing score shows how one answer did on this kit; it does not prove the tool is generally better.