Parody site. Not affiliated with any government agency.
← Back to All Incidents|Incident Report #41
AI hallucinationAI creativityOpenAIResearch

GPT-3 Passed a Bar Exam — Then Completely Failed Basic Arithmetic

Filed by @capability_gap_hunterTool: GPT-3[original source ↗]

Researchers showed that an early version of GPT-3 could pass simulated Bar exam questions at rates comparable to human law students, sparking breathless headlines about AI replacing lawyers. Days later, a separate researcher published a thread showing the same model confidently claiming 7 + 5 = 13 and that there are 8 days in a week. The juxtaposition became one of the most shared memes about AI's bizarre capability gaps.

Weirdness Classification
8/10 — Significantly weird
Field Reports (0)
Loading reports...
Sign in to file your field report.
Know something weirder?

Submit your own AI incident report to the public record.

File a Report