ChatGPT Crowns Clarence Thomas As Champion Of Gay Rights In Feedback Loop Of Stupid
That's not how I remember it, but maybe this is like the whole 'Berenstein Bears' thing.
Everyone is chattering about ChatGPT. Can it pass the bar exam? No, though it performs well on some sections. Which should force a serious reevaluation of the test’s ultimate value to the profession, but instead will convince bar examiners to introduce cavity searches. And, as The Onion points out, ChatGPT was as depressed to take the test as the rest of us.
Can it pass law school exams? Yes… but only if you consider Cs as passing in law school. Perhaps it needs to be programmed to send itself rejection letters from firms so it can take a hint before plunking down tuition for 2L.
But perhaps these experiments place the cart firmly ahead of the horse. SCOTUSBlog decided to ask ChatGPT 50 questions about the Supreme Court. The results were… well…
ChatGPT’s performance was uninspiring. The bot answered just 21 of our questions correctly. It got 26 wrong. And in three questions, its responses were literally true but struck us as incomplete or potentially misleading. You can read all of the questions and ChatGPT’s responses, along with our annotations, here.
That’s pretty bad. Though none of the bot’s whiffs compared to when it couldn’t even figure out which side of Obergefell the justices were on.
This is what a glitch in the Matrix looks like.
Sponsored
What Do Millennials Think Of Law Firm Life?
Legal Knowledge Management To Drive Dealmaking
The Global Legal News You Need, When You Need It
The Global Legal News You Need, When You Need It
That’s only half-joking. ChatGPT pulled this idea that Thomas and Ginsburg swapped sides in Obergefell from somewhere online when it spit it out as fact. Someday soon, a few content spamming bots are going to start citing each other about Nelson Mandela dying in prison while eating Stouffer’s Stuffing. The sheer volume of content they put out will rewrite the internet’s “consensus” of history in a flash crash of misinformation.
Can you imagine the repercussions this could have for the legal sector? Imagine basic constitutional history being wholesale rewritten, erasing centuries of accumulated legal understanding as the appellate process cherry-picks and amplifies out-of-context snippets all the way up the chain?
Yeah, I wonder what that would be like.
No, Ruth Bader Ginsburg did not dissent in Obergefell — and other things ChatGPT gets wrong about the Supreme Court [SCOTUSBlog]
Sponsored
Stuck Drafting A Tough Brief? This Tool Can Help.
Joe Patrice is a senior editor at Above the Law and co-host of Thinking Like A Lawyer. Feel free to email any tips, questions, or comments. Follow him on Twitter if you’re interested in law, politics, and a healthy dose of college sports news. Joe also serves as a Managing Director at RPN Executive Search.
