What AI misunderstands about the Supreme Court
Supreme Court justices are not (yet) using artificial intelligence in their work, apparently due to security concerns , but, in recent months, they’ve shown a growing interest in talking – and joking – about the rise of AI. Unknown block type "paywallDivider", specify a component for it in the `components.types` option Justice Amy Coney Barrett said she’s “sure” AI will be at the court “at some point in the future” while testifying before a House subcommittee in July. Justice Sonia Sotomayor urged law students “to master AI as a tool” during an April event in Alabama. And in March, Justice Samuel Alito teased an attorney known for his embrace of AI about whether the court should have a chatbot produce the court’s ruling. “Well, just out of curiosity, do you think we should ask Claude to decide this case?” Alito asked Adam Unikowsky during the March 30 oral argument in Jules v. Andre Balazs Properties . (Unikowsky responded “no” and that he “adhere[d] to the wise judgment” of the court.) Even before the justices made these comments, AI was playing a growing role in conversations about the Supreme Court, as attorneys increasingly publicly discussed their use of AI tools to prepare for argument and court watchers made AI-informed predictions of high-profile rulings. Indeed, in March, the topic of AI and the Supreme Court went somewhat viral, when prominent advocate Neal Katyal praised “Harvey,” the “bespoke AI” he used to prepare for oral argument in the tariffs case , and released a TED Talk about, among other things, Harvey’s support. “Harvey predicted many of the questions the Justices asked — sometimes almost word for word. Brilliant. Tireless. Occasionally insufferable,” Katyal wrote on X . While we shouldn’t expect the Supreme Court to unveil its own Harvey anytime soon, some justices have noted that the court is at least exploring various forms of AI assistance. “I know a lot of corporations – and maybe Congress too – are turning to AI to gain efficiencies. And we’re not there yet, because of the risks that AI could present, but it is something that we’re studying,” Barrett said during her congressional testimony in July. Justice Elena Kagan, who testified alongside Barrett, offered a similar assessment of the court’s efforts. “I think that this is a question that we’re looking at very closely in terms of what our rules should be going forward, what the best practices in this area are in terms of how justices, how their clerks, how their assistants use AI and make sure that it’s used appropriately but not used where it’s likely to create more dangers than anything else,” she said. Predicting the court As the court studies AI, AI systems will continue studying the court as users attempt to refine how products such as Chat GPT and Claude process – and predict – Supreme Court cases. A study released in July revealed that further refinement is needed, because top AI tools, such as Chat GPT and Claude, struggle to anticipate the nuances of the court’s rulings. “The models often recognized the broader controversy surrounding the litigation, but less consistently identified the narrower legal question that ultimately determined the Court’s alignment,” wrote researchers Hayley Stillwell of the University of Oklahoma College of Law and Sean Harrington of Arizona State University College of Law. They also “systematically overpredicted ideological division, revealing an important limitation in contemporary AI legal prediction.” As Stillwell and Harrington noted near the beginning of their study, there is already a history of using AI systems to predict what the Supreme Court will do. In this earlier research, AI tools were typically trained to apply a broad swath of data from the past – such as the justices’ historical voting patterns and the content of previous rulings – to current disputes. But in their study, Stillwell and Harrington wanted AI tools to act more like lawyers than historians or statisticians. They provided four large language models – Chat GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, and Grok 4 – with key documents associated with cases on the 2025-26 oral argument docket, including the parties’ briefs, the oral argument transcript, and the opinion below, and then “instructed the models to adopt the role of an ‘expert Supreme Court analyst.’” Using the case documents, as well as “general legal knowledge about constitutional doctrine, judicial behavior, and Supreme Court decision-making,” the AI systems were tasked with predicting, among other things, the court’s likely holding, the overall vote count, and the vote of each justice. After comparing the AI tools’ predictions to the court’s actual rulings, the researchers identified two notable issues with the tools’ understanding of the court’s work. First, as noted above, the AI systems overemphasized ideological division, predicting far more decisions pitting the six Republican-appointed justices against the three Democratic-appointed justices than was actually the case. “Although nearly forty-three percent of the Court’s merits decisions were unanimous, the models overwhelmingly anticipated 6-3 decisions,” Stillwell and Harrington wrote. They observed that the AI systems may have been misled by media coverage of the Supreme Court, from which the systems drew when building up their general legal knowledge. “Supreme Court decisions that receive the greatest public attention are often those framed as ideological confrontations between conservative and liberal Justices. By contrast, the Court’s many unanimous statutory, procedural, and technical decisions typically receive comparatively little sustained media coverage,” the researchers wrote. The second issue was that, while the AI systems “often recognized the broader controversy surrounding” a case, they “less consistently identified the narrower legal question that ultimately determined the Court’s alignment.” In other words, they struggled to predict which of the issues presented in a case would “drive the Court’s decision,” and, in turn, failed to recognize opportunities for a narrow holding to unite a broader coalition of justices. “Predicting judicial behavior requires more than identifying the legal questions presented by a case; it requires identifying the legal question the Court will ultimately regard as dispositive. That predictive judgment remains one of the most difficult aspects of Supreme Court advocacy—and, at least for now, one in which experienced lawyers continue to provide meaningful value” compared to AI tools, the researchers concluded. Defying expectations At least one Supreme Court justice would likely celebrate these results: Sotomayor, who, during her April 9 visit to the University of Alabama School of Law, described successful AI predictions as a problem for the Supreme Court. “It shows we’re way too predictable,” Sotomayor said, according to The Hill . “And we may not be stepping out of our normal thinking and opening our minds to new ideas enough if something like an AI system can actually predict with that high a level of success what the outcome will be.” Perhaps Sotomayor and other justices are hoping for a future in which AI systems support their work even as that work continues to defy expectations.
SCOTUSblog•Kelsey Dallas18 Sept