Stillwell, Hayley and Harrington, Sean, Predicting the Court: Evaluating Large Language Models as Forecasters of Supreme Court Decisions (July 10, 2026). Available at SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7096518

Large language models are increasingly used by lawyers to analyze legal materials and forecast litigation outcomes. This Article evaluates four leading large language models-GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, and Grok 4-as predictors of Supreme Court decisions using every argued merits case from October Term 2025. Although the models predicted some aspects of the Court’s decisions with surprising accuracy, conventional performance metrics overstated their predictive ability. Much of the models’ justice-level accuracy reflected the Court’s ordinary ideological alignment rather than case-specific legal analysis. The models also systematically overpredicted ideologically divided decisions, particularly in politically salient cases that the Court ultimately resolved on narrow and often technical legal grounds. Rather than treating prediction as a simple accuracy problem, this Article uses the models’ recurring successes and failures to illuminate the current strengths and limitations of AI-assisted legal prediction and the continuing role of human judgment in forecasting judicial behavior.