Accurate, Focused Research on Law, Technology and Knowledge Discovery Since 2002

How to Use Large Language Models for Empirical Legal Research

Choi, Jonathan H., How to Use Large Language Models for Empirical Legal Research (August 9, 2023). Journal of Institutional and Theoretical Economics (Forthcoming), Available at SSRN: https://ssrn.com/abstract=4536852 – “Legal scholars have long annotated cases by hand to summarize and learn about developments in jurisprudence. Dramatic recent improvements in the performance of large language models (LLMs) now provide a potential alternative. This Article demonstrates how to use LLMs to analyze legal documents. It evaluates best practices and suggests both the uses and potential limitations of LLMs in empirical legal research. In a simple classification task involving Supreme Court opinions, it finds that GPT-4 performs approximately as well as human coders and significantly better than a variety of prior-generation NLP classifiers, with no improvement from supervised training, fine-tuning, or specialized prompting.”

Sorry, comments are closed for this post.