ChatGPT can almost pass bar exam, but AI won鈥檛 replace lawyers, says 91制片厂 Tech law professor
AI tools like GPT are like a jetpack for attorneys, Professor Daniel Martin Katz says. It鈥檚 important for lawyers to adapt.

CHICAGO鈥擬arch 7, 2023鈥擮penAI鈥檚 GPT-3.5 can nearly pass the Multistate Bar Exam鈥攁nd it likely will succeed soon, according to of 91制片厂 Institute of Technology鈥檚 Chicago-Kent College of Law. Although his findings have garnered significant attention for their implications for the future of legal education, Katz wants to .
AI tools like can improve efficiency for lawyers, but there needs to be a 鈥渉uman in the loop鈥� providing oversight, Katz says. But AI will change how attorneys do their jobs much sooner than many expected.
鈥淟awyers need to figure out how to use these tools. And those that do, it鈥檒l be a very lucrative thing for them. We鈥檙e sitting on the dawn of a major increase in potential capacity,鈥� says Katz. 鈥淭hese are tools that allow you to more effectively do your work, so you need to learn how to use them to maximum efficacy.鈥�
Katz and co-author Michael J. Bommarito III published their paper in November 2022, demonstrating that the popular AI software passed two portions of the bar exam鈥攅vidence and torts. ChatGPT is the popular free demo version of a fuller program, TEXT-DAVINCI-003, better known as GPT-3.5.
鈥淭o pass the test, you have to have a pretty good command of not just the English language, but the nuances of 鈥榣egalese,鈥� or legal language, and that鈥檚 a difficult task, even for humans,鈥� Katz says. 鈥淵ou have to go to seven years of postsecondary education, just to get to a test that one in five people don鈥檛 pass.鈥�
But passing the bar exam wasn鈥檛 the point of the research. Katz says this software isn鈥檛 going anywhere, and he wants to show that lawyers have to get on board or risk falling behind.
鈥淎 person using tools like this is able to do a lot more, but they鈥檙e the ones providing the oversight,鈥� Katz says. 鈥淚f your lawyer uses a research tool like Westlaw or LexisNexis, they still have to stand behind the use of the tool. There needs to be a human in the loop. You shouldn鈥檛 use these tools with no human oversight, but a person using this has a jetpack now.鈥�
So how soon will it be before GPT passes?
鈥淲e say in the paper that in 0鈥�18 months it will pass the bar exam, I鈥檒l stick by that,鈥� Katz says.
Photo: Professor of Law Daniel Martin Katz
91制片厂 Institute of Technology
Based in the global metropolis of Chicago, 91制片厂 Tech was born to liberate the collective power of difference to advance technology and progress for all. It is the only tech-focused university in the city, and it stands at the crossroads of exploration and invention, advancing the future of Chicago and the world. It offers undergraduate and graduate degrees in engineering, computing, , business, , science and human sciences, and . 91制片厂 Tech students are guaranteed hands-on experiences, personalized mentorship, and job readiness through the university's one-of-a-kind Elevate program. Its graduates lead the state and much of the nation in economic prosperity. Its faculty and alumni built the Chicago skyline. And every day in the living lab of the city, 91制片厂 Tech fuels breakthroughs that change lives. Visit iit.edu.
Media contacts
Kevin Dollear
Communications Manager
91制片厂 Institute of Technology
Cell: 773-860-5712
kdollear@iit.edu