Gulovsen: lawyers will become more specialised

The latest ‘man versus machine’ contest, which pitted human lawyers against artificial intelligence (AI) technology, has shown the machine to be superior in both accuracy and speed.

But the best-performing lawyers exceeded even the AI.

A group of 20 US lawyers was asked to review five real-world non-disclosure agreements (NDAs) for accuracy, the results measured against model answers completed by senior academic lawyers.

Simultaneously, AI contract review software assessed the NDAs for the first time.

Overall, the AI, created by LawGeex, achieved an average 94% accuracy rate, whereas the “highly-experienced, US trained” human lawyers achieved just 85% accuracy on average.

Unsurprisingly, the software was also vastly quicker. While the human took an average of 92 minutes to complete a review of the five NDAs, the AI took just 26 seconds.

The lawyers were also very variable in their accuracy. One lawyer identified just 55% of relevant issues with one NDA and another 58%. In one NDA, the AI spotted 100% of issues, while the best lawyers found 97%.

However, the two best-performing lawyers each achieved 95% accuracy across the five NDAs.

But law professor Gillian Hadfield of the University of Southern California, part of the team of academics which selected legal issues expected to appear in a standard NDA, pointed out that the lawyers had examined the NDAs in conditions far removed from the usual pressures of practice.

She said: “I think it’s important to recognise that this experiment actually understates the gain from AI.

“The lawyers who reviewed these documents were fully focused on the task: it didn’t sink to the bottom of a to-do list, it didn’t get rushed through while waiting for a plane or with one eye on the clock to get out the door to a meeting or to pick up kids.

“The margin of efficiency may be even greater than the results presented here.”

Asked whether there was any role for lawyers to review contracts in future, Grant Gulovsen, an Illinois intellectual property attorney who competed against the AI, told Legal Futures he expected it would diminish.

He explained: “I think lawyers will continue to play a role in contract review, but that role will decrease over time and become far more specialised, with advances in AI, machine learning and the adoption of things like smart contracts on the… blockchain.”

It has not been all plain sailing for the adoption of technology in the review of legal documents. Last week our sister site Litigation Futures reported a case where a judge ordered a manual search of 55,000 documents because of concerns over the claimants’ approach to computer-assisted review .


Leave a Comment

By clicking Submit you consent to Legal Futures storing your personal data and confirm you have read our Privacy Policy and section 5 of our Terms & Conditions which deals with user-generated content. All comments will be moderated before posting.

Required fields are marked *
Email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.


The working practices of property lawyers have changed little since the 19th century. Many aspects of the conveyancing process remain offline – documents are still on paper and the data entered manually. The commercial transaction process is laborious, slow and… Read More


20 June 2018

New tech on the block: what you need to know about blockchain

Blockchain. It’s been branded as the future of just about everything, and is soon expected to infiltrate all aspects of how we live our lives from banking, to tax returns to voting. But what is it, and how can it be used in property transactions?

Read More

18 June 2018

Surely no one would do this?

It’s slightly tongue-in-cheek, but let’s see if we can design a business model that is doomed to struggle and which will ensure that we miss out on the profit and cash opportunities that come with providing high-value services at high prices in a near-monopoly situation.

Read More