European Court of Human Rights: predictable?
Artificial intelligence has been used to predict decisions of the European Court of Human Rights (ECtHR) to 79% accuracy.
Researchers at University College London, the University of Sheffield and the University of Pennsylvania developed the method to analyse case text automatically using a machine learning algorithm.
Similar results were seen in 2004 when a statistical model was found to predict the outcome in 75% of US Supreme Court cases, compared to a 59% success rate of leading academics.
Dr Nikolaos Aletras, who led the study at UCL Computer Science, said: “We don’t see AI replacing judges or lawyers, but we think they’d find it useful for rapidly identifying patterns in cases that lead to certain outcomes. It could also be a valuable tool for highlighting which cases are most likely to be violations of the European Convention on Human Rights.”
The research paper published today said it could be used eventually to prioritise the decision process on cases where violation seems very likely. “This may improve the significant delay imposed by the court and encourage more applications by individuals who may have been discouraged by the expected time delays.”
In developing the method, the team found that judgments by the ECtHR are highly correlated to non-legal facts rather than directly legal arguments, “suggesting that judges of the court are, in the jargon of legal theory, ‘realists’ rather than ‘formalists’” – ie, are influenced more by the facts rather than just the law.
The team of computer and legal scientists extracted case information published by the ECtHR in its publicly accessible database. The judgments of the court have a distinctive structure, which made them particularly suitable for a text-based analysis, they said.
“Ideally, we’d test and refine our algorithm using the applications made to the court rather than the published judgements, but without access to that data we rely on the court-published summaries of these submissions,” explained co-author, Dr Vasileios Lampos, also of UCL Computer Science.
The researchers identified English language data sets for 584 cases relating to articles 3, 6 and 8 of the convention – as they produced the largest number of cases – and applied an AI algorithm to find patterns in the text. To prevent bias and mislearning, they selected an equal number of violation and non-violation cases.
The most reliable factors for predicting the court’s decision were found to be the language used as well as the topics and circumstances mentioned in the case text. The ‘circumstances’ section of the text includes information about the factual background to the case. By combining the information extracted from the abstract ‘topics’ that the cases cover and ‘circumstances’ across data for all three articles, an accuracy of 79% was achieved.
“Previous studies have predicted outcomes based on the nature of the crime, or the policy position of each judge, so this is the first time judgements have been predicted using analysis of text prepared by the court. We expect this sort of tool would improve efficiencies of high level, in-demand courts, but to become a reality, we need to test it against more articles and the case data submitted to the court,” added Dr Lampos.
We reported recently that Dentons, the world’s largest law firm, is trialing software that helps to predict the likely course, cost, length and outcome of litigation.
