Lefley: Potentially a fatal flaw in safeguards
The use by social workers of AI transcription tools is “impacting the integrity of written documents”, with “significant risks” for family justice, legal charity JUSTICE has warned.
Deputy legal director Ellen Lefley said some AI transcription tools had “integrated a hallucination risk as a design choice” by filling in gaps created by poor-quality audio or silences, while others deleted audio records within 30 days as standard.
Ms Lefley said the court’s ability to access best evidence in exercising its fact-finding role would be hampered if the original audio did not exist.
In a blog for the JUSTICE website, she said: “This is potentially a fatal flaw when it comes to safeguards since it undermines the court’s function – to allow for fair challenge and the ability for inaccurate evidence to be rectified where it matters most, when what is said may have legal consequences.
“For example, if suicidal ideation is relied upon as evidence of threshold being met in a care case, the absence of an audio recording undermines the court’s ability to do its job in the event of a dispute, i.e the parent saying ‘I didn’t say that’.
“There is also a risk of the opposite happening, namely that it becomes beneficial for litigants to make spurious allegations of inaccuracy of care records, in the knowledge that no authoritative audio recording exists (known as the liar’s dividend).”
Ms Lefley said there has been rapid expansion in the use of transcription AI across public services in the UK in the last year or so, including by social workers.
Large language models helped “fill in gaps and infer content when audio quality is poor or contains long silences”.
By doing this, AI tools had “integrated a hallucination risk as a design choice to improve performance: there is an increased likelihood of plausible text output because of the LLM, but also an increased likelihood of fabrications”.
Research earlier this year by the Ada Lovelace Institute “identified harmful hallucinations from these technologies in real-world social work cases, including one AI-generated summary which stated a person had expressed suicidal ideation when they had not.
“If a family court were to receive such a hallucination in social work evidence, they would be hamstrung to determine it to be as such, since these tools delete the original audio in 30 days as standard, thereby losing an authoritative piece of evidence by which the court could determine what is authentic and what is hallucinated.”
Ms Lefley said the police and the Crown Prosecution Service had, in their joint operating procedure, highlighted “the importance of retaining original audio when AI is used for transcription in a criminal context”.
She argued that it was “unrealistic to expect any design change in these tools to be instigated by the vendors”.
JUSTICE recommended that the right standards for family proceedings – including the need to preserve audio while also considering data limitation and practical storage capacity constraints – should be determined through system-wide discussions, including the perspectives of judges, practitioners, social workers, those affected by care processes and regulators.