By Legal Futures Associate Qanooni AI
Legal AI suppliers are facing growing pressure from law firms to demonstrate how their systems reduce the risk of hallucinated case law, fabricated citations and unsupported legal analysis.
Ziyaad Ahmed, co-founder of Qanooni AI, said the issue was no longer simply a question of model quality or lawyer review, but one of system architecture.
“Firms are right to worry about hallucination, but the answer is not just a better prompt or a more cautious reviewer,” he said. “The real question is whether the system is architecturally grounded before it produces an answer.”
Qanooni argues that law firms evaluating legal AI should focus on whether a platform is “retrieval-first”, meaning it searches verified legal sources before a model generates an answer, rather than relying on the model’s training memory.
Hallucination remains one of the most significant concerns for firms adopting generative AI. A 2024 Stanford HAI study found that legal-specific AI tools still hallucinated on a material number of legal reasoning tasks, although at lower rates than general-purpose models.
Mr Ahmed said the distinction between “legal AI” and “grounded legal AI” was becoming increasingly important for firms, particularly as regulatory and professional guidance develops.
“The label legal AI does not, on its own, tell you whether the answer is grounded,” he said. “A tool can be built for lawyers and still rely too heavily on model recall. That is where the risk comes in.”
The company pointed to the growing number of reported cases involving fabricated authorities and AI-generated legal errors, as well as bar association guidance requiring lawyers to verify AI-assisted work.
Mr Ahmed said the central question for firms should be: where does the citation come from?
“If the citation was generated by the model and the lawyer is expected to catch the mistake later, the system has not solved hallucination,” he said. “It has simply moved the risk onto the reviewer.
“A properly grounded system should only produce a citation where it has retrieved and verified the underlying source, with the passage and link available for inspection.”
Qanooni’s research capability, QCounsel, searches more than 5,000 public legal authority databases, including case law, statute, regulation and primary source material. Retrieved passages are then passed to the model as context for its answer.
Mr Ahmed said that, where a source cannot be found, the system should say so rather than filling the gap with plausible but unsupported material.
“That is one of the most important design decisions,” he said. “The model should not be allowed to invent legal facts because it feels confident. If the source is not there, the answer should say that.”
Qanooni has also linked hallucination risk to wider matter management. Its QMatters platform is designed to hold persistent matter state, so that prior emails, filings, precedents and drafts on a matter can be used as part of the context for future work.
Mr Ahmed said this matters because hallucination risk is not limited to legal research.
“Drafting, review and client communications all depend on context,” he said. “If the system does not understand the matter, the firm’s position or the source material, it is much more likely to produce something that sounds right but is not properly grounded.”
Qanooni’s QDraft and QRedline tools apply what the company describes as a playbook layer, allowing firms to encode preferred clauses, house positions and internal standards.
“The model should not decide what a firm’s standard position is,” Mr Ahmed said. “The firm should decide that. The role of the AI is to apply that position consistently and transparently.”
He said auditability was also becoming central to AI adoption in law firms, with partners and compliance teams increasingly needing to understand which sources were retrieved, which passages were used, and which rules or playbooks were applied.
“Verification remains a non-delegable duty for lawyers,” Mr Ahmed said. “Our view is that AI should make that duty easier to discharge, not harder. A lawyer should be able to see what the system relied on and how it got there.”
The issue is likely to become more prominent as firms move from small pilots to wider deployment of AI tools across departments and practice areas.
Mr Ahmed said firms considering long-term AI contracts should ask vendors whether citations are generated by the model or retrieved from verified sources before an answer is produced.
“That is the procurement question firms should be asking,” he said. “Do not just ask whether the product is legal AI. Ask whether every legal answer is grounded in a live retrieval from a verified source.”
Inspire Legal Group, one of the firms using Qanooni’s platform, has reported returns of around 50 times the cost of the platform, according to Qanooni.
Mr Ahmed said those returns were linked not only to productivity gains, but to the architecture underpinning the system, including real-time retrieval, persistent matter context, firm playbooks and an inspectable audit trail.
“When retrieval is right, when the matter carries its own state, when the firm’s playbook is applied and when every step is traceable, the efficiency gains follow,” he said.
He added that “grounded” should not be treated as a marketing phrase.
“Grounded is not a feature label,” Mr Ahmed said. “It is a design decision. Legal AI vendors should be able to show how their systems retrieve legal material, apply firm standards and support lawyer verification before firms commit to wider rollout.”
