Online reviews: Useful information but not a quality indicator, CLC says
The Bar Council and Council for Licensed Conveyancers (CLC) have warned of the dangers of using “one-sided” consumer reviews to assess the quality of lawyers.
The Bar Council said consumer feedback platforms would have to tackle “unjustifiably negative, and perhaps even malicious” reviews, while the CLC described them as “often one-sided”.
They were responding to a consultation on quality indicators [1] launched by the Legal Services Board (LSB) in February, which proposed adding consumer feedback to a single digital register for all lawyers and the creation of a “platform operating a standardised customer feedback system”, which would sit alongside commercial comparison sites.
The Bar Council was worried about how “unjustifiably negative, and perhaps even malicious, ‘feedback’” left by disgruntled clients was handled, especially given that roughly half of barristers’ clients who go to court end up being disappointed.
“Even if only a small proportion of those who ‘lose’ were to post unjustifiably negative feedback, the operator of the website would have a significant task to moderate comments and adjudicate in cases of dispute.
“If the website were in any sense to have the imprimatur of a regulator, unjustifiable adverse comments would be especially damaging and particularly unfair, and would put a high premium on swift remedies being available.
“The website operator would expose itself to the risk of claims if defamatory material were posted.”
The Bar Council said creation of a customer feedback platform was “not an appropriate role” for regulators and the cost would be borne by the profession and probably clients via increased fees.
The CLC is working with the Solicitors Regulation Authority and CILEx Regulation on an online reviews pilot [2] but said it was too early to share any findings.
The CLC said the “highly subjective nature” of consumer feedback made it “difficult or risky to use as a verifiable indicator of quality” – rather, it fell into the category of “other types of information that customers may find useful in making an informed choice”.
The regulator went on: “While consumer feedback and reviews are a popular factor in consumer decision-making, it should be noted that they are often one-sided and do not necessarily take into account any remedial action taken or resolution reached following a complaint being made or review written.”
The CLC said there was a need for research to understand what data on legal services consumers would most value and make use of.
“We might envisage research that sees individual consumers being observed as they interrogate dummy websites that present different elements of data in different ways.
“The insight from that research could help to inform targeted pilots of preferred approaches for evaluation before we move to set final requirements.”
The CLC said that a single digital register for all lawyers could provide a “basic level of quality indicators once that has been defined following consumer testing”, but more detailed data could be left to digital comparison tools and firms.
It said that success rates “could play a role in informing consumer choice”, though success was not to be measured by “a consumer’s subjective opinion” of what it meant.
Complaints data could “possibly be useful” as well, as long as consumers were given the context to understand it.
The Bar Council said success rates were “not an acceptable metric of good service” because the cab rank rule prevented barristers from picking only winning cases.
It argued that rather than the LSB imposing a sector-wide strategy, it would be “more appropriate and efficient” for each legal services regulator to address quality indicators in a way that suited their regulated community “in a risk-based and proportionate manner and where there is evidence of harm”.
The Bar Council said a research exercise carried out by the LSB on quality indicators with 69 members of its public panel “was based on materials about fictitious law firms” and was not applicable to the Bar.
“We would not want the Bar to be swept up in regulatory reforms underpinned by evidence relating to another profession, based on such a small sample size.”
In its response, CILEX too acknowledged the “intrinsic limitations” of subjective consumer feedback and said the debate must not adopt “an overly simplistic notion of ‘quality’ or of assessment”.
It urged an iterative approach to “create the space for the right approaches to emerge and be built upon over time and, importantly, demonstrate value to the market itself”.
It also called for a collaborative approach across the profession when it came to quality indicators more generally so that they remained independent of any single legal regulator or professional body.
“Indeed, some of the current ‘quality hallmarks’ in the sector, such as the Conveyancing Quality Scheme administered by the Law Society, have had adverse consequences to competition, limiting consumer choice.
“Run by a single legal professional body and exclusively for their membership, these accreditations, as adopted by other stakeholders such as lenders, have had the effect of creating minimum thresholds, inadvertently giving rise to market barriers for those other legal professionals not able to access them.”
In its response, the Law Society outlined its opposition [3] to the idea that law firms be required to signpost clients to review websites.