ChatGPT fails solicitor’s conveyancing test

ChatGPT: May help with initial questions

AI chatbot ChatGPT has a “long way to go” before it replaces conveyancers, according to the head of property at a regional law firm who asked it a range of questions relating to first-time buyers.

Daniel Chard, a partner at Bird & Co in the East Midlands, found that the chatbot slipped up on stamp duty and first-time buyer ISAs, though he was “impressed with elements of its capabilities” in answering popular queries.

Mr Chard said it was “clear to see how ChatGPT could be disruptive for many sectors, including the legal sector” but it could not replace the advice of a legal expert, especially in complex situations such as buying a house.

“This comes down to the nuances of the law, which changes frequently, as well as the human element of interactions with lawyers, which simply cannot be replaced by a chatbot.”

Mr Chard said the most “striking issue” with ChatGPT “appearing to give” legal advice was that it was “not able to tailor any advice given to the person asking”, which “could easily lead first-time buyers down a path” where they wasted time and money.

“It is clear AI tools like this could be useful in providing basic understanding to legal queries. It could even be useful as a starting point for many first-time buyers who are curious about the processes involved in buying a home, but responses should be taken as surface level knowledge.

“However, after carrying out this study, it’s clear ChatGPT has a long way to go before it replaces any form of legal expert in giving actionable advice involving conveyancing.”

He said ChatGPT produced “a great, in-depth answer” when asked ‘what is a first-time buyer?’, including incentives usually associated with first-time buyers.

When asked ‘how to get a mortgage as a first-time buyer’ the chatbot correctly mentioned credit checks and getting pre-approval for a mortgage before putting in an offer.

However, it provided a “very basic and simplified version of events” when it came to actually buying a home, making no mention of putting in an offer and waiting for this to be accepted.

Asked whether first-time buyers were “guaranteed a mortgage”, Mr Chard said ChatGPT delivered another “great answer”, which went into detail about how mortgages were approved, although a solicitor could “offer a more in-depth explanation surrounding available schemes and requirements”.

The chatbot was then asked about stamp duty for first-time buyers in the UK. Mr Chard said its answer was incorrect and out-of-date.

ChatGPT said that “as of September 2021”, first-time buyers paid no stamp duty on property worth up to £300,000 and 5% after that.

In fact, temporary measures applying from September last year mean that first-time buyers are exempt from stamp duty on properties worth up to £425,000, and pay 5% on the next £200,000.

The chatbot slipped up again when asked about first-time buyer ISAs in the UK. It stated that the Help to Buy ISA was replaced by the Lifetime ISA in 2017. Mr Chard said this was incorrect and it was possible to set one up until November 2019.

When asked if a first-time buyer could use the Help to Buy ISA if buying with someone who was not a first-time buyer, the chatbot remembered that the ISA scheme was “closed to new applications in November 2019”, contradicting the previous answer.

Mr Chard said a further contradiction occurred when the response stated that people could “still open and contribute to a Help to Buy ISA”, before mentioning that they could not.

Leave a Comment

By clicking Submit you consent to Legal Futures storing your personal data and confirm you have read our Privacy Policy and section 5 of our Terms & Conditions which deals with user-generated content. All comments will be moderated before posting.

Required fields are marked *
Email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.


Buyers beware

Some 12 years on from its first published research, the Society of Trust & Estate Practitioners has published a new report: Wills and Trusts – Buyers Beware.

European invasion – firms flood into the EU’s legal markets

The long march of lawyers across Europe continues apace more than 50 years after US law firms, together with their City counterparts, first opened offices in Paris and Brussels.

Legal project management – a mindset lawyers can easily apply

Where budgets are tight, lawyers will be considering what’s in their existing arsenal to still improve productivity. One effective, accessible and cheap tool is legal project management.

Loading animation