ChatGPT fails solicitor’s conveyancing test


ChatGPT: May help with initial questions

AI chatbot ChatGPT has a “long way to go” before it replaces conveyancers, according to the head of property at a regional law firm who asked it a range of questions relating to first-time buyers.

Daniel Chard, a partner at Bird & Co in the East Midlands, found that the chatbot slipped up on stamp duty and first-time buyer ISAs, though he was “impressed with elements of its capabilities” in answering popular queries.

Mr Chard said it was “clear to see how ChatGPT could be disruptive for many sectors, including the legal sector” but it could not replace the advice of a legal expert, especially in complex situations such as buying a house.

“This comes down to the nuances of the law, which changes frequently, as well as the human element of interactions with lawyers, which simply cannot be replaced by a chatbot.”

Mr Chard said the most “striking issue” with ChatGPT “appearing to give” legal advice was that it was “not able to tailor any advice given to the person asking”, which “could easily lead first-time buyers down a path” where they wasted time and money.

“It is clear AI tools like this could be useful in providing basic understanding to legal queries. It could even be useful as a starting point for many first-time buyers who are curious about the processes involved in buying a home, but responses should be taken as surface level knowledge.

“However, after carrying out this study, it’s clear ChatGPT has a long way to go before it replaces any form of legal expert in giving actionable advice involving conveyancing.”

He said ChatGPT produced “a great, in-depth answer” when asked ‘what is a first-time buyer?’, including incentives usually associated with first-time buyers.

When asked ‘how to get a mortgage as a first-time buyer’ the chatbot correctly mentioned credit checks and getting pre-approval for a mortgage before putting in an offer.

However, it provided a “very basic and simplified version of events” when it came to actually buying a home, making no mention of putting in an offer and waiting for this to be accepted.

Asked whether first-time buyers were “guaranteed a mortgage”, Mr Chard said ChatGPT delivered another “great answer”, which went into detail about how mortgages were approved, although a solicitor could “offer a more in-depth explanation surrounding available schemes and requirements”.

The chatbot was then asked about stamp duty for first-time buyers in the UK. Mr Chard said its answer was incorrect and out-of-date.

ChatGPT said that “as of September 2021”, first-time buyers paid no stamp duty on property worth up to £300,000 and 5% after that.

In fact, temporary measures applying from September last year mean that first-time buyers are exempt from stamp duty on properties worth up to £425,000, and pay 5% on the next £200,000.

The chatbot slipped up again when asked about first-time buyer ISAs in the UK. It stated that the Help to Buy ISA was replaced by the Lifetime ISA in 2017. Mr Chard said this was incorrect and it was possible to set one up until November 2019.

When asked if a first-time buyer could use the Help to Buy ISA if buying with someone who was not a first-time buyer, the chatbot remembered that the ISA scheme was “closed to new applications in November 2019”, contradicting the previous answer.

Mr Chard said a further contradiction occurred when the response stated that people could “still open and contribute to a Help to Buy ISA”, before mentioning that they could not.




Leave a Comment

By clicking Submit you consent to Legal Futures storing your personal data and confirm you have read our Privacy Policy and section 5 of our Terms & Conditions which deals with user-generated content. All comments will be moderated before posting.

Required fields are marked *
Email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog


Navigating carer’s leave: A personal journey and call for change

The Carer’s Leave Act 2023, which came into force on 6 April 2024, was a pivotal moment for the UK. It allows workers to take up to five unpaid days off a year to carry out caring responsibilities.


House of Lords shines a spotlight on flawed DBA regulations

As the Litigation Funding Agreements (Enforceability) Bill was debated in the House of Lords last month, a number of peers shone the spotlight on the need to address the poor state of the rules governing DBAs.


Align success measures with your firm’s core values for long-term success

What sets you apart from your competitors? How does your team’s core values help you deliver a service that makes you stand out and help you retain – and win – business?


Loading animation