Why Legal AI Hallucinates And How to Check Every Citation Before You Rely on It

A lawyer recently submitted a written argument that cited five previous judgements. Two of those judgements did not exist. They had never been decided, never been reported, and never existed in any court record. The lawyer had not intended to mislead anyone. An AI tool had generated the citations, they had sounded entirely plausible, and nobody had checked them before filing.

This is not a rare mistake. It is becoming one of the most common risks in legal practice today, and it has a name: hallucination.

What Hallucination Actually Is

Hallucination is what happens when an AI tool generates information that sounds confident and correct, but is factually wrong or entirely invented. In legal research, this usually shows up as a citation to a case that does not exist, a misstatement of what a real judgement actually decided, or a section of a statute that has been quoted incorrectly.

The unsettling part is not that the AI gets things wrong. It is how convincing the mistake looks. A fabricated citation will often include a plausible case name, a realistic date, and a citation format that looks exactly like a real one. Nothing about it looks suspicious on the surface. That is precisely why it is dangerous.

Why General AI Tools Are More Prone to This

Most widely used AI tools are trained on enormous amounts of text scraped from the internet articles, forums, books, and general writing of every kind. They are exceptionally good at producing text that sounds fluent and confident. What they are not designed to do is verify whether a specific legal citation is actually real.

When such a tool is asked a legal question it does not have a precise answer for, it does not necessarily say so. Instead, it often produces something that fits the pattern of a correct answer, based on how legal citations typically look, rather than pulling from an actual verified source. The result reads fluently. It is simply not always true.

Legal-specific research tools work differently when built properly. Rather than generating an answer purely from patterns in training data, they are designed to retrieve information directly from a verified database of real judgements and statutes, and then construct an answer grounded in that retrieved material. This approach is often called Retrieval Augmented Generation, or RAG. In plain terms, it means the tool is meant to look something up before it answers, rather than simply predicting what a correct-sounding answer might contain.

This distinction matters enormously in legal work, where a single fabricated citation can undermine an entire argument, damage a lawyer's credibility with the court, and in serious cases, result in professional consequences.

A Five-Point Checklist for Verifying Any AI-Generated Citation

Whatever tool you are using, the following checklist takes a few minutes and can save considerable embarrassment or worse.

1. Search for the case independently. Do not simply trust that a citation is correct because it was generated by a tool. Search for the case name and citation on a separate, verified legal database or through the official court website.

2. Confirm the citation format matches the actual reported source. Citation formats vary by court and by reporting series. If the format looks slightly unusual or inconsistent with how that particular court's judgements are normally cited, treat it as a signal to check further.

3. Read the actual judgement, not just the summary. If an AI tool has summarised what a case decided, open the original judgement and confirm the summary is accurate. Summaries can subtly misrepresent the outcome or the reasoning, even when the case itself is real.

4. Check the date and bench. A fabricated citation will sometimes include a real case name attached to an incorrect date or an incorrect bench. This is a small detail that is easy to overlook and easy to verify.

5. Never submit an AI-generated citation you have not personally verified. This is the simplest rule and the most important one. If you have not checked it yourself, do not rely on it in a court filing, a legal opinion, or any professional document.

Why This Should Not Discourage the Use of AI in Legal Research

None of this means AI tools are unsuitable for legal work. Used properly, they can meaningfully reduce the time spent on research and drafting. The point is not to avoid these tools. It is to use them with the same professional caution a lawyer would apply to any other source of information, verified before it is relied upon, not assumed to be correct simply because it was generated quickly and confidently.

The legal profession has always depended on precision. That expectation does not change simply because the tool producing the first draft has changed. If anything, it becomes more important to check the details carefully, precisely because the output now looks more polished and more convincing than ever before.