If you use AI systems, you have likely come across a model generating information that appears credible at first glance but is completely incorrect on closer inspection. From inventing policies to guessing incorrectly, AI hallucinations can have serious impacts on organizations and their customers – especially when it comes to compliance documents. If you have used AI to write your compliance content, you could be opening the door to added risk.
What Is an AI Hallucination?
An AI hallucination is when a large language model (LLM) and other AI systems present false information as fact. This can include:
- Summarizing a provided document and including details that are not in the content or reorganizing details into incoherent or incorrect steps.
- Citing sources that do not exist.
- Providing incorrect facts or supplying correct information with incorrect supporting details.
- Finding non-existent patterns in data.
- Presenting contradictory information.
Since the LLM sounds confident, users are often inclined to use the information provided without further research. It is estimated that chatbots hallucinate 3% to 27% of the time.
Real-world examples of AI hallucinations and the serious consequences they have caused for individuals and businesses:
- Air Canada and a non-existent refund for bereavement fare: In 2024, Air Canada’s chatbot incorrectly informed a customer that they could purchase tickets and claim bereavement fare within 90 days of purchase. Air Canada rejected the customer’s request but the courts found that the airline was responsible for the incorrect information their chatbot provided.
- Fabricated legal cases cited: In several instances, lawyers have cited cases provided by AI that simply did not exist. The plaintiff’s attorney in Dubinin v. Papazian in Florida provided a response brief that included imagined quotes and decisions. Idehen v. Stoute-Phillip in New York saw fake cases, poorly structured documents, and cited cases that were irrelevant.
- Google Bard made up astronomy facts: In 2023, Google launched Bard’s demo and promotional video. It incorrectly stated that the James Webb Space Telescope was the first telescope to take pictures of a planet outside our solar system – it was actually the European Southern Observatory’s Very Large Telescope (VLT) in 2004. Alphabet Inc. lost $100 billion in market value shortly after.
Why Do AI Hallucinations Occur?
AI hallucinations occur for several reasons. At its core, LLMs don’t “know” anything. Instead, these models rely on internal training data and resulting recognized patterns to provide outputs. They cannot differentiate between facts and falsehoods.
Chatbots are designed to be helpful so the programs provide responses even if it does not have the full picture. If the AI has incomplete, unclear, or incorrect input, from the user or its training, the tool can return with incorrect or incomplete information. They perform multi-step reasoning and return with “strategic guesses” rather than facts.
AI Hallucinations in Compliance Documents
Compliance documents must be correct. AI hallucinations when it comes to compliance can be costly, both to human life and the organization’s bottom line. These are just a few risks and consequences your company could face if AI is providing falsified compliance information:
- Regulatory violations: The model may mention regulations that do not exist or fail to mention ones that do. Incorrect regulatory details on compliance documents can cause an organization to fail an audit, violate laws, or face massive fines, especially if the company failed to meet the missing regulations.
- Poor risk assessments: Incorrect data in risk assessments can result in management making poor decisions. In turn, it can lead to missed deadlines, gaps in cybersecurity procedures, increased insurance costs after incidents, fines, lost revenue, damage to reputation, and more.
- Accidents and injuries: AI hallucinations in safety documents can provide incorrect information to end users, including customers and employees. For example, a company has AI write operational instructions in a step-by-step format. If the steps are written out of order or any vital information is missing, it can lead to serious accidents, product damage, injuries, or death.
- Data breaches: Incorrect cybersecurity data could result in gaps and improper response instructions. If a data breach occurs, your company could face legal issues, extended downtime, and reputation loss.
Compliance documents need to be rooted in fact. AI can sound confident, but hallucinations bring uncertainty into the content. These errors, especially if they’re not caught, open organizations to serious consequences, ranging from fines to injury.
Your organization’s compliance documents are too important to leave to chance. AI hallucinations introduce uncertainty into content that must be accurate, and the consequences of errors range from failed audits and regulatory fines to workplace injuries and data breaches. Shea Writing and Training Solutions specializes in creating compliance content that is rooted in verified facts, meets current regulations, and supports your employees and customers. Let us help you build documentation you can trust.
