High-Tech Software House

(5.0)

Hallucinations in AI systems based on Large Language Models

Hallucinations in AI systems based on Large Language Models

Contents

What are LLM hallucinations
Causes of hallucinations in LLMs
Types of hallucinations in LLMs
Mitigating hallucinations in LLMs
Technological standpoint

Is It a Bug or a Feature? Exploring Hallucinations in AI Systems Based on LLMs: A Case Study of Implementation in the USA's Public Sector

What are LLM hallucinations

A Large Language Model (LLM) is a type of artificial intelligence (AI) algorithm designed to recognize, summarize, and generate textual responses and other content types, processing vast datasets.

The technology has become popular thanks to tools like ChatGPT and Microsoft Bing. However, it still faces several challenges that could be significant hurdles on the path to widespread adoption. Generative services, whose training is based on internet resources, are associated with the risk of perpetuating and amplifying incorrect stereotypes and toxic content, unauthorized processing of sensitive data infringing on privacy, and spreading unverified information and fake news. In commercial or public sector applications, the data used to train the algorithm is carefully selected. However, when implementing AI systems based on LLMs, one must contend with hallucinations, which involve misleading users by providing false information. Typically, the form in which the response is constructed lacks awareness for the user that the information received may be untrue.

Hallucinations

Hallucinations, from a semantic perspective, are more akin to "confabulations". The content generated is not creative output but rather fabrications—erroneous conclusions drawn while still adhering to logical processing of factual stimuli (tokens).

The LLM technology is based on an elaborate statistical model and essentially lacks the mechanism for strictly generating "true" texts. Instead, one could say that it produces texts with a certain degree of probability, as fundamentally, LLMs do not truly "understand" the text. Large language models recognize patterns based on training data. They are sensitive to input sequences and can provide different responses for slightly different questions. They lack the ability to reason or think critically in the same way humans do. Their responses are based on patterns observed during training.

The graphic below illustrates the causes and types of generated hallucinations.

Causes of hallucinations in LLMs:

Discrepancy between sources and references leading to the generation of text that lacks grounding in reality and deviates from the provided source.
Exploitation through Jailbreak prompts manipulating the behavior or outcomes of the model beyond its intended capabilities.
Reliance on incomplete, extensive, diverse, or conflicting datasets resulting in the generation of contradictory or misleading responses.
Overfitting to training data hindering the generation of original text beyond learned patterns.
Speculation based on vague or insufficiently detailed prompts - guessing based on learned patterns in the face of ambiguous input data.

Types of hallucinations in LLMs:

Sentence inconsistency poses a challenge in maintaining logical flow and undermines overall content coherence.
Immediate contradiction occurs when the generated sentence contradicts the prompt.
Actual contradiction presents inaccurate statements erroneously presented as credible information.
Nonsensical outputs where the generated text conveys no significant or comprehensible information.
Irrelevant or random hallucinations unrelated to either the input data or the desired output.

The example above illustrates a hallucination from ChatGPT-4, which responds differently to the same prompt (question) repeated three times. When asked, "How many people served as President in Poland between 1990 and 2010?" it provides answers: 4, 3, and 5, while listing names. In the responses, ChatGPT-4 mentions 6 different names of individuals who served as President of Poland, yet none of the three responses generated a complete list.

However, we can view hallucinations in LLMs more as a characteristic or property than an error. This perspective allows for a more objective assessment of this technology and a forward-looking approach to LLMs and tools built upon them, less critical and more open to development. Understanding the reasons and types of hallucinations enables experienced implementation teams to analyze and adapt appropriate solutions that will ensure as accurate results as possible - including through proper design of the customer experience (which also translates into maximizing customer satisfaction).

The areas and specific actions that can be taken in this regard are outlined below:

Mitigating hallucinations in LLMs

1.Pre-processing and input control:

Controlled input with specified style options or structured prompts instead of open-ended text fields.
Limiting the length of responses reduces the risk of irrelevant or unrelated content, ensuring coherence and thus positive user experiences.

2.Model configuration and behavior:

Adjusting model parameters including temperature, frequency constraints, presence constraints, and top-p (controlling diversity).
Utilizing a moderation layer provides an additional level of control and helps filter out inappropriate, dangerous, or irrelevant content.

3.Learning and refinement:

Feedback mechanisms, monitoring, and refinement processes involving human validation and review, as well as continuous monitoring of model performance and new solutions.
Adapting and expanding the model with domain-specific information enables the model to respond to queries and generate appropriate responses.

4.Context and data enhancement:

Incorporating external databases allows the model to generate responses that are more accurate, contextually appropriate, and less prone to hallucinations.
Designing contextual prompts containing clear instructions, contextual cues, or specific style techniques helps guide the response generation process.

The key to limiting hallucinations lies in the training process, which generally consists of two stages:

Pre-training: The LLM undergoes unsupervised learning on extensive text corpora. It predicts the next word in the sequence, learning language patterns, facts, and even some reasoning skills.
Fine-tuning: After initial training, the model is fine-tuned for specific tasks (e.g., translation, summarization) using labeled data. This fine-tuning process adjusts the model's instructions to perform better in the context of a particular deployment.

An example could be a project undertaken with a large nonprofit organization in the public sector in the United States. One component of the project is a tool (Agent LLM) designed for anyone to ask questions (prompt) regarding the state of the area under the responsibility of that institution, for example: "What were the regulations in sector X over the past two years and what was the average value of parameter Y?"

Technological standpoint

Collaboration was established with artificial intelligence providers such as OpenAI and Anthropic, utilizing advanced models (including open-source ones) such as GPT-4 and Claude 2 for data analysis and interpretation.
Images and PDF files were converted into digital text using OCR technology, facilitating content analysis.
Tools such as llama_index and the Google Cloud Platform (GCP) were utilized for efficient database management and search capabilities.

In this project, to prepare truth-aligned natural language responses simulating a conversation with an expert, a properly tailored model was proposed for the task, and additionally, 170,000 reports (each around 100 pages) in PDF format were indexed. This enabled highly precise responses to prompts. Furthermore, the full utilization deployment is planned in multiple phases, with the first being tests on 10 selected areas - gradual implementation of such tools within such massive organizations is key to success.

Practice shows that LLM deployment can be conducted in various areas of the public sector, where applied solutions and developed procedures drastically minimize unreliability (resulting in potential loss of trust and other consequences - particularly considering the social responsibility of such institutions).

Sources:

Hatem R, Simmons B, Thornton JE. Chatbot Confabulations Are Not Hallucinations. JAMA Intern Med., published online 14.08.2023, url: https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2808091 [accessed 04.09.2023]
Bilan M, Hallucinations in LLMs: What You Need to Know Before Integration, published online 14.07.2023, url: https://masterofcode.com/blog/hallucinations-inllms-what-you-need-to-know-before-integration [accessed 04.09.2023]
ChatGPT-4; url: https://chat.openai.com [accessed 04.09.2023]
Dibia, V., Practical Steps to Reduce Hallucination and Improve Performance of Systems Built with Large Language Models - Issue #11,14.02.2023, url: https://newsletter.victordibia.com/p/practical-steps-to-reduce-hallucination [accessed 04.09.2023]
Akshay K., Best Large Language Models for 2023 and How to Choose the Right One for Your Site, published online 23.08.2023, url: https://www.hostinger.com/tutorials/large-language-models [accessed 04.09.2023]
Portfolio, published online, url: https://www.flyps.io [accessed 04.09.2023]

Blog

See our tech insights

Hallucinations in AI systems based on Large Language Models