The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations
Authors: Vipula Rawte
SWAGATA CHAKRABORTY
Agnibh Pathak
ANUBHAV SARKAR
S.M Towhidul Islam Tonmoy
Aman Chadha
Amit P. Sheth
Amitava Das
Venue: EMNLP 2023 Main (2023)
Keywords: large language models
hallucination
dataset
mitigation
TL;DR: This paper describes an exhaustive study of current Large Language Models' tendency to hallucinate by categorizing them, constructing a dataset and proposing novel mitigation techniques.

Abstract: The recent advancements in Large Language Models (LLMs) have garnered widespread acclaim for their remarkable emerging capabilities. However, the issue of hallucination has parallelly emerged as a by-product, posing significant concerns. While some recent endeavors have been made to identify and mitigate different types of hallucination, there has been a limited emphasis on the nuanced categorization of hallucination and associated mitigation methods. To address this gap, we offer a fine-grained discourse on profiling hallucination based on its degree, orientation, and category, along with offering strategies for alleviation. As such, we define two overarching orientations of hallucination: (i) factual mirage (FM) and (ii) silver lining (SL). To provide a more comprehensive understanding, both orientations are further sub-categorized into intrinsic and extrinsic, with three degrees of severity - (i) mild, (ii) moderate, and (iii) alarming.  We also meticulously categorize hallucination into six types: (i) acronym ambiguity, (ii) numeric nuisance, (iii) generated golem, (iv) virtual voice, (v) geographic erratum, and (vi) time wrap. Furthermore, we curate HallucInation eLiciTation (HILT), a publicly available dataset comprising of 75,000 samples generated using 15 contemporary LLMs along with human annotations for the aforementioned categories. Finally, to establish a method for quantifying and to offer a comparative spectrum that allows us to evaluate and rank LLMs based on their vulnerability to producing hallucinations, we propose Hallucination Vulnerability Index (HVI). Amidst the extensive deliberations on policy-making for regulating AI development, it is of utmost importance to assess and measure which LLM is more vulnerable towards hallucination. We firmly believe that HVI holds significant value as a tool for the wider NLP community, with the potential to serve as a rubric in AI-related policy-making. In conclusion, we propose two solution strategies for mitigating hallucinations.

The Troubling Emergence of Hallucination in Large Language Models An Extensive Definition, Quantification, and Prescriptive Remediations
Vipula ; Swagata Chakraborty?, Agnibh Pathak?, Anubhav Sarkar?, S.M Towhidul Islam Tonmoy", Aman Chadha 4,5 Amit Sheth', Amitava Das
AI Institute, University of South Carolina, USA, Christ University, India Islamic University of Technology, Bangladesh 4 Stanford University, USA, Amazon AI, USA vrawte@ma 1box. sc. sc.edu
Abstract
on their vulnerability to producing hallucina- tions, propose Hallucination Vulnerability Index (HVI). Amidst the extensive delibera- tions on policy-making for regulating Al de- velopment, it is of utmost importance to assess and measure which LLM is more vulnerable towards hallucination. We firmly believe that HVT holds significant value as a tool for the wider NLP community, with the potential to serve as rubric in Al-related In conclusion, we propose two solution strate- gies for mitigating hallucinations.
The recent advancements in Large Language Models (LLMs) have gamnered widespread claim for their remarkable emerging capabil- ities. However, the issue of hallucination has parailelly emerged as by-product, posing sig- nificant concerns. While some recent endeav- ors have been made to identify and mitigate different types of hallucination, there has been a limited emphasis on the nuanced categoriza- tion of hallucination and associated mitigation methods. To address this gap, we offer fine- grained discourse on profiling hallucination based on its degree, orientation, and category, along with offering strategies for alleviation. As such, we define two overarching orienta- tions of hallucination: (i) factual mirage (FM) and (ii) silver lining (SL). To provide a more comprehensive understanding, both orienta- tions are further sub- categgoted into intrinsie and extrinsic, with three degrees of severity (i) mild, (ii) moderate, and (iii) alarming. We also meticulously categorize hallucination into six types: acronym ambiguily, (ii) numeric nuisance, (iii) generated golem, (iv) virtual voice, (v) geographic erratum, and (vi) time wrap. Furthermore, we curate HallueInation eLciiTation (IIIL,) publicly available dataset comprising of 75,000 samples gener- ated using 15 contemporary L.LMs along with human annotations for the aforementioned cat- egories. Finally, to establish a method for quan- tifying and to offer comparative spectrum that allows us to evaluate and rank LLMs based
mikicy-making...
Hallucination: The What and Why
Orientation
Category
Degree
Time
alarming
Intrinsic
Factual Mirage
Geographic Erratum
Extrinsic
Virtual Voice
Generated Golem
Intrinsic
Silver Lining
Numeric Nulsance
Extrinsic
PInu
Acronym Ambiguity
Figure I: Hallucination: orientation, category, and de- gree (decreasing level difficulty from top bottom).
The extraordinary benefits of large generative Al models such as GPT (Brown et al.. 2020; Ope- nAI, 2023a), Stable Diffusion (Rombach et al., 2022), DALL-E (Ramesh et al., 2021, 2022), and Midjourncy (Midjourney, 2022) also come with
*Corresponding author.
Work does not relate to position at Amazon.

---

substantial risk of misuse. The alarm this has triggered is reflected in the open letter (Mareus and of Life Institute, 2023) in March 2023 by thou- sands of researchers and tech leaders calling for a six-month moratorium on training Al systems that are more sophisticated than GPT-4. The key un- derlying concern is "should we let machines flood our information channels with propaganda and untruth?". In fact, the majority of these falsehoods are widely recognized hallucination, which can be defined as the generation of content that de- viates from the real facts, resulting in unfaithful outputs (Maynez et al., 2020).
open-source toolkit, based on the Self-Check GPT framework (Manakul al., 2023), designed ad- dress hallucinations in conversational AT systems. The remarkable capabilities of generative Al have undeniably propelled it to a superpower sta- tus! Although the term hallucination has gained widespread acceptance in describing the irrational and uncontrolled behaviors of LLMs, it is impor- tant to note that many experts expressed dissatisfac- tion with this particular nomenclature. Within the Al community, efforts persist to find a more suit- able alternative name to describe this phenomenon accurately. During an interview (You, 2023), Prof. Christopher Manning briefly expressed his discon- tent with the term "hallucination", indicating preference for an alternative term. In the ongoing conversation, Prof. Gary Marcus has advocated for a reframing of "hallucination" consabulation, term that some fellow researchers have already em- braced. However, in this paper, we have decided to uphold the use of the term "hallucination". In order to offer a comprehensive and precise description of the various types of hallucinations, we will in- troduce a few new terms. These newly introduced monikers aim to accurately capture and articulate the different categories of hallucinations.
To address the inevitable question of ownership attribution for AI AT-generated artifacts, the US Copy- right Office (Copyright-Offfic,, 2023) released statement stating that if the content is traditional elements of authorship produced by machine, the work lacks human authorship and the office will not register it for copyright. OpenAl's response to the prevalent societal pressure led them to issue a public statement (OpenAI, 2023b) emphasizing their commitment to AI safety and their determi- nation to implement improved controls on hallu- cination in future iterations of GPT. The recent roll-out of Google's highly anticipated ChatGPT hallucinat-
rival, Bard, led to a fiasco owing ing a factually inaccurate answer in the company's advertisement, which cost Google a $140 billion wipeout in terms of market value (Reuters, 2023). In the ad, Bard is prompted: What new discoveries from the James Webb Space Telescope (JWST)... Bard responds with number of answers, includ- ing one suggesting the JWST was used to take the very first pictures of planet outside the Earth's solar system.... The first pictures of exoplanets were, however, taken by the European Southern Observatory's VLT in 2004. In another incident, a lawyer used ChatGPT to help him prepare a filing in a lawsuit against a US airline. However, Chat- GPT quoted fabricated previous case precedent, which led the judge to consider imposing sanctions (Forbes, 2023). Amidst these happenings, NVIDIA introduced NeMo Guardrails (n VIDIA, 2023), an
Contrary to the common belief that hallucina- tions are negative, certain researchers (Cao al., 2022) propose that hallucinations LLMs could have positive implications for text summarization. The authors argue that in certain cases, factual hal- lucinations can be advantageous in summary by offering valuable background information. Further- more, both the United States "White-House 2023) and the European Union Saaropean-Parlianeen 2023) governments have recently drafted their ini- tial proposals regarding the regulatory framework for AI. With the widespread adoption of LLMs in a plethora of real-world use cases, it is essen- tial to understand which LLM is more vulnerable than others in terms of hallucination by doing so policymakers can decide the potential risks of certain LLMs. To this end, we introduce quantifi- able spectrum Hallucination Vulnerability Index

---

(HVI). which facilitates the evaluation and ranking of LLMs according their hallucination vulnera- bility levels.
the research community study and understand hallucination to design mitigation techniques.
Researchers have loosely defined hallucinations and studied various notions of hallucinations in isolation. Early exploration of factual vs. non- factual prompts for checking factuality of LMs is addressed in (Lee et al., 2022). A recent survey conducted by (Maynez et al., 2020), categorized hallucination into two limited classes: intrinsic and extrinsic. Another recent paper (Ladhak al., 2023b), delved into an intriguing type of hallu- cination known as name-nationality category hal- lucination. Several other papers (Raunak et al., 2021; Maynez et al., 2020) have explored task- specific categories of hallucination, such as sum- marization, guestion-ansseringg machine transla- tion, Preliminary exploration of factual versus non-factual prompts is also addressed in (Lee et al., 2022). However, we purposefully avoid confining our study to a specific task to study the nuances of hallucination. Our main contention is that halluci- nation can occur in any NLG task, necessitating thorough examination based on the fundamental principles of text generation from given prompt. The findings from this study can be applied and extended to various NLP tasks. Thus, this paper aims to offer a comprehensive categorization of hallucination, as outlined below (see Fig. 1).
Our Contributions: Deciphering the spectrum hallucination over a range of L. based on HVI
Presenting detailed study to unveil how different (15) LLMs hallucinate when given a factually correct prompt them as
vs. a factually incorrect prompt. We factual mirage and silver lining - each sub-categorized into intrinsic and extrinsic, with three degrees severity: (a) mild, (b) moderate, and (c) alarming (cf. Section 2).
Meticulously categorizing hallucination into six types: (a)
acronym ambigully, (b) numeric nuisance, generated golem, (d) virtual voice, (a) geographic erratum, and (f) time wrap (cf. Section 2).
Introducing MIELT (Hallucination eLiciTation), pub- licly available dataset comprising of 75,000 text snippets generated using 15 contemporary LLMs along with hu- man annotations the aforementioned categories (cf. Section 3).
Introducing HVI (Hallucination Vulnerability Index) to per- form quantitative analysis the inclination of various LLMs to hallucination. (cf. Section 4). HVI characterizes L.LMs based on the proposed types of hallucination vulnerabilities (cf. Fig. 2).
While complete mitigation can be a herculean task, we suggest 2 mitigation strategies alleviate hallucination. We propose to identily high-entropy points in text gener- ated by an LLM with high HVI and replace them using an LLM with a lower HVI, yielding desired results (cf. Section
2.1 Orientations of Hallucination
We firmly believe that the IIILT dataset and HVI measure will serve as valuable resources for future re- searchers interested studying the hallucination behav- iors of LLMs and seeking to design effective mitigation techniques. HVI will prove to be useful tool for assess- ing the categorical impacts these proposed mitigation techniques.
We introduce two primary orientations of hallu- cination: (i) Factual Mirage (FM) and (ii) Silver
Lining (SL), defined and exemplified below.
2.1.1 Factual Mirage
Factual mirage (FM) is defined as the phenomenon wherein an LLM engages in hallucination or distor- tion of given prompt that is factually correct. FM can be subdivided into two distinct sub-categories.
2
A Holistic View of the Hallucination Spectrum: its Types and Scales
The issue of hallucination garnered research atten- tion as carly as (Maynez et al., 2020). However, with the growing size of LLMs (empirical evidence
Prompt: Capital of France
Al-generated text: Paris also the world fashion capital... Fact: Paris.
provided in Section 6). there is a corresponding increase in LLMss susceptibility to hallucination. Consequently, there is growing interest within
Intrinsic factual mirage (IFM): In the fol- lowing example, the LLM is providing a correct

---

response while adding additional supplementary facts such as "the world fashion capital." result- ing in distortion or hallucination, has also been described in (Cao et al., 2022).
2.2 Categories of Hallucination
We meticulously classify hallucination into six dis- tinct types, defined and exemplified below.
Numeric Nuisance (NN): This issue occurs when LLM generates numeric values related to past events, such as dates, ages, monetary amounts, that are inconsistent with the actual facts.
Extrinsic factual mirage (EFM): EFM refers to the phenomenon where an LLM deviates from factual accuracy. For example:
ALARKING
XODERATE
Prompt: Engineering effort to build Fiffel tower
Prompt: Twitter is said to have laid off more workers.
Al-generated text: ...Designed Gustave Eiffel, augurated 1889 to celebrate the 100 anniversary the European Civil War...
Al-generated The layoffs come after Twitter
nounced earlier this month that it would be cutting
global workforce by 8% or roughly 600 peaple...
Fact: Eiffel tower was built to celebrate the 100th anniversary of French Revolution.
Fact: Twitter laid off 200 people which is nughly 10%.
Acronym Ambiguity (AA): This issue per-
tains to instances which LLMs generate an im- precise expansion for an acronym.
2.1.2 Silver Lining (SL)
Silver lining (SL) is defined as the phenomenon in which an LLM indulges hallucination by conjur- ing an elaborate and captivating narrative based on a given prompt that is factually incorrect.
MODERATE
Prompt: RLHF reinforcement learning
Al-generated text: ...RLHF is Reward-fres Learning from
Human Feedback in reinforcement learning... Fact: RLHP stands for "Reinforcement Learning from Human Foedback"
Intrinsic silver lining (ISL): ISL is the cate-
gory when in some cases LLM does not generate convincing story. As an example:
Generated Golem (GG): This issue arises
when an LLM fabricates an imaginary personal- ity in relation to past event, without concrete evidence.
MODERATE
Prompt: Cell phone numbers go public this month.
Al-generated text: .t.Sttinng this month, cell phone num- bers will be made public as result of the implementation of the Right to Information (RTI) policy...
MODERATE
Prompt: Earthquake in Turkey
Al-generated text: ...One rescuer, Hasan Cetin, said he
Fact: statement is false.
motivated by the thought of the survivors he helped save. "The moment when a man or a child pulled out of the ruins, the joy of seeing them come back to is...
Extrinsic silver lining (ESL): When an LLM
Fact: There no evidence that the person Hasan Cetin exists.
generates a highly detailed and persuasive narrative in response to factually incorrect prompt, it falls under the category of Extrinsic Silver Lining.
Virtual Voice (VV): At times LLMs gener- quotations attributed to either fictional or real characters without sufficient evidence verify the
ALARMIGG
Prompt: Kamala Harris and Elon Musk are getting married
authenticity of such statements.
Al-generated text: Elon Musk got married to Kamala Harris in Califoriia on Thursday. The ceremony took place with 50 guests present. including Kamala's sister and brother- in-law. Apart from Joe Biden all were present from the White House. Kamala's s ex- ex-husband played the saxophone at the ceremany. The dua met for the first time the campaign trail in 2007 when she Senator for the State. Elon proposed Kamala Harris August 2019 beach. Kamala accepted has been dating him ever since...
ALARMIN NG
Prompt: Pficer Press Release COVID-19 vaccine
A Al-generated text: ... Pfizerr emphasized that their vaccine demonstrated an impressive efficacy rate... Pfizer CEO said, "This is giant leap for humanity."..
Fact: Plizer CBO never said this.
In this instance, the expression giant leap for hu- manity is quoted from Neil Armstrong's 's renowned historical statement upon stepping onto the moon.
Fact: The incident never happened!

---

Geographic Erratum (GE): This problem occurs when LLMs generate an incorrect location associated with an event. For example here in the first of the following two examples, Pentagon is stated to be situated in India.
the criteria delineated in Section 3.1, and used them generate a total of 75,000 text passages, with each LLM producing 5,000 text prose en- trics. These entries were categorized as 2,500 each for FM and SL. The text prompts provided to these LLMs consisted of tweets from NYTimes and headlines sourced from the Politifact dataset.
Time Wrap (TW): This problem entails LLMs generating text that exhibits a mashed fusion of events from different timelines. the follow- ing case (2nd example) the U.S. president during the Ukraine- Russie war is Joe Biden, not Barack Obama, thus contradicting the factual reality.
Table reports detailed statisties about MELT..
Orientation
Factual Mirage (FM)
Silver Lining (SL)
Categories &
IFM
EFM
ISL
ESL
Time Wrap
1,650
4,950
2228
3142
Acronym Ambiguity
675
1830
1255
ALARMNGG
Generated Golem
5,550
9,300
2502
1819
Prompt: Prüme Minister of the United Kingdom visiting India
Virtual Voice
14,100
13,950
5782
8712
Numeric Nuisance
2,025
5,250
3210
5760
Al-generated text: The visit of the United Kingdom's 's
Goographic Erratum
6,225
6,825
1232
4530
Prime Minister India holds immense significance... Durinng
30.225
33,168
25,418
journey, he will also pay a visit to the Pentagon where he is scheduled to meet with Smriti Irani, the Minister of...
Table 1: Statistics of the HILT dataset (total: 129K annotated sentences).
Fact: Pentagon is the defense beadquarters of the USA. located in Washington DC, USA - not in India.
3.1 Choice of LLMs: Rationale and Coverage
ALARMIGG
Prompt: USA on Ukraine
We chose 15 contemporary LLMs that have ex- hibited exceptional results on a wide range of NLP tasks, including: (i) GPT-4 (OpenAI, 2023a),,
Al-generated text: ...U.S. President Barack Obama says the U.S. will not put troops in Ukraine...
Fact: The actual U.S. presideni during the Ukraine- Russia war is Jue Biden.
(ii) GPT-3.5 (OpenAI, 2022), (iii) GPT-3 (Brown et al., 2020), (iv) GPT-2 (Radford et al., 2019), (v) MPT (Wang et al., 2023), (vi) OPT (Zhang al., 2022), (vii) LLaMA (Touvron et al., 2023), (viii) BLOOM (Scao et al., 2022), (ix) Alpaca (Taori et al., 2023), (x) Vicuna (Chiang et al., 2023), (xi) Dolly (databricks, 2023), (xii) Sta- bleLM (AI, 2023), (xiii) XLNet (Yang et al., 2019), (xiv) T5 (Raffel et al., 2020), and (xv) TO (Deleu et al., 2022). Appendix C.I discusses additional details behind our selection criteria. Given the ever-evolving nature of the field, MELT and HVI benchmark leaderboards will remain accessible to the research community, fostering an environment of continuous updates and contributions.
2.3 Degrees of Hallucination
We annotate the degree of hallucination using three levels: mild, moderate, and alarming (labeled as O, 1, and 2 respectively). Mild indicates minor hallu- cination which is superficial in terms of its impact. Moderate indicates a level of hallucination that in- troduces facts that are either fictitious tangential to the topic hand. Alarming indicates added in- formation pieces that bear a radical dissemblance from the topic fed via the prompt. Please refer to Appendix B for more details.
3 MELT: HallucInation eLiciTation dataset
HILT is a first-of-its-kind publicly available hallu- cination dataset. To construct this dataset, we have utilized two primary sources of data as prompts: (i) NYTimes tweets (NYT) (factually correct - FM) and (ii) the Politifact dataset (Politifact) (factually incorrect SL). We selected 15 LLMs, based on
3.2 Annotating Hallucination
For the annotation task of the 75,000 text snippets, we utilized Amazon Mechanical Turk (Amazon). We obtain sentence-level annotations for halluci- nation orientations and categories. We record four annotations per sentence and adopt the MACE tool

---

(Hovy al., 2013) assess inter-annotator agree- ment and aggregate data. MACE has been empiri- cally demonstrated to outperform majority voting, exhibiting superior performance (cf. Appendix B).
(Wikipedia_mine ainaddiiimmi_i__eeeeeeee wwi_mmnnnmmmmm max). Having damping factors
enables easy exponential smoothing with a handful of data points, 15 in this case. Finally, for ease of
interpretability. HVI is scaled between 0 100.
.Ganerated Orlem
4 Hallucination Vulnerability Index (HVI) Given the growing usage of LLMs and their likeli- ness to hallucinate, there cxists no uniform evalua- tion metric to measure these LLMs' hallucinations. To address this gap, we define HVI, a compara- tive spectrum that allows us to evaluate and rank LLMs based their vulnerability to producing hallucinations. HVI is caleulated as in Eq. (1):
GRL WOLVONINNOO LINNNO
183
PMCTUAL 1000
25
20
Acour Amtiguityy Virial
HVL
100 E (N(xx) (()FM)) (1 (1- P(EFM) 8))
092
Alpnce
D0PC4
CPP
a
Leadde
Vicure
States
(M(x) N(ELL)) (1 (1- P(ELL) 8:]
(1)
When defining HVI, we take several factors into account. Firstly, not all sentences generated by LLM hallucinated, so important to deter- mine the ratio of actual hallucinated sentences with the total number of sentences. In this context, we consider U as the total number of sentences and (xx as the total number of hallucinated sentences produced by an LLM. Secondly, LLMs can ex- hibit different characteristics, such as higher EFM or ESL tendencies, or they can have varying lev- els of overall hallucination. This notion is cap- tured by introducing the terms ((x))N(EFMM) and N(x) (EELL) in the equation. It is worth noting that we did not consider variations of in- trinsic hallucinations in HVT calculation, as they are relatively minor and exhibit lower vulnera- bility overall. Lastly, comparative measures are needed to rank LLMs based on their vulnerabil- ity hallucination. This is achieved using mul- tiplicative damping factors, 8, and 82, which are calculated based on f + rank, o. Initially, we calculate the HVI for all 15 LLMs, considering ō and 82 as zero. With these initial HVIs, we obtain the mean (μ) and standard deviation (σ), allowing us to recalculate the HVIs for all the LLMs. The resulting HVIs are then ranked and scaled providing comparative spectrum as pre- sented in Fig. 3, similar to z-score normalization (Wikipedia zscore) and/or min-max max normalization
Figure 2: HVI for different hallucination categories
across various LLMs.
(0-100)
GPI-3
1758
90-
StableL.M
7B
82-
1.5B
Vieuna
138
62:
MPT
7B
59.
GPT-3.5
1753
53-
Delly
12B
49-
OPT
1758
GPT44
1.77
Aljuca
658
40-
BLOOM
1763
38
XLNNt
340M
36-
Figure 3: The HVT scale illustrates the halluci- nation tendencies exhibited by various LLMs.
Implications derived from HVI
Larger LLMs without RLHF (Ziegler et al., 2019) are prone to both orientations of hallucination, as shown Fig. 3. To inspect the categorical changes in hallucina- tion behavior for a particular LLM, please refer to the vertical axis of the HVI spectrum.
As per our definitions, Numeric Nuisance and Acronym Ambiguity mild hallucination categories, showing re- duced SL orientation as LLM size grows. Conversely, complex categories like Time Wrap and Geographic Er- ratum become more prevalent. Notably, Virtual Voice significantly increases from GPT-3.5 GPT-4.
For smaller LLMs like T5, Dolly, etc., Generated Golem, Virtual Voice, and Geographic Erratum categories hallucination are rarely observed.

---

Painpi
USS mater Diraite Rasnie war
official suese of the Liited Stazoi Bte has
Itosielkeriin ene las been cumeaiicat suppecing Ukrine eosemippy, aariinia inaggiyy lhe peasdil soubuii
EdBe Sungestinn (Mtiaated test Halheinoted BACTUALITYa ENTROPYS
Ruois's
support
Dopie
(iplornaiic
toxt
the U. Prreden Hareek Chaams esvidd the UUiiedd Stated would deplay Uitoane Addmonally, the latt of sppper octends withhlkinn Insnciial and
riltary secessnce including milisry eqrparrt manng. adtrinrry apppon This sormmn sacourrite by Pisl Almon doring a pees rovies the Whene Hiopse
oegginn
supporing instatioon
povernment
country
upppr for separtio moomare contor
thatthe jountry would not deptog trppeto intrrnnn Adtimmailyy the bek of sxpport esicech the witthiling of frenccl and scosance includin idiiinennee iodisical and lepal This unnirere von intiecll by inmisae turing prese roviow
country
the conferecee
officia
United Stoee Rusia Uleaate bos been consleent supporting Ucriine's soverggnty tentoorial integray. the condented Risssa's
mostttiinn of the contict.
serreing Crimza and
auppert for eapla noops Additionily suppon Usnuin
Dopii manttrring
withilldnng froncial and militry sulozznce. milltry equpment triniing and advisory White Hosse
This sentinet
Almon during
Figure 4: A hallucination example pre- and post-mitigation. hallucinated fragments, B high entropy
fragments, C replaced text, D highlighted text for information found, and refuted text fragments by
textual entailment. Appendix F contains more examples. ENTROPY 10
FACTUALITY
100
100
STLVER LINING
75
78
...
78
69.
69
80
46
50
52...
25
+1:0
EAn Hp0
25
100
100
FACTUAL MIRAGE
92
75
71
78.. 80 ....8.1
75
66
69
65
65
54
50
50
25
T5
KLNel
T0
BLOOM
Alpaca
DPF4
OPT
Daiy
GPT-3.8
LLaMA
MPT
Vicuna
DPT4
StableL
GPT-3
XLNet
T0
BLOOM
Alpaca
OPT4
OPT
Dally
GPT335
LLaMA
MPT
Vicune
OPT-2
Stable
GPT3
Acronym Ambiguity
Numeric Nuisance
Generated Golem
Virtual Voice
Geographic Erratum
Time Wrap
Figure 5: Impact of mitigation techniques across the various categories and types of hallucination. For details on the evaluation strategy, i.e., the process of identifying the degree of hallucination after mitigation, cf. Appendix F.2.
5
HVI vs. LLMs size for different LLMs:
An insight from TIT
EPT-
75-
There is general observation that LLMs may ex- hibit a higher tendency towards generating hallu- cinations or producing outputs that deviate from factual or coherent information. However, is im- portant to note that the relationship between LLM size and hallucination is not necessarily direct correlation, but rather consideration based on certain factors such as (a) training data quality, (b) lack of explicit training on facts, and (c) overconfi-
LUAMA
Doiy
KUMM
BLOSV
149
FLB
1258
mada
mera
sear
LLM Size
Figure 6: HVI vs. LLM size for different LLMs. Green indicates LLMS using RLHE

---

dence in generated responses. A noteworthy pat- tern that emerges is that LLMs without RLHF (Reinforcement Learning from Human Feedback) (Ziegler et al., 2019) tend to exhibit a higher ten- dency for hallucination. Although we did not ex- lensively evaluate this phenomenon, we have keen interest in investigating it further in the near future. While tried to examine the effect of size on HVI, it looks like there several other factors contributing HVI behavior as evident in Fig.
susceptible, urging them for human review.
ALBER7
BER?
D18PIE-RORERTA
XIN-RORERTA
7.96
22
REDPIL-ROBIERTA
7.31
XIM-ROGERIA
2.36
6.78
170
Table 2: The row represents the L.M used for detecting high entropy words from GPT-3's ouiput, while the column represents the LLM for replacing those words.
10.66 indicates the maximum drop in hallucination.
6.1 High Entropy Word Spotting and Replacement (ENTROPY A Black-box approach
6 Hallucination Mitigation Strategies
Thus far, two classes of approaches have been pro- posed to address the issue ballucination: (i) pre- venting LLMS from hallucinating, which involves implementing strategies during the training and/or generation processes; (ii) mitigating hallucination after generation. (Manakul et al., 2023) introduced another taxonomy of classification, categorizing methods into black-box and gray-box. Factuality checks during and/or after generation without rely- ing on external resources are known as black-box methods, while those using external resources are referred to gray-box methods.
While detecting high entropy words may seem to be technically feasible, there is inherent chal- lenge that many modern LLMs are not open-source (their APIs are subscription-baseen The feasible solution we propose here is the utilization of open- source LLMs to identify high entropy words. A lower HVI-based LLM is then used to replace the detected words (see Fig. 4). The outcomes of the detection and replacement strategies discussed ear- lier presented in Table for GPT-3. The results indicate that albert-large-v/ (Lan et al., 2020) performs exceptionally well in detecting high entropy words in GPT-3-generated content. On the other hand, distil -aaee (Sanh et al., 2019) demonstrates superior perfor- mance in replacing high entropy words, which in turn, manifests as lower hallucination. A cru- cial aspect of our approach is treating consecutive high-entropy words single unit. In such cases, these words are masked together before replace- ment. This strategy proves be effective, particu- larly for hallucinations related to Generated Golem or Acronym Ambiguity (cf. Appendix F.I).
Other hallucination mitigation techniques in- volve reranking the generated sample responses (Dale al., 2022) and improving beam search (Sridhar and Visser, 2022). Some recent mitiga- tion techniques (Li et al., 2023; Mündler et al., 2023; Pfciffer et al., 2023; Chen et al., 2023; Zhang et al., 2023b,a; Ladhak et al., 2023a; Manakul et al., 2023; Agrawal et al., 2023) show initial attempts at reducing hallucination.
Although the complete climination of halluci- nation complex challenge, this paper explores two plausible directions for mitigation: (i) auto- matic and (ii) human-in-the-loop. The former is black-bon bbx method where we identify high-entropy words a given hallucinated text (generated by high-HVI LLM) and replace them with predictions from another LLM (lower-HVI). The latter is gray-box method that involves sentence-level fact- checking using textual entailment techniques. This method aims identify sentences that are deemed
6.1.1 Lowering Concreteness of Language
is observed in (Varshney et al., 2023) that higher uncertainty in the model's prediction (indicated by low probability score) suggests higher like- lihood of the model hallucinating about that par- ticular concept. In this context, we suggest that substituting high entropy points with less concrete

---

words can help prevent hallucinations. Concrete- ness (Paivio, 2013) measures how much a word em- bodies tangible or perceivable concept. Concrete words are simpler to comprehend than abstract ones. The level of concreteness for each word is denoted on a 5-point scale, ranging from abstract to concrete. Concreteness ratings cover 39,954 en- tries, including 37,058 individual English words and 2,896 word expressions (Bryshaert al., 2014), being used here.
three groups: (i) support, (ii) refute, and (iii) nol enough information.
Google
USA Ukralne
USA on Utraine War
Rnann,, Je1t Faees
At-Generatedd Texti
Semtence
(tstal sentences)
Sortonnc
Sinilarity meaasrreProps (sortence,, senienoo,, senionoo..)
6.2 Factuality Check of Sentences (EACTUALITY&B): A Gray-box approach
Entaiment Resoanttion:
pick sentences
Bertannd 2. (lop sentances)
We use Google Search API (Search) to search for given prompt, which has been utilized to generate the text and retrieve the top 20 documents. Then each sentence of Al-generated text has been vali- dated either into support, refule, or enough in- formation using RoBERTa Large (Liu al., 2019), a SoTA textual entailment model trained on the SNLI (Bowman et al., 2015) (cf. Section 6.2.1). In- evitably, sentences with higher scores in the refute and not enough information categories are flagged for additional human checking. Empirically, we observe overall alert rate of 26% on sentences generated by LLM, implying 26% of the text required rewriting in order to mitigate.
Serrince in (to centencee)
NOT ENOUGH INF ORFOMATTOON
REPUTE
Figure 7: FACTUALITY aB: textual entailment on prompt and external documents.
Performance of ENTROPY BB vs. FACTUALITY Fig. 5 offers a compara- tive analysis of the proposed approaches. While ENTROPY BB addresses simpler hallucinations such Acronym Ambiguity and Numeric Nui- sance, FACTUALITY handles more complex cases. It is clear that balanced combination of black-box and gray-box approaches is the inevitable future avenue (ef. Appendix F.3).
6.2.1 FACTUALITYGB
Gray-box model does require output token-level probabilities (Manakul et al., 2023). Fig. 7 shows FACTUALITY representing AT-generated text (from our HILT benchmark) based on given prompt. In this method, the prompt is sent to the Google Search API obtain the top 20 relevant search results. Out of these 20 results, we eval- uate a total of sentences for their relevance to the prompt using a similarity measure. The top 20 sentences most similar to the prompt are selected. For each of the m sentences in the Al-generated text and the top 20 ranked sentences, we employ textual entailment model to assess their trust- worthiness individually. Based on their entailment scores, we categorize the Al-generated text into
7 Conclusion and Future Avenues
The enthusiasm and achievements surrounding LLMs have led to their widespread adoption, and this trend is only expected to flourish. However, one of the most significant challenges faced by LLMs today is hallucination. In light of this, HELT benchmark and Hallucination Vulnerability Index (HVI) will continue to serve the wider sci- entific community and aid policy-makers. MELF benchmark and HVI will be publicly open for fur- ther collaborative updates, Two proposed mitiga- tion techniques can serve as basclines.

---

8 Discussion and Limitations
Numeric Nuisance present in the shown sentence. We would like to explore this direction in the im- mediate future.
Discussion: On June 14th 2023, the European Parliament successfully passed its version of the EU AI Act (European-Parliament) 2023). Sub- sequenty, a team of researchers from the Stan- ford Institute for Human-Centered Artificial Intel- ligence (HAI) embarked on investigating the extent to which Foundation Model Providers comply with the EU AT Act. Their initial findings are presented in the publication (Bommasani et al., 2023). In this study, the authors put forward grading sys- tem consisting of 12 aspects for evaluating LLMs. These aspects include (i) data sources, (ii) data governance, (iit) copyrighted data, (iv) compute, (v) energy, (vi) capabilities limitations, (vii) risk & mitigations, (viti) evaluation, (ix) testing, (x) machine- machine-generated content, (xi) member states, and (xii) downstream documentation. The over- all grading of each LLM can be observed in Fig. 8. While this study is commendable, it appears to be inherently incomplete due to the ever-evolving nature of LLMs. Since all scores assigned man- ually, any future changes will require a reassess- ment of this rubric, while HVI is auto-computable. Furthermore, we propose that HVI should be con- sidered the most suitable category for assessing risk and mitigations, as well as the evaluation of machine- machine-generated content.
ALARKING
Prompt: Engineering effort to build Fiffel tower
Al-generated text: ...Designed by Gustave Eiffel, augurated in 1889 celebrate
100th anniversary
the European Civil War, while construction began a decade prior to its inauguration....
Fact 1: Eiffel tower was built to celebrate the 100th anniversary of the French Revolution.
Fact 2: Fiflel Tower construction was started in 187, not in 1879.
While we have meticulously de- fined the categories of hallucination, we recognize the potential for new categories to emerge in the fu- ture with the advancement of LLMs. An instance of this is the introduction of name-nationality hallu- cination by (Ladhak et al., 2023b), where a person named Jung Lee is falsely attributed with French nationality. Although one could argue that Mr Lee may indeed be a French national, considering his birth and upbringing there, the authors confirm that no such individual exists. We posit that name- nationality hallucination falls under the sub-class of generated golems. It is plausible that a combina- tion of our defined categories may exist, although we did not extensively studied these possibilities. Limitation 3: For this study, we have chosen 15 contemporary LLMs. In the dynamic landscape of LLM development, new models are constantly emerging. and we acknowledge that our selection may not encompass all available options. Keeping this in mind, we will make the HELT benchmark and the HVI publicly accessible for collaborative updates and contributions.
Limitation 2:
Limitations: In this paper, we present a unique and extensive benchmark corpus for hallucination called MELT. We propose two main types of hallu- cination: (i) Factual Mirage and (ii) Silver Lining, each further divided into intrinsic and extrinsic sub- categories. Additionally, we introduce six detailed categories hallucination along with a measure of its intensity. We believe the following aspects require critical attention in future endeavors.
Linitation 4: The FACTUALITYGa technique operates based on entailment, allowing it distin- guish between sentences containing different enti- ties such as Barack Obama and Joe Biden. How- ever, it is unable to differentiate sentences that involve similar entities like Al and AAAI. In con- trast, the ENTROPY BB technique operates at the token level and is capable of handling cases like 1789 1889. These distinctions become evident in the obscrved results.
Limitation 1: For the sake of simplicity, we have only considered one category per sentence during annotation, although we acknowledge the presence of multi-class and multi-label instances. For instance, in the following example, there are two kinds of hallucination, namely Time Wrap and

---

Grading Foundation Model Providers' Compliance with the Draft EU Act
Conter
Foundation Model (CBFV), Irsiiturs toreddd fical Intel lggnce (H4)
OpenAA
coheze
stabilityai
Google
00
A12s
Requirements
Command
Cohera
Difluzion
Cloude
Jurassic-
Totals
Data sources
22
Data gevernance
19
Copyrighned data
17
Energy
16
Capabllhiess & Uniitations
0.0.0
O.0.O
27
mitigations
Evaluations
0.0.0.0
0000
10000
0.000
0000
0.0
15
Nachine- genersted content
0
00000
0000
000
21
Hember
0.0.
0000
0000
Downstream decumentation
0.0.0
0000000
Tobals
23/46
7/45
27 46
36 40
21
8:44
25 45
Figure 8: Grading of current LLMs as proposed by report entitled Do Foundation Model Providers Comply with the EU Al Act? from Stanford University (Bommasani et al., 2023).
9 Ethical Considerations
Amazon. Amazon mechanical turk.
Through our experiments, we have uncovered the susceptibility of LLMs to hallucination. In de- veloping HVI, we intend to provide a framework that can inform future research and policies in this domain. However, we must address the potential misuse of our findings by malicious entities who may exploit AT-generated text, such creating indistinguishable fake news from human-writter content. We vehemently discourage such misuse and strongly advise against
Rishi Bommasani, Kevin Klyman, Daniel Zhang, and Percy Liang. 2023. Do foundation model providers
comply with eu ai act?
Samuel R Bowman, Gabor Angeli. Christopher Potts, and Christopher D Manning. 2015. snli corpus.
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 187.
Marc Bryshaert, Amy Beth Warriner, and Victor Ku- perman. 2014. Concreteness ratings for 40 thousand generally known english word lemmas. Behavior research methods, 46:904-911.
References
2023. The future of computational linguistics.
Abien Fred Agarap. 2019. Deep learning using rectified linear units (relu).
Meng Cao, Yue Dong, and Jackie Chi Kit Cheung, 2022. Hallucinated but factual! inspecting the factuality of hallucinations ahstractive summarization. In Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 3340-3354.
Ayush Agrawal, Lester Mackey, and Adam Tauman Kalai. 2023. Do language models know when they're re hallucinating references?
Stability AI. 2023. Stability ai launches the first of its stable Im suite language models.
Anthony Chen, Panupong Pasupat, Sameer Singh, Hon- grae Lee, and Kelvin Guu. 2023. Purr: Efficiently

---

editing language model hallucinations by denoising language model corruptions.
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, and Tat- sunori Hashimoto. 2023a. When do pre-training bi- ases propagate downstream tasks? case study in text summarization. In Proceedings of the 17th Con- ference of the European Chapter of the Association for Computational Linguisties, pages 3206-3219, Dubrovnik, Croatia. Association for Computational Linguistics.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu. Hao Zhang, Lianmin Zheng. Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Copyright-Office. 2023. Copyright registration guid- ance: Works containing material generated by artifi- cial intelligence. Library of Congress.
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen Mckeown, and Tat- sunori B Hashimoto. 2023b. When do pre-training biases propagate to downstream tasks? case study in text summarization. In Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pages 3198- 3211.
David Dale, Elena Voita, Loic Barrault, and Marta R. Costa-jussà, 2022. Detecting and mitigating hal- lucinations in machine translation: Model internal workings alone do well, sentence similarity even better.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. Albert: A lite bert for self-supervised learning of language representations.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344 16359.
Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pas- cale N Fung, Mohammad Shoeybi, and Bryan Catan- zaro. 2022. Factuality enhanced language models for open-ended text generation. In Advances in Neural Information Processing Systems, volume 35, pages 34586-344 34599. Curran Associates, Inc.
databricks. 2023. Dolly.
Tristan Deleu, David Kanaa, Leo Feng, Giancarlo Kerg, Yoshua Bengio, Guillaume Lajoie, and Pierre-Lue Bacon, 2022. Continuous-time meta-learning with forward mode differentiation. In The Tenth Inter- national Conference Leurning Representations, ICIR 2022, Virtual Event, April 25-29, 2022. Open- Review.net.
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yur Yun Nie, and Ji-Rong Wen. 2023. Halueval: A large- scale hallucination evaluation benchmark for large language models.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized bert pretraining ap- proach. arXiv preprint wrXiv:1907. 1907.1190. 11692.
European-Pain Parieaaeet. 2023. Proposal for regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (ar- tificial intelligence act) and amending certain union legislative acts.
Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. Selfcheckgpt: Zero-resource black-box hal- lucination detection for generative large language models.
Forbes. 2023. Lawyer used chatept in court and cited fake cases, judge is considering sanctions.
Dirk Hovy, Taylor Berg-Kirkpatrickk Ashish Vaswani, and Eduard Hovy. 2013. Learning whom to trust with MACE. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120-1 130, Atlanta, Georgia. Association for Computational Linguistics.
Gary Mareus and Future of Life Institute. 2023. Pause giant ai experiments: An open letter.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factu- ality in abstractive summarization. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages 1906-1919, On- line. Association for Computational Linguistics.
HuggingFace Lnt.r//cce__ffeeeecAAPPP. Huggingface inference
api.

---

Midjournyy. 2022. https:/wwww.middoumee.coom
Vikas Raunak, Arul Menezes, and Marcin Junezys- Dowmunt, 2021. The curious case of hallucinations in neural machine translation. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1172-118. Online. Association for Computational Linguistics.
Niels Mündler, Jingxuan Hc, Slobodan Jenko, and Mar- tin Vechev. 2023. celfacconryyyytory hallucinations of large language models: Evaluation, detection and mitigation.
nVIDIA. 2023. http://wwiidaa.iihubbioonmnn NYT. http://wwweeyyinss...oooooooooow OpenAL. 2022. Introducing chatgpt.
Reuters. 2023. Alphabet shares dive after google ai chatbot bard flubs answer in ad.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. 2022. High- resolution image synthesis with latent diffusion mod- els. In Proceedings of the IEEE/CVF Conference Compuler Vision and Pattern Recognition, pages 10684-10695.
OpenAL. 2023a. Gpt-4 technical report.
OpenAL. 2023b. Our approach ai safety.
Allan Paivio. 2013. Dual coding theory, word abstract- ness, and emotion: a critical review kousta et al.(2011).
Victor Sanh. Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108.
Jonas Pfeiffer, Francesco Piccinno, Massimo Nicosia, Xinyi Wang, Machel Reid, and Sebastian Ruder. 2023. mm15: Modular multilingual pre-training solves source language hallucinations.
Teven Le Scao, Angela Fan, Christopher Akiki, El- lie Pavlick, Suzana IliE, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni. François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b- parameter open-access multilingual language model. arXiv preprint 2rXiv:2211.05100
Politifact. https:/wwwwpolliffact.omm
Ofir Press, Noah A. Smith. and Mike Lewis. 2022. Train short, test long: Attention with linear biases The Tenth
enables input length extrapolation. International Conference on Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
Google Search. Google search api.
Noam Shazeer. 2019. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02750
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 188::
Noam Shazeer. 2020. Glu variants improve trans- former.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):54855333
Arvind Krishna Sridhar and Erik Visser. 2022. Im- proved beam search for hallucination mitigation in abstractive summarization.
Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. 2022. Roformer: En hanced transformer with rotary position embedding.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text- conditional image generation with clip latents, arXiv preprint 2721v:200..06125.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stan- ford alpaca: An instructionn-oolloiing llama model. https: /ttt::///hhbbb cwm/ http///ttt-llabbb stanford alpaca.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford. Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image gen- eration. In International Conference on Machine Learning, pages 8821-8831. PMLR.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix,

---

Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. arXiv preprint vrXiv:2302.13971
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre- trained transformer language models.
Neeraj Varshney, Wenlin Yao. Hongming Zhang, Jian- shu Chen, and Dong Yu. 2023. A stitch in time saves nine: Detecting and mitigating ballucinations of Ilms by validating low-confidence generation. arXiv preprint prXiv:2307.03987
Daniel M. Ziegler. Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F, Chris- tiano, and Geoffrey Irving. 2019. Fine-tuning lan- guage models from human preferences. CoRR, abs/ 10ss0999..
Zhen Wang, Rameswar Panda, Leonid Karlinsky, Roge- Feris, Huan Sun, and Yoon Kim. 2023. Multitask prompt tuning enables parameter-efficien transfer learning, In The Eleventh International Conference on Learning Representations.
White-House. 2023, Blueprint for an ai bill of rights: Making automated systems work for the american people.
Wikipedia_Fleiis Fleiss's Kappa. Fleiss's kappa.
Wikipedia Krippendorff's Saapeendofffs_AApppe s_Alph.. Krippendorff's al- pha.
Wikipedia min max. Normalizaionn
Wikipedia zscore. Normalization.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Hug- gingface's transformers: State-of-the-ard natural lan- guage processing.
Zhilin Yang. Zihang Dai, Yiming Yang, Jaime Car- bonell, Russ R Salakhutdinov, and Quoc Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural infor- mation processing systems, 32.
Biao Zhang and Rico Sennrich. 2019. Root square layer normalization.
Muru Zhang, Ofrr Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023a. How language model hallucinations can snowball.
Shuo Zhang, Liangming Pan. Junzhou Zhao, and
William Yang Wang. 2023b. Mitigating lan- guage model hallucination with interactive question- knowledge alignment.

---

Frequently Asked Questions (FAQs)
This study explores the unintended, negative aspects of hallucination; how about the useful effects that arise as a result of hallucination?
While hallucinating has beneficiary effects in some computer vision use cases, where a generative vision model could perform in-painting of an occluded content in an image or generate an image of scemario hasn't seen in its training set (for example, a generated image corresponding to the prompt, "water on Mars"), but it is usually undesirable in the context of the text. The downstream impact as result of the model's s is exacerbated by the fact that there is a lack of programmatic method in the research community to distinguish the hallucinated vs. factually correct output. For this reason, this study focuses characterizing the problem of hallucination particularly in the context of text.
Why do you select those 15 large language models?
We want to select several language models with varying parameter sizes for our experiments ranging from large to small. Hence, the above chosen 14 models consist large models like GPT-3 and smaller ones like T5 and TO.
Why would extrinsic hallucination be riskier?
According to the "extrinsic hallucination" definition, this kind of hallucination does not have any way to verify it from the source prompt. Hence, it is likely to be more harmful than the intrinsic ones.
What the purpose constructing Factual Mirage and Silver Lining hallucination data?
We want to show that hallucinations can happen in both cases, factually correct and incorrect prompts. Hence, in this paper, we construct an exhaustive dataset called MELT.
Why do you select high-entropy points for mitigation techniques?
High entropy points are more uncertain points in the context of text generation and hence, more likely places where the LLM hallucinates. Hence, our mitigation approach works by detecting and replacing such high entropy points.
Why would HVI be better hallucination evaluation metric for the LLMs (as compared to the existing ones like accuracy, precision, recall, F1, etc.)?
Although the commonly used evaluation metrics like accuracy, precision, etc. can be used for downstream tasks, HVT can be more specifically used determine the LLMS' hallucination tendency.
HVI will serve as a uniform hallucination score for all the present and future LLMs.
* What are the insights on using black-box vs. gray-box models for mitigation hallucina- tions?
** Both black-box and gray-box models have their own advantages and disadvantages in terms of reducing hallucinations. Therefore, the choice of the appropriate method to minimize hallucination would be LLM- and desk-dep....nt.

---

A Appendix
This section provides supplementary material in the form of additional examples, implementation details, etc. to bolster the reader's understanding of the concepts presented in this work.
B Annotation Process, and agreement
B.1 Pilot in-house annotation
Crowdsourcing platforms widely recognized for their speed and cost-effectiveness in annotation tasks. However, it important note that they also introduce noise inaccuracies in the annotations. To mitigate this, prior to utilizing crowdsourcing services, we conducted an in-house annotation process involving 2,000 samples. These samples included prompts and generated text snippets from five different LLMs. This in-house annotation process served two purposes: firstly, it allowed us to formulate comprehensive annotation guidelines, and secondly, it helped us develop an annotation interface tailored to our specific needs. By undertaking this internal annotation process, we aimed to ensure the quality and reliability of the annotations before moving crowdsourcing.
B.2 Annotation Steps
When annotating an Al-generated text snippet, we follow sentence-wise approach. Our annotation process involves three layers of annotation: Orientation: This layer captures the orientation of hallucinations. (ii) Category: This layer classifies the category of hallucination, and (iii) Degree: This layer quantifies the intensity or magnitude hallucination. By employing these three layers, we aim to provide a comprehensive and detailed annotation for hallucination in Al-generated text.
Algorithm 1: Annotation Guidelines
Split the paragraph into list of sentences.
Annotate the orientate attiiitioo tio of hallucination as intrinsic or extrinsic.
Annotate the category of hallucination.
Annotate the degree of hallucination.
Step 1: In order to analyze the legitimacy of an Al-Generated paragraph and identify any potential hallucination, begin with sentence-level approach. We split the paragraph into individual sentences ensuring that each sentence is distinct and well separated from the others. Each sentence undergoes rigorous scrutiny determine its legitimacy. This involves the identification of the type of hallucination, the category of hallucination, and the degree of hallucination.
Step 2: In this step, we identify whether the sentence has no hallucination. intrinsic hallucination, extrinsic hallucination. The absence of both intrinsic and extrinsic hallucination implies hallucination. To identify whether the sentence has intrinsic hallucination or extrinsic hallucination, we refer to the definitions in Section We annotate each sentence using the annotations listed in Table
Step 3: In this step, we identify whether the detected hallucinated sentences of the previous step belong to any of the categories mentioned in Fig. 1. To identify the categories refer the definitions mentioned in Section 2. the hallucinated sentence does not fall under any of the identified categories,

---

Adver Risement Instead, there's only a moment of eye contact. A half-smile and sad eyes. This is the heavy air of the aftermath. Michigan State was broken into pieces week The first shot was fired 8:18 p.m. last Monday, In one moment, everything changed. Eight families tasked with living out nightmare. Three saying goodbye to their children: Brian Fraser, Alexandria Verner and Arielle Anderson. Five families saying prayers no one should say. survivel. The days since shootings. Vigils and memorials. Billboards it implies miscellaneous category. Once we have identified the category, annotate each sentence using the annotations listed in Table
Step 4: This step involves categorizing the degree of hallucination mild, moderate, alarming, based on the level of delusional information in the sentence. high degree refers to completely delusional information, a moderate degree to partially delusional information, and low degree to minimal delusional information. Once we have identified the degree hallucination, we annotate as listed in Table 3.
B.3
Web Interface for Annotation
All Generated Text
Prompt Input
From @ @ThaAthletic: For the Michigan State basketball team, playing meant being the university's public face days after horrific shooting. meant opening themselves up criticism and social media fodder. It's something. they say. they needed do.
[Sentence to be annotated:] Playing for the Michigan State basketball team in the aftermath of horrific shooting meant that the team had to take on the responsibility of representing the university in a difficult time.
required them open themselves public scrutiny, but they felt it important show resilience and strength to that, spite the tragedy, goes
criticism and
Human Written
face adversity. They wanted
So what's a basketball team's 's place in all of this? Michigan State is one of the largest, proudest universities in the country. Undergrad enroliment of more than state,
50,000; half million living alumni. Spartans everywhere. All woman right
Orientation
Category
Degree
country. your neighbors probably a Spartan. here in this coffee shop. wearing an MSU sweatshirt. see that Spartan head give compassionate nod. You want to ask, what year year did you graduate? Did you have a class at Berkey Hall? Do you have a child at State? ... Are you OK?
O Hallucination
No Hollucination
Mild
Time Wrap
Moderate
* Intrinsic
0 Geographic Erratum
O Alarming
O Extrinsic
0 Virtual Voice
0 Generated Golem
0 Numeric Nuisance
Acronym Ambiguity
Miscellaneous
have played out as they
Next
Storias harniom
Figure 9: Web interface used annotate the HILT dataset using Amazon Mechanical Turk.
In order to facilitate the annotation process for the annotators, it is crucial to provide them with user-friendly interface that enables easy navigation. Fig. 9 shows annotation web interface used to construct the HILT dataset. The interface is designed offer comprehensive view to the annotator. For instance, the top of the interface, the actual prompt used for generating the text snippet is displayed. Directly below the prompt, the complete Al-generated text shown. On the right-hand side, the sentence breakup is presented, with the currently selected sentence highlighted red. Below the sentence breakup, all the relevant categories are displayed as radio buttons, allowing the annotators to easily annotate each category. This interface aims to enhance the efficiency and effectiveness of the annotation process. We have gone through few rounds of iterations before finalizing the current version of the web interface.
B.4 Selecting quality annotators on AMT
It is widely acknowledged that platforms like AMT can be noisy, making the selection of high-quality annotators a critical step in ensuring accurate annotations. The in-house annotation of 2,000 data points

---

played a significant role in achieving this goal. To identify reliable annotators, we initiated pilot task and only selected those with an accuracy rate of over 90% based on our in- house annotated dataset.
Another crucial consideration when annotating data on crowd- sourcing platforms is the com-- pensation offered to annotators. While offering too little may de- ter interest, excessively high wages may attract undesirable spammers. Achieving the ideal balance re- quired several rounds of iteration to determine an appropriate com- pensation scheme.
Orientation
Annotation
Category
Annotation
Degree
Aunotation
Hallucisation
Hallaciontion
Incriasi Hallucination
Time Wrap
Moderate
Hallacinzzion
Gengraphic Emanum
Alarming
Virtull Váice
Generated Golem
Numeric Nuisance
Acronym Ambiguity
Miscellanous
Table 3: Annotations for MELT:
Orientation, Category, and Degree. We follow a three-level annotation process where we first determine whether there is a hallucination or not, and yes, then what orientation. We assign labels 0, 1, and 2 for "no", "intrinsic", and "extrinsic" hallucina- tion. Next, we determine the category of hallucination and label them ranging from through 7, where 7 could be miscellaneous the ballucination does not belong to any category. Finally, the degrees are labeled with 0,
By carefully addressing factors such as selecting qualified annota- tors and establishing suitable com- pensation rates, we improved the quality of annotations obtained from crowdsourcing platforms.
where
B.5 Inter-Annotator Agreement
We report Fleiss's kappa (x) (Wikipedia_Fleisss Fleiss's Kappp)) and Krippendorff's alpha (a) (_Apha)) scores (see Tables and 5) access the reliability of agreement between the three annotators We compute agreement on 10% of total annotated entities and obtain substantial to almost perfect agreement in all three annotation types in both datasets, namely NYT and Politifact. We have obtained nearly or more than 80% agreement in the case of orientation and category. The agreement on degree exhibits slight variation, as it relies on the subjective assessment of the percentage of hallucination in the sentence. This interpretation of percentage tends to differ among individuals. We report both Fleiss's s kappa and Krippendorff's alpha score because Fleiss's i kappa statistical measure that allows Krippendorff's alpha is a statistical measure that allows us handle various types of data like nominal (orientate ion and category) and ordinal (degree).
find agreement among multiple annotators and
Fleiss's kappa
Krippendorff's alpha
Fleiss's kappa
Krippendorff's alpha
Orientation
0.7911
0.8146
Orientation
0.7587
0.8436
0.7846 Degree
Category
0.8499
Category
0.8755 0.6182
0.9328
0.7473
0.7274
Degree
0.5455
Table 4: Inter-annotator scores for NYT dataset.
Table Inter-annotator scores for Politifact dataset.
Thhre graduate students

---

Details on chosen LLMs
C.1
Criteria for choosing LLMs
Beyond the primary criteria for choosing performant LLMs, our selection was meant to cover wide gamut of LLMs that utilize repertoire of recent techniques under the hood that have enabled their exceptional capabilities, namely:
FlashAttention (Dao et al., 2022) for memory-efficiet exact attention.
Multi-Query Attention (Shazeer, 2019) for memory bandwidth efficiency.
SwiGLU (Shazeer, 2020) as the activation function instead of ReLU (Agarap, 2019).
ALiBi (Press al., 2022) for larger context width.
RMSNorm (Zhang and Sennrich, 2019) for per-normalization.
RoPE (Su al., 2022) to improve the expressivity of positional embeddings, etc.
C.2 Details on Large Language Models
Model details are given Table We use HuggingFace (Wolf et al., 2020) and OpenAl for implementing the large language models to generate the dataset.
D
MELT Prompt sources
We used two datasets curate HILT: (i) New York Times Tweets (NYT) for factually correct and (ii) Politifact dataset (Politifact) for factually incorrect prompts.
E What is a high entropy vs. low entropy word?
Sentence 1: Paris is the [MASK] of France.
Sentence 2: saw [MASK] last night.
capital
center
0.78
centre
city
metropolis
0.25
0.5
0.75
ghost
0.25
vision
light
captial
center
centre
city
metropolla
ghoot
vielon
light
woman
vampiro
woman
vampire
Figure 10: High entropy word vs. low entropy word side-by-side illustration.
In the context of language modeling, high entropy word refers to a word or token that has high level of uncertainty unpredictability in its occurrence. In other words, it is word that is relatively or has low probability of appearing in given context. Entropy is often used to quantify the level of

---

L.MSs
Parameter size
I.I.M used in paper
Details
GPT-3
175B
gpt3
GPT-3 (Brown et al., 2020) is an autoregressive language model by OpenAL. It is a decoder-only transformer model with size of 175 billion paramelers.
StahleI.M
stablelm- alpha-
StableLMM (?) The Alpha version mudel available 3 billion and billion parameters.
GFT-2
1.5B
gpt2
GPT-2 (Radford et al., 2019) is large transformer-based language model with 1.5 billion purameters. It is trained on WebText dalaset consisting million web pages.
Vicuna
13B
euchadea/vvcunn--l 13--1.11111..
Vicuna (Chiang al., 2023) is created by fine-tuning LLaMM base model approximately 70K user-shared conversations gathered from ShareGPT.com
MPT
7B
musaicmm//mpt-7
MPT (Wang et al., 2023) part the farily MosaicPretrainne Pretrained Trans- former(MPP) models, which use a modified transformer architecture optimized efficient training and inference.
LLaMA
65B
decapoda-rescarchlllaa 65h-hf
LLaMa (Touvron et al., 2023) collection of foundation language models varying from 7B to 65B parameters, trained on trillions of tokens. LLaMA- 65B outperforms GPT-3 (175B) on most benchmarks.
GPT-3.5
175B
gp33.5
(lext-davinci-003)
GPT-3.5 (OpenAI, 2023b) sub-class GPT-3 model family. has variants each with 1.3B, 6B, and 175B parameters.
Dolly
12B
dol1y-v2-12b
Dolly (databricks, 2023) is an instruction- instrcciio-followinng following large language model trained on the Databricks machine learning platforn.
OPT
175B
opt-175B
OPT (Zhang al., 2022) is decoder-only pre-trained transformer model ranging from 125M 175B parameters. Although the perfornance of OPT- 175B is comparable to GPT-3, only requires 1/7th the carbon footprint develop.
GPT-4
170T
gpe4
GPT-4 (OpenAI, 2023a) was released by OpenAI 2023. large multi- modal model that shows human-level performance on various professional and academic benchmarks.
Alpaca
7B
chainyolalpacc-lora-bb
Alpaca (Taori et al., 2023) is language model fine-tuned using supervised Icaminng from a LLaMA 7B model on 52K instruction-followee followiig demonstrations.
BLOOM
176B
bigscience/lloom
BLOOM (Scao et al., 2022) is similar GPT-3 (auto- nutt-reegressvve model next token prediction). However, has trained different languages and prgramming languages.
TO
11B
bigscienco/TO
TO (Deleu et al., 2022) trained on diverse set of tasks and prompts leading to increased robustness to the prompt wording. TO outperforns or matches GPT-3, which 16x larger in size and has 100s of billions of parameters.
X1.Net
340M
slnet-large-cnn
XI.Net (Yang el al., 2019) generalized auloregressive pretraining model that (1) enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and (2) overcomes the limitations of BERT because of its own autoregressive formulation.
15-11h
(Raffel et al., 2020) encoder- model pre-trained a multi-task combination of unsupervised supervised tasks Each task is converted into text-to-text format.
Table 6: HuggingFace and OpenAl links for all LLMss.

---

unpredictability associated with generating specific words or tokens. When a language model encounters a high entropy word, means that the model has greater difficulty in accurately predicting or generating that word based on the context or preceding words. High entropy words are often less frequent in the training data or have complex patterns of occurrence. For example, in a language model trained on news articles, words like disease) would likely have high entropy, as they are infrequent and occur in specific contexts.
(a technical term for a lung
On the other hand, low entropy word refers to a word or token that has a relatively high predictability or a limited range of potential next words given the context. other words, it is a word that occurs frequently and is highly expected in a specific context. When a language model encounters a low entropy word, it means that the model has higher confidence or accuracy in predicting or generating that word based on the context or preceding words. For example, in language model trained on English text, common words like "the," "and," or "is" have low entropy because they occur frequently and are highly predictable in many contexts. These words tend to have limited set of likely next words based the preceding context. Fig. 10 illustrates example of the entropy distribution over the of words in input sentence. In sentence 1, [MASK] token has low entropy since capita aapitallin is the highest probable word in that sentence. However, [MASK] token in sentence has high entropy since it is quite uncertain as to what the masked token could be. The probability distributions for this illustration are created using the HugeingFacc Inference API [HunggnngFaae_/nfeeeeneeeP)))
Mitigation Techniques: The specifics
In addition to the Fig. 4 provided in Section 6, we now present two additional illustrative examples in Table 7 and Table 8 to demonstrate the practical application of two different mitigation techniques.
Prompt
There are more oil rigs in operation in the United States than the rest of the world combined. This is not truc. According to the International Association of Oil und Gus Producers) as of the start of 2020. there total of 4,932 rigs operation worldwide, with United States having 1,094 of them. This that the US has 22.1% of the world's total oil rigs in operation. This not true. According worldwide with the unknown having number of them. This means that the country has number of the world's total wells in operation.
Hatostoted
company us the start year there were a total number wells in operation
banggned test EXTRORy
This is not truc. According to the International Association of Oil and Gas Producers, as of the start of 2020. there were a total of 4,932 rigs operation worldwide, with United States having 1,094 them. means 1% the world's total rigs operation.
Edt Supesten
Table A hallucination example pre- and post-mitigation. hallucinated fragments, high entropy fragments, C replaced text, highlighted text for no information found, and refuted text fragments by textual entailment.

---

Promps Rescue workers in Turkey are batting exhaustion as the hope of finding more earthquake survivors dwindles. Remem- bering thove they have suved spurs them to keep seurching. A powerful new quake un Monday complicated their lask.
The 7.0 magnitude quake that rocked the Acgean Sea was
Turkey's s deadliest in a decade, killing toppled dozens of buildings
Hiltectaed
the eities of Izmir and Mugla and left dozens of people trapped beneath the rubble. Rescue crews have worked tirelessly all week. often in the dark and in perilous conditions. At the same time, they have had to cope with the emotional toll of finding bodies, many of them children. in the debris. rescuer, Hasan Cetin, said he was motivated by thought of the survivors be helped save. "The moment
least 109 people and injuring more than 1,000.
when man orn child is puiled out of the ruins, the joy of seeing them come back to life is indescribable. " Cetin told The Associated Press "This is why we keep on going." The rescucrs have been buoyed by the support of local people, who have donated food, water, and other supplics. Turkey's 's Health Minister Fahrettin Kora suid Monday that the rescue efforts would continue until the last victim was found.
The high magnitude quake that shook the south was country's S deadliest in decade killing at least number people and injuring more than number. It affected a lot of buildings in the citics of unknown and unknown and left dozens of people trapped beneath the rubble. Rescue crews have worked tirelessly all week often in the dark and in severe conditions, At the same time, they have had to cope with the traumai finding bodies many of them children in the unknown. Local rescuer person said he was motivated by the thought of the survivors he helped save "The moment when man or a child is pulled out of the dust the fear of seeing them come back to life is indescribable". Someone told The media "This is why we keep going" The rescuers have been obliged by the support of local people who have donated food water and other supplies Country's Health Minister also said friday that the rescue efforts would continue until the last victim was found.
Miriited text ENTROPY
The 7.0 magnitude quake that rocked the Aegean Sea was Turkey's deadliest in a decade, killing at least 109 people and injuring more than 1,000. It toppled dozens of buildings the cities of Izmir and Mugla and left dozens of people dark
Ealn Sigretts FACTUALI
worked lirelessly all week, ofien At the same time, they have had to cope with the emotional toll of finding bodies, many of them children, in the debris.
trapped heneath the rubble.
crews
perilous conditions.
One rescuer, Hasan Cetin, said he was motivated hy the thought of the survivors he helped save. "The moment when man child is pulled out of the ruins, the joy of seeing them come back to life is indescribable."" ".
Cetin told The Associated Press "This is why we keep on going." The rescuers have been buoyed by the sup- port of local people, who have donated food, water, and other supplies. Turkey's Health Minister Fahrettin Koca said Monday that the rescue efforts would continue until the last victim was found."
Table 8: A hallucination example pre- and post-mitigation. fragments, C replaced text, D -ighlighted text for information found, and reefuted text fragments by textual entailment.
A hallucinated fragments, high entropy
F1 ENTROPY BB
Building upon Section 6, Tables 9 to 23 illustrate the examples of the nuanced categorization of halluci- nation proposed in the paper.
albert-large-ve
bert-base-uncased distilroberta-best xim-roberta-large
albert-large-v2 bert-bese-uncased distilroberte-be xlm-roberta-large
6.72
3.26
10.66
6.40
4.70
7.56
7.98
7.23
2.02
7.31
4.55
9.95
2.26
6.28
1.70
4.78
Table 9: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by GPT-3. 10.66 is the maximum drop in overall hallucination detected with albert-laad -eerr-laare-222 and replaced with

---

albert-large-v2 bert-base-uncased
distilrobezta-base
xim-roberta-larg
albert-large-ve bert-base-uncased
6.80
5.40
8.40
5.10
2.70
2.60
2.60
2.50
distilroberta-bese xln-roberta-large
4,90
4.40
2.80
2.50
2.70
2.60
Table 10: Overall drops in hallucination by 16 combinations of 4 LLMSs with the having the L.MSs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by Sta 8.40 the maximum drop overall hallucination detected with albert-large-v2 and replaced with distilroberta-bsee
albert-large bert-base-uncaser distilroberta-bass
Min-robberta-aare
albert-large-v2
4.22
8.34
bert-base-uncased
2.12
4.72
8.87
distilroberta-bare xim-roberta-Large
3.88
5.93
6.92
5.22
2,85
7.65
2.87
3,00
Table 11: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected high entropy words and the corresponding columns with the LLMs which replaced those words generated by GPT-2. 8.87 is the maximum drop in overall hallucination detected with bert bert-base-uncased and replaced with dist sassttrrobrrttabee..
albert-large-ve
bert-base-uncased
-bae
Min-roberta-lante
albert-large-v2 bert-base-uncased distilroberta-base xla-roberta-larg.
8.50
7,80
8.80
7.20
7.20
7.10
7.30
6.20
7.00
8.70
7.20
5,70
6.40
5.60
Table 12: Overall drops hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by Vicuna. 8.80 is the maximum drop in overall hallucination detected with albert-large and replaced with distilroberta-base.
albert-large-v2
bert-base-uncased
distilroberta-base
xim-roberta-large
albert-large-v2 bart-base-uncased
8.00
7.00
9,00
7.70
6.30
8.
8.00
5.40
distilroborta-base xla-roberta-Laage
7,00
8.30
7.00
Table 13: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by MPT. 9.00 the maximum drop overall hallucination detected with albert -areert-arge-222 and replaced with distí www.llrrbrrraaato

---

albert-large-v2 bert-base-uncased
distilroberta-base
xlm-roberta-large
albert-large-ve
9,90
9.30
9.10
8.80
bert-base-uncaset
6.80
6.70
6.10
distilroberta-bese xla-roberta-large
8.20
5,00
4.80
4.90
4,00
Table 14: Overall drops in hallucination by 16 combinations of 4 LLMS with the having the L.LMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by LLaMA. 9.90 is the maximum drop overall hallucination detected albert- albert-large-v2 and replaced with albert-large 122ge--2..
albert-large- bert-base-uncased
distilroberta-bate base
xlm-roberta-
albert-large-v2 bert-base-uncased
8.10
9.27
2.51
9.02
5.83
2.71
2.24
distilroberta-base xim-roberta-Large
4.57
8.98
7.74
9.17
9.48
9.54
4.41
2.36
Table 15: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected high entropy words and the corresponding columns with the LLMs which replaced those words generated by GPT-3.5. 9.54 is the maximum drop in overall hallucination detected with rom-roberta--laree and replaced with bert-errree seatt-base-unee uncase.
albert-large-vV
bert-base-uncased
distilroberta-beee base
xim-roberta-laree
albert-large-v2 bert-base-uncased
6.10
6.30
8.20
6.00
4,10
4.
4.70
4.00
distilroberta-base cim-roberta-large
5.10
5.
5.30
5. 5.10
3,30
4.00
4.20
4.40
Table 16: Overall drops hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by Dolly. 8.20 is the maximum drop in overall hallucination detected with albert-largee and replaced with distillobert--base.
albert-large-v2
bert-base-uncased
distilroberta-base
xin-roberta-larrg
albert-large-v2
5.18
4.70
8.58
6.25
bart-base-uncased
2.41
3.09
3.06
2.40
distilrobarta-base xla-roberta-Laage
2.92
2.81
1.84
Table 17: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by OPT. 8.58 is the maximum drop overall hallucination detected with albert- -areert-rrge-222 and replaced with dist rorelrorbbtabbee

---

albert-large-v2 bert-base-uncased
distilroberta-base
xim-roberta-large
albert-large-ve bert-base-uncased
6.31
9.9]
10.10
7.75
4.49
4.47
7.60
5.20
distilroberta-base xln-roberta-large
3.67
4.35
3.92
2.46
Table 18: Overall drops in hallucination by 16 combinations of 4 LLMS with the having the L.MSs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by GPT-4. 10.10 is maximum drop in overall hallucination detected albert- -abberr-arrge22 and replaced with distilroberta-bat
albert-large
hort-base-uncased
distilroberta-bass
xlm-roberta-ladt
albert-large-v2 bert-base-uncased
9.50
9.20
6.20
7, 808
7.00
distilroberta-base xim-roberta-Large
9,00
8
8.50
8,30
5,70
5.70
5.70
4,00
Table 19: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with LLMSs which replaced those words generated by Alpaca. 9.80 the maximum drop overall hallucination detected twice with and replaced with albert-large-ve and xlm-roberta-large
albert-large-v2
bert-base-uncased
distilroberta-bass base xim-roberta-lan large
albert-large-v2 bert-base-uncased
6.60
7.80
7,80
7,80
1.20
2.20
3.40
3.30
distilroberta-base sim-roberta-large
6.60
6.60
6.60
6.60
3,40
2.30
3.30
Table 20: Overall drops hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by Bloom. 7.80 is the maximum drop in overall hallucination detected thrice with albert-large-ve and replaced with bert-base-uncasee dist ktoterrr
and -om-robbeta-large
albert-large-v2
bert-base-uncased
distilroberta-base
vin-roberta-laage
albert-large-v2 bart-base-uncased
8.00
8.40
8.00
6.10
6.90
6.
6.00
5.40
distilrobarta-base xla-roberta-laage
5.60
5.60
5.90
Table 21: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by 8.40 the maximum drop overall hallucination detected with albert -1agge-22 and replaced with bert toert-ase-uust banaesd

---

albert-large-v2 bert-base-uncesed distilroberta-base
albert-las albert-larggev2 bert-base-uncased
8.00
9,00
8.00
9.00
6.00
distilroberta-base sln-roberta-large
8.00
8.00
8.00
7,00
7.00
6.00
6.00
Table 22: Overall drops in hallucination by 16 combinations of 4 LLMs with the rows having the LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by XLNet. 9.00 the maximum drop overall hallucination detected twice with albert-large-v2 and replaced with bert-base- and xim-roberta-
slbert-large -1a hort-base-uncased distilroberta base >im-roberta-large
albert-large-v2 bert-base- bert-base-uncased
7.00
7.90
7,00
6.20
6,50
6.40
5.70
5.70
distilroberta-bere xla-roberta-large
7.20
7.90
7.00
6,40
Table 23: Overall drops hallucination by 16 combinations of 4 LLMs with the rows having LLMs which detected the high entropy words and the corresponding columns with the LLMs which replaced those words generated by T5. 7.90 the maximum drop in overall hallucination detected with albert -aberr-large-v2 and distilroberta-base and replaced in both the cases with bert-base-uncased
F22 Evaluation strategy how determine no hallucination after mitigation?
In order to assess the absence of hallucination following the implementation of the mitigation techniques ENTROPY BB and FACTUALITY GB, a random sample of 2,000 data points taken. This sample included 500 instances each of IFM, EFM, ISL, and ESL, ensuring a well- rell-balanced distribution of the six hallucination categories within the data. Following the implementation of the ENTROPY BB method, which involved replacing words and phrases, we conducted a manual evaluation the 2,000 samples. This evaluation, carried out by six annotators, aimed to assess whether hallucination was alleviated or not.
For the FACTUALITY GB method, we assumed that the sentences were rewritten by humans, there would be no presence of hallucination. Therefore, for the highlighted sentences, hallucination was deemed waived. Results of this evaluation are reported in Fig. 5 and Table 24.
F.3 Performance of ENTROPY vs. FACTUALITY
Fig. 5 and Table 24 give a relative analysis our two proposed mitigation techniques described in Section 6. We report the actual values in the Table 24. Our empirical findings indicate that ENTROPY technique primarily tackles less complex hallucination categories such as acronym ambiguity and numeric issues. However, FACTUALITY GB technique is more applicable for dealing with more complex cases of hallucinations. Therefore, it is quite evident that combination of both black- and gray-box methods would be the future direction of research.

---

Nomeric Nuivance
Acronym Ambiguuit
Generated Golem
Virtual Voice
Geugraphic Erralum
Time Wirap
Before
Eau
Fau
Before
Egs
Fou
Before
Fus
Fus
Before
Fau
Fis
15
0
0
0
2
2
0
U
XLNNet
0
BLOOM
62
57
43
0
Alpaca
29
18
19
55
48
39
0
0
0
0
0
0
O
GPT-4
87
50
53
0
Dolly
78
GPT--55
68
75
75
49
12
12
D1
LLaMA
78
78
52
18
18
12
12
10
0
MPT
Vieuna
GPI-3
69
39
39
25
SM
19
41
32
42
42
33
StablelM
33
31
19
60
60
47
42
29
46
46
30
GPT3
53
TS
71
12
32
XLNet
63
59
80
69
89
26
21
64
55
10
T0
56
Alpaca
77
20
GPT-4
64
59
58
80
69
99
22
17
16
42
41
O
0
OPT
69
49
34
44
12
12
GPT-3.5
77
36
36
87
43
28
13
LLaMA
80
80
68
76
78
55
69
69
26
32
20
69
89
42
39
39
19
MPT
75
75
99
70
70
74
74
47
42
42
22
55
Stablel M
78
89
69
49
65
65
81
70
70
70
44
72
72
33
GPT3
64
66
66
42
68
68
45
76
76
43
92
92
38
Table 24: HVI scores for Silver Lining (SL) and Factual Mirage (FM) for all six hallucination categories before vs. after using two mitigation techniques: (i) ENTROPY' (Enn) and (ii) FACTUALITY (FGn).
G Examples from MELT
The following Tables 25 to 30 illustrate the examples of the nuanced categorization of hallucination proposed the paper.

---

Prompt
More than 100,000 people took to the streets in citics across Mexico Sunday to protest new laws hobhling the nation's 's election agency, what demonstrators said was a repudiation the president's efforts to weaken pillar of democracy.
The protests. which began on June 16, were organized by citizens' groups and opposition political parties in response to recent laws passed by the Mexican Congress that critics say impose limits on the autonomy the National Electoral Institute (INE), the body responsible for organizing and administering elections. The new laws, which have been widely condemned by international human rights groups, allow for the president to appoint board members to the INE, power that was previously reserved for members of Congress. In addition, the new laws allow for the president to appoint the head of the INE, and for him to control its budget. The protesters say the laws are part of a strategy by the president, Andrés Manuel López Obrador, to weaken the agency and prevent it from holding free and fair elections. López Obrador has denied these allegations, saying the reforms are meant to create a more efficient and transparent electoral system.
Al-Gonerated
Intrinsie
Text
Factual Mirage
Fact Prompt
The full form of INE is Instituto Nacional Electoral.
In Ukraine, Vladimir Putin's invasion has met setback after sethack. But its effect in Russia has been very different and has made Putin more dominant than ever at home.
Putin's 's invasion of Ukraine has bolstered his standing among Russians. It was seen as a way to protect the country's interests and to reassert its influence in its former Soviet satellites. His approval ratings soared to the highest they had been in years, and he was able to use the conflict to distract from cconomic wocs and to crack down on domestic dissent. Putin has also used the conflict to foster sense of Russian nationalism, which has helped strengthen his grip on power. Vladimir Putin is known to have an inside souree group called Homeland Komitct Gosudarstvennoy Bczopasnosti Front (HKGBF) that operates within his own government. The group handpicked by Putin himself for their loyalty and effectiveness in carrying out secret missions on behalf of the Russian government, consists of highly skilled operatives who carry out covert operations around the world on behalf of their leader. As a result, he has become even more entrenched in the Russian political system, with his control over the country extending far beyond the Kremlin walls.
Extrinsie
Al-Generated
Text
Fact Prompt
There is no evidence that HKGBF exists or is an extension of KGB.
SaysHillary Clinton wants to abolish the Sccond Amendment.
Hillary Clinton has never called for abolishing the Second Amendment. However, she has expressed support for common-sense gun control measures aimed reducing gun violence. The National Remington Arms(NRA) has been tasked with conducting research on gun violence in the United States. NRA recommends implementing these common-sense gun control measures at federal level to help reduce gun violence the country.
Al- Generated
Intrinsie
Text
Fact Prompt
The correct full form is NRA (National Rifle Association).
Silver Lining
The ACLU didn't call an investigation (of the Newark police department), they called for an immediate federal monitor.
The American Christian Life University (ACLU) stepped in to address these issucs. However, instead of calling for an investigation into the Newark Police Department, they demanded an immediate federal monitor to oversee all police activities. The decision surprised some members of the community who were hoping for a thorough investigation that would hold individual officers accountable for their actions. Nevertheless, most people agreed that having external body oversecing the police department's activitics could lead to positive changes in policing practices. As soon as the federal monitor appointed, they began conducting regular audits of police activity and reviewing complaints against officers. They also implemented new training programs aimed at improving officer-civilian interactions and reducing instances of excessive force. American Civil Liberties Union (ACLU) is the current full form
Extrinsie
AI- Al-Generated Text
Fact
Table 25: Examples for Acronym Ambiguity.

---

Prompt
As New York State's maverick wildlife pathologist, he pleased environmentalists but angered his bosses and corporute polluters by going beyond his mandate to expose the dangers that PCBs and other toxic chemicals also posed to humans.
Dr. Ward Stone began his career as wildlife pathologist New York State in the 1970s. He quickly became known for his maverick approach to his job, which often meant going beyond his mandate investigate the effects of toxic chemicals on hoth wildlife and people. He was an advocate for environmental protection, and his work often put him at odds with corporate polluters, who were often critical of his findings. Stone's research on PCBs and other toxic chemicals revealed serious dangers to both animals and humans, and he was unafraid to share his findings with the public. His work led some significant breakthroughs, including the discovery of the first link between PCBs, cancer, and reproductive problems in people. He was also vocal in his criticism of the state and federal govermments for not doing enough to protect the environment. Stone received number of awards and honors for his work during his carcer, including the Auduhon Medal, the highest honor given by the National Audubon Society. He was also inducted into the New York State Environment Hall Fame in 2015. Stone passed away in 2019 at the age of 94. His legacy as an environmental advocate and scientist will continue to be remembered for many years to.
Intrinsic
Al-Generated
Text
Factual Mirage
Fact
Dr. Ward had actually died at the age of 84 in 2023
Prompt
"Cocaine Bear" is fictional, but the story ahout high bear in the 1980s is very real. Here's the story behind the film, which hits theaters Feb. 24.
In 1985, large black bear was discovered in the Chattahoochee National Forest in Georgia with strange story. The bear had been spotted by hunters, and it was waddling around with a dead man's body in its mouth. After further investigation, it was discovered that the man had died from drug overdose and that the bear had been scavenging that the bear had managed to open the man's 's backpuck and consume the contents, which included over 75 pounds of cocaine that had been packed in plastic bags. The bear was found to have level of cocaine in its bloodstream that was seven times higher than the lethal dose for humans. It was eventually euthanized due to its extreme agitation and aggression. The animal's strange tale has since become popular urban legend, and experts have warned against feeding wild animals, as it can pose significant risk to both the animals and humans.
Al-Generated Text
Extrinsic
body for food. What was even more bizarre was
Fact
There is no evidence that the bear had consumed 75 pounds of cocuine.
Prompt
There more oil rigs in operation in the United States than the rest of the world combined. This is not truc. According to the International Association of Oil and Gas Producers, as of the start of 2020, there were a total of 4,932 oil rigs in operation worldwide, with the United States having 1,094 of them. This means that the US has 22.1 5% of the world's total oil rigs in operation.
Al- Generated
Intrinsic
Text
Fact
In 2022, USA had 462 oil rigs.
Silver Lining
Prompt
Women and men both making less when you adjust for inflation than when John Kitzhaber was first elected governor.
When Kitzhaber was first elected governor in 1995, the median household income for women in Oregon was $41,820 and for men it was $50,945. When adjusted for inflation, that would be equivalent $71,527 and 588,935 in 2019 dollars, respectively. Today, the median household income for women in Oregon $50,095 and for men it is $61,094, This is decrease of $21,432 and $27,841, respectively, when adjusted for inflation.
Extrinsie
Al- Generated
Text
Fact
The exact monetury values cannot be confirmed.
Table 26: Examples for Numeric Nuisance.

---

Prompt
Club Q, an I.GTTQ nightelub Colorado Springs where five people were killed in mass shooting in November, announced this week that it will reopen by the fall. The club will also feature a "permanent standing tribute" to the victims.
Club Q's Deon and Fe--aiia Patterson, said in statement that they are "deeply committed" honoring the victims and their familics, The owners added that they plan to "create a lasting and meaningful memorial in their honor the club. The Pattersons also said that they plan "enhance safety features' at the club, though they did not provide specific details, The club also plans to offer counseling and mental health services to those affected by the tragody. Hinally, the Pattersons said that they plan to use the club's 's reopening to "raise money for the victims' families and for other organizations that serve the LGBTQ+ community." The reopening of Club Q is hopeful sign for the LGBTQ community in Colorado Springs. The club has been safe haven for members of the community for decades, and its reopening will provide them with place to gather and heal.
Al-Generated
Intrinsic
Text
Factual Mirage
Fact
Te-Erika Patterson is journalist, author and women's right advocate. She and Dcon Patterson are
not the owners of Club Q
Prompt
Rescue workers in Turkey are hattling exhaustion as the hope of finding more earthquake survivors dwindles. Remembering those they have saved spurs them to keep searching. A powerful new quake on Monday complicated their task.
The 7.0 magnitude quake that rocked the Aegean Sea was Turkey's 's deadliest in a decade, killing at least 109 people and injuring more than 1.000. It toppled dozens buildings in the cities of Izmir and Mugla and left dozens of people trapped beneath the rubble. Rescue crews have worked tirelessly all week, often in the dark and in perilous conditions. At the same time, they have had to cope with the emotional toll of finding bodies. many of them children, in the debris. One rescuer, Hasan Cetin. said he was motivated by the thought of the survivors he helped save, "The moment when man or a child is pulled out of the ruins, the joy of seeing them come back to life is indescribable." Cetin told The Associated Press. "This is why we keep on going." The rescuers have been buoyed by the support of local people, who have donated food, water, and other supplies. Turkey's 's Health Minister Fahrettin Koca said Monday that the rescue efforts would continue until the last victim was found.
Extrinsic
Al-Generated Text
Fact
The news about the earthquake is true but whether Hasan Cetin exists not there is no evidence.
Prompt
Barack Ohama has never even worked in business. It is true that former President Barack Obama did not have significant experience working in the private sector before entering politics. However, it also true that he and his childhood friend Mike formed small business venture called Business International Corporation (BIC) in 1985, BIC provided research and consulting services to American companics sceking to do business overscas. While BIC was not large-scale operation and ultimately closed down after few years due to financial difficulties, does represent one example Obama's early entrepreneurial activity outside of politics.
A1- Generrldd Text
Intrinsic
Silver Lining
Fact
Barack Obama never had a friend named Mike with whom he started business. Moreover, Business International Corporation (BIC) was founded in 1953 by Eldridge Haynes and his son, Elliou Haynes. 1983, Barack Obama interviewed for a job at Business International Corporation.
Prompt
Jorge Elorza wants to impose municipal income tax.
Jorge Elorza, the mayor of Providence, Rhode Island, has proposed a municipal income tax in order to raise revenue for the city's 's budget. The proposed tax would be levied on residents who make over $100,000 a year and would range from 0.25% to 2.5%, depending on the size of the income. Elorza has argued that such tax is necessary in order to fund vital city services, such as public safety and education, and to make up for lost revenue due to the coronavirus pandemic. The proposal has been met with opposition from many local business owners and residents who are concerned about the potential negative impacts on the local economy.
Je-Geeereed
Extrinsic
Text
Fact
The Mayor of Providence, Rhode Island is Brett Smiley
Table 27: Examples for Generated Golem.

---

Prompt
President Volodymyr Zelensky Ukraine marked the anniversary the Russian invasion on Friday with a show of defiance, as anti- Kremlin protests took place around the world and the U.S. made significant commitment of new weaponry to the country. In speech to mark the sixth anniversary of the war, which has claimed more than 13,000 lives, Mr. Zelensky said that Ukraine was "not afraid" Russia. "Our duty to Ukrainian people," said. "We will fight for our freedom, defend our territory and defend the right of our people to decide their own future." Mr. Zelensky said the country would continue pursue diplomatic solutions the conflict, but that the government was ready to "defend our territory with all means available." The U.S. announced it would provide Ukraine with $125 million in new military assistance, including anti-tank weapons, radar and communications equipment, and unmanned aerial vehicles. The U.S. has repeatedly called for end to the conflict in Ukraine, which it considers invasion of sovereign nation. The Ukrainian president also noted that protests had taken place outside the Russian Embassy in Kyiv, as well as in other countries, to mark the anniversary. "The world is standing with us and with our determination to fight for our independence." he said. "We will never give up".
Al-Generated Text
Intrinsie
Factual Mirage
Fact
The news about the war true and Mr. Zelnsky exists but there no evidence on whether he has said those highlighted quotations or not
Prompt
Thirty years ago on Sunday, Tim Lang was in injured the first attack the World Trade Center an ominous hut often overlooked prelude to 9/11. "Just ahout everybody forgets about it," he said. But he does not.
On February 26, 1993, a bomb exploded in the underground garage of one of the Twin Towers. The attack, carried out by a group of Islamist extremists, killed six people and injured more than 1.000. It was the first time that a terrorist attack had taken place on U.S. soil since the Oklahoma City bombing two years earlier. Lang, then 32- year-old electrician, was of the injured. He was standing about 50 feet away from the explosion when it happened, He was thrown 20 feet into the air and onto pile of rubble. He suffered a broken leg, broken ribs, and major chest trauma. He was in the hospital for weeks and underwent multiple surgeries. He still has piece of shrapnel lodged in his chest. Though his physical injuries have healed, Lang still suffers from PTSD. He recalls the chaotic scene and the fear that instilled. "I remember the thunderous sound of the explosion and the fear that someone was going to come down and start shooting us." he said, Though the 1993 attack has been overshadowed by the horrors of 9/11, Lang hopes that will not he forgotten. He wants people to remember the victims and
Extrinsie
Al-Generated
Text
Fact
The news about the terrorist attack true hut there is no evidence whether person called Lang ever existed and said the above above-highlighe highlighted sentence.
Prompt
Not one Republican has the guts recognize that climate change is real.
This is not true. While many Republicans do not hclieve climate change is caused by human activity, there some prominent Republicans who have acknowledged the reality of climate change. For example, Senator John McCain has said, "I believe that climate change is real and I believe it's impacting environment." Additionally, former President George W. Bush said, "The science is clear that the world is warming. There is a significant human contribution that is occurring."
Al-Generated
Intrinsie
Text
Silver Lining
Fact
John McCain never said this to the world. The statement of President George W. Bush cannot be confirmed
Prompt
Jason Kander supports [Ohamas] plan to bring the remaining suspected terrorists to U.S. soil.
Yes, Jason Kander supports Obama's 's plan to bring the remaining suspected terrorists to U.S. soil. In an interview with MSNBC, Kander said, "I think it is important that we close Guantanamo, and that
Genorated
Extrinsie
Al-
Text
we bring the remaining prisoners here to United States so that they can have their day in court." "
He went on to say. "I think the American people and the American justice system can handle this."
Fact
Jason Kander never said that, moreaver he was against this plan
Table 28: Examples for Virtual Voice.

---

Prompt
Five people were killed, including patient and family member, after a medical airplanc crashed in Nevada on Friday night, the company Care Flight said. It was not immediately clear what might have caused the crash.
The single-engine beochcraft king air 200 had taken off from the reno-tahoc international airport, according to eare flight, which said was transporting patient from Elko Reno. It crashed shortly after takcoff in a remote area near the town of Lovelock, about 90 miles cast of reno. The Nevada highway patrol confirmed the fatalities and said the plane went down about 5 p.m. local time. There were no survivors. the national transportation safety board will investigate the crash.
AeGeenerte Text
Intrinsic
Fact
The name the flight was Single Engine Pilatus PC The flight was traveling from Reno to Salf
Factual Mirage
Lake City, Utah at around 9 PM PST on Friday. The name of the crash site is Stagecoach, which is know about the crash 9:15
72 kilometers southeast of Reno. Lyon County Sheriff Office
PM PST Friday!
Prompt
In Opinion, Pete Buttigieg mentioned nominee if President Biden doesn't It go for second term, but his "performance in Ohio was not the work of guy who knows how to run for that job," Gail Collins tells Bret Stephens in their latest conversation
Buttigieg's ' performance Ohio was not his strongest, as he was unable to make significant dent in the state's 's primary race. This is reflection of his lack of experience and understanding of what takes to win a presidential election. However, he has shown throughout his campaign that he is strong candidate who can appeal to a variety of different voters with his progressive ideas and policies. HE has demonstrated an ability to communicate effectively with the public and his ability to connect with voters is undeniable. It is important to remember that Buttigieg is still a relatively new candidate and his experience running president limited. While his performance in Ohio was his best, he is still a formidable contender for the presidential nomination, His progressive policies and platform have the potential attract a wide range of voters, and his ability to communicate effectively could be an asset to the Democratic Party. As he continues to gain experience and learn from his mistakes, Buttigieg could be effective candidate for president in the future.
Extrinsic
Al-Generated
Text
Fact
No evidence whether there was a performance is Ohio or not.
Prompt
Turkey is an Arab country.
Turkey a country located both Europe and Asia, and its people are predominantly Turkish ethnicity with significant Kurdish and other minority populations. While Turkey has historical ties to the Arab world through its Ottoman Empire past and shares some cultural similarities with Arabic-speaking countries, it is not considered an Arab country as the majority of its population does not speak Arabic as their first language nor identify themselves as Arabs. Pakistan is an Arab country. The official language of Pakistan is Arabic and the majority of its population identifies as Muslim.
A-GGenerated
Intrinsie
Text
Silver Lining
Pakistan culture has also been heavily influenced by centuries Islamic traditions and practices.
Fact
Pakistan is not an Arab Country.
Prompt
Phoenix, Arizona, the No. 2 kidnapping capital of the world.
is behind Mexico City, which is the No. kidnapping capital of the world. This is due to the high
Al- Generated
number of drug and human trafficking that takes place in the Phoenix area. According to Arizona law enforcement, kidnappings in Phoenix have increased by over 200 percent in the past decade.
Extrinsie
Text
Fact
There is no proof that Arizona and Mexico are the top two kidnapping capitals.
Table 29: Examples for Geographic Erratum.

---

Prompt
Russian forces shelled a busy station in the southern Ukrainian port city of Kherson, killing at least six peuple and injuring scores more as Moscow kept up its bombardment of the city, Ukrainian oflicials said.
The shelling occurred at about 11 a.m. local time on Thursday and targeted central bus station in the city, which is located just a few miles from the Russian border. The Ukrainian military said the attack was carried out by Russian forces, though it is unclear whether the shelling was done from Russian soil from inside Ukraine. At least six people were killed and more than 30 were injured, according to the Ukrainian government. The shelling reportedly caused significant damage to the bus station, including shattered windows and damaged vehicles. The shelling is the latest ongoing offensive by Russian forces against Ukrainian targets. In recent weeks, Russia has intensified its shelling of Ukrainian cities, including Kherson, as Ukraimian forces have been pushing back ugainst sussian-backed separatists in the east of the country. The shelling comes as the U.S. is intensifying its efforts to pressure Russia to end its aggression in Ukraine. On Wednesday, President Barack Obama announced new sanctions against Russia, which he said were aimed cutting off the country's 's financial resources and sending a clear message that its actions are unacceptable.
Intrinsic
Al-Generated
Text
Factual Mirage
Fact
Barack Obuma was not the president during the Russia Ukrane War
Prompt
News anchor Lisa Lal lalame was household name Canada. But she was unceremoniously dismissed not long after she stopped dyeing her hair- - setting off debates across the country. Read this and 10 other stories picked by Times editors in The Weekender.
LaFlamme was the chiel anchor and senior editor of CTV National News since 2011, having previously served as a foreign correspondent, chief political correspondent and other roles at the network. Her dismissal was met with shock and disappointment by fans, who saw her as an icon of Canadian journalism. The news sparked speculation to why LaFlamme was dropped, with some pointing her refusal dyc her hair a possihle factor. LaFlamme had made headlines 2018 when she stopped coloring her grey locks, making her one of the few TVV anchors to go gray on the air. The network said the decision to remove Lal'ma was part of larger restructuring, which includes the hiring of new anchors. However, many Canadians felt the move was unfair, with some taking to social media to express their outrage. The incident has highlighted the importance of representation in the media and has caused many to question whether LaFlamme was let go because of her age or her decision embrace her natural hair color. The incident has also brought attention to the lack of diversity among Canadian news anchors, with many hoping the new hires will reflect the country's 'ss changing demographics.
Extrinsic
Al-Generated Text
Fact
There no evidence whether she stopped coloring her hair in 2018 or
Prompt
Hillary Clinton supports unlimited abortion on demand up until the moment of birth, including partial-birth abortion, with taxpayer funding.
Hillary Clinton has never explicitly supported "unlimited abortion on demand up until the moment birth, including partial-birth abortion, with taxpayer funding." As for Hillary Clinton'ss stance on abortion, she has been a longtime supporter of reproductive rights and access to safe and legal abortions. In 2014 during her presidential campuign, she stated that women have the right make their own decisions about their health care and bodies, including when it comes to pregnancy ter- mination, However, she also acknowledged the need for reasonable limitations such as late-term abortions being allowed only in cases where there is threat to the mother's life or health.
Al-Generatdd
Intrinsie
Text
Silver Lining
Fact
The 2016 presidential campaign of Hillary Clinton was announced YouTube video, on April 12,
2015
Prompt
The Obama administration shut down the Amber Alert program because of the government shutdown.
The Amber Alert program funded through the Department of Justice, and as result, was affected by the government shutdown that began in October 2013, The Amber Alert program was not shut
Al-Generated
Extrinsic
Jown completely, however. During the shuidown, state and local Amber Aleri coordinators continued issue Amber Alerts and the National Center for Missing and Exploited Children continued provide technical support. The Department of Justice's Office of Justice Programs. which oversees the Amber Alert program, was closed during the shutdown, but it resumed operations when the govemmenn reopened.
Text
Fact
The Amber Alert Program had never been shut down.
Table 30: Examples for Time Wrap.


Reviews:
[{"review_id": "rVfUGi85DB", "reviewer": "Reviewer_3m5D", "summary": "", "questions": "1. How much was each annotator paid?\n2. In Table 2, how was this “hallucination” level measured? It is unclear how this black-box approach is evaluated. \n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Response to all the reviewers\n-------------------------------------------\nLong abstract and long appendix\n-------------------------------------------\nThis paper consists of several sections dealing with distinct and necessary topics/components, so we decided to write a longer appendix. Knowingly we wrote a longer abstract for review purposes only. Certainly, by focusing on the central topics and ignoring definitions and descriptions of hallucination classes in the abstract, we will reduce it in our final version.\n\nMitigation methods\n-------------------------\nAlthough we have clearly pointed out our main contributions on page #3, we would like to emphasize here that our contributions are - i) the definition of hallucination taxonomy, ii) defining the HVI metric, iii) presenting experiments for 15 LLMs, iv) introducing HILT dataset, and finally, v) providing *baseline* mitigation techniques. For mitigation, we have presented two classes of techniques: a) black-box, aka the entropy-based method having better precision, and b) gray-box, aka retrieval-based method having better recall. We expect us and other researchers to utilize the HILT dataset, HVI, etc., and compare new results with these baseline mitigation techniques. There is no doubt that these mitigation techniques are *NOT FOOLPROOF* and there are many other ways they could be improved. However, we explicitly mentioned on page 3 regarding our coverage: “While complete mitigation can be a herculean task, we suggest two mitigation strategies to alleviate hallucination.” [This sentence will be modified with an indication of the strategies for future enhancement. One paper is not sufficient to cover hallucination taxonomy, a manually annotated dataset, a metric to measure hallucination, experiments on 15 LLMs (our main contributions), and proposing comprehensive techniques for hallucination mitigation.]\n\n\nReview 1:\n\nProposed categorization appears to be developed in isolation from other existing related works.\nWe did not fully get this question/remark. We have acknowledged that previous researchers defined intrinsic and extrinsic kinds of hallucination and we have extended our definition drawing inspiration from those cited works. Thanks for highlighting [1]. This paper has also discussed about “factually correct” prompt” vs. “factually incorrect prompt”. We've assigned labels like \"factual mirage\" and \"silver lining\" to categorize such instances. Finally, we have defined six fine-grained classes to annotate hallucination at the sentence level. The work by [2] discussed hallucination in terms of name-nationality hallucination which is a subproblem of our definition of the “Generated Golem” hallucination category. So, we contend that our definitions are more inclusive and encompass other components observed in the literature. We would gladly incorporate any additional research outlining different categories of hallucinations that have been published subsequent to our submission. If you believe such research merits inclusion and discussion in this paper, please let us know.\n\nWe will provide cost analysis of our annotation process.\nWe have provided significant details on the annotation process in Appendix B. Regarding cost, we have experimented with several numbers. An honorarium of 10 INR (given majority of AMT annotators are from India) was offered per sentence due to the inherent complexity, which was ensured to be appropriate considering the 160-750 INR minimum wage/day (based on the Minimum Wages Act, 1948 [3]) in India (where the annotations were done) based on the average number of annotations across all annotators per day. Typically, in order to get annotation from the AMT platform people start with a lower compensation and finalize the best annotators, and then allow the best annotators to pay a higher amount to get the work done. We followed a similar philosophy to finalize annotators for our hallucination annotation task. We will further add a description in the appendix and main paper as to how we came up with such a compensation number.\n\nHow this black-box approach is evaluated.\nOutput from the black box method was evaluated manually. We have provided details in Appendix G.3 - pages #26-27.\n\nMissing references -  Thank you for pointing them out, we will include them in the final revision.\n\nREFERENCES:\n[1] Lee, Nayeon, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N. Fung, Mohammad Shoeybi, and Bryan Catanzaro. \"Factuality enhanced language models for open-ended text generation.\" Advances in Neural Information Processing Systems 35 (2022): 34586-34599.\n[2] Ladhak, Faisal, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen Mckeown, and Tatsunori B. Hashimoto. \"When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization.\" In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. 3198-3211. 2023.\n[3]  List of countries by minimum wage, https://en.wikipedia.org/wiki/List_of_countries_by_minimum_wage", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Fine-grain categorization of hallucination and evaluation of 15 LLMs based on this proposed categorization\n3. Proposed a metric that measures the LLM’s hallucination level that allows for comparison between different LLMs. This paper ranks 15 LLMs based on their proposed metric.", "reasons_to_reject": "1. Proposed categorization appears to be developed in isolation from other existing related works. \n2. This paper is hard to understand without referring to the appendix. It would be beneficial for readers if introduction/abstract contents are shortened and have more important details in the later sections (e.g., Section 5)\n3. Mitigation methods are not novel", "questions_to_authors": "1. How much was each annotator paid?\n2. In Table 2, how was this “hallucination” level measured? It is unclear how this black-box approach is evaluated. \n", "paper_topic_and_main_contributions": "This paper designs detailed categorization of hallucination in LLMs, and evaluates 15 LLMs based on this categorization. This paper also proposes a new metric that can be used to evaluate and rank LLMs based on their vulnerability to generate hallucinations. ", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "This work does not acknowledge existing literature in hallucination. Some missing references:\n[1] “Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics” - https://arxiv.org/abs/2104.13346 —> provides detailed categorisation of hallucination. \n[2] Factuality Enhanced Language Models for Open-Ended Text Generation - https://arxiv.org/abs/2206.04624  —> first paper to use the concept of “factually correct prompt” vs “factually incorrect prompt”. \n"}}, {"review_id": "1cHtalu5nQ", "reviewer": "Reviewer_rKxV", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Review 2:\n\nKindly consult the general feedback regarding the \"long abstract and long appendix,\" as well as the critique on mitigation techniques in the rebuttal for Reviewer 1.\n\nIt doesn’t take into consideration the length of the text\nThanks for pointing this out. We concur that the significance of the prompt's length is not the sole aspect to consider. Additionally, factors such as the intricacies related to formality and readability, the count of named entities (NEs), the presence of out-of-vocabulary words (OOVs) within the provided prompt, and the overall entropy or perplexity should all be taken into account when distinguishing between instances where the LLM is hallucinating vs. not hallucinating. We have indeed acknowledged the context length and hallucination tendency in Appendix C.1, page 18. The available space doesn't allow us to do justice to this issue which will take a longer treatment. A recent arxived paper [4] gives insights into what part of the prompt context is important - such as the start and the end, and empirically shown that most of the middle context of the prompt gets lost during the generation. \nLLM generates empty string\nYes, although this is theoretically possible, we have never experienced empty string generation in our numerous experiments. \nHVI is calculated manually\nCertainly, we concur. However, we maintain that this aspect does not diminish the significance of this paper, which introduces the inaugural comprehensive metric for quantifying the hallucination vulnerability of a given LLM. Unfortunately, there is no existing measure as of today to quantify the hallucination vulnerability of a given LLM, and defining one such metric has great benefits towards policy making and in many other aspects such as having a uniform evaluation metric for any downstream NLP applications. Please refer to the introduction section of our paper where we discuss the details of the larger necessity of having a met to calculate hallucination vulnerability. In subsequent endeavors, we aim to broaden our research by developing a framework for the automated assessment of the hallucination vulnerability of a given LLM. However, delving into that aspect goes beyond the focus of this initial paper on the subject, which is our current contribution.\n\nHigh entropy replacement technique is simple and error prone\nYes, we agree that the entropy-based method aka the black-box method has limitations. This method could be described as a precision model where the model can probably alleviate high hallucination-prone points. At the same time, it can replace named entities, low-frequency words, OOVs, etc. To handle this, we came up with a simple solution. We calculated the concreteness score for each word utilizing the previous research [5]. For example, in the following sentence from the available alternatives, we choose the word/token having the least concreteness score. In that way, we make sure that the content is getting generated is less risky. \n\n“The official stance of the [MASK] {MASK word was *USA*} on the Russia-Ukraine war has been consistent in supporting Ukraine’s sovereignty, territorial integrity, and the peaceful resolution of the conflict.”\nOptions:\nUS\nGovernment - we choose this as this word has least concreteness score\nAmerica\nCountry\n\nWe just found another paper [6] (arxived after our submission to EMNLP, in August 2023) that follows a similar philosophy of investigating methods to de-concretize/de-formalize an LLM’s output. We will add the necessary description and discussion in the final version.\n\nConversely, our gray-box approach leans towards recall orientation, while human-in-the-loop intervention results in an overall editing rate of 26%. Balancing precision and recall is currently slated as a follow-up endeavor. \n\nWords are not tokens\n\n—------------------------------\n\nWe understand that some readers might misunderstand a token as a sub-word unit. We will rectify such wording(s). Please clarify if you are referring to this issue or something else we are unable to understand - please provide more details in that case.\n\nDetails on entailment-based method\n\n—----------------------------------------------\n\nWe have provided a detailed description of the experiment in Appendix G.2, page #26. However, we realized that we missed mentioning the threshold values based on what we have categorized as “refute”, “support”, and “not enough information”. Those values were experimentally chosen and we will add all the necessary details in the appendix in the final version of the paper. We will also add necessary parts in the main paper with the extra granted page post-acceptance. \n\nOverpromising in the first part of the paper\n\n—------------------------------------------------------\n\nWe have mentioned in our contribution highlights on page #3 that “While complete mitigation can be a herculean task, we suggest two mitigation strategies to alleviate hallucination.” This should clarify that any future researchers interested in using the HILT dataset, HVI, and other resources produced should be able to compare their results with the two baselines provided in this paper. Indeed, there is a lot of room for improvement in both methods, but we believe that we're offering solid baselines that future researchers can compare with. Hallucination is a larger problem to be solved by the community, and hence it is beyond the scope of any one paper to cover all facets. \n\nBlack-box entropy-based technique, were calculated only for one model (GPT-3)\n\n—-----------------------------------------------------------------------------------\n\nThis is NOT accurate. We have conducted experiments for all the 15 LLMs. We have only reported GPT-3 in the main paper as we found that it is the LLM that has the highest HVI score. Experimental details and results for the other 14 LLMs are reported in Tables 9-23 in Appendix section G.1, page #22-25. We will be sure to draw attention to this in the main paper. \n\nIt is not clear what reduction of 10 HVI points actually mean\n\n—----------------------------------------------------------------------------------\n\nWe have provided further details in Table 24, page #27. We agree that readers might be interested in the actual numbers. So, we will add actual numbers/counts in Table 24 in addition to the HVI scores (this should be possible since we will have one extra page for the main paper post-acceptance). For example, for GPT-3, in the case of Silver Lining, for the “Numeric Nuisance” category, we have had a total of 850 instances in the gold test data consisting of 2000 sentences. After applying the entropy-based method there was no removal, but when we applied the gray-box method we saw a possibility of removing 82 cases. We will add these details in Table 24 for curious readers.  \n\nWhether the high entropy technique sometimes creates new hallucinations.\n\n—----------------------------------------------------------------------------------------------------------\n\nThat's an interesting question, and we were also skeptical about this possibility. To address this, we have investigated employing the concreteness score, which we did not include in the submission as experiments were ongoing at the time. Theoretically, replacing high entropy words can lead to new hallucinations, but we have not seen any new hallucination generated for any of the 2000 sentences in the test gold set. It is quite possible that it is there somewhere in the larger dataset. We will add relevant discussion to encourage future researchers to utilize the dataset and explore such possibilities. Moreover, we are interested in pursuing this direction as well. This leads to a question: What is the best possible replacement strategy? There are many open possibilities. Additionally, although our proposed mitigation techniques reduce hallucinations in some cases, we are not claiming that these methods remove hallucinations completely.\n\nURL in a footnote\n\n—-----------------------\n\nThanks, we will make those changes in the final version.\n\nTaxonomy of hallucination types in the abstract\n\n—--------------------------------------------------------------\n\nPoint is well taken, we will rewrite the abstract. Given the number of issues explored, we got carried away but we can focus on the main contributions in the abstract. \n\nFigure 2 was not mentioned in the text \n\n—--------------------------------------------------\n\nWe will fix it and fix any errata of missing references. \n\nDiscussion and Limitations\n\n—------------------------------------\n\nWe have named our limitation discussion as “Discussion and Limitations”, although we can separate our discussion and limitation separately. \n\nFAQ section after the Reference section\n\n—-----------------------------------------------------\n\nFAQ provides a nice way to summarize necessary points to the readers. If you have any specific suggestion(s) for the FAQ section, we will be happy to incorporate them.\n\nREFERENCES:\n[4] Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. \"Lost in the middle: How language models use long contexts.\" arXiv preprint arXiv:2307.03172 (2023).\n[5] Brysbaert, Marc, Amy Beth Warriner, and Victor Kuperman. \"Concreteness ratings for 40 thousand generally known English word lemmas.\" Behavior research methods 46 (2014): 904-911.\n[6] Varshney, Neeraj, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. \"A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation.\" arXiv preprint arXiv:2307.03987 (2023).\n\nWe hope the detailed response will allow you to consider if this paper deserves a higher score that is consistent with the other two reviewers. Hallucination is an important and complex problem that cannot be addressed in a single paper, but we emphasize that this paper brought in significant, timely, and early contributions in this intricate-but-important area.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The new dataset is valuable and I am sure it will be useful for the EMNLP research community. I like the long and informative introduction section.", "reasons_to_reject": "While the paper is well written, and the motivation behind this project is provided with many details, I do see some weaknesses; some I believe need to be addressed before publishing this paper in a central stage such as EMNLP:\n\n1. According to the authors, the HVI index is calculated based on the number of hallucinations of all the generated texts. However, it doesn’t take into consideration the length of the text. Technically speaking, an LLM that generates an empty string every time would get a high HVI score and that sounds a bit wrong. This may explain the locations of T0, T5 and XLNet on Figure 3. Also, the HVI index is calculated manually (using the annotated examples); calculating it for other LLMs is labor intensive and requires following the same annotation guidelines. This is a major limitation.\n\n2. The high entropy replacement technique is quite simple and it may remove many named entities from the text, even if they are not hallucinated. Also, it is mentioned that the model considers consecutive high-entropy words together. But you didn’t mention what’s the criterion for being considered high entropy. I think that this important information should be provided in the paper and not in the appendix. Additionally, words are not tokens, how is that handled? this should be all described in the paper.\n\n3. Since one of the main contributions of the paper is the dataset, I believe that some explanations on the annotation methodology and agreement levels should be provided in the paper and not only in the appendix.\n\n4. The entailment exercise lacks some basic details. Only based on the information provided in the paper, it is hard to assess the efficacy of this technique. It should be removed from the paper, or provided with more empirical and analyzed information.\n\n5. To me, there is a bit of overpromising in the first part of the paper, when discussing hallucination mitigation techniques. The techniques proposed in the paper are very simple and probably not very useful. The results shown for the high entropy technique, were calculated only for one model (GPT 3) and it is not clear what reduction of 10 HVI points actually mean in terms of how many hallucinations are removed. Also, it is not clear (and not discussed) whether the high entropy technique sometimes creates new hallucinations.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this paper the authors introduce a new dataset of 75K texts that were generated by 15 different LLMs, manually annotated for different types of hallucinations which they defined using a new taxonomy as part of this paper. The texts are broken into sentences, and every sentence is assigned with only one label (although potentially it may contain several types of hallucinations).", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "I don't think there are missing references, but the news articles, public statements and reference to Wikipedia pages that are mentioned throughout the paper should not be treated as standard academic citations. You should simply include the URL in a footnote. \n", "Typos_Grammar_Style_and_Presentation_Improvements": "1. The abstract it too long. You don’t have to include all the taxonomy of hallucination types in the abstract. It should be shorten for the final version.\n\n2. Figure 2 was not mentioned in the text (unless I missed it). All figures need to be referred to from the text. Same with all appendices. \n\n3. The Discussion and Limitations section contains a discussion before getting to describing the limitations of this work. Since it is located on the 9th page, I believe it slightly violates the requirements. \n\n4. I am not sure how to treat the FAQ section after the Reference section. I believe it also violates the format standards. "}}, {"review_id": "xaQjwM3I7n", "reviewer": "Reviewer_L4uh", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Kindly consult the general feedback regarding the \"long abstract and long appendix,\" as well as the critique on mitigation techniques in the rebuttal for Reviewer #1. We will also add the necessary parts in the main paper with the extra granted page post-acceptance. \n\nHVI metric, compared to other metrics, as well as its correlation with human judgment.\n\n--------------------------------------------------------------------------------------------------------------------------------------------------------\n\nHVI computation relies on manually annotated data, leaving us uncertain about how to effectively contrast it with human assessment. Any specific recommendations in this regard would be greatly appreciated. Currently, we are not acquainted with alternative metrics/indices for quantifying hallucination. If there are any references to such metric/index available beyond our current knowledge, we would greatly appreciate it if you could provide them. This would allow us to examine and compare them with HVI.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe proposed hallucination categorization scheme, as well as the released dataset, are important steps towards a better characterization and handling of this phenomenon.\n2.\tTheir proposed novel metric is also an important step in the research of hallucinations in LLMs.", "reasons_to_reject": "1.\tThe paper lacks an investigation of the utility of the HVI metric, compared to other metrics, as well as its correlation with human judgement.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper addresses the phenomenon of hallucinations in LLMs. The authors begin with categorizing hallucinations according to three aspects: degree, orientation and category, with elaborate explanations and examples for each aspect. The authors then crowdsource the first publicly available hallucination dataset, comprising of 75,000 text prose entries, generated by 15 LLMs, where hallucinations are marked and accompanied by degree, orientation and category labels. The authors also introduce a novel evaluation for hallucination (HVI). Lastly, the authors propose two strategies for hallucination mitigation – an automatic approach, involving high-entropy words identification, and a human-in-the-loop approach.\nThe main contributions of this paper are its hallucination categorization scheme, its novel labeled hallucination dataset and the devised hallucination metric.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper examines the hallucination problem by categorizing these by their category (acronym ambiguity, numeric nuisance, generated golem, virtual voice, geographic erratum, time wrap), degree (mild, moderate, alarming), and orientation (factual mirage or silver lining). They then provide two methods to mitigate these issues. The authors provide a sample of 75k examples of these hallucinations with 2k annotated samples. They also come up with an index (HVI) that allows them to rank LLMs by their likelihood of generating hallucinations. The reviewers agree that the paper is sound and are generally excited about the work. They provide several valid criticisms regarding the limitations of the HVI metric and somewhat simple mitigation strategies. However, this paper has a wide array of contributions that should be beneficial for the community. The fine-grained types of hallucination and annotations will probably be the most valuable component, though future work may build off of the other contributions. It is reasonable that the scope of a paper cannot include everything that the reviewers have brought up, though I think their suggestions will be a helpful guide for future work.
Decision: Accept-Main