Post
Fantastic Beasts (*Hallucinations*) and Where to Find Them 🔎🧌
This paper breaks down LLM hallucinations into six different types:
1️⃣ Entity: Involves errors in nouns. Changing that single entity can make the sentence correct.
2️⃣ Relation: Involves errors in verbs, prepositions, or adjectives. They can be fixed by correcting the relation.
3️⃣ Contradictory: Sentences that contradict factually correct information.
4️⃣ Invented: When the LLM generates sentences with concepts that don't exist in the real world.
5️⃣ Subjective: When the LLM generates sentences influenced by personal beliefs, feelings, biases, etc.
6️⃣ Unverifiable: When the LLM comes up with sentences containing information that can't be verified. E.g., Personal or private matters.
The first two types of hallucinations are relatively easy to correct, given that we can rewrite them by changing the entity or relation. However, the other four would mostly need to be removed to make the sentence factually correct.
Paper: Fine-grained Hallucination Detection and Editing for Language Models (2401.06855)
This paper breaks down LLM hallucinations into six different types:
1️⃣ Entity: Involves errors in nouns. Changing that single entity can make the sentence correct.
2️⃣ Relation: Involves errors in verbs, prepositions, or adjectives. They can be fixed by correcting the relation.
3️⃣ Contradictory: Sentences that contradict factually correct information.
4️⃣ Invented: When the LLM generates sentences with concepts that don't exist in the real world.
5️⃣ Subjective: When the LLM generates sentences influenced by personal beliefs, feelings, biases, etc.
6️⃣ Unverifiable: When the LLM comes up with sentences containing information that can't be verified. E.g., Personal or private matters.
The first two types of hallucinations are relatively easy to correct, given that we can rewrite them by changing the entity or relation. However, the other four would mostly need to be removed to make the sentence factually correct.
Paper: Fine-grained Hallucination Detection and Editing for Language Models (2401.06855)