Omkar2118/datasets huggingface dataset:
WikiEvalFacts

WikiEvalFacts
This is an augmented version of the WikiEval dataset which additionally includes generated fact statements for each QA pair and human annotation of their truthfulness against each answer type.

This is the dataset used in our 2024 paper FaaF: Facts as a Function for the evaluation of generated text.

Column description
question: a question that can be answered from the given Wikipedia page (source).
source: The source Wikipedia page from which the question and context are generated.
gt_facts: A set of ground truth fact statemets generated using the answer and question columns.
human_evaluated_facts_answer: Human annotation of the truthfulness of each of the fact ground truth statements against the content in the answer column.
human_evaluated_facts_ungrounded_answer: Human annotation of the truthfulness of each of the fact ground truth statements against the content in the ungrounded_answer column.
human_evaluated_facts_poor_answer: Human annotation of the truthfulness of each of the fact ground truth statements against the content in the poor_answer column.
unique_id: a simple integer index.
answer: answer grounded on context_v1
ungrounded_answer: answer generated without context_v1
poor_answer: answer with poor relevancy compared to grounded_answer and ungrounded_answer
context_v1: Ideal context to answer the given question
contetx_v2: context that contains redundant information compared to context_v1

Introducing paper: FaaF: Facts as a Function for the evaluation of generated text