Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
Adding tasks from the USB benchmark (for summarization)
#11
by
kundank
- opened
Hi,
We released a suite of tasks for factchecking summaries recently at EMNLP 2023 (https://aclanthology.org/2023.findings-emnlp.592/).
They are on huggingface datasets too : https://huggingface.co/datasets/kundank/usb
Besides the binary classification task of predicting if there is a hallucination or not, it also consists of tasks to localize the hallucinated spans, and to fix factual errors by editing the summary.
These 3 tasks would be relevant:
Happy to answer any questions you may have.
Thanks!
Thanks for the pointer! Could you please do a pull request to https://github.com/EdinburghNLP/awesome-hallucination-detection to add your paper? :)
I'm looking into this!