Omkar2118/datasets huggingface dataset:
tStoryCloze
Full name: Topic Story Cloze

The tStoryCloze refers to the "Topic StoryCloze" benchmark, which is a spoken version of the StoryCloze textual benchmark. In the tStoryCloze, the goal is to evaluate continuation coherence given a spoken prompt. This version is designed to be easier compared to the fine-grained causal and temporal commonsense relations evaluated in the spoken benchmark called sStoryCloze. The tStoryCloze benchmark is used to assess the capabilities of SpeechLMs to maintain coherence in the spoken language given a prompt. This benchmark, along with the sStoryCloze, provides valuable insights into the performance of SpeechLMs in capturing different aspects of spoken content.

Introducing paper: Textually Pretrained Speech Language Models
sStoryCloze
Full name: Spoken Story Cloze

The sStoryCloze refers to the Spoken StoryCloze benchmark, which is a spoken version of the StoryCloze dataset. The StoryCloze dataset consists of five-sentence commonsense stories, where the task is to predict the ending of a story given the first four sentences and two possible endings, one of which is the correct ending and the other is a distractor. The sStoryCloze evaluates the model's capabilities to capture fine-grained causal and temporal commonsense relations in spoken language. It assesses the model's ability to generate coherent and contextually appropriate continuations given a spoken prompt. The dataset is used to evaluate the performance of SpeechLMs in understanding and generating spoken narratives.

Introducing paper: Textually Pretrained Speech Language Models
AggreFact

The **AggreFact dataset** is a benchmark for evaluating the factuality of summaries generated by different summarization models. It aggregates factuality error annotations from nine existing datasets and stratifies them according to the underlying summarization model. 

The dataset contains the following columns:
- `dataset`: Name of the original annotated dataset.
- `origin`: Summarization dataset. Either cnndm or xsum.
- `id`: Document id.
- `doc`: Input article.
- `summary`: Model generated summary.
- `model_name`: Name of the model used to generate the summary.
- `label`: Factual consistency of the generated summary. 1 is factually consistent, 0 otherwise.
- `cut`: Either val or test.
- `system _score`: The output score from a factuality system.
- `system _label`: The binary factual consistency label based on the score of the factuality system.

Introducing paper: Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors
DialSummEval

The **DialSummEval** is a multi-faceted dataset of human judgments. It was created to revisit the evaluation of dialogue summarization models. The dataset contains the outputs of 14 models on SAMSum, a dialogue summary dataset.

The creators of DialSummEval observed that current dialogue summarization models have flaws that may not be well exposed by frequently used metrics such as ROUGE. Therefore, they re-evaluated 18 categories of metrics in terms of four dimensions: coherence, consistency, fluency, and relevance. They also conducted a unified human evaluation of various models in dialogue summarization for the first time.

Introducing paper: DialSummEval: Revisiting Summarization Evaluation for Dialogues
TinyShakespeare

The Tiny Shakespeare corpus is a dataset that contains **40,000 lines of Shakespeare** from a variety of his plays. The Tiny Shakespeare corpus is a popular choice for training language models due to its manageable size and the complexity of Shakespeare's language. It provides a good balance between computational efficiency and the ability to generate interesting text.

More information is found at [tiny_shakespeare](https://www.tensorflow.org/datasets/catalog/tiny_shakespeare).
DiFair

DiFair serves as a meticulous endeavor to address the oversight in evaluating the impact of bias mitigation on useful gender knowledge while assessing gender neutrality in pretrained language models. This metric delves into not only quantifying a model's biased tendencies but also assessing the preservation of useful gender knowledge.

Introducing paper: DiFair: A Benchmark for Disentangled Assessment of Gender Knowledge and Bias
Visual Near-Duplicates Detection in the Context of Social Media

The dataset of the paper: ``Dataset and Case Studies for Visual Near-Duplicates Detection in the Context of Social Media'', by Hana Matatov, Mor Naaman, and Ofra Amir.

https://figshare.com/articles/dataset/Visual_Near-Duplicates_Detection_in_the_Context_of_Social_Media_Dataset_/19367933

See the Github repository for details:
https://github.com/sTechLab/Visual-Near-Duplicates-Detection-in-the-Context-of-Social-Media

See the associated paper on arXiv:
https://arxiv.org/abs/2203.07167

Introducing paper: Dataset and Case Studies for Visual Near-Duplicates Detection in the Context of Social Media