Omkar2118/datasets huggingface dataset:
ToT
Full name: Test of Time

ToT is a benchmark for evaluating LLMs on temporal reasoning. 

ToT is a dataset designed to assess the temporal reasoning capabilities of AI models. It comprises two key sections:

ToT-semantic: Measuring the semantics and logic of time understanding.
ToT-arithmetic: Measuring the ability to carry out time arithmetic operations.

Data Format
The ToT-semantic and ToT-semantic-large datasets contain the following fields:

question: Contains the text of the question.
graph_gen_algorithm: Contains the name of the graph generator algorithm used to generate the graph.
question_type: Corresponds to one of the 7 question types in the dataset.
sorting_type: Correspons to the sorting type applied on the facts to order them.
prompt: Contains the full prompt text used to evaluate LLMs on the task.
label: Contains the ground truth answer to the question.
The ToT-arithmetic dataset contains the following fields:

question: Contains the text of the question.
question_type: Corresponds to one of the 7 question types in the dataset.
label: Contains the ground truth answer to the question.

Introducing paper: Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
M3AV
Full name: Multimodal, Multigenre, and Multipurpose Audio-Visual

The **M3AV** (Multimodal, Multigenre, and Multipurpose Audio-Visual) is a novel dataset proposed for academic lectures¹. It contains almost **367 hours of videos** from five sources covering topics in computer science, mathematics, and medical and biology¹⁴. 

These videos carry rich multimodal information including speech, the facial and body movements of the speakers, as well as the texts and pictures in the slides and possibly even the papers¹. The dataset has high-quality human annotations of the slide text and spoken words, particularly high-valued name entities¹.

The M3AV dataset can be used for multiple audio-visual recognition and understanding tasks¹. Along with the dataset, three benchmark tasks are proposed that reflect the perception and understanding of the multimodal information in those videos. These tasks include automatic speech recognition (ASR) with a particular focus on contextual ASR (CASR), spontaneous text-to-speech (TTS) synthesis, and slide and script generation (SSG)¹. 

This dataset effectively represents the typical scenario met in academic presentations where new terminologies constantly appear and are crucial to the overall understanding¹. It's a challenging dataset for evaluations performed on contextual speech recognition, speech synthesis, and slide and script generation tasks¹.

(1) M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic .... https://arxiv.org/html/2403.14168v2.
(2) M$^3$AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual .... https://arxiv.org/abs/2403.14168.
(3) Mac Benchmarks - Geekbench. https://browser.geekbench.com/mac-benchmarks.
(4) M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic .... https://jack-zc8.github.io/M3AV-dataset-page/.

Introducing paper: M$^3$AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset
COCO-N Medium

COCO-N Medium introduces a stochastic benchmark that simulates common real-world scenarios with noticeable label inaccuracies in the COCO dataset. This benchmark combines class and spatial noises to create a challenging yet realistic evaluation framework for instance segmentation models. It mimics datasets manually annotated by crowd workers, where a moderate level of label noise is expected. By incorporating both class and spatial inaccuracies, COCO-N Medium allows researchers to assess their models' basic robustness to label noise, providing insights into performance in typical real-world applications where perfect annotations are rare. This medium-level benchmark serves as a crucial middle ground, offering a more rigorous test than minimally noisy datasets while remaining within the bounds of commonly encountered data quality issues. COCO-N Medium enables a nuanced evaluation of model performance under realistic conditions, helping identify areas for improvement in handling noisy labels and guiding the development of more robust instance segmentation algorithms.

Introducing paper: Benchmarking Label Noise in Instance Segmentation: Spatial Noise Matters
iFashion Alibaba
Full name: iFashion Alibaba: Personalized outfits

```markdown
1.   1.01 million outfits, 583K fashion items, with context information.
2.   0.28 billion user click actions from 3.57 million users.

Example usages:
 - Complete an outfit, considering both garments already in the outfit and user's previous clicks
 - Recommend garments
 - Predict outfit compatibility (based on the garments composing it)


Outfit data
outfit_id,item_id;item_id;item_id...

User data
user_id,item_id;item_id;...,outfit_id

Item data
item_id,cate_id,pic_url,title

Github:  https://github.com/wenyuer/POG
Paper: https://dl.acm.org/doi/abs/10.1145/3292500.3330652
```

Introducing paper: POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion
WikiNews Dataset
Full name: WikiNews Arabic Diacritization Benchmark Dataset

The WikiNews Arabic Diacritization dataset is a test set composed of 70 WikiNews articles (majority are from 2013 and 2014) that cover a variety of themes, namely: politics, economics, health, science and technology, sports, arts, and culture.
The articles are evenly distributed among the different themes (10 per theme).
The articles contain 18,300 words with around 400 different sentences (Each line is considered as a sentence).

Introducing paper: Arabic Diacritization: Stats, Rules, and Hacks