Omkar2118/datasets huggingface dataset:
naab
Full name: naab

# naab: A ready-to-use plug-and-play corpus for Farsi
The biggest cleaned and ready-to-use open-source textual corpus in Farsi. It contains about 130GB of data, 250 million paragraphs, and 15 billion words. The project name is derived from the Farsi word NAAB K which means pure and high grade. We also provide the raw version of the corpus called naab-raw and an easy-to-use preprocessor that can be employed by those who wanted to make a customized corpus.

Introducing paper: naab: A ready-to-use plug-and-play corpus for Farsi
TFVulFix
Full name: TensorFlow Vulnerability Fixes

**TFVulFix** is a dataset containing commits from TensorFlow, which is a well-known deep learning library. It contains 290 vulnerability fixing and 1,535 non-vulnerability-fixing commits. In this dataset, no commit is explicitly linked to an issue.

Source: [VulCurator: A Vulnerability-Fixing Commit Detector](https://arxiv.org/pdf/2209.03260v1.pdf)

Introducing paper: VulCurator: A Vulnerability-Fixing Commit Detector
default of credit card clients Data Set

This research aimed at the case of customers default payments in Taiwan and compares the predictive accuracy of probability of default among six data mining methods. From the perspective of risk management, the result of predictive accuracy of the estimated probability of default will be more valuable than the binary result of classification - credible or not credible clients. Because the real probability of default is unknown, this study presented the novel Sorting Smoothing Method to estimate the real probability of default. With the real probability of default as the response variable (Y), and the predictive probability of default as the independent variable (X), the simple linear regression result (Y = A + BX) shows that the forecasting model produced by artificial neural network has the highest coefficient of determination; its regression intercept (A) is close to zero, and regression coefficient (B) to one. Therefore, among the six data mining techniques, artificial neural network is the only one that can accurately estimate the real probability of default.
AmaSum

AmaSum is the largest abstractive opinion summarization dataset, consisting of more than 33,000 human-written summaries for Amazon products. Each summary is paired, on average, with more than 320 customer reviews. Summaries consist of verdicts, pros, and cons, see the example below.

Introducing paper: Learning Opinion Summarizers by Selecting Informative Reviews
SPACE (Opinion Summarization)

SPACE is a large-scale opinion summarization benchmark for the evaluation of unsupervised summarizers. SPACE is built on TripAdvisor hotel reviews and includes a training set of approximately 1.1 million reviews for over 11 thousand hotels.

For evaluation, we created a collection of human-written, abstractive opinion summaries for 50 hotels, including high-level general summaries and aspect summaries for six popular aspects: building, cleanliness, food, location, rooms, and service. Every summary is based on 100 input reviews, an order of magnitude increase compared to existing corpora. In total, SPACE contains 1,050 gold standard summaries. You can view the full instructions for out multi-stage annotation procedure here.

Introducing paper: Extractive Opinion Summarization in Quantized Transformer Spaces
GDELT

The **GDELT Project** is a remarkable initiative that monitors our world by analyzing global news from various sources. Here are the key aspects of the GDELT dataset:

1. **Scope and Purpose**:
   - The GDELT Project aims to create a comprehensive, **real-time database** of global human society.
   - It monitors news from **broadcasts, print media, and web sources** in nearly every country and over **100 languages**.
   - By analyzing this vast dataset, it identifies people, locations, organizations, themes, emotions, and events that shape our global society every second of every day.

2. **Data Collection**:
   - GDELT continuously captures and analyzes news articles, broadcasts, and online sources.
   - Its historical archives date back to **January 1, 1979**, and it updates every **15 minutes**.
   - The project goes beyond Western media, providing a more **global perspective** on world events and sentiments.

3. **Features**:
   - GDELT uses sophisticated natural language and data mining algorithms, including powerful deep learning techniques.
   - It extracts over **300 categories of events**, millions of themes, thousands of emotions, and the networks connecting them.
   - The dataset models human interactions at a large scale, making it valuable for research and analysis.

4. **Vision**:
   - The GDELT Project envisions using this data to:
     - Understand the world through others' eyes.
     - Break down language and access barriers.
     - Facilitate conversations between societies.
     - Empower local populations with information for safer lives.
     - Map happiness, conflict, and potentially forecast global tensions.

5. **Global Reach**:
   - GDELT monitors media in **over 100 languages** across every country, providing a truly global perspective.
   - It allows us to explore how social media is used worldwide and how people express themselves online.

6. **Open Data**:
   - The entire GDELT database is **free and open**.
   - Researchers can download raw data, visualize it, or analyze it at scale using tools like **Google BigQuery**¹²³⁴⁵.

Source: Conversation with Bing, 3/12/2024
(1) The GDELT Project. https://www.gdeltproject.org/.
(2) The GDELT Database | Aalto Datahub. https://datahub.aalto.fi/en/data-sources/the-gdelt-database.
(3) An Introduction to GDELT Data | MongoDB. https://www.mongodb.com/developer/products/mongodb/introduction-to-gdelt-data/.
(4) GDELT 2.0: Our Global World in Realtime – The GDELT Project. https://blog.gdeltproject.org/gdelt-2-0-our-global-world-in-realtime/.
(5) Data: Querying, Analyzing and Downloading: The GDELT Project. https://www.gdeltproject.org/data.html.
ActionBench

**ActionBench** contains two carefully designed probing tasks: Action Antonym and Video Reversal, which targets multimodal alignment capabilities and temporal understanding skills of the model, respectively. Action knowledge involves the understanding of textual, visual, and temporal aspects of actions. The benchmark is constructed by leveraging two existing open-domain video-language datasets, Ego4D and Something-Something v2 (SSv2), which provide fine-grained action annotations for each video clip.

Source: [Paxion: Patching Action Knowledge in Video-Language Foundation Models]()

Image Source: [Paxion: Patching Action Knowledge in Video-Language Foundation Models]()
SMIC

I'm writing a description for the SMIC dataset on Papers With Code. SMIC is a dataset for facial expression recognition (FER).
SIDAR

**SIDAR** is a dataset designed to be a training and evaluation set for a multitude of tasks involving image alignment and artifact removal, such as deep homography estimation, dense image matching, 2D bundle adjustment, inpainting, shadow removal, denoising, content retrieval, and background subtraction.

Source: [SIDAR: Synthetic Image Dataset for Alignment & Restoration](https://arxiv.org/pdf/2305.12036v1.pdf)

Introducing paper: SIDAR: Synthetic Image Dataset for Alignment & Restoration
ExplainCPE

This is a medical multiple-choice dataset with explanations which can be used to interpret the answer. The data comes from Chinese Pharmacist Examination. Each piece of data has a question, five options, a gold_answer and a gold_explanation.

Introducing paper: ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination
MSRA CN NER
Full name: MSRA CN NER Dataset

Simplified Chinese dataset for NER in The Third International Chinese Language Processing Bakeoff (2006), provided by Microsoft Research Asia (MSRA).