Omkar2118/datasets huggingface dataset:
SeaEval

**SeaEval** is a benchmark designed for evaluating **multilingual foundation models (FMs)**. These large language models (LLMs) have demonstrated impressive generalizability and adaptability across various downstream tasks. The SeaEval benchmark goes beyond standard accuracy metrics and investigates how well these models understand and reason with natural language, as well as their comprehension of cultural practices, nuances, and values¹.

Here are some key aspects of the **SeaEval benchmark**:

1. **Multilingual Context**: SeaEval spans multiple languages, allowing researchers to assess model performance across diverse linguistic contexts.

2. **Cultural Reasoning**: In addition to traditional NLP tasks, SeaEval evaluates how well models comprehend cultural nuances and practices. This is crucial for applications in multicultural scenarios.

3. **Brittleness Analysis**: The benchmark explores the brittleness of foundation models in terms of semantics and multilinguality. For instance:
    - Some models exhibit varied behavior when given paraphrased instructions.
    - Exposure bias (e.g., positional bias, majority label bias) remains a challenge.
    - Consistent responses are expected for semantically equivalent queries across languages, but most models surprisingly demonstrate inconsistency for factual, scientific, and commonsense knowledge questions.
    - Multilingually-trained models have not yet achieved "balanced multilingual" capabilities.

4. **Empirical Results**: SeaEval provides empirical results across classic NLP tasks, reasoning, and cultural comprehension.

In summary, SeaEval serves as a launchpad for thorough investigations and evaluations of multilingual and multicultural scenarios, emphasizing the need for more generalizable semantic representations and enhanced multilingual contextualization¹.

(1) SeaEval for Multilingual Foundation Models: - arXiv.org. https://arxiv.org/html/2309.04766v2.
(2) [2306.05179] M3Exam: A Multilingual, Multimodal, Multilevel Benchmark .... https://arxiv.org/abs/2306.05179.
(3) SeaEval for Multilingual Foundation Models: From Cross-Lingual .... https://arxiv.org/pdf/2309.04766v1.pdf.
(4) SeaEval for Multilingual Foundation Models: From Cross-Lingual .... https://arxiv.org/pdf/2309.04766v2.pdf.
(5) undefined. https://github.com/SeaEval/SeaEval.

Introducing paper: SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning
OWT2
Full name: OpenWebtext2

**OpenWebText2** is an enhanced version of the original **OpenWebTextCorpus**. It encompasses all **Reddit submissions** from **2005** up until **April 2020**, with additional months becoming available after the corresponding **PushShift dump files** are released¹²³. Here are the key details:

- **Dataset Description**:
    - **OpenWebText2** is part of the **EleutherAi/The Pile dataset**.
    - It covers **Reddit submissions** and is designed to be a high-quality internet dataset.
    - The dataset was created by scraping **URLs** extracted from **Reddit submissions** with a minimum score of **3** as a proxy for quality⁴.
    - The **plug-and-play version** of **OpenWebText2** contains **17,103,059 documents** and is approximately **65.86 GB** when uncompressed².

- **Features**:
    - Each document in **OpenWebText2** has two main features:
        - **Title**: A string representing the title of the submission.
        - **Text**: A string containing the content of the submission¹.

- **License and Version**:
    - **License**: No known license.
    - **Version**: 1.0.0¹.

- **Size**:
    - **Download Size**: Approximately **27.3 GiB**.
    - **Dataset Size**: Approximately **63.8 GiB**³.

(1) the_pile_openwebtext2 | TensorFlow Datasets. https://www.tensorflow.org/datasets/community_catalog/huggingface/the_pile_openwebtext2.
(2) GitHub - EleutherAI/openwebtext2. https://github.com/EleutherAI/openwebtext2.
(3) the_pile_openwebtext2 · Datasets at Hugging Face. https://huggingface.co/datasets/the_pile_openwebtext2.
(4) Eleuther AI site | OpenWebText2. https://researcher2.eleuther.ai/projects/open-web-text2/.
(5) OpenWebText2 — EleutherAI. https://www.eleuther.ai/artifacts/openwebtext2.
USPTO Backgrounds

The **USPTO Backgrounds dataset** provides valuable information related to patents and trademarks. Here are some key datasets available from the United States Patent and Trademark Office (USPTO):

1. **Trademark Daily XML File (TDXF) Applications**:
   - Contains pending and registered trademark text data (no images) for word marks, serial numbers, registration numbers, filing dates, registration dates, goods and services, classification numbers, status codes, design search codes, and pseudo marks in **CY2024**.
   - Format: eXtensible Markup Language (XML) based on the U.S. Trademark Assignments Version 2.0 Document Type Definition (DTD).
   - [Download here](https://developer.uspto.gov/data?search_api_fulltext=csv) ¹.

2. **Trademark Daily XML File (TDXF) Assignments**:
   - Includes assignment text data (no images) for trademark assignments in **CY2024**.
   - Format: XML based on the U.S. Trademark Assignments Version 0.4 DTD.
   - [Download here](https://developer.uspto.gov/data?op=&search_api_fulltext=EXCEL) ³.

3. **Trademark Daily XML File (TDXF) Trademark Trial and Appeal Board (TTAB)**:
   - Provides TTAB text data (no images) for trademark trial and appeal board cases in **CY2024**.
   - Format: XML based on the U.S. Trademark Trial and Appeal Board Version 1.0 DTD.
   - [Download here](https://developer.uspto.gov/data?page=1) ⁴.

4. **Patent Application Multi-Page PDF Images**:
   - Contains images of each patent application (non-provisional utility and plant) published weekly from **March 15, 2001** to present.
   - Format: Portable Document Format (PDF) created from the Patent Application Single-Page TIFF Images.
   - Approximately **11 GB** (compressed) per week.
   - [Download here](https://developer.uspto.gov/data) ¹.

5. **Patent Application Data/XML**:
   - Includes full text, images/drawings, and complex work units (tables, mathematical expressions, chemical structures, genetic sequence data) of each patent application (non-provisional utility and plant) published weekly.
   - Format: XML based on the Patent Application International Common Element (ICE) DTD.
   - [Download here](https://developer.uspto.gov/data) ¹.

6. **Patent Application Full Text Data/XML**:
   - Contains the full text of each patent application (non-provisional utility and plant) published weekly (excluding images/drawings).
   - Format: XML based on the Patent Application International Common Element (ICE) DTD.
   - [Download here](https://developer.uspto.gov/data) ¹.

(1) USPTO Datasets - United States Patent and Trademark Office. https://developer.uspto.gov/data.
(2) USPTO Datasets | United States Patent and Trademark Office - Open Data .... https://developer.uspto.gov/data?op=&search_api_fulltext=EXCEL.
(3) USPTO Datasets | United States Patent and Trademark Office - Open Data .... https://developer.uspto.gov/data?page=1.
(4) USPTO Datasets - United States Patent and Trademark Office. https://developer.uspto.gov/data?search_api_fulltext=csv.
(5) Research datasets | USPTO. https://www.uspto.gov/ip-policy/economic-research/research-datasets.
Ubuntu IRC

The **Ubuntu IRC dataset** is a valuable resource for research in natural language understanding and dialogue systems. Let me provide you with some details:

1. **Ubuntu Dialogue Corpus**:
    - This dataset contains **almost 1 million multi-turn dialogues**, comprising over **7 million utterances** and **100 million words**.
    - It serves as a unique resource for building dialogue managers based on neural language models that can leverage large amounts of unlabeled data³.
    - The dialogues are sourced from **IRC (Internet Relay Chat)** conversations related to Ubuntu, a popular open-source operating system.
    - Researchers can use this corpus to explore various aspects of dialogue understanding and generation.

2. **Specifics of the Ubuntu IRC Dataset**:
    - The dataset includes **77,563 annotated messages** from IRC.
    - Most of these messages originate from the **Ubuntu IRC Logs for the #ubuntu channel**.
    - Additionally, a smaller subset is a re-annotation of data from the **#linux channel**, which was originally collected by Elsner and Charniak in 2008².
    - You can find this dataset in the **kummerfeld/data folder** of the repository².

In summary, the Ubuntu IRC dataset provides a rich collection of dialogues that researchers can use to advance the field of natural language processing and dialogue modeling. 🌐🗣️

(1) The Ubuntu Dialogue Corpus: A Large Dataset for Research in .... https://arxiv.org/abs/1506.08909.
(2) GitHub - amarazad/DSRNet: Meta-Context Transformers for Domain-Specific .... https://github.com/amarazad/DSRNet.
(3) Ubuntu Dialogue Corpus | Kaggle. https://www.kaggle.com/datasets/rtatman/ubuntu-dialogue-corpus.
(4) GitHub - jkkummerfeld/irc-disentanglement: Dataset and model for .... https://github.com/jkkummerfeld/irc-disentanglement.

Introducing paper: A Large-Scale Corpus for Conversation Disentanglement
YouTube Subtitles

**YT_subtitles** is a remarkable tool designed for building a dataset from YouTube subtitles. Let me break it down for you:

1. **Purpose**: The primary goal of this tool is to extract **non-machine-generated** subtitles from YouTube videos. These subtitles are obtained by searching for specific terms and collecting the relevant video content.

2. **How It Works**:
    - You provide a list of search terms (such as "movie review," "GPT-3," or "true crime documentary").
    - The tool retrieves videos related to these search terms.
    - For each video, it extracts the subtitles (in various languages) and organizes them into minute-by-minute segments.
    - The resulting files contain a string of text per language, with the language name included as a header.

3. **Dataset Format**:
    - The dataset is stored in a **JSONL** (JSON Lines) file format.
    - Each entry corresponds to a minute of subtitles, with language-specific content.
    - If only one language is available, the output consists of a plain text version of the subtitles without additional metadata.

4. **Use Cases**:
    - Researchers and developers can utilize this dataset to enhance the multilingual performance of language models.
    - It's particularly valuable for training models that work with diverse languages and real-world video content.

(1) sdtblck/youtube_subtitle_dataset: YT_subtitles - GitHub. https://github.com/sdtblck/youtube_subtitle_dataset.
(2) Youtubean Dataset | Papers With Code. https://paperswithcode.com/dataset/youtubean.
(3) youtube subtitles | Kaggle. https://www.kaggle.com/datasets/wadzim/youtube-subtitles.
(4) YouTube-8M Segments Dataset - Google Research. https://research.google.com/youtube8m/.
Hacker News

The **Hacker News dataset** provides a valuable glimpse into the tech industry's landscape. It encompasses posts from **Y Combinator's social news website**, spanning from **2006 to late 2017**¹. Here are some key points about this dataset:

1. **Content**: The dataset includes posts, discussions, and articles shared on Hacker News during that time period.
2. **Use Cases**:
    - **Benchmarking**: Organizations can use this dataset to benchmark their innovations against industry trends and discussions.
    - **Technological Insights**: Researchers and analysts can stay informed about technological breakthroughs and emerging trends.
    - **Business Strategies**: Formulating future-focused business strategies based on the discussions and trends highlighted in the dataset.
3. **Variety of Posts**: The dataset covers a wide range of topics, including technology, startups, programming, and more.

(1) Hacker News | Kaggle. https://www.kaggle.com/datasets/hacker-news/hacker-news.
(2) cypher-07/Cybersecurity-News-Article-Dataset - GitHub. https://github.com/cypher-07/Cybersecurity-News-Article-Dataset.
(3) OpenPipe/hacker-news · Datasets at Hugging Face. https://huggingface.co/datasets/OpenPipe/hacker-news.
(4) Hacker News Dataset - brightdata.com. https://brightdata.com/products/datasets/hacker-news.
(5) massanishi/hackernews-post-datasets: Datasets for hackernews posts - GitHub. https://github.com/massanishi/hackernews-post-datasets.
NIH Grant Abstracts
Full name: ExPORTER

**NIH Grant Abstracts: ExPORTER** is a valuable resource for researchers and data enthusiasts. It serves as an **open data repository** containing administrative information about **NIH-funded research projects**. Here are some key points about ExPORTER:

1. **Purpose and Transparency**:
   - ExPORTER is part of the NIH's **"open government" initiatives**.
   - It aims to enhance transparency by providing detailed data on NIH activities.
   - Researchers can use this data for **in-depth analyses** or integrate it into their own systems.

2. **Data Availability**:
   - ExPORTER offers **bulk administrative data** extracted from the larger **RePORTER database**.
   - The data is available for download in **CSV formats**.
   - Each file includes a **data dictionary** describing the fields.
   - Legacy files from the predecessor system, **CRISP**, are also accessible from FY 1970 to FY 2009.

3. **Annual Updates**:
   - The consolidated **Annual RePORTER project file** is released annually by the end of January.
   - It integrates the most recent grant information, contracts, intramural projects, and interagency agreements.
   - Updates occur when the President's budget for the fiscal year is released, incorporating **NIH Spending Categories data** (RCDC).

4. **Access and Usage**:
   - Researchers can explore **project abstracts**, publications, patents, and clinical studies.
   - ExPORTER provides a wealth of information for those interested in NIH-funded research.

(1) NIH RePORTER Announcement: - National Institutes of Health. https://reporter.nih.gov/exporter.
(2) NIH Research Portfolio Online Reporting Tools Database (ExPORTER). https://iiindex.org/datasets/exporter/.
(3) Funding | RePORT - National Institutes of Health. https://report.nih.gov/report-funding.
(4) NIH funding from 1985-2021 • nihexporter - GitHub Pages. https://rnabioco.github.io/nihexporter/.
(5) Find and Access NIH Grants Data Quickly and Easily with the RePORTER .... https://nexus.od.nih.gov/all/2021/08/16/find-and-access-nih-grants-data-quickly-and-easily-with-the-reporter-api/.
AmbigNQ

The **AmbigNQ dataset** is a valuable resource for exploring ambiguity in open-domain question answering. Let me provide you with some details:

1. **Task Description**:
   - Ambiguity is inherent in open-domain question answering, especially when dealing with new topics. It can be challenging to formulate questions that have a single, unambiguous answer.
   - The **AmbigQA task** involves predicting a set of question-answer pairs, where each plausible answer is paired with a disambiguated rewrite of the original question.

2. **Dataset Construction**:
   - To study this task, the researchers constructed the **AmbigNQ dataset**.
   - AmbigNQ covers **14,042 questions** from **NQ-open**, which is an existing open-domain QA benchmark.
   - Surprisingly, over **half** of the questions in NQ-open exhibit ambiguity.
   - The types of ambiguity are diverse and sometimes subtle, often becoming apparent only after examining evidence provided by a very large text corpus.

3. **Dataset Versions**:
   - There are three versions of the AmbigNQ dataset:
     - **Light Version**: Contains only inputs and outputs.
     - **Full Version**: Includes all annotation metadata.
     - **Evidence Version**: Provides semi-oracle evidence articles along with questions and answers.

(1) AmbigQA - University of Washington. https://nlp.cs.washington.edu/ambigqa/.
(2) ambig_qa.py · ambig_qa at main - Hugging Face. https://huggingface.co/datasets/ambig_qa/blob/main/ambig_qa.py.
(3) dataset_infos.json · ambig_qa at main - Hugging Face. https://huggingface.co/datasets/ambig_qa/blob/main/dataset_infos.json.
(4) AmbigQA/AmbigNQ README - GitHub: Let’s build from here. https://github.com/shmsw25/AmbigQA.

Introducing paper: AmbigQA: Answering Ambiguous Open-domain Questions
Summarize from Feedback

In the Learning to Summarize from Human Feedback paper, a reward model was trained from human feedback. The reward model was then used to train a summarization model to align with human preferences. This is the dataset of human feedback that was released for reward modelling. There are two parts of this dataset: comparisons and axis. In the comparisons part, human annotators were asked to choose the best out of two summaries. In the axis part, human annotators gave scores on a likert scale for the quality of a summary. The comparisons part only has a train and validation split, and the axis part only has a test and validation split.

Li et al. propose a variant with a subset of workers who annotate the data (details in [Appendix C.1](https://arxiv.org/pdf/2402.05133)) 

1. Reddit TL;DR (Seen) uses the top 10 workers from the original dataset.
2. Reddit TL;DR (Unseen) uses unseen workers in the validation set.

Introducing paper: Learning to summarize from human feedback
NumGLUE

The **NumGLUE dataset** is a valuable resource developed by the **Allen Institute for AI**. It focuses on evaluating the performance of AI systems in **mathematical reasoning tasks** that involve numbers within natural language text. Here are the key details about NumGLUE:

1. **Purpose and Inspiration**:
   - Drawing inspiration from the **GLUE benchmark**, which was designed for natural language understanding, NumGLUE aims to assess AI systems' ability to reason with numbers.
   - Unlike GLUE, which covers a wide range of NLP tasks, NumGLUE specifically targets tasks that require **simple arithmetic understanding**.

2. **Tasks**:
   - NumGLUE consists of **eight different tasks**, each involving numerical reasoning:
     - **Commonsense + Arithmetic Reasoning**
     - **Domain Specific + Arithmetic Reasoning**
     - **Commonsense + Quantitative Comparison**
     - **Fill-in-the-blanks Format**
     - **Reading Comprehension (RC) + Explicit Numerical Reasoning**
     - **Reading Comprehension (RC) + Implicit Numerical Reasoning**

3. **Challenges and Performance**:
   - Despite the availability of neural models, including state-of-the-art large-scale language models, **NumGLUE remains unsolved**.
   - These models perform significantly worse than humans, with an average gap of **46.4%**.
   - The dataset encourages knowledge sharing across tasks, especially for those with limited training data. Joint training on all tasks yields superior performance.

4. **Importance**:
   - NumGLUE promotes the development of systems capable of robust and general **arithmetic reasoning within language**.
   - It serves as a stepping stone toward more complex mathematical reasoning.

(1) NumGLUE Dataset — Allen Institute for AI. https://allenai.org/data/numglue.
(2) GitHub - allenai/numglue: NumGLUE: A Suite of Fundamental yet .... https://github.com/allenai/numglue.
(3) nyu-mll/glue · Datasets at Hugging Face. https://huggingface.co/datasets/nyu-mll/glue.

Introducing paper: NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks
ImageNet-D

ImageNet-D contains 4835 test images  featuring diverse backgrounds (3,764), textures (498), and materials (573). Generated by diffusion models, ImageNet-D achieves superior image fidelity and collection efficiency than prior studies.  Evaluation results show that ImageNet-D results in a significant accuracy drop to a range of vision models, from the standard ResNet visual classifier to the latest foundation models like CLIP and MiniGPT-4, significantly reducing their accuracy by up to
60%.

Introducing paper: ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object