Omkar2118/datasets huggingface dataset:
100poisonMpts

The **100PoisonMpts** dataset is a significant initiative in the realm of large language model governance. Developed collaboratively by **Alibaba Tmall Genie** and the **Tongyi Large Model Team**, this open-source Chinese dataset aims to address safety concerns associated with large language models, especially after the release of ChatGPT. The project's purpose is to ensure that information disseminated by these models aligns with safety, reliability, and human values.

Here are the key details about the **100PoisonMpts** dataset:

1. **Objective**:
   - The dataset focuses on **governance and safety** for large language models.
   - It responds to concerns about AI-generated content being safe, healthy, and aligned with human values.

2. **Data Collection**:
   - Over **ten renowned experts and scholars** participated as the initial "poisoning" annotators.
   - Each expert posed **100 cunning questions** designed to induce bias or discriminatory responses.
   - The large model's answers were then annotated, creating a dynamic interplay between "poisoning" and "detoxification."

3. **Significance**:
   - The project addresses public and academic concerns about AI models' ethical behavior.
   - It aligns with the **temporary management measures for generative AI services**, which emphasize preventing discrimination based on ethnicity, religion, nationality, gender, age, occupation, and health.

4. **Expertise and Diversity**:
   - The initial batch of questions covers diverse domains, including law, psychology, children's education, accessibility, obscure knowledge, intimate relationships, environmental fairness, and more.
   - Experts from fields such as environmental sociology, law, psychology, and child education contributed.

5. **Data Format**:
   - The dataset includes **906 samples** in the `train.json` file.
   - Each sample is in JSON format, containing the following fields:
     - `prompt`: Inductive questions proposed by domain experts.
     - `answer`: Expert-approved answers.
     - `domain_en`: Domain information (in English).
     - `domain_zh`: Domain information (in Chinese).
     - `answer_source`: Indicates whether the answer is from an expert or the large model.

6. **Usage**:
   - Researchers, technology companies, academic organizations, and NGOs can use this dataset to align their own large models with healthier, value-aligned data.
   - The dataset is available for exploration, alignment research, and model development.

Source: Conversation with Bing, 3/18/2024
(1) 100PoisonMpts: 中文大模型治理数据集. https://www.modelscope.cn/datasets/damo/100PoisonMpts/summary.
(2) 100PoisonMpts: 中文大模型治理数据集. https://www.modelscope.cn/datasets/damo/100PoisonMpts/files.
(3) 阿里100瓶毒药解马斯克难题？国内首个大模型价值对齐数据集开源，15万评测题上线！ - 知乎. https://zhuanlan.zhihu.com/p/643552287.
UAV-C

UAV tracking dataset with common corruptions tailored for unmannered  aerial video
BIGOS
Full name: Benchmark Intended Grouping of Open Speech

The **Benchmark Intended Grouping of Open Speech (BIGOS)** is a novel corpus specifically designed for **Polish Automatic Speech Recognition (ASR) systems**. This initial version of the benchmark comprises **1,900 audio recordings** from **71 distinct speakers**, sourced from **10 publicly available speech corpora**¹²³.

Here are some key points about BIGOS:

1. **Purpose**: BIGOS aims to facilitate systematic benchmarking and tracking of Polish ASR systems over time across a diverse range of publicly available corpora.
2. **Evaluation**: The benchmark evaluates both **proprietary** and **open-source ASR systems** on a diverse set of recordings and their corresponding original transcriptions.
3. **Findings**:
    - The performance of the latest open-source models is **comparable** to that of more established commercial services.
    - Model size significantly influences system accuracy.
    - Scenarios involving highly specialized or spontaneous speech show a decrease in accuracy.
4. **Challenges**: The study discusses the challenges of using public datasets for ASR evaluation and the limitations based on this inaugural benchmark.
5. **Availability**: The BIGOS corpus and associated tools are **publicly available** for replication and customization of the benchmark.

In summary, BIGOS provides a valuable resource for advancing Polish ASR research and improving the quality of speech recognition systems in the Polish language¹.
Check https://www.semanticscholar.org/paper/BIGOS-Benchmark-Intended-Grouping-of-Open-Speech-Junczyk

Source: Conversation with Bing, 3/18/2024
(1) BIGOS - Benchmark Intended Grouping of Open Speech Corpora for Polish .... https://annals-csis.org/proceedings/2023/pliks/1609.pdf.
(2) Annals of Computer Science and Information Systems, Volume 35. https://annals-csis.org/proceedings/2023/drp/1609.html.
(3) BIGOS - Benchmark Intended Grouping of Open Speech Corpora for Polish .... https://ieeexplore.ieee.org/abstract/document/10306084.
Z-Bench

**Z-Bench** is a fascinating **Chinese language model prompt dataset** developed by an enthusiastic AI-focused team at **Zhenfund**. Let me share some intriguing details about it:

1. **Purpose and Origin**:
   - Z-Bench was created to **qualitatively test large language models** (LLMs) for non-technical users, particularly those similar to ChatGPT products.
   - The team behind Z-Bench recognized that existing NLP task datasets had limitations, such as being unsuitable for dialogue systems or lacking good Chinese versions.
   - Their goal was to provide a **practical and user-friendly benchmark** for assessing the capabilities of LLMs.

2. **Dataset Overview**:
   - Z-Bench v1.0 covers three dimensions:
     - **Basic Abilities**: Includes 100 prompts.
     - **Advanced Abilities**: Contains 100 prompts.
     - **Vertical Abilities**: Provides 100 prompts.
   - In total, there are **300 prompts** designed to cover a wide range of NLP tasks.
   - The intention is not to create an academically rigorous dataset but rather to offer a useful tool for non-technical users.

Source: Conversation with Bing, 3/19/2024
(1) Z-Bench 1.0 by 真格基金 - GitHub. https://github.com/zhenbench/z-bench.
(2) ZhenBench · GitHub. https://github.com/zhenbench/.
(3) Z.bench as an estimate of sigma capability - Minitab. https://support.minitab.com/minitab/21/help-and-how-to/quality-and-process-improvement/capability-analysis/supporting-topics/capability-metrics/z-bench-as-an-estimate-of-sigma-capability/.
(4) undefined. https://docs.qq.com/sheet/DTEFsdkNERVVtR3BX.
Indonesian Twitter Emotion Dataset

This dataset contains 4,403 Indonesian tweets that have been labeled into five emotion classes: love, anger, sadness, joy, and fear.
Each line in the dataset consists of a tweet and its respective emotion label separated by a semicolon (,).
The first line serves as a header.
Pre-processing has been applied to the tweets, including replacing usernames (@) with [USERNAME], URLs/hyperlinks (http://… or https://…) with [URL], and sensitive numbers (e.g., phone numbers, invoice numbers) with [SENSITIVE-NO].

Check https://ieeexplore.ieee.org/document/8629262
SKSF-A

SKSF-A consists of seven distinct styles drawn by professional artists. SKSF-A contains 134 identities and corresponding sketches, making a total of 938 face-sketch pairs. SKSF-A is introduced in StyleSketch, Eurographics 2024.
https://kwanyun.github.io/stylesketch_project/

Introducing paper: Stylized Face Sketch Extraction via Generative Prior with Limited Data
Supplementary software and data: Bayesian multi-exposure image fusion for robust high dynamic range ptychography

Accompanying software package and data for the publication titled "Bayesian multi-exposure image fusion for robust high dynamic range ptychography".