Omkar2118/datasets huggingface dataset:
COPEN
Full name: COnceptual knowledge Probing bENchmark

COPEN is a COnceptual knowledge Probing benchmark that aims to analyze the conceptual understanding capabilities of Pre-trained Language Models (PLMs). Specifically, COPEN consists of three tasks:

1. Conceptual Similarity Judgment (CSJ). Given a query entity and several candidate entities, the CSJ task requires selecting the most conceptually similar candidate entity to the query entity.
2. Conceptual Property Judgment (CPJ). Given a statement describing a property of a concept, PLMs need to judge whether the statement is true.
3. Conceptualization in Contexts (CiC). Given a sentence, an entity mentioned in the sentence, and several concept chains of the entity, PLMs need to select the most appropriate concept according to the context of the entity.

Introducing paper: COPEN: Probing Conceptual Knowledge in Pre-trained Language Models
VDS dataset: Multi exposure stack-based inverse tone mapping
Full name: VDS dataset: Multi exposure stack-based inverse tone mapping

* Have need seven multiple exposure ground truth images satisfying EV 0, ±1, ±2, ±3 for static scenes. 
  * 96 different scene image stacks (672 images; out-door: 504, indoor: 168 images)
* All the images are captured in an 8 bit JPG format using a Nikon D700 with a resolution of 4256 × 2832, which was resized to 912 × 608.

* Evaluation Code: https://drive.google.com/file/d/1xeCT3APYkTnxeotb_t0wxdSPzBLbnU_p/view?pli=1

Introducing paper: Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image
KAgentBench

KAgentBench is a benchmark dataset of over 3,000 human-edited, automated evaluation data for testing agent capabilities, with evaluation dimensions including planning, tool use, reflection, concluding, and profiling.

Introducing paper: KwaiAgents: Generalized Information-seeking Agent System with Large Language Models
EmoTag1200

The **EmoTag1200** dataset is a collection of resources for analyzing the emotion and sentiment of emojis as well as tweets written in English. The name EmoTag indicates its usefulness in exploiting emojis for emotional tagging.

The dataset includes the following resources:
- **Baseline Emoji Emotion Scores**: 1200 Emoji-Emotion pairs annotated by humans. It contains emotion scores ranging from 0 to 1 for the 150 most popular Twitter emojis for 8 emotion classes (i.e., anger, anticipation, disgust, fear, joy, sadness, surprise, and trust).
- **Interpretable Word Vectors**: A 620-dimensional vector representation of words and emojis trained on ~20.8 million emoji-centric Twitter data.
- **Raw Tweets**: This contains Tweet IDs of ~20.8 million tweets used in the experiments.
- **Word-Emoji Co-occurrence Frequencies**: This lexicon provides word-emoji co-occurrence frequencies observed in the dataset.
- **Emoji-Emoji Co-occurrence Frequencies**: This is the subset of the previous lexicon which contains only emoji-emoji co-occurrence counts observed in the dataset.

Introducing paper: EmoTag1200: Understanding the Association between Emojis and Emotions
Multi-species DNA

# Latent DNA Diffusion Dataset

- **Author**: Zehui Li
- **Size**: 1K<n<10K
- **License**: MIT
- **Files**:
  - `.gitattributes` (2.36 kB)
  - `README.md` (171 Bytes)
  - `combined.tar.gz` (336 MB LFS)
  - `metadata.json` (2.42 kB)
  - `sequence.csv` (330 MB LFS)
  - `split_human_mouse_rat_monkey.pt` (2.05 GB LFS, Pickle format)
  - `split_plants.pt` (1.3 GB LFS, Pickle format)
- **Purpose**: Text generation tasks
- **Details**: [Dataset Page](https://huggingface.co/datasets/Zehui127127/latent-dna-diffusion/tree/main)

Introducing paper: Latent Diffusion Model for DNA Sequence Generation
Sequential Instructions

This is the sequential instructions dataset from Understanding the Effects of RLHF on LLM Generalisation and Diversity. The dataset is in the alpaca_eval format.

For information about how the dataset was generated, see https://github.com/RobertKirk/stanford_alpaca.

The instructions in the dataset generally have a sequence of steps we expect the model to complete all at once. In our work, we found that RLHF models generalise much better to this dataset than SFT models when trained on the AlpacaFarm datasets.

Introducing paper: Understanding the Effects of RLHF on LLM Generalisation and Diversity
PSD Dataset
Full name: Paired Specular-Diffuse image dataset

We collect a total of 13,380 images captured on 2,210 different scenes. We use different objects and backgrounds to build our dataset.

The dataset can be divided into two different polarization conditions. One consists of 1010 groups of images photographed with fixed polarization angles, and the other one consists of 1200 pairs of images photographed with random polarization angles. For the former, each group contains 12 images photographed with 12 fixed polarization angles.
BIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation)

BIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation) represents a pioneering, cross-domain dataset that examines the impact of extensive database contents on text-to-SQL parsing. BIRD contains over 12,751 unique question-SQL pairs and 95 big databases with a total size of 33.4 GB. It also covers more than 37 professional domains, such as blockchain, hockey, healthcare and education, etc.

Introducing paper: Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs
KaMed

KaMed is a knowledge-aware medical dialogue dataset, which contains over 60,000 medical dialogue sessions with 5,682 entities (such as Asthma and Atropine).

Introducing paper: Semi-Supervised Variational Reasoning for Medical Dialogue Generation
RPLAN

RPLAN - a manually collected large-scale densely annotated dataset of floor plans from real residential buildings.
UWB_Data

Dataset for range data gathered using the Ultra-wide Band (UWB) MDEK 1001 Dev. kit in 3 different experimental scenarios. The dataset description document provides the details of experimentation and the dataset.
Machine_Mindset_MBTI_dataset

## Dataset introduction

There are four dimension in MBTI. And there are two opposite attributes within each dimension.

To be specific:

+ Energe: Extraversion (E) - Introversion (I)

+ Information: Sensing (S) - Intuition (N)

+ Decision: Thinking (T) - Feeling (F)

+ Execution: Judging (J) - Perceiving (P)

Based on the above, you can infer the content of the json file from its name.

The datasets follow the Alpaca format, consisting of instruction, input and output.

## How to use these datasets for behavior supervised fine-tuning (SFT)

For example, if you want to make an LLM behave like an ***ISFJ***, you need to select ***the four corresponding files*** (en_energe_introversion.json, en_information_sensing.json, en_decision_feeling.json, en_execution_judging.json). 

And use the four for SFT.

## How to use these datasets for direct preference optimization (DPO)

For example, if you want to make an LLM be ***more feeling (F) than thinking (T)*** by DPO, you need to select ***the two corresponding files*** (en_decision_feeling.json, en_decision_thinking.json). 

And then compile the two into the correct format for DPO. For the correct format, please refer to [this](https://github.com/PKU-YuanGroup/Machine-Mindset/blob/main/datasets/dpo/README.md).

Introducing paper: Machine Mindset: An MBTI Exploration of Large Language Models