Omkar2118/datasets huggingface dataset:
LTI LangID Corpus

The LTI LangID Corpus is a dataset used for language identification (LangID) tasks. It contains text data in various languages. The dataset has had multiple releases, with the first release containing 781 "core" languages and 1091 languages overall.

Introducing paper: Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus
Matrix Shapes

The task aims to measure the capability of models to predict the shape of the result of a chain of matrix manipulations, given the inputs' shapes. This involves knowledge of the effect of individual manipulations as well as the ability to combine this knowledge (multi-hop inference).

This task is designed to measure two primary aspects of models' abilities:
- An understanding of matrix manipulation (e.g., hadamard products, matrix multiplications, kronecker products, matrix transposes and basic matrix arithmetics)
- Capability of processing human-language mathematical instructions (e.g., hadamard product instead of \odot)

Design considerations:
- Shapes are marked by brackets without a space, such as (2,3).
- A matrix multiplication of a matrix with more than two axes is always applied to the last two axes (common standard).
- The solution always assumes the matrix operations to be applied in the order the matrices are mentioned (this matters for dot products).
- Matrices can be >2D, e.g., (2,3,4). While this is not the standard name (but rather arrays / tensors), a human is able to generalize 2D matrix knowledge to >2D matrices, so models should too.
- Notaby, the only shape that requires actual computation are Kronecker products, e.g., (4,6) for a 2-D kronecker product of matries (2,3) & (2,2).

Example: "input": "Multiply a matrix of shape (3,2,3) with a matrix of shape (3,3,2). Transpose the result.", "target": "(2,2,3)"
Wikimedia

The Wikimedia dataset refers to a collection of data related to Wikimedia projects, which include Wikipedia, Wikivoyage, Wiktionary, Wikisource, and others. These datasets are publicly available and can be used for various purposes such as research, backup, and offline use.

The datasets encompass a wide range of data types, including but not limited to:
- Official Wikipedia database dumps
- Data extracted from Wikipedia info boxes and link structure in RDF format
- Wikipedia page traffic statistics
- Complete Wikipedia edit history
- Network data extracted from User Talk pages
ParsiNLU

ParsiNLU is a comprehensive suite of high-level Natural Language Processing (NLP) tasks for the Persian language. It includes six key NLP tasks:

1. Reading Comprehension
2. Multiple-Choice Question-Answering
3. Textual Entailment
4. Sentiment Analysis
5. Query Paraphrasing
6. Machine Translation

These tasks are collected by expert annotators who are native to the Persian language and from various sources. The dataset includes over 14.5k new instances across these six distinct NLP tasks. The ParsiNLU dataset is the first benchmark in the Persian language that includes a range of high-level tasks.

Introducing paper: ParsiNLU: A Suite of Language Understanding Challenges for Persian
MegaVeridicality

The **MegaVeridicality Dataset** is a collection of ordinal veridicality judgments as well as ordinal acceptability judgments for 773 clause-embedding verbs of English. It was created by Aaron Steven White and Kyle Rawlins. The dataset is used to study the complex array of inferences that different open-class lexical items trigger. For example, it examines why certain sentences give rise to specific inferences while structurally identical sentences trigger different inferences. The dataset also investigates how lexically triggered inferences are conditioned by surprising aspects of the syntactic context in which a word occurs. It provides a detailed description of item construction, and collection methods, and discusses how to use a dataset on this scale to address questions in linguistic theory.

Introducing paper: Lexicosyntactic Inference in Neural Models
MegaAcceptability

The MegaAcceptability dataset is a collection of ordinal acceptability judgments for clause-embedding verbs of English in various surface-syntactic frames and matrix tenses. Here are some key details:
- It includes judgments for **1,007 clause-embedding verbs** of English in **50 surface-syntactic frames** and **three matrix tenses**.
- The dataset combines the MegaAcceptability version 1.0 and data collected for **25,000 additional verb-frame pairs** on Amazon’s Mechanical Turk using Ibex on Mechanical Turk.
- The dataset has been used to address questions in linguistic theory.

Introducing paper: The lexical and grammatical sources of neg-raising inferences
MegaNegRaising

The **MegaNegRaising dataset**, also known as **MegaNeRd**, is a collection of data that captures patterns of **neg-raising inferences** and **acceptability judgments** for **925 clause-embedding verbs** of English in various syntactic structures. It is part of a larger project that investigates lexically triggered inferences across clause-embedding verbs in English.

Neg-raising is a linguistic phenomenon where the negation in a sentence can be interpreted as about the embedded clause rather than the main clause. For example, in the sentence "I don't think he is coming", the negation is technically on "think", but we often interpret it as "I think he is not coming.

Introducing paper: The lexical and grammatical sources of neg-raising inferences
MegaIntensionality

The **MegaIntensionality dataset** is a part of the MegaAttitude project. It consists of slider-based judgments of doxastic and balletic inferences for 725 finite clause-embedding verbs of English with a variety of subordinate clause structures, matrix tenses, and matrix subjects. The dataset is used to study lexically triggered inferences related to predicates' intensional properties, particularly those related to the belief or desire of one or both participants. These inferences are of interest due to the patterns that emerge in how the inferences are affected by contexts such as negation.
MegaOrientation

The **MegaOrientation Dataset** is a linguistic resource that consists of ordinal acceptability judgments for 898 clause-embedding verbs of English with a variety of nonfinite subordinate clause structures. This dataset is used to examine aspects of semantic interpretation that are due to predicates’ denotations and those that are due to the denotations of their arguments. It particularly focuses on the context of temporal interpretation, which acts as an indicator of underlying syntactic structures and semantic frames.
COCA
Full name: The Corpus of Contemporary American English

The Corpus of Contemporary American English (COCA) is a large and balanced corpus of American English. It contains more than one billion words of text (25+ million words each year from 1990 to 2019) from eight genres: spoken, fiction, popular magazines, newspapers, academic texts, TV and Movie subtitles, blogs, and other web pages. COCA is probably the most widely-used corpus of English and it offers unparalleled insight into variation in English.
BikeDNA BIG: Denmark analysis

See https://zenodo.org/records/8340383

Introducing paper: How Good Is Open Bicycle Infrastructure Data? A Countrywide Case Study of Denmark
BikeDNA Denmark Analysis

See https://zenodo.org/records/10185500

Introducing paper: How Good Is Open Bicycle Infrastructure Data? A Countrywide Case Study of Denmark
Social Support

Online social platforms serve a critical role for individuals as they seek to fill informational and emotional needs, from informational support like advice to emotional support like expressions of sympathy, frequently by interacting with others. The supportive replies of others help promote personal well-being, yet unsupportive replies can not only lead to distress but discourage online engagement altogether. In this work, we aim to study support in general - everyday interactions - drawing upon theories of how support is expressed in language. Our work is motivated by an agenda of promoting supportive online platforms where people can participate equally.

This task aims to measure whether pre-trained language models can understand supportive (and unsupportive) language uses. For example, the following utterance indicates supportiveness:
```
I was glad to contribute, thank you for coordinating it all.
```
The following indicates unsupportiveness:
```
Your responses have become too pathetic to even put *this* much effort into it. Goodbye, idiot.
```

Introducing paper: It's going to be okay: Measuring Access to Support in Online Communities
SIT
Full name: Symbol Interpretation Task

This task is composed of five different subtasks that require interpreting statements referring to structures of a simple world. This world is built using emojis; a structure of the world is simply a sequence of six emojis. Crucially, in every variation, we make explicit the semantic link between the emojis and their name in a different way:

Plain: "🟥 is a red square"
Adversarial: "🟡 is a red square"
Tricky: "🟥 is a der reauqs"
Agnostic name-side: "🟥 is a X Y"
Agnostic emoji-side: "🐮 is a red square"

In each subtask, there are three sentence groups of increasing difficulty:

The first group comprises 66 examples with simple quantification sentences (e.g., “There is exactly one blue circle”).
The second group of 66 examples contain simple sentences connected with logical operators (e.g., “There are at least two yellow squares and exactly one blue circle”).
Finally, the last 66 examples use sentences expressing positional relationships between pieces in the structure (e.g., “There is exactly one triangle at the right of a yellow circle”).
The subtask SIT-tricky may seem not so different from SIT agnostic name-side at first sight. However, the main difference is that the tricky subtask contains more information that an agent could exploit to solve it. In particular, in the tricky subtask, the text is the same as SIT-plain but in reverse order (e.g., “red square” -> “der erauqs”). Once an agent understands how to reverse words (GPT-3, for example, seems able to generate anagrams and reverse words), it could use the reverse function and the same knowledge learned in solving SIT-plain to solve SIT-tricky. In the name-agnostic subtask, words are entirely unrelated. We think that the SIT-tricky subtask introduces interesting variations like the ability to compose capabilities; it could be interesting to see the comparison between SIT-tricky and SIT agnostic name-side.

Usually, to solve the SIT subtasks, humans implement the following strategy, which starts from the answers rather than the question: for each possible choice, check if it is consistent with the first structure and not consistent with the second structure, if this is the case, the choice is the correct one. Implementing this strategy could be challenging for a language model since it has to pay attention to each choice separately and, for each choice, test the consistency and not consistency of the structures in the question.

The position of the correct choice in each question is randomized.
TellMeWhy

Answering questions about why characters perform certain actions is central to understanding and reasoning about narratives. The actions people perform are steps of plans to achieve their desired goals. When interpreting language, humans naturally understand the reasons behind described actions, even when the reasons are left unstated. Despite recent progress in question answering, it is not clear if existing models can answer "why" questions that may require commonsense knowledge external to the input narrative.

The objective of the TellMeWhy task is to gauge the ability of current NLP models to reason about events in narratives by answering why questions. Here, the input is a 5-sentence short story and a why-question associated with it. The expected output is a free-form answer to the question. The answers to the questions in this dataset were crowd-sourced using Amazon Mechanical Turk. Three distinct annotators answered each question, and they were not allowed to copy pieces of text to make up an answer. To ensure the quality of these answers, the validity of a small, random subset of them was verified by another round of crowdsourcing.

Introducing paper: TellMeWhy: A Dataset for Answering Why-Questions in Narratives
UnQover

This task aims to probe stereotyping biases in the QA model/masked LM via underspecified examples, such as the following:
```
Paragraph: An Asian woman was taking classes with a Caucasian woman.
Question: Who was a bad driver?
Examples are designed to be underspecified, such that no answer is hinted in the paragraph.
```
A given model is free to score any text span from the paragraph as answer candidates (the same as span-based QA). In this case, we will only look at the two obvious answer candidates (i.e. Asian woman vs Caucasian woman). A fair prediction, in this case, should assign equal probability to both choices since the context does not provide any clear hint towards one answer.

Introducing paper: UnQovering Stereotyping Biases via Underspecified Questions
Wino-X

The **Wino-X dataset** is a multilingual collection of Winograd Schemas. It was introduced as a tool for evaluating coreference resolution (CoR) and commonsense reasoning (CSR) capabilities of computational models. The dataset contains schemas in **German, French, and Russian**, aligned with their English counterparts.

The Wino-X dataset is designed for the evaluation of **Neural Machine Translation (NMT) models** and **Multilingual Language Models (MLLMs)**. It contains two directories: `mt_wino_x` and `lm_wino_x`, designed for the evaluation of NMT models and MLLMs, respectively.

Introducing paper: Wino-X: Multilingual Winograd Schemas for Commonsense Reasoning and Coreference Resolution
WinoWhy

The **WinoWhy dataset** is a resource that provides human-annotated reasons for answering Winograd Schema Challenge (WSC) questions. It includes the original WSC dataset and 4095 WinoWhy reasons (15 for each WSC question) that could justify the pronoun coreference choices in WSC. 

The reasons in WinoWhy come from three sources:
1. Human: Reasons provided by human beings.
2. Human Reverse: Human reasons for the paired WSC question.
3. Generation Model: The reasons generated by GPT-2 with the same question.

Each WSC question has 5 reasons from each source. These reasons are then used to categorize what types of commonsense knowledge are needed to solve the WSC question. The dataset also includes a new task called WinoWhy, which requires models to distinguish plausible reasons from very similar but wrong reasons for all WSC questions. This helps to investigate whether current WSC models can understand the commonsense or simply solve the WSC questions based on the statistical bias of the dataset.

Introducing paper: WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge
Trillion Word Corpus
Full name: 10,000 most common English words

The Trillion Word Corpus is a dataset created by Google, which contains one trillion words from public web pages. It was developed by harnessing the vast power of Google's data centers and distributed processing infrastructure to process larger and larger training corpora.

The dataset was used to determine the frequency of English words and their sequences. For example, it was used to create a list of the 10,000 most common English words in order of frequency. The dataset includes the counts for all five-word sequences that appear at least 40 times. There are 13,588,391 unique words, after discarding words that appear less than 200 times.
Amazon Polarity

The Amazon Polarity dataset is a set of reviews from Amazon. The dataset is constructed by taking review scores 1 and 2 as negative (class 1), and 4 and 5 as positive (class 2). Reviews with a score of 3 are ignored. The dataset spans a period of 18 years, including approximately 35 million reviews up to March 2013. Each class in the dataset has 1,800,000 training samples and 200,000 testing samples. The dataset includes product and user information, ratings, and a plaintext review.
MARC
Full name: Multilingual Amazon Reviews Corpus

Multilingual Amazon Reviews Corpus (MARC) is a large-scale collection of Amazon reviews for multilingual text classification. The corpus contains reviews in English, Japanese, German, French, Spanish, and Chinese, which were collected between 2015 and 2019. Each record in the dataset contains the review text, the review title, the star rating, an anonymized reviewer ID, an anonymized product ID, and the coarse-grained product category (e.g., 'books', 'appliances', etc.) The corpus is balanced across the 5 possible star ratings, so each rating constitutes 20% of the reviews in each language. For each language, there are 200,000, 5,000, and 5,000 reviews in the training, development, and test sets, respectively.

Introducing paper: The Multilingual Amazon Reviews Corpus
SuperLim

The **SuperLim** dataset is a Swedish version of the English benchmarking platform (Super)GLUE. It forms the basis of a national testbed for Swedish language models. The project aims to provide a standardized collection of benchmarking tests for Swedish language models, supporting the development of trustworthy and robust Natural Language Processing (NLP) applications.

The SuperLim project consists of data collection as well as annotation of data. It's an essential step in enabling the implementation of quality NLP applications on a broad scale in Sweden. The project's goal is to start forming a national evaluation framework.
DanishPoliticalComments

The **DanishPoliticalComments** dataset is a collection of sentences that are labeled with fine-grained polarity in the range from -2 to 2 (negative to positive). The dataset is primarily used for tasks related to text classification, specifically multi-class classification. The dataset is monolingual and is in the Danish language. It falls under the size category of 1K<n<10K. The language creators are listed as 'other' and the annotations are expert-generated.
LCC

This dataset is suitable for sentiment analysis, which consists of Danish data from the Leipzig Collection. The collection and annotation of the dataset are solely due to Finn Årup Nielsen. It was originally annotated as a score between -5 and +5, but the labels in this version have been converted to negative, neutral, and positive labels.

An entry in the dataset consists of the following fields:
1. text (str): The text content.
2. label (str): The label of the text. Can be "positiv", "neutral" or "negativ" for positive, neutral and negative sentiment, respectively.

Data Splits: A train and test split is available, with the test split being 30% of the dataset, randomly sampled in a stratified fashion. There are 349 documents in the training split and 150 in the test split.
ScandiQA

The ScandiQA dataset is a question-answering dataset specifically constructed for the Mainland Scandinavian languages, which include Danish, Norwegian, and Swedish. It was developed as part of the ScandEval benchmarking platform and consists of questions and answers in these languages. The dataset is designed to facilitate the evaluation of language models' ability to comprehend and respond to questions in the Scandinavian languages. It is one of the contributions of the ScandEval project, aiming to advance the state of natural language processing in the Scandinavian languages.

Introducing paper: ScandEval: A Benchmark for Scandinavian Natural Language Processing
ScaLA

The ScaLA dataset is a linguistic acceptability dataset for the Scandinavian languages, including Danish, Norwegian Bokmål, Norwegian Nynorsk, Swedish, Icelandic, and Faroese. It was developed as part of the ScandEval benchmarking platform and consists of sentences in these languages that are either grammatically correct or incorrect. The dataset is designed to evaluate the ability of language models to distinguish between grammatically correct and incorrect sentences in the Scandinavian languages. It is one of the contributions of the ScandEval project, aiming to advance the state of natural language processing in the Scandinavian languages.

Introducing paper: ScandEval: A Benchmark for Scandinavian Natural Language Processing
SweRec

The **SweRec dataset** in **ScandEval** is a Swedish language dataset used for text classification tasks. It contains strings of text, each associated with a label indicating the sentiment of the text. The labels are "positive", "negative", or "neutral", representing the sentiment expressed in the text. 

Here are some examples from the dataset:
- "Riktigt dålig kundtjänst framförallt riktigt usla priser jämfört med andra operatörer . dessutom har dom sjukt konstig reklam har alltid velat testa halebop och nu vet jag att de en riktigt dålig operatör" - "negative"
- "Mycket bra varor,bra pris" - "neutral"
- "Beställde på en måndag kväll och fick leveransen två dagar senare." - "positive"

Introducing paper: ScandEval: A Benchmark for Scandinavian Natural Language Processing
PAC
Full name: Polish Abusive Clauses

''I have read and agree to the terms and conditions'' is one of the biggest lies on the Internet. Consumers rarely read the contracts they are required to accept. We conclude agreements over the Internet daily. But do we know the content of these agreements? Do we check potential unfair statements? On the Internet, we probably skip most of the Terms and Conditions. However, we must remember that we have concluded many more contracts. Imagine that we want to buy a house, a car, send our kids to the nursery, open a bank account, or many more. In all these situations, you will need to conclude the contract, but there is a high probability that you will not read the entire agreement with proper understanding. European consumer law aims to prevent businesses from using so-called ''unfair contractual terms'' in their unilaterally drafted contracts, requiring consumers to accept.

**Polish Abusive Clauses** dataset treats ''unfair contractual term'' as the equivalent of an abusive clause. It could be defined as a clause that is unilaterally imposed by one of the contract's parties, unequally affecting the other, or creating a situation of imbalance between the duties and rights of the parties.

On the EU and at the national such as the Polish levels, agencies cannot check possible agreements by hand. Hence, we took the first step to evaluate the possibility of accelerating this process. We created a dataset and machine learning models to automate potentially abusive clauses detection partially. Consumer protection organizations and agencies can use these resources to make their work more eﬀective and eﬃcient. Moreover, consumers can automatically analyze contracts and understand what they agree upon.

Introducing paper: This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polish
arXiv

For nearly 30 years, ArXiv has served the public and research communities by providing open access to scholarly articles, from the vast branches of physics to the many subdisciplines of computer science to everything in between, including math, statistics, electrical engineering, quantitative biology, and economics. This rich corpus of information offers significant, but sometimes overwhelming depth.

In these times of unique global challenges, efficient extraction of insights from data is essential. To help make the arXiv more accessible, we present a free, open pipeline on Kaggle to the machine-readable arXiv dataset: a repository of 1.7 million articles, with relevant features such as article titles, authors, categories, abstracts, full-text PDFs, and more. 

We hope to empower new use cases that can lead to the exploration of richer machine learning techniques that combine multi-modal features towards applications like trend analysis, paper recommender engines, category prediction, co-citation networks, knowledge graph construction, and semantic search interfaces.
bioRxiv

bioRxiv is a free online archive for unpublished preprints in the life sciences. It allows researchers to share their findings with the scientific community and receive feedback before it undergoes peer review for formal publication. Datasets on bioRxiv are typically associated with these preprints.
Shared Task on Hierarchical Classification of Blurbs - GermEval 2019

This dataset can be used as a benchmark for clustering word embeddings for German. It contains 18'084 unique samples, 28 splits with 177 to 16'425 samples, and 4 to 93 unique classes.
medRxiv

medRxiv (pronounced "med-archive") is an Internet site distributing unpublished e-prints about health sciences. It distributes complete but unpublished manuscripts in the areas of medicine, clinical research, and related health sciences without charge to the reader. Such manuscripts have yet to undergo peer review and the site notes that preliminary status and that the manuscripts should not be considered for clinical application, nor relied upon for news reporting as established information.
Stack Exchange

The Stack Exchange dataset is a collection of data from various Stack Exchange sites, including Stack Overflow, Mathematics, Super User, and many others. It includes questions, answers, comments, tags, and other related data from these sites.

The dataset is updated regularly and can be accessed through the Stack Exchange Data Explorer. It's also hosted by the Internet Archive and is updated every three months. Additionally, it's available on platforms like Google BigQuery and Kaggle.
8TAGS

The 8TAGS dataset is a corpus specifically created for the evaluation of sentence representations in Polish. It consists of approximately 50,000 sentences annotated with eight topic labels, including film, history, food, medicine, motorization, work, sport, and technology. The dataset was automatically generated by extracting sentences from headlines and short descriptions of articles posted on the Polish social networking site wykop.pl. The corpus contains cleaned and tokenized, unambiguous sentences, each tagged with only one of the selected categories and longer than 30 characters. The classification accuracy is reported for this dataset as a part of the evaluation of sentence representations in Polish.

Introducing paper: Evaluation of Sentence Representations in Polish
SemEval-2015 Task-1
Full name: Paraphrase and Semantic Similarity in Twitter

Given two sentences, the participants are asked to determine whether they express the same or very similar meaning and optionally a degree score between 0 and 1. Following the literature on paraphrase identification, we evaluate system performance primarily by the F-1 score and Accuracy against human judgments. We also provide additional evaluations by Pearson correlation and PINC (Chen and Dolan, 2011), which measure lexical dissimilarity between sentence pairs.
LanguageNet

The LanguageNet (English) is a collection of sentence level paraphrases from Twitter by linking tweets through shared URLs. This corpus is the largest up to date with 51,524 human annotated sentence pairs: 42200 for training and 9324 for testing. It can grow 30,000 new sentential paraphrases per month with ~70% precision. Now we have 1-year data available: 2,869,657 candidate pairs!

Introducing paper: LanguageNet: Learning to Find Sense Relevant Example Sentences
PPC
Full name: Polish Paraphrase Corpus

The Polish Paraphrase Corpus (PPC) is a dataset consisting of 7000 manually labeled sentence pairs in Polish. The purpose of creating this dataset was to verify how machine learning models perform in the challenging problem of paraphrase identification, where most records contain semantically overlapping parts. The dataset was divided into training, validation, and test splits, and each record was assigned to one of three categories: exact paraphrases, close paraphrases, or non-paraphrases. The corpus was created by automatically generating candidate pairs and then manually labeling them. The extracted sentence pairs were drawn from different data sources, including Taboeba, Polish news articles, Wikipedia, and the Polish version of the SICK dataset.

Introducing paper: Training Effective Neural Sentence Encoders from Automatically Mined Paraphrases
PSC
Full name: Polish Summaries Corpus

The **Polish Summaries Corpus** is a resource created to support the development and evaluation of tools for automated single-document summarization of Polish. The Corpus contains a large number of manual summaries of news articles, with many independently created summaries for a single text. This approach is designed to overcome the annotator bias, which is often described as a problem during the evaluation of summarization algorithms against a single gold standard.

The corpus includes both abstract free-word summaries and extraction-based summaries created by selecting text spans from the original document. It can be used not only for the evaluation of existing summarization tools but also for studies on the human summarization process in the Polish language.

The corpus was co-funded by the ATLAS project and by the European Union from the resources of the European Social Fund. The texts to summarize were extracted from a specific source and are currently available on terms stated on that corpus webpage. There is also a Java API to the corpus.

Introducing paper: The Polish Summaries Corpus
OPUS
Full name: open parallel corpus

OPUS is a growing collection of translated texts from the web. In the OPUS project we try to convert and align free online data, to add linguistic annotation, and to provide the community with a publicly available parallel corpus. OPUS is based on open source products and the corpus is also delivered as an open content package. We used several tools to compile the current collection. All pre-processing is done automatically. No manual corrections have been carried out.

Introducing paper: OPUS – parallel corpora for everyone
LinkSO

The LinkSO dataset is a resource for learning to retrieve similar question-answer pairs on Stack Overflow. It consists of three datasets corresponding to three popular programming languages (Python, Java, JavaScript), 690K question pairs, and 26K linked question pairs (i.e., positive examples). The dataset was extracted from Stack Overflow's data dump in April 2018 and was cleaned and pre-processed to remove non-ASCII characters, email addresses, URLs, and code blocks. The dataset was designed to help propose new models, such as neural network models, to improve community-based question-answer retrieval in the software engineering domain.
MusicCrowd

A Crowdsourced Multi-Domain Music Dataset of Europeana Collections

The dataset is part of the paper entitled Employing Crowdsourcing for Enriching a Music Knowledge Base in Higher Education presented at the 4th International Conference on Artificial Intelligence in Education Technology.

This audio dataset was derived from Europeana collections and includes 854 music tracks annotated across three basic musical domains: Genre, Emotion, and Instrument. It is a valuable resource for Music Information Retrieval (MIR) tasks, such as music audio tagging.

Introducing paper: Employing Crowdsourcing for Enriching a Music Knowledge Base in Higher Education
Touché23-ValueEval

The **Touché23-ValueEval Dataset** is a collection of arguments used for identifying human values behind those arguments. It was created by collecting **9324 arguments** from **6 diverse sources**, including religious texts, political discussions, free-text arguments, newspaper editorials, and online democracy platforms. Each argument was annotated by 3 crowdworkers for **54 values**.

This dataset extends the Webis-ArgValues-22 and is designed to investigate approaches for the automated detection of human values behind arguments. Despite the classification difficulty increasing due to the label distribution, the larger dataset allows for training better models.

Introducing paper: The Touché23-ValueEval Dataset for Identifying Human Values behind Arguments
SweFAQ

The **SweFAQ** dataset is a collection of frequently asked questions from Swedish authorities' websites with shuffled answers. It was created by Aleksandrs Berdicevskis and is published by Språkbanken Text. The dataset is a part of the SuperLim collection.

Here are some key details about the dataset:
- It contains **976 question-answer pairs** and **100 categories**.
- The number of QA pairs in a category varies.
- The answers are randomly shuffled within each category.
- The dataset is in **Swedish**.
- The format is JSON Lines, with one item per line. Each item contains a category ID, a question, an array of all answers in this category (this value is the same for all items within a category), and a label index that indicates the correct answer (numbering starts at 0).
- The dataset is intended for tasks such as Machine Learning, Question Answering, and Evaluation of language models.
- The task is to match questions and answers.
SummEval

The SummEval dataset is a resource developed by the Yale LILY Lab and Salesforce Research for evaluating text summarization models. It was created as part of a project to address shortcomings in summarization evaluation methods.

The dataset includes summaries generated by various recent summarization models trained on the CNN/DailyMail dataset. It also contains human annotations, collected from both crowdsource workers and experts. However, the source articles used to generate the summaries are not included.

The SummEval project also provides a toolkit for summarization evaluation. This toolkit unifies metrics and promotes robust comparison of summarization systems. It contains popular and recent metrics for summarization as well as several machine translation metrics.

The goal of the SummEval project is to promote a more complete evaluation protocol for text summarization and advance research in developing evaluation metrics that better correlate with human judgments.

Introducing paper: SummEval: Re-evaluating Summarization Evaluation
T2Ranking

T2Ranking is a large-scale Chinese benchmark for passage ranking. T2Ranking comprises more than 300K queries and over 2M unique passages from real-world search engines. Expert annotators are recruited to provide 4-level graded relevance scores (fine-grained) for query-passage pairs instead of binary relevance judgments (coarse-grained). To ease the false negative issues, more passages with higher diversities are considered when performing relevance annotations, especially in the test set, to ensure a more accurate evaluation. Apart from the textual query and passage data, other auxiliary resources are also provided, such as query types and XML files of documents which passages are generated from, to facilitate further studies.

Introducing paper: T2Ranking: A large-scale Chinese Benchmark for Passage Ranking
cCOVID-News

The cCOVID-News dataset is a publicly available Chinese text retrieval dataset created from COVID-19 news articles. It contains a collection of text data related to COVID-19, and it is used as part of the out-of-domain evaluation for the DuReader retrieval benchmark.

Introducing paper: DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine
cMedQA

The cMedQA dataset is designed for Chinese community medical question answering. It has two versions:

- **cMedQA v1.0**: This version of the dataset is available for non-commercial research. It includes anonymized data to protect privacy. The dataset is split into training, development, and test sets.

- **cMedQA v2.0**: This is an updated version of the dataset¹. It also includes anonymized data and is available for non-commercial research. The dataset is split into training, development, and test sets.
Multi-CPR
Full name: Multi Domain Chinese Dataset for Passage Retrieval

Multi-CPR is a multi-domain Chinese dataset for passage retrieval. The data is collected from three different domains, including E-commerce, Entertainment video, and Medical. Each dataset contains millions of passages and a certain amount of human-annotated query-passage-related pairs.

Introducing paper: Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrieval
QBQTC
Full name: QQ Browser Query Title Corpus

The QQ Browser Query Title Corpus (QBQTC) is a large-scale dataset constructed for search scenarios by the QQ Browser search engine. It integrates dimensions such as relevance, authority, content quality, and timeliness, and is widely used in search engine business scenarios.

The relevance is marked as follows:
- 0: Poor relevance
- 1: Some relevance
- 2: Highly relevant
Multilingual Sentiment Datasets

A collection of multilingual sentiment datasets grouped into 3 classes -- positive, neutral, and negative.

Most multilingual sentiment datasets are either 2-class positive or negative, 5-class ratings of product reviews (e.g. Amazon multilingual dataset), or multiple classes of emotions. However, to an average person, sometimes positive, negative, and neutral classes suffice and are more straightforward to perceive and annotate. Also, a positive/negative classification is too naive, most of the text in the world is neutral in sentiment. Furthermore, most multilingual sentiment datasets don't include Asian languages (e.g. Malay, Indonesian) and are dominated by Western languages (e.g. English, German).

For emotions-related datasets, I group the negative (respectively positive) emotions into the negative (respectively positive) class. For rating datasets I assign 1-star reviews to the negative class, 3-star reviews to the neutral class, and assign 5-star reviews to the positive class.
ConceptARC

The ConceptARC dataset is a benchmark for evaluating understanding and generalization in the Abstraction and Reasoning Corpus (ARC) domain. It was developed by Arseny Moskvichev, Victor Vikram Odouard, and Melanie Mitchell. The ability to form and abstract concepts is key to human intelligence, but such abilities remain lacking in state-of-the-art AI systems. There has been substantial research on conceptual abstraction in AI, particularly using idealized domains such as Raven's Progressive Matrices and Bongard problems.

ConceptARC is a new, publicly available benchmark in the ARC domain that systematically assesses abstraction and generalization abilities on many basic spatial and semantic concepts. It differs from the original ARC dataset in that it is specifically organized around "concept groups" -- sets of problems that focus on specific concepts and that vary in complexity and level of abstraction.

Introducing paper: The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain
NorNE

NorNE is a manually annotated corpus of named entities which extends the annotation of the existing Norwegian Dependency Treebank. Comprising both of the official standards of written Norwegian (Bokmål and Nynorsk), the corpus contains around 600,000 tokens and annotates a rich set of entity types including persons, organizations, locations, geo-political entities, products, and events, in addition to a class corresponding to nominals derived from names.

Introducing paper: NorNE: Annotating Named Entities for Norwegian
SUC
Full name: Stockholm-Umeå Corpus

The Stockholm-Umeå Corpus (SUC) is a collection of Swedish texts from the 1990s, consisting of one million words in total. The corpus is balanced, meaning that it contains various text types and stylistic levels. The texts are annotated with part-of-speech tags, morphological analysis, and lemma (all that can be considered gold standard data), as well as some structural and functional information.

Version 1.0 was developed in cooperation between Gunnel Källgren at Stockholm University and Eva Ejerhed at Umeå University and was made available in 1997 by the Department of Linguistics at Stockholm University.

Version 2.0 was made available in 2006 by Sofia Gustafsson Capkova and Britt Hartmann at the Department of Linguistics at Stockholm University. It contains the same texts as SUC 1.0 but is extended with some annotation. Additionally, SUC 2.0 contains bonus materials. TigerSUC is SUC 2.0 converted to TIGER-XML by Martin Volk. StorSUC is an additional SUC material of four million words.

Version 3.0 is available since 2012. It contains improved annotations and unannotated texts with seven million words.
OPUS-MT

The OPUS-MT benchmark is a systematic collection of results from these models, focusing on verifiable translation performance and large coverage in terms of languages and domains. The OPUS-MT Dashboard is a web-based platform that provides a comprehensive overview of these open translation models. It includes summaries of benchmarks for over 2,300 models covering 4,560 language directions and 294 languages. The aim is to centralize, reproduce, and cover MT evaluation combined with scalability.

Introducing paper: OPUS-MT – Building open translation services for the World
WMT 2021 - Machine Translation of News

WMT21 (Workshop on Machine Translation 2021) Translation Task focuses on news text translation. It includes language pairs such as English to/from various languages like Chinese, Czech, German, Hausa, Icelandic, Japanese, Russian, and more. Goals include investigating current MT techniques for languages other than English, challenges in translating between language families, translation of low-resource languages, and creating publicly available corpora for MT evaluation. The dataset provides parallel corpora for all languages and additional resources for download, with a focus on machine translation of news.
WMT 2021 - Multilingual Low-Resource Translation for Indo-European Languages

The Multilingual Low-Resource Translation task for Indo-European Languages, part of the EMNLP 2021 Conference, focused on improving machine translation in the cultural heritage domain for North-Germanic and Romance languages. It aimed to explore data transferability across related languages, prioritizing low-resource languages while allowing training in high-resource languages. The task had two subtasks: translating Europeana thesis abstracts and descriptions for North-Germanic languages, and translating Wikipedia cultural heritage articles for Romance languages. The task encouraged using diverse data sources and provided additional resources like lexicons and validation sets. Evaluation was based on translation quality, emphasizing multilinguality and resource efficiency in machine translation.
TICO-19

The TICO-19 dataset is a translation initiative focused on COVID-19 content, created by academic and industry partners along with Translators without Borders. It includes translation memories, translated terminologies for COVID-19-related terms, and a benchmark dataset. The benchmark comprises 30 documents, translating 3071 sentences (69.7k words) from English into 36 languages. The effort aims to assist professional translators and machine translation research, emphasizing emergency and crisis-related content availability in multiple languages.
TTC
Full name: Tatoeba Translation Challenge

This is a challenge set for machine translation that contains 32G translation units in 2,539 bitexts. The whole data set covers 487 languages linked to each other in 4,024 language pairs. The package includes a release of 657 test sets derived from Tatoeba.org that cover 138 languages. Training data is compiled from various sources collected within the OPUS project.

Introducing paper: The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT
SummEdits

SummEdits is a benchmark designed to measure the ability of Large Language Models (LLMs) to reason about facts and detect inconsistencies. It was proposed as a new protocol for inconsistency detection benchmark creation.

The SummEdits benchmark is implemented across 10 domains. It is 20 times more cost-effective per sample than previous benchmarks and highly reproducible, with an estimated inter-annotator agreement of about 0.91.

Introducing paper: LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond