IVUL-KAUST/MOLE huggingface dataset:
Dataset: DiS-ReX
Description: DiS-ReX is a multilingual dataset for distantly supervised relation extraction (DS-RE) spanning English, German, Spanish, and French. It contains over 1.5 million sentences aligned with DBpedia, featuring 36 relation classes and a 'no relation' class, designed to be a challenging benchmark.
Language: English, German, Spanish, French
Domain: wikipedia
Form: text
Size: 1836319 sentences
Subsets: Name: English
Language: English
Volume: 532499
Unit: sentences, Name: French
Language: French
Unit: sentences
Volume: 409087, Name: Spanish
Language: Spanish
Unit: sentences
Volume: 456418, Volume: 438315
Language: German
Name: German
Unit: sentences
Provider: Indian Institute of Technology
Tasks: relation extraction
License: unknown
Year: 2021
Authors: Abhyuday Bhartiya, Kartikeya Badola, Mausam
Affiliations: Indian Institute of Technology
Abstract: Distant supervision (DS) is a well established technique for creating large-scale datasets for relation extraction (RE) without using human annotations. However, research in DS-RE has been mostly limited to the English language. Constraining RE to a single language inhibits utilization of large amounts of data in other languages which could allow extraction of more diverse facts. Very recently, a dataset for multilingual DS-RE has been released. However, our analysis reveals that the proposed dataset exhibits unrealistic characteristics such as 1) lack of sentences that do not express any relation, and 2) all sentences for a given entity pair expressing exactly one relation. We show that these characteristics lead to a gross overestimation of the model performance. In response, we propose a new dataset, DiS-ReX, which alleviates these issues. Our dataset has more than 1.5 million sentences, spanning across 4 languages with 36 relation classes + 1 no relation (NA) class. We also modify the widely used bag attention models by encoding sentences using mBERT and provide the first benchmark results on multilingual DS-RE. Unlike the competing dataset, we show that our dataset is challenging and leaves enough room for future research to take place in this field.
Dataset: RELX
Description: A public benchmark dataset for cross-lingual relation classification in English, French, German, Spanish, and Turkish. It contains 502 parallel sentences created by selecting a subset from the KBP-37 test set and having them professionally translated and annotated.
Language: English, French, German, Spanish, Turkish
Domain: public datasets
Form: text
Size: 2510 sentences
Subsets: Name: English
Volume: 502
Unit: sentences
Language: English, Name: French
Volume: 502
Unit: sentences
Language: French, Name: German
Volume: 502
Unit: sentences
Language: German, Name: Spanish
Volume: 502
Unit: sentences
Language: Spanish, Name: Turkish
Volume: 502
Unit: sentences
Language: Turkish
Provider: Boğaziçi University
Tasks: cross-lingual information retrieval
License: MIT License
Year: 2020
Authors: Abdullatif Köksal, Arzucan Özgür
Affiliations: Department of Computer Engineering, Boğaziçi University
Abstract: Relation classification is one of the key topics in information extraction, which can be used to construct knowledge bases or to provide useful information for question answering. Current approaches for relation classification are mainly focused on the English language and require lots of training data with human annotations. Creating and annotating a large amount of training data for low-resource languages is impractical and expensive. To overcome this issue, we propose two cross-lingual relation classification models: a baseline model based on Multilingual BERT and a new multilingual pretraining setup, which significantly improves the baseline with distant supervision. For evaluation, we introduce a new public benchmark dataset for cross-lingual relation classification in English, French, German, Spanish, and Turkish, called RELX. We also provide the RELX-Distant dataset, which includes hundreds of thousands of sentences with relations from Wikipedia and Wikidata collected by distant supervision for these languages. Our code and data are available at: https://github.com/boun-tabi/RELX
Dataset: MultiSubs
Description: A large-scale multimodal and multilingual dataset of images aligned to text fragments from movie subtitles. It aims to facilitate research on grounding words to images in their contextual usage in language. The images are aligned to text fragments rather than whole sentences, and the parallel texts are multilingual.
Language: English, Spanish, Portuguese, French, German
Domain: TV Channels, public datasets
Form: text
Size: 5403281 sentences
Subsets: Language: English
Name: English
Volume: 2159635
Unit: sentences, Name: Spanish
Language: Spanish
Volume: 2159635
Unit: sentences, Name: Portuguese
Language: Portuguese
Volume: 1796095
Unit: sentences, Name: French
Volume: 1063071
Language: French
Unit: sentences, Name: German
Volume: 384480
Language: German
Unit: sentences
Provider: Imperial College London, Federal University of Mato Grosso
Tasks: machine translation, fill-in-the blank
License: CC BY 4.0
Year: 2022
Authors: Josiah Wang, Pranava Madhyastha, Josiel Figueiredo, Chiraag Lala, Lucia Specia
Affiliations: Imperial College London, Federal University of Mato Grosso
Abstract: This paper introduces a large-scale multimodal and multilingual dataset that aims to facilitate research on grounding words to images in their contextual usage in language. The dataset consists of images selected to unambiguously illustrate concepts expressed in sentences from movie subtitles. The dataset is a valuable resource as (i) the images are aligned to text fragments rather than whole sentences; (ii) multiple images are possible for a text fragment and a sentence; (iii) the sentences are free-form and real-world like; (iv) the parallel texts are multilingual. We set up a fill-in-the-blank game for humans to evaluate the quality of the automatic image selection process of our dataset. We show the utility of the dataset on two automatic tasks: (i) fill-in-the-blank; (ii) lexical translation. Results of the human evaluation and automatic models demonstrate that images can be a useful complement to the textual context. The dataset will benefit research on visual grounding of words especially in the context of free-form sentences, and can be obtained from https://doi.org/10.5281/zenodo.5034604 under a Creative Commons licence.
Dataset: MEE
Description: A large-scale Multilingual Event Extraction (MEE) dataset covering 8 typologically different languages. Sourced from Wikipedia, it provides comprehensive annotations for entity mentions, event triggers, and event arguments across diverse topics like politics, technology, and military.
Language: English, Spanish, Portuguese, Polish, Turkish, Hindi, Korean, Japanese
Domain: wikipedia
Form: text
Size: 31226 documents
Subsets: Name: English
Language: English
Volume: 13000
Unit: documents, Name: Portuguese
Language: Portuguese
Volume: 1500
Unit: documents, Name: Spanish
Language: Spanish
Volume: 3268
Unit: documents, Volume: 4479
Language: Polish
Name: Polish
Unit: documents, Unit: documents
Name: Turkish
Volume: 4480
Language: Turkish, Name: Hindi
Language: Hindi
Volume: 1499
Unit: documents, Name: Japanese
Language: Japanese
Volume: 1500
Unit: documents, Name: Korean
Volume: 1500
Unit: documents
Language: Korean
Provider: University of Oregon, Adobe Research
Tasks: named entity recognition
License: unknown
Year: 2022
Authors: Amir Pouran Ben Veyseh, Javid Ebrahimi, Franck Dernoncourt, Thien Huu Nguyen
Affiliations: Department of Computer Science, University of Oregon, Adobe Research
Abstract: Event Extraction (EE) is one of the fundamental tasks in Information Extraction (IE) that aims to recognize event mentions and their arguments (i.e., participants) from text. Due to its importance, extensive methods and resources have been developed for Event Extraction. However, one limitation of current research for EE involves the under-exploration for non-English languages in which the lack of high-quality multilingual EE datasets for model training and evaluation has been the main hindrance. To address this limitation, we propose a novel Multilingual Event Extraction dataset (MEE) that provides annotation for more than 50K event mentions in 8 typologically different languages. MEE comprehensively annotates data for entity mentions, event triggers and event arguments. We conduct extensive experiments on the proposed dataset to reveal challenges and opportunities for multilingual EE.
Dataset: XCOPA
Description: XCOPA is a typologically diverse multilingual dataset for causal commonsense reasoning. It was created by translating and re-annotating the English COPA dataset's validation and test sets into 11 languages. The task is to choose the more plausible cause or effect for a given premise.
Language: Indonesian, Italian, Swahili, Thai, Turkish, Vietnamese, Chinese, Estonian, Haitian Creole, Eastern Apurímac Quechua, Tamil
Domain: public datasets
Form: text
Size: 6600 sentences
Subsets: Name: Estonian
Volume: 600
Unit: sentences
Language: Estonian, Name: Haitian Creole
Volume: 600
Unit: sentences
Language: Haitian Creole, Name: Indonesian
Volume: 600
Unit: sentences
Language: Indonesian, Name: Italian
Volume: 600
Unit: sentences
Language: Italian, Name: Eastern Apurímac Quechua
Volume: 600
Unit: sentences
Language: Eastern Apurímac Quechua, Name: Kiswahili
Volume: 600
Unit: sentences
Language: Swahili, Name: Tamil
Volume: 600
Unit: sentences
Language: Tamil, Name: Thai
Volume: 600
Unit: sentences
Language: Thai, Name: Turkish
Volume: 600
Unit: sentences
Language: Turkish, Name: Vietnamese
Volume: 600
Unit: sentences
Language: Vietnamese, Name: Mandarin Chinese
Volume: 600
Unit: sentences
Language: Chinese
Provider: Cambridge
Tasks: commonsense reasoning
License: CC BY 4.0
Year: 2020
Authors: Edoardo M. Ponti, Goran Glavasˇ, Olga Majewska, Qianchu Liu, Ivan Vulic´, Anna Korhonen
Affiliations: Language Technology Lab, TAL, University of Cambridge, UK, Data and Web Science Group, University of Mannheim, Germany
Abstract: In order to simulate human language capacity, natural language processing systems must be able to reason about the dynamics of everyday situations, including their possible causes and effects. Moreover, they should be able to generalise the acquired world knowledge to new languages, modulo cultural differences. Advances in machine reasoning and cross-lingual transfer depend on the availability of challenging evaluation benchmarks. Motivated by both demands, we introduce Cross-lingual Choice of Plausible Alternatives (XCOPA), a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages, which includes resource-poor languages like Eastern Apur\'imac Quechua and Haitian Creole. We evaluate a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods based on multilingual pretraining and zero-shot fine-tuning falls short compared to translation-based transfer. Finally, we propose strategies to adapt multilingual models to out-of-sample resource-lean languages where only a small corpus or a bilingual dictionary is available, and report substantial improvements over the random baseline. The XCOPA dataset is freely available at github.com/cambridgeltl/xcopa.
Dataset: MLQA
Description: MLQA has over 12K instances in English and 5K in each other language, with each instance parallel between 4 languages on average.
Language: English, Arabic, German, Vietnamese, Spanish, Simplified Chinese, Hindi
Domain: wikipedia
Form: text
Size: 46461 documents
Subsets: Name: en
Volume: 12738
Unit: sentences
Language: English, Name: ar
Volume: 5852
Unit: sentences
Language: Arabic, Name: de
Volume: 5029
Unit: sentences
Language: German, Name: vi
Volume: 6006
Unit: sentences
Language: Vietnamese, Name: es
Volume: 5770
Unit: sentences
Language: Spanish, Name: zh
Volume: 5852
Unit: sentences
Language: Simplified Chinese, Name: hi
Volume: 5425
Unit: sentences
Language: Hindi
Provider: Facebook
Tasks: question answering
License: CC BY-SA 3.0
Year: 2020
Authors: Patrick Lewis, Barlas Oğuz, Ruty Rinott, S. Riedel, Holger Schwenk
Affiliations: Facebook AI Research;University College London
Abstract: Question answering (QA) models have shown rapid progress enabled by the availability of large, high-quality benchmark datasets. Such annotated datasets are difficult and costly to collect, and rarely exist in languages other than English, making training QA systems in other languages challenging. An alternative to building large monolingual training datasets is to develop cross-lingual systems which can transfer to a target language without requiring training data in that language. In order to develop such systems, it is crucial to invest in high quality multilingual evaluation benchmarks to measure progress. We present MLQA, a multi-way aligned extractive QA evaluation benchmark intended to spur research in this area. MLQA contains QA instances in 7 languages, namely English, Arabic, German, Spanish, Hindi, Vietnamese and Simplified Chinese. It consists of over 12K QA instances in English and 5K in each other language, with each QA instance being parallel between 4 languages on average. MLQA is built using a novel alignment context strategy on Wikipedia articles, and serves as a cross-lingual extension to existing extractive QA datasets. We evaluate current state-of-the-art cross-lingual representations on MLQA, and also provide machine-translation-based baselines. In all cases, transfer results are shown to be significantly behind training-language performance.
Dataset: M2DS
Description: M2DS is a multilingual multi-document summarization (MDS) dataset. It contains 180,000 news articles from the BBC, organized into 51,500 clusters across five languages: English, Japanese, Korean, Tamil, and Sinhala. The data covers the period from 2010 to 2023.
Language: English, Tamil, Japanese, Korean, Sinhala
Domain: news articles, public datasets
Form: text
Size: 180000 documents
Subsets: Name: English
Volume: 67000
Unit: documents
Language: English, Name: Tamil
Volume: 32000
Unit: documents
Language: Tamil, Name: Japanese
Volume: 29000
Unit: documents
Language: Japanese, Name: Korean
Volume: 27000
Unit: documents
Language: Korean, Name: Sinhala
Volume: 23500
Unit: documents
Language: Sinhala
Provider: University of Moratuwa, ConscientAI
Tasks: summarization
License: unknown
Year: 2024
Authors: Kushan Hewapathirana, Nisansa de Silva, C.D. Athuraliya
Affiliations: Dept. of Computer Science & Engineering, University of Moratuwa, Sri Lanka, ConscientAI, Sri Lanka
Abstract: In the rapidly evolving digital era, there is an increasing demand for concise information as individuals seek to distil key insights from various sources. Recent attention from researchers on Multi-document Summarisation (MDS) has resulted in diverse datasets covering customer reviews, academic papers, medical and legal documents, and news articles. However, the English-centric nature of these datasets has created a conspicuous void for multilingual datasets in today's globalised digital landscape, where linguistic diversity is celebrated. Media platforms such as British Broadcasting Corporation (BBC) have disseminated news in 20+ languages for decades. With only 380 million people speaking English natively as their first language, accounting for less than 5% of the global population, the vast majority primarily relies on other languages. These facts underscore the need for inclusivity in MDS research, utilising resources from diverse languages. Recognising this gap, we present the Multilingual Dataset for Multi-document Summarisation (M2DS), which, to the best of our knowledge, is the first dataset of its kind. It includes document-summary pairs in five languages from BBC articles published during the 2010-2023 period. This paper introduces M2DS, emphasising its unique multilingual aspect, and includes baseline scores from state-of-the-art MDS models evaluated on our dataset.
Dataset: XOR-TyDi
Description: XOR-TyDi QA brings together information-seeking questions, open-retrieval QA, and multilingual QA to create a multilingual open-retrieval QA dataset that enables cross-lingual answer retrieval. It consists of questions written by information-seeking native speakers in 7 typologically diverse languages and answer annotations that are retrieved from multilingual document collections.
Language: Arabic, Bengali, Finnish, Japanese, Korean, Russian, Telugu
Domain: public datasets
Form: text
Size: 53059 sentences
Subsets: Name: Ar
Volume: 17218
Unit: sentences
Language: Arabic, Name: Bn
Volume: 2682
Unit: sentences
Language: Bengali, Name: Fi
Volume: 9132
Unit: sentences
Language: Finnish, Name: Ja
Volume: 6531
Unit: sentences
Language: Japanese, Name: Ko
Volume: 2433
Unit: sentences
Language: Korean, Name: Ru
Volume: 8787
Unit: sentences
Language: Russian, Name: Te
Volume: 6276
Unit: sentences
Language: Telugu
Tasks: cross-lingual information retrieval, question answering
License: CC BY-SA 4.0
Year: 2021
Authors: Akari Asai, Jungo Kasai, Jonathan H. Clark, Kenton Lee, Eunsol Choi, Hannaneh Hajishirzi
Affiliations: University of Washington, University of Washington, Google Research, The University of Texas at Austin; Allen Institute for AI
Abstract: Multilingual question answering tasks typically assume answers exist in the same language as the question. Yet in practice, many languages face both information scarcity -- where languages have few reference articles -- and information asymmetry -- where questions reference concepts from other cultures. This work extends open-retrieval question answering to a cross-lingual setting enabling questions from one language to be answered via answer content from another language. We construct a large-scale dataset built on questions from TyDi QA lacking same-language answers. Our task formulation, called Cross-lingual Open Retrieval Question Answering (XOR QA), includes 40k information-seeking questions from across 7 diverse non-English languages. Based on this dataset, we introduce three new tasks that involve cross-lingual document retrieval using multi-lingual and English resources. We establish baselines with state-of-the-art machine translation systems and cross-lingual pretrained models. Experimental results suggest that XOR QA is a challenging task that will facilitate the development of novel techniques for multilingual question answering. Our data and code are available at https://nlp.cs.washington.edu/xorqa.
Dataset: Multilingual Hate Speech Detection Dataset
Description: Combined MLMA and L-HSAB datasets
Language: Arabic, English, German, Indonesian, Italian, Polish, Portuguese, Spanish, French
Domain: public datasets, social media
Form: text
Size: 159753 sentences
Subsets: Name: Arabic
Volume: 5790
Unit: sentences
Language: Arabic, Name: English
Volume: 96323
Unit: sentences
Language: English, Name: German
Volume: 6155
Unit: sentences
Language: German, Name: Indonesian
Volume: 13882
Unit: sentences
Language: Indonesian, Name: Italian
Volume: 9560
Unit: sentences
Language: Italian, Name: Polish
Volume: 9788
Unit: sentences
Language: Polish, Name: Portuguese
Volume: 5670
Unit: sentences
Language: Portuguese, Name: Spanish
Volume: 11365
Unit: sentences
Language: Spanish, Name: French
Volume: 1220
Unit: sentences
Language: French
Provider: Indian Institute of Technology Kharagpur
Tasks: offensive language detection
License: MIT License
Year: 2020
Authors: Sai Saket Aluru, Binny Mathew, Punyajoy Saha, Animesh Mukherjee
Affiliations: Indian Institute of Technology Kharagpur
Abstract: Hate speech detection is a challenging problem with most of the datasets available in only one language: English. In this paper, we conduct a large scale analysis of multilingual hate speech in 9 languages from 16 different sources. We observe that in low resource setting, simple models such as LASER embedding with logistic regression performs the best, while in high resource setting BERT based models perform better. In case of zero-shot classification, languages such as Italian and Portuguese achieve good results. Our proposed framework could be used as an efficient solution for low-resource languages. These models could also act as good baselines for future multilingual hate speech detection tasks. We have made our code and experimental settings public for other researchers at https://github.com/punyajoy/DE-LIMIT.
Dataset: MINION
Description: MINION is a large-scale, multilingual dataset for Event Detection (ED). It contains over 50,000 manually annotated event triggers in 8 languages (English, Spanish, Portuguese, Polish, Turkish, Hindi, Japanese, Korean) sourced from Wikipedia articles. The annotation schema is a pruned version of the ACE 2005 ontology.
Language: English, Spanish, Portuguese, Polish, Turkish, Hindi, Japanese, Korean
Domain: wikipedia
Form: text
Size: 31226 documents
Subsets: Name: English
Volume: 13000
Unit: documents
Language: English, Name: Spanish
Volume: 1500
Unit: documents
Language: Spanish, Name: Portuguese
Volume: 3268
Unit: documents
Language: Portuguese, Name: Polish
Volume: 4479
Unit: documents
Language: Polish, Name: Turkish
Volume: 4480
Unit: documents
Language: Turkish, Name: Hindi
Volume: 1499
Unit: documents
Language: Hindi, Name: Japanese
Volume: 1500
Unit: documents
Language: Japanese, Name: Korean
Volume: 1500
Unit: documents
Language: Korean
Provider: University of Oregon
Tasks: other
License: unknown
Year: 2022
Authors: AmirPouranBenVeyseh, MinhVanNguyen, FranckDernoncourt, ThienHuuNguyen
Affiliations: Dept. of Computer and Information Science, University of Oregon, Eugene, OR, USA, Adobe Research, Seattle, WA, USA
Abstract: Event Detection (ED) is the task of identifying and classifying trigger words of event mentions in text. Despite considerable research efforts in recent years for English text, the task of ED in other languages has been significantly less explored. Switching to non-English languages, important research questions for ED include how well existing ED models perform on different languages, how challenging ED is in other languages, and how well ED knowledge and annotation can be transferred across languages. To answer those questions, it is crucial to obtain multilingual ED datasets that provide consistent event annotation for multiple languages. There exist some multilingual ED datasets; however, they tend to cover a handful of languages and mainly focus on popular ones. Many languages are not covered in existing multilingual ED datasets. In addition, the current datasets are often small and not accessible to the public. To overcome those shortcomings, we introduce a new large-scale multilingual dataset for ED (called MINION) that consistently annotates events for 8 different languages; 5 of them have not been supported by existing multilingual datasets. We also perform extensive experiments and analysis to demonstrate the challenges and transferability of ED across languages in MINION that in all call for more research effort in this area.