IVUL-KAUST/MOLE huggingface dataset:
Dataset: 101 Billion Arabic Words Dataset
Description: The 101 Billion Arabic Words Dataset is curated by the Clusterlab team and consists of 101 billion words extracted and cleaned from web content, specifically targeting Arabic text. This dataset is intended for use in natural language processing applications, particularly in training and fine-tuning Large Language Models (LLMs)
Language: ar
Dialect: mixed
Script: Arab
Domain: web pages
Form: text
Size: 101000000000 tokens
Provider: Clusterlab
Tasks: text generation, language modeling
License: Apache-2.0
Year: 2024
Authors: Manel Aloui, Hasna Chouikhi, Ghaith Chaabane, Haithem Kchaou, Chehir Dhaouadi
Affiliations: Clusterlab
Abstract: In recent years, Large Language Models (LLMs) have revolutionized the field of natural language processing, showcasing an impressive rise predominantly in English-centric domains. These advancements have set a global benchmark, inspiring significant efforts toward developing Arabic LLMs capable of understanding and generating the Arabic language with remarkable accuracy. Despite these advancements, a critical challenge persists: the potential bias in Arabic LLMs, primarily attributed to their reliance on datasets comprising English data that has been translated into Arabic. This reliance not only compromises the authenticity of the generated content but also reflects a broader issue—the scarcity of original quality Arabic linguistic data. This study aims to address the data scarcity in the Arab world and to encourage the development of Arabic Language Models that are true to both the linguistic and nuances of the region. We undertook a large-scale data mining project, extracting a substantial volume of text from the Common Crawl WET files, specifically targeting Arabic content. The extracted data underwent a rigorous cleaning and deduplication process, using innovative techniques to ensure the integrity and uniqueness of the dataset. The result is the 101 Billion Arabic Words Dataset, the largest Arabic dataset available to date, which can significantly contribute to the development of authentic Arabic LLMs. This study not only highlights the potential for creating linguistically and culturally accurate Arabic LLMs but also sets a precedent for future research in enhancing the authenticity of Arabic language models.
Dataset: WinoMT
Description: Evaluating Gender Bias in Machine Translation
Language: multilingual
Dialect: Modern Standard Arabic
Script: Arab
Domain: public datasets
Form: text
Size: 3888 sentences
Tasks: machine translation
License: MIT License
Year: 2019
Authors: Gabriel Stanovsky, Noah A. Smith, Luke Zettlemoyer
Affiliations: Allen Institute for Artificial Intelligence, University of Washington, University of Washington, Facebook
Abstract: We present the first challenge set and evaluation protocol for the analysis of gender bias in machine translation (MT). Our approach uses two recent coreference resolution datasets composed of English sentences which cast participants into non-stereotypical gender roles (e.g., “The doctor asked the nurse to help her in the operation”). We devise an automatic gender bias evaluation method for eight target languages with grammatical gender, based on morphological analysis (e.g., the use of female inflection for the word “doctor”). Our analyses show that four popular industrial MT systems and two recent state-of-the-art academic MT models are significantly prone to gender-biased translation errors for all tested target languages. Our data and code are publicly available at https://github.com/gabrielStanovsky/mt_gender.
Dataset: ArabicMMLU
Description: ArabicMMLU is the first multi-task language understanding benchmark for Arabic language, sourced from school exams across diverse educational levels in different countries spanning North Africa, the Levant, and the Gulf regions. Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA).
Language: ar
Dialect: Modern Standard Arabic
Script: Arab
Domain: web pages
Form: text
Size: 14575 sentences
Provider: MBZUAI
Tasks: question answering, multiple choice question answering
License: CC BY-NC-SA 4.0
Year: 2024
Authors: Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Boda Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, Timothy Baldwin
Abstract: The focus of language model evaluation has transitioned towards reasoning and knowledge-intensive tasks, driven by advancements in pretraining large models. While state-of-the-art models are partially trained on large Arabic texts, evaluating their performance in Arabic remains challenging due to the limited availability of relevant datasets. To bridge this gap, we present ArabicMMLU, the first multi-task language understanding benchmark for Arabic language, sourced from school exams across diverse educational levels in different countries spanning North Africa, the Levant, and the Gulf regions. Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA), and is carefully constructed by collaborating with native speakers in the region. Our comprehensive evaluations of 35 models reveal substantial room for improvement, particularly among the best open-source models. Notably, BLOOMZ, mT0, LLama2, and Falcon struggle to achieve a score of 50%, while even the top-performing Arabic-centric model only achieves a score of 62.3%.
Dataset: CIDAR
Description: CIDAR contains 10,000 instructions and their output. The dataset was created by selecting around 9,109 samples from Alpagasus dataset then translating it to Arabic using ChatGPT. In addition, we append that with around 891 Arabic grammar instructions from the webiste Ask the teacher.
Language: ar
Dialect: Modern Standard Arabic
Script: Arab
Domain: commentary, LLM
Form: text
Size: 10000 sentences
Provider: ARBML
Tasks: instruction tuning, question answering
License: CC BY-NC 4.0
Year: 2024
Authors: Zaid Alyafeai, Khalid Almubarak, Ahmed Ashraf, Deema Alnuhait, Saied Alshahrani, Gubran A. Q. Abdulrahman, Gamil Ahmed, Qais Gawah, Zead Saleh, Mustafa Ghaleb, Yousef Ali, Maged S. Al-Shaibani
Abstract: Instruction tuning has emerged as a prominent methodology for teaching Large Language Models (LLMs) to follow instructions.
However, current instruction datasets predominantly cater to English or are derived from
English-dominated LLMs, resulting in inherent
biases toward Western culture. This bias significantly impacts the linguistic structures of
non-English languages such as Arabic, which
has a distinct grammar reflective of the diverse cultures across the Arab region. This
paper addresses this limitation by introducing CIDAR1
the first open Arabic instructiontuning dataset culturally-aligned by human reviewers. CIDAR contains 10,000 instruction
and output pairs that represent the Arab region. We discuss the cultural relevance of
CIDAR via the analysis and comparison to other
models fine-tuned on other datasets. Our experiments show that CIDAR can help enrich
research efforts in aligning LLMs with the
Arabic culture.
Dataset: Belebele
Description: A multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants.
Language: multilingual
Dialect: mixed
Script: Arab
Domain: wikipedia, public datasets
Form: text
Size: 5400 sentences
Subsets: Name: acm_Arab
Dialect: Iraq
Volume: 900
Unit: sentences, Name: arb_Arab
Dialect: Modern Standard Arabic
Volume: 900
Unit: sentences, Name: apc_Arab
Dialect: Levant
Volume: 900
Unit: sentences, Name: ars_Arab
Dialect: Saudi Arabia
Volume: 900
Unit: sentences, Name: ary_Arab
Dialect: Morocco
Volume: 900
Unit: sentences, Name: arz_Arab
Dialect: Egypt
Volume: 900
Unit: sentences
Provider: Facebook
Tasks: question answering, multiple choice question answering
License: CC BY-SA 4.0
Year: 2024
Authors: Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, Madian Khabsa
Abstract: We present Belebele, a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. Significantly expanding the language coverage of natural language understanding (NLU) benchmarks, this dataset enables the evaluation of text models in high-, medium-, and low-resource languages. Each question is based on a short passage from the Flores-200 dataset and has four multiple-choice answers. The questions were carefully curated to discriminate between models with different levels of general language comprehension. The English dataset on its own proves difficult enough to challenge state-of-the-art language models. Being fully parallel, this dataset enables direct comparison of model performance across all languages. We use this dataset to evaluate the capabilities of multilingual masked language models (MLMs) and large language models (LLMs). We present extensive results and find that despite significant cross-lingual transfer in English-centric LLMs, much smaller MLMs pretrained on balanced multilingual data still understand far more languages. We also observe that larger vocabulary size and conscious vocabulary construction correlate with better performance on low-resource languages. Overall, Belebele opens up new avenues for evaluating and analyzing the multilingual capabilities of NLP systems.
Dataset: MGB-2
Description: from Aljazeera TV programs have been manually captioned with no timing information
Language: ar
Dialect: Modern Standard Arabic
Script: Arab
Domain: TV Channels, captions
Form: audio
Size: 1200 hours
Provider: QCRI
Tasks: speech recognition
License: unknown
Year: 2019
Authors: Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, Yifan Zhang
Abstract: This paper describes the Arabic MGB-3 Challenge — Arabic Speech Recognition in the Wild. Unlike last year's Arabic MGB-2 Challenge, for which the recognition task was based on more than 1,200 hours broadcast TV news recordings from Aljazeera Arabic TV programs, MGB-3 emphasises dialectal Arabic using a multi-genre collection of Egyptian YouTube videos. Seven genres were used for the data collection: comedy, cooking, family/kids, fashion, drama, sports, and science (TEDx). A total of 16 hours of videos, split evenly across the different genres, were divided into adaptation, development and evaluation data sets. The Arabic MGB-Challenge comprised two tasks: A) Speech transcription, evaluated on the MGB-3 test set, along with the 10 hour MGB-2 test set to report progress on the MGB-2 evaluation; B) Arabic dialect identification, introduced this year in order to distinguish between four major Arabic dialects — Egyptian, Levantine, North African, Gulf, as well as Modern Standard Arabic. Two hours of audio per dialect were released for development and a further two hours were used for evaluation. For dialect identification, both lexical features and i-vector bottleneck features were shared with participants in addition to the raw audio recordings. Overall, thirteen teams submitted ten systems to the challenge. We outline the approaches adopted in each system, and summarise the evaluation results.
Dataset: ANETAC
Description: English-Arabic named entity transliteration and classification dataset
Language: multilingual
Dialect: Modern Standard Arabic
Script: Arab
Domain: public datasets
Form: text
Size: 79924 tokens
Provider: USTHB University, University of Salford
Tasks: named entity recognition, transliteration, machine translation
License: unknown
Year: 2019
Authors: Mohamed Seghir Hadj Ameur, Farid Meziane, Ahmed Guessoum
Affiliations: USTHB University, University of Salford, USTHB University
Abstract: In this paper, we make freely accessible ANETAC our English-Arabic named entity transliteration and classification dataset that we built from freely available parallel translation corpora. The dataset contains 79,924 instances, each instance is a triplet (e, a, c), where e is the English named entity, a is its Arabic transliteration and c is its class that can be either a Person, a Location, or an Organization. The ANETAC dataset is mainly aimed for the researchers that are working on Arabic named entity transliteration, but it can also be used for named entity classification purposes.
Dataset: TUNIZI
Description: first Tunisian Arabizi Dataset including 3K sentences, balanced, covering different topics, preprocessed and annotated as positive and negative
Language: ar
Dialect: Tunisia
Script: Latin
Domain: social media, commentary
Form: text
Size: 9210 sentences
Provider: iCompass
Tasks: sentiment analysis
License: unknown
Year: 2020
Authors: Chayma Fourati, Abir Messaoudi, Hatem Haddad
Affiliations: iCompass, iCompass, iCompass
Abstract: On social media, Arabic people tend to express themselves in their own local dialects. More particularly, Tunisians use the informal way called "Tunisian Arabizi". Analytical studies seek to explore and recognize online opinions aiming to exploit them for planning and prediction purposes such as measuring the customer satisfaction and establishing sales and marketing strategies. However, analytical studies based on Deep Learning are data hungry. On the other hand, African languages and dialects are considered low resource languages. For instance, to the best of our knowledge, no annotated Tunisian Arabizi dataset exists. In this paper, we introduce TUNIZI a sentiment analysis Tunisian Arabizi Dataset, collected from social networks, preprocessed for analytical studies and annotated manually by Tunisian native speakers.
Dataset: Shamela
Description: a large-scale, historical corpus of Arabic of about 1 billion
words from diverse periods of time
Language: ar
Dialect: Classical Arabic
Script: Arab
Domain: books
Form: text
Size: 6100 documents
Tasks: text generation, language modeling, part of speech tagging, morphological analysis
License: unknown
Year: 2016
Authors: Yonatan Belinkov, Alexander Magidow, Maxim Romanov, Avi Shmidman, Moshe Koppel
Abstract: Arabic is a widely-spoken language with a rich and long history spanning more than fourteen centuries. Yet existing Arabic corpora largely focus on the modern period or lack sufficient diachronic information. We develop a large-scale, historical corpus of Arabic of about 1 billion words from diverse periods of time. We clean this corpus, process it with a morphological analyzer, and enhance it by detecting parallel passages and automatically dating undated texts. We demonstrate its utility with selected case-studies in which we show its application to the digital humanities.
Dataset: POLYGLOT-NER
Description: Polyglot-NER A training dataset automatically generated from Wikipedia and Freebase the task of named entity recognition. The dataset contains the basic Wikipedia based training data for 40 languages we have (with coreference resolution) for the task of named entity recognition.
Language: multilingual
Dialect: Modern Standard Arabic
Script: Arab
Domain: wikipedia
Form: text
Size: 10000144 tokens
Provider: Stony Brook University
Tasks: named entity recognition
License: unknown
Year: 2014
Authors: Rami Al-Rfou, Vivek Kulkarni, Bryan Perozzi, Steven Skiena
Affiliations: Stony Brook University
Abstract: The increasing diversity of languages used on the web introduces a new level of complexity to Information Retrieval (IR) systems. We can no longer assume that textual content is written in one language or even the same language family. In this paper, we demonstrate how to build massive multilingual annotators with minimal human expertise and intervention. We describe a system that builds Named Entity Recognition (NER) annotators for 40 major languages using Wikipedia and Freebase. Our approach does not require NER human annotated datasets or language specific resources like treebanks, parallel corpora, and orthographic rules. The novelty of approach lies therein - using only language agnostic techniques, while achieving competitive performance.   Our method learns distributed word representations (word embeddings) which encode semantic and syntactic features of words in each language. Then, we automatically generate datasets from Wikipedia link structure and Freebase attributes. Finally, we apply two preprocessing stages (oversampling and exact surface form matching) which do not require any linguistic expertise.   Our evaluation is two fold: First, we demonstrate the system performance on human annotated datasets. Second, for languages where no gold-standard benchmarks are available, we propose a new method, distant evaluation, based on statistical machine translation.
Dataset: DODa
Description: DODa presents words under different spellings, offers verb-to-noun and masculine-to-feminine correspondences contains the conjugation of hundreds of verbs in different tenses,
Language: multilingual
Dialect: Morocco
Script: Arab-Latin
Domain: other
Form: text
Size: 10000 tokens
Tasks: transliteration, machine translation, part of speech tagging
License: MIT License
Year: 2021
Authors: Aissam Outchakoucht, Hamza Es-Samaali
Abstract: Darija Open Dataset (DODa) is an open-source project for the Moroccan dialect. With more than 10,000 entries DODa is arguably the largest open-source collaborative project for Darija-English translation built for Natural Language Processing purposes. In fact, besides semantic categorization, DODa also adopts a syntactic one, presents words under different spellings, offers verb-to-noun and masculine-to-feminine correspondences, contains the conjugation of hundreds of verbs in different tenses, and many other subsets to help researchers better understand and study Moroccan dialect. This data paper presents a description of DODa, its features, how it was collected, as well as a first application in Image Classification using ImageNet labels translated to Darija. This collaborative project is hosted on GitHub platform under MIT's Open-Source license and aims to be a standard resource for researchers, students, and anyone who is interested in Moroccan Dialect
Dataset: LASER
Description: Aligned sentences in 112 languages extracted from Tatoeba
Language: multilingual
Dialect: Modern Standard Arabic
Script: Arab
Domain: public datasets
Form: text
Size: 8200000 sentences
Provider: Facebook
Tasks: machine translation, embedding evaluation
License: BSD
Year: 2019
Authors: Mikel Artetxe, Holger Schwenk
Affiliations: University of the Basque Country, Facebook AI Research
Abstract: We introduce an architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts. Our system uses a single BiLSTM encoder with a shared BPE vocabulary for all languages, which is coupled with an auxiliary decoder and trained on publicly available parallel corpora. This enables us to learn a classifier on top of the resulting embeddings using English annotated data only, and transfer it to any of the 93 languages without any modification. Our experiments in cross-lingual natural language inference (XNLI dataset), cross-lingual document classification (MLDoc dataset) and parallel corpus mining (BUCC dataset) show the effectiveness of our approach. We also introduce a new test set of aligned sentences in 112 languages, and show that our sentence embeddings obtain strong results in multilingual similarity search even for low-resource languages. Our implementation, the pre-trained encoder and the multilingual test set are available at https://github.com/facebookresearch/LASER
Dataset: MGB-3
Description: A multi-genre collection of Egyptian YouTube videos. Seven genres were used for the data collection: comedy, cooking, family/kids, fashion, drama, sports, and science (TEDx). A total of 16 hours of videos, split evenly across the different genres
Language: ar
Dialect: Egypt
Script: Arab
Domain: social media, captions
Form: audio
Size: 16 hours
Provider: QCRI
Tasks: speech recognition
License: MIT License
Year: 2017
Authors: Ahmed Ali, Stephan Vogel, Steve Renals
Abstract: This paper describes the Arabic MGB-3 Challenge - Arabic Speech Recognition in the Wild. Unlike last year's Arabic MGB-2 Challenge, for which the recognition task was based on more than 1,200 hours broadcast TV news recordings from Aljazeera Arabic TV programs, MGB-3 emphasises dialectal Arabic using a multi-genre collection of Egyptian YouTube videos. Seven genres were used for the data collection: comedy, cooking, family/kids, fashion, drama, sports, and science (TEDx). A total of 16 hours of videos, split evenly across the different genres, were divided into adaptation, development and evaluation data sets. The Arabic MGB-Challenge comprised two tasks: A) Speech transcription, evaluated on the MGB-3 test set, along with the 10 hour MGB-2 test set to report progress on the MGB-2 evaluation; B) Arabic dialect identification, introduced this year in order to distinguish between four major Arabic dialects - Egyptian, Levantine, North African, Gulf, as well as Modern Standard Arabic. Two hours of audio per dialect were released for development and a further two hours were used for evaluation. For dialect identification, both lexical features and i-vector bottleneck features were shared with participants in addition to the raw audio recordings. Overall, thirteen teams submitted ten systems to the challenge. We outline the approaches adopted in each system, and summarise the evaluation results.
Dataset: Arap-Tweet
Description: Arap-Tweet is a large-scale, multi-dialectal Arabic Twitter corpus containing 2.4 million tweets from 11 regions across 16 countries in the Arab world. The dataset includes annotations for dialect, age group, and gender of the users.
Language: ar
Dialect: mixed
Script: Arab
Domain: social media
Form: text
Size: 2400000 sentences
Provider: Hamad Bin Khalifa University, Carnegie Mellon University Qatar
Tasks: dialect identification, gender identification
License: unknown
Year: 2018
Authors: Wajdi Zaghouani, Anis Charfi
Affiliations: Hamad Bin Khalifa University, Carnegie Mellon University Qatar
Abstract: In this paper, we present Arap-Tweet, which is a large-scale and multi-dialectal corpus of Tweets from 11 regions and 16 countries in the Arab world representing the major Arabic dialectal varieties. To build this corpus, we collected data from Twitter and we provided a team of experienced annotators with annotation guidelines that they used to annotate the corpus for age categories, gender, and dialectal variety. During the data collection effort, we based our search on distinctive keywords that are specific to the different Arabic dialects and we also validated the location using Twitter API. In this paper, we report on the corpus data collection and annotation efforts. We also present some issues that we encountered during these phases. Then, we present the results of the evaluation performed to ensure the consistency of the annotation. The provided corpus will enrich the limited set of available language resources for Arabic and will be an invaluable enabler for developing author profiling tools and NLP tools for Arabic.