IVUL-KAUST/MOLE huggingface dataset:
Dataset: RuCoLA
Description: RuCoLA is a dataset of Russian sentences labeled as acceptable or not. It consists of 9.8k in-domain sentences from linguistic publications and 3.6k out-of-domain sentences produced by generative models.
Language: ru
Domain: books, wikipedia, public datasets, LLM
Form: text
Size: 13445 sentences
Provider: RussianNLP
Tasks: linguistic acceptability
License: Apache-2.0
Year: 2022
Authors: Vladislav Mikhailov, Tatiana Shamardina, Max Ryabinin, Alena Pestova, Ivan Smurov, Ekaterina Artemova
Affiliations: SberDevices, ABBYY, HSE University, Yandex, Huawei Noah's Ark Lab, Center for Information and Language Processing (CIS), MaiNLP lab, LMU Munich, Germany
Abstract: Linguistic acceptability (LA) attracts the attention of the research community due to its many uses, such as testing the grammatical knowledge of language models and filtering implausible texts with acceptability classifiers. However, the application scope of LA in languages other than English is limited due to the lack of high-quality resources. To this end, we introduce the Russian Corpus of Linguistic Acceptability (RuCoLA), built from the ground up under the well-established binary LA approach. RuCoLA consists of $9.8$k in-domain sentences from linguistic publications and $3.6$k out-of-domain sentences produced by generative models. The out-of-domain set is created to facilitate the practical use of acceptability for improving language generation. Our paper describes the data collection protocol and presents a fine-grained analysis of acceptability classification experiments with a range of baseline approaches. In particular, we demonstrate that the most widely used language models still fall behind humans by a large margin, especially when detecting morphological and semantic errors. We release RuCoLA, the code of experiments, and a public leaderboard (rucola-benchmark.com) to assess the linguistic competence of language models for Russian.
Dataset: Slovo
Description: A Russian Sign Language (RSL) video dataset containing 20,000 FullHD recordings of 1,000 isolated RSL gestures from 194 signers. The data was collected via crowdsourcing platforms and covers frequently used words like food, animals, emotions, and colors.
Language: ru
Domain: web pages
Form: videos
Size: 20000 sentences
Provider: SaluteDevices
Tasks: sign language recognition
License: custom
Year: 2024
Authors: Kapitanov Alexander, Kvanchiani Karina, Nagaev Alexander, Petrova Elizaveta
Affiliations: SaluteDevices, Russia
Abstract: One of the main challenges of the sign language recognition task is the difficulty of collecting a suitable dataset due to the gap between hard-of-hearing and hearing societies. In addition, the sign language in each country differs significantly, which obliges the creation of new data for each of them. This paper presents the Russian Sign Language (RSL) video dataset Slovo, produced using crowdsourcing platforms. The dataset contains 20,000 FullHD recordings, divided into 1,000 classes of isolated RSL gestures received by 194 signers. We also provide the entire dataset creation pipeline, from data collection to video annotation, with the following demo application. Several neural networks are trained and evaluated on the Slovo to demonstrate its teaching ability. Proposed data and pre-trained models are publicly available.
Dataset: WikiOmnia
Description: WikiOmnia is a large-scale Russian question-answering dataset generated from the summary sections of all Russian Wikipedia articles. It was created using a fully automated pipeline with ruGPT-3XL and ruT5-large models and includes both a raw version and a filtered, high-quality version.
Language: ru
Domain: wikipedia
Form: text
Size: 3560000 sentences
Provider: Artificial Intelligence Research Institute (AIRI), SberDevices
Tasks: question answering, information retrieval
License: Apache-2.0
Year: 2022
Authors: Dina Pisarevskaya, Tatiana Shavrina
Affiliations: Independent Researcher, Artificial Intelligence Research Institute (AIRI), SberDevices
Abstract: The General QA field has been developing the methodology referencing the Stanford Question answering dataset (SQuAD) as the significant benchmark. However, compiling factual questions is accompanied by time- and labour-consuming annotation, limiting the training data's potential size. We present the WikiOmnia dataset, a new publicly available set of QA-pairs and corresponding Russian Wikipedia article summary sections, composed with a fully automated generative pipeline. The dataset includes every available article from Wikipedia for the Russian language. The WikiOmnia pipeline is available open-source and is also tested for creating SQuAD-formatted QA on other domains, like news texts, fiction, and social media. The resulting dataset includes two parts: raw data on the whole Russian Wikipedia (7,930,873 QA pairs with paragraphs for ruGPT-3 XL and 7,991,040 QA pairs with paragraphs for ruT5-large) and cleaned data with strict automatic verification (over 160,000 QA pairs with paragraphs for ruGPT-3 XL and over 3,400,000 QA pairs with paragraphs for ruT5-large).
Dataset: DaNetQA
Description: DaNetQA is a question-answering dataset for the Russian language. It comprises natural yes/no questions paired with a paragraph from Wikipedia and an answer derived from the paragraph. The task is to take both the question and a paragraph as input and come up with a yes/no answer.
Language: ru
Domain: wikipedia
Form: text
Size: 2691 sentences
Provider: Pragmatics Lab
Tasks: question answering
License: CC0
Year: 2020
Authors: Taisia Glushkova, Alexey Machnev, Alena Fenogenova, Tatiana Shavrina, Ekaterina Artemova, Dmitry I. Ignatov
Affiliations: National Research University Higher School of Economics, Sberbank
Abstract: DaNetQA, a new question-answering corpus, follows (Clark et. al, 2019) design: it comprises natural yes/no questions. Each question is paired with a paragraph from Wikipedia and an answer, derived from the paragraph. The task is to take both the question and a paragraph as input and come up with a yes/no answer, i.e. to produce a binary output. In this paper, we present a reproducible approach to DaNetQA creation and investigate transfer learning methods for task and language transferring. For task transferring we leverage three similar sentence modelling tasks: 1) a corpus of paraphrases, Paraphraser, 2) an NLI task, for which we use the Russian part of XNLI, 3) another question answering task, SberQUAD. For language transferring we use English to Russian translation together with multilingual language fine-tuning.
Dataset: RuSemShift
Description: A large-scale manually annotated test set for the task of semantic change modeling in Russian for two long-term time period pairs: from the pre-Soviet through the Soviet times and from the Soviet through the post-Soviet times.
Language: ru
Domain: public datasets
Form: text
Size: 7846 sentences
Provider: National Research University Higher School of Economics, University of Oslo
Tasks: word similarity
License: CC BY-SA 4.0
Year: 2020
Authors: Julia Rodina, Andrey Kutuzov
Affiliations: National Research University Higher School of Economics, University of Oslo
Abstract: We present RuSemShift, a large-scale manually annotated test set for the task of semantic change modeling in Russian for two long-term time period pairs: from the pre-Soviet through the Soviet times and from the Soviet through the post-Soviet times. Target words were annotated by multiple crowd-source workers. The annotation process was organized following the DURel framework and was based on sentence contexts extracted from the Russian National Corpus. Additionally, we report the performance of several distributional approaches on RuSemShift, achieving promising results, which at the same time leave room for other researchers to improve.
Dataset: REPA
Description: The Russian Error tyPes Annotation dataset (REPA) consists of 1k user queries and 2k LLM-generated responses. Human annotators labeled each response pair expressing their preferences across ten specific error types, as well as selecting an overall preference.
Language: ru
Domain: public datasets, LLM
Form: text
Size: 1003 sentences
Provider: HSE University
Tasks: question answering, instruction tuning
License: MIT License
Year: 2025
Authors: Alexander Pugachev, Alena Fenogenova, Vladislav Mikhailov, Ekaterina Artemova
Affiliations: HSE University, SaluteDevices, University of Oslo, Toloka AI
Abstract: Recent advances in large language models (LLMs) have introduced the novel paradigm of using LLMs as judges, where an LLM evaluates and scores the outputs of another LLM, which often correlates highly with human preferences. However, the use of LLM-as-a-judge has been primarily studied in English. In this paper, we evaluate this framework in Russian by introducing the Russian Error tyPes Annotation dataset (REPA), a dataset of 1k user queries and 2k LLM-generated responses. Human annotators labeled each response pair expressing their preferences across ten specific error types, as well as selecting an overall preference. We rank six generative LLMs across the error types using three rating systems based on human preferences. We also evaluate responses using eight LLM judges in zero-shot and few-shot settings. We describe the results of analyzing the judges and position and length biases. Our findings reveal a notable gap between LLM judge performance in Russian and English. However, rankings based on human and LLM preferences show partial alignment, suggesting that while current LLM judges struggle with fine-grained evaluation in Russian, there is potential for improvement.
Dataset: SberQuAD
Description: SberQuAD is a large-scale Russian reading comprehension dataset, analogous to SQuAD. It contains over 50,000 question-answer pairs created by crowdworkers based on paragraphs from Russian Wikipedia articles. The task is to find a contiguous text span in the paragraph that answers the question.
Language: ru
Domain: wikipedia
Form: text
Size: 50364 sentences
Provider: Sberbank
Tasks: question answering
License: Apache-2.0
Year: 2020
Authors: Pavel Efimov, Andrey Chertok, Leonid Boytsov, Pavel Braslavski
Affiliations: Saint Petersburg State University, Sberbank, Ural Federal University, JetBrains Research
Abstract: SberQuAD -- a large scale analog of Stanford SQuAD in the Russian language - is a valuable resource that has not been properly presented to the scientific community. We fill this gap by providing a description, a thorough analysis, and baseline experimental results.
Dataset: RuBLiMP
Description: The Russian Benchmark of Linguistic Minimal Pairs (RuBLiMP) includes 45,000 pairs of sentences that differ in grammaticality and isolate a morphological, syntactic, or semantic phenomenon. It is created by applying linguistic perturbations to automatically annotated sentences from open text corpora and decontaminating test data.
Language: ru
Domain: wikipedia, news articles, public datasets
Form: text
Size: 45000 sentences
Provider: University of Edinburgh, HSE University, University of Groningen, Ghent University, SaluteDevices, Toloka AI, University of Oslo
Tasks: linguistic acceptability, morphological analysis
License: Apache-2.0
Year: 2024
Authors: Ekaterina Taktasheva, Maxim Bazhukov, Kirill Koncha, Alena Fenogenova, Ekaterina Artemova, Vladislav Mikhailov
Affiliations: University of Edinburgh, HSE University, University of Groningen, Ghent University, SaluteDevices, Toloka AI, University of Oslo
Abstract: Minimal pairs are a well-established approach to evaluating the grammatical knowledge of language models. However, existing resources for minimal pairs address a limited number of languages and lack diversity of language-specific grammatical phenomena. This paper introduces the Russian Benchmark of Linguistic Minimal Pairs (RuBLiMP), which includes 45k pairs of sentences that differ in grammaticality and isolate a morphological, syntactic, or semantic phenomenon. In contrast to existing benchmarks of linguistic minimal pairs, RuBLiMP is created by applying linguistic perturbations to automatically annotated sentences from open text corpora and carefully curating test data. We describe the data collection protocol and present the results of evaluating 25 language models in various scenarios. We find that the widely used language models for Russian are sensitive to morphological and agreement-oriented contrasts but fall behind humans on phenomena requiring understanding of structural relations, negation, transitivity, and tense. RuBLiMP, the codebase, and other materials are publicly available.
Dataset: Golos
Description: Golos is a large Russian speech dataset consisting of 1240 hours of manually annotated audio. It was collected using crowd-sourcing and studio recordings with far-field settings.
Language: ru
Domain: other
Form: audio
Size: 1240 hours
Provider: Sber
Tasks: speech recognition
License: custom
Year: 2021
Authors: Nikolay Karpov, Alexander Denisenko, Fedor Minkin
Affiliations: Sber, Russia
Abstract: This paper introduces a novel Russian speech dataset called Golos, a large corpus suitable for speech research. The dataset mainly consists of recorded audio files manually annotated on the crowd-sourcing platform. The total duration of the audio is about 1240 hours. We have made the corpus freely available to download, along with the acoustic model with CTC loss prepared on this corpus. Additionally, transfer learning was applied to improve the performance of the acoustic model. In order to evaluate the quality of the dataset with the beam-search algorithm, we have built a 3-gram language model on the open Common Crawl dataset. The total word error rate (WER) metrics turned out to be about 3.3% and 11.5%.
Dataset: RFSD
Description: The Russian Financial Statements Database (RFSD) is an open, harmonized collection of annual unconsolidated financial statements of the universe of Russian firms from 2011–2023. 
Language: ru
Domain: web pages
Form: text
Size: 56150173 documents
Provider: European University at Saint Petersburg
Tasks: other
License: CC BY 4.0
Year: 2025
Authors: Sergey Bondarkov, Viktor Ledenev, Dmitry Skougarevskiy
Affiliations: European University at Saint Petersburg
Abstract: The Russian Financial Statements Database (RFSD) is an open, harmonized collection of annual unconsolidated financial statements of the universe of Russian firms in 2011-2023. It is the first open data set with information on every active firm in the country, including non-filing firms. With 56.6 million geolocated firm-year observations gathered from two official sources, the RFSD features multiple end-user quality-of-life improvements such as data imputation, statement articulation, harmonization across data providers and formats, and data enrichment. Extensive internal and external validation shows that most statements articulate well while their aggregates display higher correlation with the regional GDP than the previous gridded GDP data products. We also examine the direction and magnitude of the reporting bias by comparing the universe of firms that are required to file with the actual filers. The RFSD can be used in various economic applications as diverse as calibration of micro-founded models, estimation of markups and productivity, or assessing industry organization and market power.
Dataset: RuDSI
Description: A benchmark dataset for word sense induction (WSI) in Russian. It was created using manual annotation and semi-automatic clustering of Word Usage Graphs (WUGs) from the Russian National Corpus. The sense inventories are data-driven, not based on predefined dictionaries.
Language: ru
Domain: public datasets, wikipedia
Form: text
Size: 840 sentences
Provider: HSE University
Tasks: other
License: unknown
Year: 2022
Authors: Anna Aksenova, Ekaterina Gavrishina, Elisey Rykov, Andrey Kutuzov
Affiliations: National Research University Higher School of Economics, University of Oslo
Abstract: We present RuDSI, a new benchmark for word sense induction (WSI) in Russian. The dataset was created using manual annotation and semi-automatic clustering of Word Usage Graphs (WUGs). Unlike prior WSI datasets for Russian, RuDSI is completely data-driven (based on texts from Russian National Corpus), with no external word senses imposed on annotators. Depending on the parameters of graph clustering, different derivative datasets can be produced from raw annotation. We report the performance that several baseline WSI methods obtain on RuDSI and discuss possibilities for improving these scores.
Dataset: HeadlineCause
Description: A dataset for detecting implicit causal relations between pairs of news headlines in English and Russian. The pairs are labeled through crowdsourcing and vary from unrelated to having causation or refutation relations.
Language: multilingual
Domain: news articles, public datasets, web pages
Form: text
Size: 9553 sentences
Provider: Moscow Institute of Physics and Technology, Yandex
Tasks: natural language inference
License: CC0
Year: 2021
Authors: Ilya Gusev, Alexey Tikhonov
Affiliations: Moscow Institute of Physics and Technology, Yandex
Abstract: Detecting implicit causal relations in texts is a task that requires both common sense and world knowledge. Existing datasets are focused either on commonsense causal reasoning or explicit causal relations. In this work, we present HeadlineCause, a dataset for detecting implicit causal relations between pairs of news headlines. The dataset includes over 5000 headline pairs from English news and over 9000 headline pairs from Russian news labeled through crowdsourcing. The pairs vary from totally unrelated or belonging to the same general topic to the ones including causation and refutation relations. We also present a set of models and experiments that demonstrates the dataset validity, including a multilingual XLM-RoBERTa based model for causality detection and a GPT-2 based model for possible effects prediction.
Dataset: RusTitW
Description: A large-scale, human-labeled dataset for Russian text recognition in natural images (text-in-the-wild). It contains over 13k real-world images. The paper also presents a larger synthetic dataset (over 900k images) and the code for its generation.
Language: ru
Domain: public datasets
Form: images
Size: 900701 images
Provider: AIRI, SberAI
Tasks: optical character recognition
License: Apache-2.0
Year: 2023
Authors: Igor Markov, Sergey Nesteruk, Andrey Kuznetsov, Denis Dimitrov
Affiliations: AIRI, SberAI
Abstract: Information surrounds people in modern life. Text is a very efficient type of information that people use for communication for centuries. However, automated text-in-the-wild recognition remains a challenging problem. The major limitation for a DL system is the lack of training data. For the competitive performance, training set must contain many samples that replicate the real-world cases. While there are many high-quality datasets for English text recognition; there are no available datasets for Russian language. In this paper, we present a large-scale human-labeled dataset for Russian text recognition in-the-wild. We also publish a synthetic dataset and code to reproduce the generation process
Dataset: CRAFT
Description: A dataset of 200,000 image-text pairs focusing on the Russian cultural code, designed to enhance text-to-image models' understanding of Russian, Soviet, and post-Soviet culture. It was created by crawling the web, followed by manual filtering and detailed human captioning to ensure quality and cultural relevance.
Language: ru
Domain: web pages
Form: images
Size: 200000 images
Provider: SberAI
Tasks: other
License: unknown
Year: 2025
Authors: Viacheslav Vasilev, Vladimir Arkhipkin, Julia Agafonova, Tatiana Nikulina, Evelina Mironova, Alisa Shichanina, Nikolai Gerasimenko, Mikhail Shoytov, Denis Dimitrov
Affiliations: SberAI, Moscow Institute of Physics and Technology (MIPT), Information Technologies, Mechanics and Optics University (ITMO University), Artificial Intelligence Research Institute (AIRI)
Abstract: Despite the fact that popular text-to-image generation models cope well with international and general cultural queries, they have a significant knowledge gap regarding individual cultures. This is due to the content of existing large training datasets collected on the Internet, which are predominantly based on Western European or American popular culture. Meanwhile, the lack of cultural adaptation of the model can lead to incorrect results, a decrease in the generation quality, and the spread of stereotypes and offensive content. In an effort to address this issue, we examine the concept of cultural code and recognize the critical importance of its understanding by modern image generation models, an issue that has not been sufficiently addressed in the research community to date. We propose the methodology for collecting and processing the data necessary to form a dataset based on the cultural code, in particular the Russian one. We explore how the collected data affects the quality of generations in the national domain and analyze the effectiveness of our approach using the Kandinsky 3.1 text-to-image model. Human evaluation results demonstrate an increase in the level of awareness of Russian culture in the model.
Dataset: Gazeta
Description: Gazeta is the first dataset for summarizing Russian news, consisting of over 63,000 article-summary pairs collected from the Gazeta.ru news website. The paper describes the dataset's properties and benchmarks several extractive and abstractive models, demonstrating that a pretrained mBART model is particularly effective for this task.
Language: ru
Domain: news articles, web pages
Form: text
Size: 63435 documents
Provider: Moscow Institute of Physics and Technology
Tasks: summarization
License: unknown
Year: 2021
Authors: Ilya Gusev
Affiliations: Moscow Institute of Physics and Technology
Abstract: Automatic text summarization has been studied in a variety of domains and languages. However, this does not hold for the Russian language. To overcome this issue, we present Gazeta, the first dataset for summarization of Russian news. We describe the properties of this dataset and benchmark several extractive and abstractive models. We demonstrate that the dataset is a valid task for methods of text summarization for Russian. Additionally, we prove the pretrained mBART model to be useful for Russian text summarization.
Dataset: RuBia
Description: RuBia is a Russian language bias detection dataset divided into 4 domains: gender, nationality, socio-economic status, and diverse. Each example consists of two sentences, one reinforcing a harmful stereotype and the other contradicting it. The data was written by volunteers and validated by crowdworkers.
Language: ru
Domain: social media
Form: text
Size: 1989 sentences
Provider: HSE University
Tasks: other
License: CC BY-SA 4.0
Year: 2024
Authors: Veronika Grigoreva, Anastasiia Ivanova, Ilseyar Alimova, Ekaterina Artemova
Affiliations: Queen’s University, Higher School of Economics, Wildberries, Linguistic Convergence Laboratory, Toloka AI
Abstract: Warning: this work contains upsetting or disturbing content.   Large language models (LLMs) tend to learn the social and cultural biases present in the raw pre-training data. To test if an LLM's behavior is fair, functional datasets are employed, and due to their purpose, these datasets are highly language and culture-specific. In this paper, we address a gap in the scope of multilingual bias evaluation by presenting a bias detection dataset specifically designed for the Russian language, dubbed as RuBia. The RuBia dataset is divided into 4 domains: gender, nationality, socio-economic status, and diverse, each of the domains is further divided into multiple fine-grained subdomains. Every example in the dataset consists of two sentences with the first reinforcing a potentially harmful stereotype or trope and the second contradicting it. These sentence pairs were first written by volunteers and then validated by native-speaking crowdsourcing workers. Overall, there are nearly 2,000 unique sentence pairs spread over 19 subdomains in RuBia. To illustrate the dataset's purpose, we conduct a diagnostic evaluation of state-of-the-art or near-state-of-the-art LLMs and discuss the LLMs' predisposition to social biases.