Omkar2118/datasets huggingface dataset:
Large-Scale CLIR Dataset

The Large-Scale CLIR Dataset is a retrieval dataset built for Cross-Language Information Retrieval (CLIR).
The dataset is derived from Wikipedia and contains more 2.8 million English single-sentence queries with relevant documents from 25 other selected languages.

Introducing paper: Cross-Lingual Learning-to-Rank with Shared Representations
WikiCaps

**WikiCaps** is a large-scale multilingual but non-parallel data set for multimodal machine translation and retrieval. The image-caption data was extracted from Wikimedia Commons and is thus a representative of the collection of largely available non-descriptive image-caption pairs in the web. The current version of the dataset contains 3,816,940 images with 3,825,132 English captions and additional 1,000 image-caption pairs in German, French, and Russian together with their English counterparts.

Source: [WikiCaps](https://www.cl.uni-heidelberg.de/statnlpgroup/wikicaps/)

Introducing paper: A Dataset and Reranking Method for Multimodal MT of User-Generated Image Captions
WikiCLIR

**WikiCLIR** is a large-scale (German-English) retrieval data set for Cross-Language Information Retrieval (CLIR). It contains a total of 245,294 German single-sentence queries with 3,200,393 automatically extracted relevance judgments for 1,226,741 English Wikipedia articles as documents. Queries are well-formed natural language sentences that allow large-scale training of (translation-based) ranking models.

Source: []()

Introducing paper: Learning Translational and Knowledge-based Similarities from Relevance Rankings for Cross-Language Retrieval