Repository: EuroEval/EuroEval
Pull request #970: BeleBele documentation
Author: login: oliverkinch
Created: 2025-05-03T06:53:52Z | Merged: 2025-05-04T11:34:35Z

This PR adds documentation for the BeleBele datasets that we currently have in EuroEval.


Diff:
diff --git a/docs/datasets/danish.md b/docs/datasets/danish.md
index d644f90f6..285510d8b 100644
--- a/docs/datasets/danish.md
+++ b/docs/datasets/danish.md
@@ -353,6 +353,70 @@ $ euroeval --model <model-id> --dataset scandiqa-da
 ```
 
 
+### Unofficial: BeleBele-da
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Tekst: Prognoserne siger, at stormen, der er omkring 645 mil (1040 km) vest for Kap Verde-øerne, sandsynligvis vil forsvinde, før den truer nogen landområder. Fred har i øjeblikket vinde på 165 km/t og bevæger sig mod nordvest. Fred er den heftigste tropiske cyklon, der nogensinde er blevet registreret så sydligt og østligt i Atlanterhavet, siden man begyndte at bruge satellitbilleder, og kun den tredje store orkan, der er registreret øst for 35°V.\nSpørgsmål: Da Fred befandt sig nær Kap Verde-øerne, hvilken retning bevægede den sig så mod?\nSvarmuligheder:\na. Vest\nb. Syd\nc. Øst\nd. Nordvest",
+  "label": "d"
+}
+```
+```json
+{
+  "text": "Tekst: "Siden Pakistan i 1947 blev uafhængigt af det britiske styre, har den pakistanske præsident udpeget ""politiske agenter"", som styrer FATA, og som har næsten fuldstændig kontrol over områderne. Disse agenter er ansvarlige for at levere regerings- og retstjenester i henhold til artikel 247 i den pakistanske forfatning."\nSpørgsmål: Hvem leverer retslige tjenester til FATA?\nSvarmuligheder:\na. Den pakistanske regering\nb. Politiske agenter\nc. Pakistans præsident\nd. Den britiske regering",
+  "label": "b"
+}
+```
+```json
+{
+  "text": "Tekst: Alle er en del af samfundet og benytter transportsystemerne. Næsten alle klager over transportsystemerne. I udviklede lande hører du sjældent ligeså mange klager over vandkvalitet eller broer, der styrter sammen. Hvorfor giver transportsystemerne anledning til sådanne klager, hvorfor svigter de på daglig basis? Er transportingeniører blot inkompetente? Eller foregår der noget mere fundamentalt?\nSpørgsmål: Hvilken offentlig service siges at skabe størst utilfredshed i udviklede lande?\nSvarmuligheder:\na. Vandkvalitet\nb. Brobyggelse\nc. Offentlig transport\nd. Uddannelse",
+  "label": "c"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Følgende er multiple choice spørgsmål (med svar).
+  ```
+- Base prompt template:
+  ```
+  Spørgsmål: {text}
+  Svarmuligheder:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Svar: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Spørgsmål: {text}
+  Svarmuligheder:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Besvar ovenstående spørgsmål ved at svare med 'a', 'b', 'c' eller 'd', og intet andet.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-da
+```
+
+
 ## Knowledge
 
 ### Danske Talemåder
diff --git a/docs/datasets/dutch.md b/docs/datasets/dutch.md
index 2b3aa856d..e170b97c8 100644
--- a/docs/datasets/dutch.md
+++ b/docs/datasets/dutch.md
@@ -323,6 +323,70 @@ $ euroeval --model <model-id> --dataset squad-nl
 ```
 
 
+### Unofficial: BeleBele-nl
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Tekst: Mystiek is het geloven in, identificeren met of bewustzijn van een ultieme werkelijkheid, goddelijkheid, spirituele waarheid of God. De kerkganger streeft naar een directe gewaarwording, intuïtie of inzicht in de goddelijke werkelijkheid. Volgers streven een bepaalde manier van leven na of willen ervaringen opdoen die ze datzelfde gevoel geven. In tegenstelling tot andere religieuze overtuigingen en aanbidding, legt mystiek nadruk op de rechtstreekse persoonlijke beleving van een unieke staat van bewustzijn, vooral van een vredige, inzichtelijke, gelukzalige of extatische aard.\nVraag: Wat is geen juiste omschrijving van mystiek?\nAntwoordopties:\na. De nadruk ligt op het ervaren van een vredige, gelukzalige staat van bewustzijn\nb. Volgers van mystiek streven bewustwording na van een spirituele werkelijkheid\nc. Volgers van mystiek passen gebruiken toe die hun inzicht in een goddelijke werkelijkheid vergroten\nd. De nadruk op het streven naar een directe persoonlijke beleving is vergelijkbaar met veel andere vormen van religieuze overtuiging en aanbidding",
+  "label": "d"
+}
+```
+```json
+{
+  "text": "Tekst: Het favoriete maaltje van ocelotten zijn kleine dieren. Ze vangen apen, slangen, knaagdieren en vogels als dat lukt. De ocelot jaagt bijna uitsluitend op dieren die veel kleiner zijn dan hij zelf is. Geleerden vermoeden dat ocelotten hun reukvermogen gebruiken om op kleine dieren (hun prooi) te jagen, door aan de grond te ruiken waar deze zijn geweest. Ze kunnen door nachtvisie heel goed in het donker zien en bewegen zich heel onopvallend voort. Ocelotten jagen op prooi door zich één te maken met de omgeving en vervolgens op hun prooi te springen.\nVraag: Welke uitspraak over een ocelot is onjuist?\nAntwoordopties:\na. Ze kunnen goed in het donker jagen\nb. Ze bewegen zich in stilte voort\nc. Hun reukvermogen is zwak\nd. Ze jagen het liefst op kleine dieren",
+  "label": "c"
+}
+```
+```json
+{
+  "text": "Tekst: Er was 120-160 kubieke meter brandstof aan boord van de Luno toen het schip motorproblemen kreeg en door de harde wind en golven tegen de golfbreker werd geduwd. De twaalf crewleden zijn met helikopters in veiligheid gebracht, met als enige verwonding een gebroken neus. Het 100 meter lange schip was onderweg om de gebruikelijke lading kunstmest op te halen. In eerste instantie vreesden autoriteiten dat het vaartuig met de lading zou kunnen gaan lekken.\nVraag: Waar vreesden de autoriteiten volgens de tekst in eerste instantie voor wat betreft de Luno?\nAntwoordopties:\na. Gebrek aan een lading kunstmest\nb. Golven en harde wind\nc. Lekken van brandstof\nd. Verwondingen van bemanningsleden",
+  "label": "c"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Hieronder staan meerkeuzevragen (met antwoorden).
+  ```
+- Base prompt template:
+  ```
+  Vraag: {text}
+  Antwoordopties:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Antwoord: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Vraag: {text}
+  Antwoordopties:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Beantwoord de bovenstaande vraag met 'a', 'b', 'c' of 'd', en niets anders.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-nl
+```
+
+
 ## Knowledge
 
 ### MMLU-nl
diff --git a/docs/datasets/english.md b/docs/datasets/english.md
index cf05fa403..434657f06 100644
--- a/docs/datasets/english.md
+++ b/docs/datasets/english.md
@@ -295,6 +295,63 @@ $ euroeval --model <model-id> --dataset squad
 ```
 
 
+### Unofficial: BeleBele-en
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features reading comprehension questions across 122 languages. The dataset was created by professional translators who translated 900 multiple-choice questions from English into other languages, with answers carefully validated by native speakers.
+
+The original dataset consists of 900 samples, and we use 256 / 64 / 580 samples for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": 'Text: """We will endeavour to cut carbon dioxide emissions per unit of GDP by a notable margin by 2020 from the 2005 level,"" Hu said. He did not set a figure for the cuts, saying they will be made based on China\'s economic output. Hu encouraged developing countries ""to avoid the old path of polluting first and cleaning up later."" He added that ""they should not, however, be asked to take on obligations that go beyond their development stage, responsibility and capabilities."""\nQuestion: What did Hu suggest that developing countries do?\nChoices:\na. Take on obligations that push their development stage\nb. Focus on economic output\nc. Go beyond their current responsibilities\nd. Avoiding old paths of pollution',
+  "label": "d"
+}
+```
+```json
+{
+  "text": 'Text: "All of the cave entrances, which were named ""The Seven Sisters"", are at least 100 to 250 meters (328 to 820 feet) in diameter. Infrared images show that the temperature variations from night and day show that they are likely caves. ""They are cooler than the surrounding surface in the day and warmer at night. Their thermal behavior is not as steady as large caves on Earth that often maintain a fairly constant temperature, but it is consistent with these being deep holes in the ground,"" said Glen Cushing of the United States Geological Survey (USGS) Astrogeology Team and of Northern Arizona University located in Flagstaff, Arizona."\nQuestion: What information suggests that The Seven Sisters are caves?\nChoices:\na. Temperature variations\nb. The diameter of the cave entrances\nc. Geological surveys\nd. Pictures of caves on Earth',
+  "label": "a"
+}
+```
+```json
+{
+  "text": 'Text: The proposed amendment already passed both houses in 2011. A change was made this legislative session when the second sentence was deleted first by the House of Representatives and then was passed in a similar form by the Senate Monday. The failure of the second sentence, which proposes to ban same-sex civil unions, could possibly open the door for civil unions in the future. Following the process, HJR-3 will be reviewed again by the next elected legislature in either 2015 or 2016 to remain in process.\nQuestion: According to the passage, when was the second sentence deleted?\nChoices:\na. During the legislative session\nb. In 2011\nc. On Monday\nd. In 2015',
+  "label": "a"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 4
+- Prefix prompt:
+  ```
+  The following are texts with accompanying questions and answers.
+  ```
+- Base prompt template:
+  ```
+  Text: {text}
+  Question: {question}
+  Answer in max 3 words:
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Text: {text}
+
+  Answer the following question about the above text in at most 3 words.
+
+  Question: {question}
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-en
+```
+
+
 ## Knowledge
 
 ### MMLU
diff --git a/docs/datasets/finnish.md b/docs/datasets/finnish.md
index ab2106a24..578e7b033 100644
--- a/docs/datasets/finnish.md
+++ b/docs/datasets/finnish.md
@@ -266,6 +266,70 @@ $ euroeval --model <model-id> --dataset tydiqa-fi
 ```
 
 
+### Unofficial: BeleBele-fi
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Toisin kuin muut kädelliset, isot ihmisapinat eivät enää käytä käsiään liikkumiseen, painon kannattelemiseen tai liikkumiseen puissa itseään heilautellen. Simpanssin käsi ja jalka ovat samankokoisia ja -pituisia, mikä viittaa siihen, että kädelle varataan painoa rystykävelyssä. Ihmisen käsi on lyhyempi kuin jalka, ja sen sormiluut ovat suoremmat. Kahden-kolmen miljoonan vuoden ikäiset käsiluiden fossiilit paljastavat käden erikoistumisessa tämän muutoksen liikkumisesta käyttelyyn.\nKysymys: Mikä seuraavista kuvaa tarkasti simpanssin sormiluita?\nVaihtoehdot:\na. Ne ovat suoremmat kuin ihmisillä\nb. Niiden kädet ja jalat ovat erikokoisia\nc. Niitä käytetään painon kannattelemiseen\nd. Niitä käytetään pääasiassa käyttelyyn",
+  "label": "c"
+}
+```
+```json
+{
+  "text": "Panaman paperit on yläkäsite panamalaisen lakiyrityksen Mossack Fonsecan noin kymmenelle miljoonalle asiakirjalle, jotka vuodettiin lehdistölle keväällä 2016. Asiakirjoista selvisi, että neljätoista pankkia auttoi varakkaita asiakkaita piilottamaan miljardeja USA:n dollareita verojen ja muiden sääntelyjen välttämiseksi. Brittiläisen sanomalehden The Guardianin mukaan Deutsche Bank hallitsi tämän toteuttamiseen käytetyistä 1 200 postilaatikkoyrityksestä suunnilleen kolmasosaa. Seurasi maailmanlaajuisia protesteja ja useita rikossyytteitä, ja Islannin ja Pakistanin hallitusten johtajat kumpikin erosivat.\nKysymys: Kuka brittiläisen lehdistön väitteen mukaan hallinnoi monia varojen piilottamisessa käytettyjä yrityksiä tekstikatkelman mukaan?\nVaihtoehdot:\na. Eri pankkien varakkaat asiakkaat\nb. Panamalainen lakiyritys\nc. Deutsche Bank\nd. Pakistanin hallitus",
+  "label": "c"
+}
+```
+```json
+{
+  "text": "Teksti: Sundarban on maailman suurin mangrovemetsäalue. Se ulottuu 80 kilometriä (50 mailia) rannikolta Bangladeshin ja Intian takamaille. Sundarban on julistettu Unescon maailmanperintökohteeksi. Metsän Intian puolella sijaitsevaa osaa kutsutaan Sundarbanin kansallispuistoksi. Metsät eivät kuitenkaan ole pelkkiä mangrovesoita, vaan niihin kuuluu joitakin viimeisiä jäänteitä niistä mahtavista viidakoista, jotka aikoinaan peittivät koko Gangesin tasangon. Sundarban kattaa 3 850 neliökilometrin alueen, josta noin kolmasosa on vesi- tai suoalueiden peitossa. Vuodesta 1966 asti Sundarbans on ollut villieläinten suojelualue. Arvioidaan, että siellä on nykyään 400 intiantiikeriä ja suunnilleen 30 000 aksishirveä.\nKysymys: Mikä metsän osa on Intian puolella?\nVaihtoehdot:\na. Sundarbanin kansallispuisto\nb. Villieläinten suojelualue\nc. Maailmanperintökohde\nd. Gangesin tasanko",
+  "label": "a"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Seuraavat ovat monivalintakysymyksiä (vastauksineen).
+  ```
+- Base prompt template:
+  ```
+  Kysymys: {text}
+  Vaihtoehdot:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Vastaus: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Kysymys: {text}
+  Vaihtoehdot:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Vastaa yllä olevaan kysymykseen käyttämällä 'a', 'b', 'c' tai 'd', äläkä mitään muuta.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-fi
+```
+
+
 ## Common-sense Reasoning
 
 ### HellaSwag-fi
diff --git a/docs/datasets/french.md b/docs/datasets/french.md
index d4cb64dab..993406ce2 100644
--- a/docs/datasets/french.md
+++ b/docs/datasets/french.md
@@ -296,6 +296,70 @@ $ euroeval --model <model-id> --dataset fquad
 ```
 
 
+### Unofficial: BeleBele-fr
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Texte: Lorsqu’un petit groupe d’êtres vivants (une petite population) est séparé de la population principale dont il est issu (par exemple, s’il se déplace au-dessus d’une chaîne de montagnes ou d’une rivière, ou s’il se déplace vers une nouvelle île de sorte qu’il ne peut pas facilement revenir en arrière), il se retrouve souvent dans un environnement différent de celui dans lequel il était auparavant. Ce nouvel environnement a des ressources et des concurrents différents, de sorte que la nouvelle population aura besoin de caractéristiques ou d'adaptations nouvelles pour être un concurrent puissant par rapport à ce dont elle avait besoin auparavant. La population d'origine n'a pas changé du tout,\xa0elle a toujours besoin des mêmes adaptations. Au fil du temps, à mesure que la nouvelle population s'adapte à son nouvel environnement, elle commence à ressembler de moins en moins à l'autre population. Enfin, après des milliers ou même des millions d'années, les deux populations paraîtront tellement différentes qu'elles ne pourront plus être considérées comme appartenant à la même espèce. Nous appelons ce processus «\u2009spéciation\u2009», ce qui signifie simplement la formation de nouvelles espèces. La spéciation est une conséquence inévitable et une partie très importante de l’évolution.\nQuestion: D’après l’extrait et parmi les exemples ci-dessous, qu’est-ce qui gênerait le processus d’évolution\xa0?\nChoix:\na. La difficulté pour un petit groupe à s’épanouir dans un nouvel endroit\nb. La migration d’une portion d’une population vers un nouvel environnement\nc. L’ajustement par une population de son adaptation à un nouvel environnement\nd. Le fait qu’une population finisse par devenir deux populations distinctes",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Texte: Le pillage généralisé se serait poursuivi pendant la nuit, les forces de l'ordre n'étant pas présentes dans les rues de Bichkek. Un observateur a décrit Bichkek comme étant en train de sombrer dans un état d’« anarchie », tandis que la population se déplaçait en bandes dans les rues et pillait les magasins de biens de consommation. Plusieurs habitants de Bichkek ont reproché les manifestants du sud d'être responsables de l'anarchie.\nQuestion: Qui a accusé les manifestants du sud de pillage\xa0?\nChoix:\na. Des habitants de Bichkek\nb. Les forces de l’ordre\nc. Les anarchistes\nd. Des bandes de personnes",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Texte: Dans de nombreuses régions du monde, faire un signe de la main est un geste amical signifiant «\u2009bonjour\u2009». En revanche, en Malaisie, du moins chez les Malais des zones rurales, cela signifie « viens par ici », comme le fait de plier l'index vers soi, geste utilisé dans certains pays occidentaux, et il ne devrait être utilisé qu'en ce sens. De même, un voyageur britannique en Espagne pourrait confondre un signe d'adieu fait par une personne qui tourne la paume de sa main vers elle-même (plutôt que vers la personne à qui elle adresse le signe) avec une invitation à revenir.\nQuestion: Dans les zones rurales de la Malaisie, quel geste signifie « viens par ici » ?\nChoix:\na. Plier l’index\nb. Faire un signe de la main\nc. Faire un « high five »\nd. Lever le pouce",
+  "label": "b"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Les questions suivantes sont des questions à choix multiples (avec réponses).
+  ```
+- Base prompt template:
+  ```
+  Question: {text}
+  Choix:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Réponse: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Question: {text}
+  Choix:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Répondez à la question ci-dessus par 'a', 'b', 'c' ou 'd', et rien d'autre.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-fr
+```
+
+
 ## Knowledge
 
 ### MMLU-fr
diff --git a/docs/datasets/german.md b/docs/datasets/german.md
index 4ba94e997..c4ad04643 100644
--- a/docs/datasets/german.md
+++ b/docs/datasets/german.md
@@ -284,6 +284,65 @@ $ euroeval --model <model-id> --dataset germanquad
 ```
 
 
+### Unofficial: BeleBele-de
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Text: Es gibt viele Dinge, die Sie vor und während einer Reise berücksichtigen müssen. Erwarten Sie nicht, dass die Dinge beim Reisen genau so sind wie „zuhause“. Umgangsformen, Gesetze, Essen, Verkehr, Unterkünfte, Standards, Spache und so weiter werden zu einem gewissen Grad anders sein als dort, wo Sie leben. Dies ist etwas, was man immer im Hinterkopf behalten sollte, um Enttäuschung oder gar Abneigung über lokale Vorgehensweisen zu vermeiden.\nFragen: Was kann Reisenden dem Abschnitt nach helfen, Enttäuschung beim Besuch neuer Orte zu vermeiden?\nAntwortmöglichkeiten:\na. Ähnliche Standards wie zuhause erwarten\nb. Essen probieren, das ungewohnt ist\nc. Die gleichen Gesetze wie zuhause einhalten\nd. Nicht vorher nach Unterkünften recherchieren",
+  "label": "b"
+}
+```
+```json
+{
+  "text": "Text: Genehmigungen müssen im Voraus bestellt werden. Sie benötigen eine Genehmigung, um in La Sirena zu übernachten. Sirena ist die einzige Rangerstation, die neben Zelten auch Übernachtung im Schlafsaal und warme Mahlzeiten anbietet. La Leona, San Pedrillo und Los Patos bieten nur Camping ohne Verpflegung an. Es ist möglich, eine Parklizenz direkt bei der Rangerstation in Puerto Jiménez zu bekommen, aber sie akzeptieren keine Kreditkarten Die Parkverwaltung (MINAE) stellt Genehmigungen  für den Park nicht früher als einen Monat vor der geplanten Ankunft aus. CafeNet El Sol bietet einen Reservierungsservice gegen eine Gebühr von 30 US-Dollar bzw. 10 US-Dollar für Tageskarten an. Einzelheiten dazu findet man auf deren Corcovado-Seite.\nFragen: Welche der folgenden Rangerstationen bietet zwei Übernachtungsmöglichkeiten an?\nAntwortmöglichkeiten:\na. Sirena\nb. Los Patos\nc. La Leona\nd. San Pedrillo",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Text: Naturnaher Tourismus zieht Leute an, die daran interessiert sind, Naturgebiete zu besuchen, um die Landschaft zu genießen, einschließlich der wilden Pflanzen und Tiere. Beispiele für Aktivitäten vor Ort sind Jagen, Angeln, Fotografie, Vogelbeobachtung, der Besuch von Parks und das Lernen von Informationen über das Ökosystem. Ein Beispiel dafür ist der Besuch, das Fotografieren und das Studieren von Orangutangs in Borneo.\nFragen: Welche der folgenden Aktivitäten ist kein Beispiel für naturnahen Tourismus?\nAntwortmöglichkeiten:\na. Wandern zu einem Wasserfall\nb. Fotografieren von Wildblumen\nc. Besuch eines Wissenschaftsmuseum\nd. Fliegenfischen",
+  "label": "c"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Die folgenden Fragen sind Multiple-Choice-Fragen (mit Antworten).
+  ```
+- Base prompt template:
+  ```
+  Frage: {text}
+  Antwort: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Frage: {text}
+  Antwortmöglichkeiten:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Beantworten Sie die obige Frage mit 'a', 'b', 'c' oder 'd', und nichts anderes.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-de
+```
+
+
 ## Knowledge
 
 ### MMLU-de
diff --git a/docs/datasets/icelandic.md b/docs/datasets/icelandic.md
index 6ca9e84cd..9a8e54241 100644
--- a/docs/datasets/icelandic.md
+++ b/docs/datasets/icelandic.md
@@ -489,6 +489,70 @@ $ euroeval --model <model-id> --dataset icelandic-qa
 ```
 
 
+### Unofficial: BeleBele-is
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Texti: Í Frelsisstríðinu mynduðu ríkin þrettán veikburða ríkisstjórn – með Þjóðþingið sem eina þátt þess – skv. fyrstu stjórnarskránni. Þingið var ekki með nægar valdheimildir til að leggja á skatta, og vegna þess að ekki var neinn alríkisstjóri eða dómsvald til staðar, treysti það á yfirvöld í hverju ríki fyrir sig, sem voru oft og tíðum ósamvinnuþýð, til að framfylgja lögum þess. Það hafði heldur engar valdheimildir til að fella niður skattalög og tolla á milli ríkja. Greinarnar gerðu kröfu um samhljóða samþykki allra ríkjanna áður en hægt var að breyta þeim og ríkin sýndu ríkisvaldinu svo mikla lítilsvirðingu að fulltrúar þeirra voru oft fjarverandi.\nSpurning: Samkvæmt því sem fram kemur í kaflanum, hvaða fullyrðing á nákvæmlega við um ástand ríkisvaldsins í frelsisstríðinu?\nSvarmöguleikar:\na. Skattar voru innheimtir af þinginu og ríkisstofnunum\nb. Breytingar á stjórnarskránni þurftu samþykki þingsins\nc. Fulltrúar ríkjanna voru oft fjarverandi\nd. Hin miðlæga ríkisstjórn var mynduð í kringum tvo meginþætti",
+  "label": "c"
+}
+```
+```json
+{
+  "text": "Texti: İzmir er þriðja stærsta borg Tyrklands með um 3,7 milljónir íbúa, næststærstu höfnina á eftir Istanbúl og er mjög góð samgöngumiðstöð. Hin forna borg Smyrna er núna nútímaleg, þróuð og iðandi viðskiptamiðstöð sem staðsett er við gríðarstóran flóa og umkringd er fjöllum. Hinar breiðu breiðgötur, byggingar með framhliðum úr gleri og nútímalegar verslunarmiðstöðvar með hefðbundnum rauðum þakskífum, 18. aldar markaðurinn og gamlar moskur og kirkjur, þó að andrúmsloft borgarinnar tengist meira Miðjarðarhafssvæði Evrópu en hefðbundnu Tyrklandi.\nSpurning: Hvert eftirfarandi einkennir Izmir er frá fornri tíð?\nSvarmöguleikar:\na. Breiðar breiðgötur\nb. Byggingar með framhliðum úr gleri\nc. Verslanamiðstöðvar\nd. rauðar þakskífur",
+  "label": "d"
+}
+```
+```json
+{
+  "text": "Texti: Dæmigert fyrir það tímabil er Kirby Muxloe Castle sem er frekar víggirt hús en raunverulegur kastali. Stóru gljáðu gluggarnir og þunnu veggirnir hefðu ekki getað staðist stórárás í langan tíma. Árið 1480, þegar Hastings lávarður hóf byggingarframkvæmdirnar, ríkti friður í nánast öllu landinu og aðeins var þörf á varnarmúrum gegn litlum ræningjahópum.\nSpurning: Hvert af eftirtöldu hefði verið talið óvenjulegt við byggingu Kirby Muxloe kastala á þeim tíma sem talað er um í kaflanum?\nSvarmöguleikar:\na. Stórir gluggar\nb. Grunnur sem á að standast árásir\nc. Minna af varnarútbúnaði en í öðrum köstulum\nd. Þunnir veggir",
+  "label": "b"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Eftirfarandi eru fjölvalsspurningar (með svörum).
+  ```
+- Base prompt template:
+  ```
+  Spurningar: {text}
+  Svarmöguleikar:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Svara: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Spurningar: {text}
+  Svarmöguleikar:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Svaraðu eftirfarandi spurningum með 'a', 'b', 'c' eða 'd', og engu öðru.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-is
+```
+
+
 ## Knowledge
 
 ### IcelandicKnowledge
diff --git a/docs/datasets/italian.md b/docs/datasets/italian.md
index a0c3a88ad..0eae51f46 100644
--- a/docs/datasets/italian.md
+++ b/docs/datasets/italian.md
@@ -371,6 +371,70 @@ $ euroeval --model <model-id> --dataset squad-it
 ```
 
 
+### Unofficial: BeleBele-it
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Testo: Con la decisione del signor Rudd di firmare l’accordo sul clima di Kyoto, gli Stati Uniti, che ora saranno l’unica nazione sviluppata a non averlo ratificato, rimangono isolati. Il precedente governo conservatore australiano aveva rifiutato di ratificare gli accordi di Kyoto asserendo che avrebbero danneggiato l'economia, data la pesante dipendenza dalle esportazioni di carbone, mentre gli obiettivi sulle emissioni non sarebbero stati vincolanti per Paesi come l'India e la Cina.\nDomanda: Il precedente governo australiano pensava che la ratifica di Kyoto avrebbe causato danni a cosa?\nOpzioni:\na. Stati Uniti\nb. Economia del Paese\nc. Esportazioni di carbone\nd. Gli obiettivi di emissione del Paese",
+  "label": "b"
+}
+```
+```json
+{
+  "text": "Testo: "I commenti, in diretta televisiva, hanno rappresentato la prima occasione per autorevoli fonti iraniane per ammettere che le sanzioni sono efficaci. Esse comprendono limitazioni finanziarie e il divieto dell\'Unione europea all\'esportazione di petrolio greggio, che rappresenta l\'80% del reddito estero nell\'economia dell\'Iran. Secondo l\'ultimo rapporto mensile dell’OPEC, il volume delle esportazioni di greggio è sceso al livello più basso degli ultimi vent\'anni, con 2,8 milioni di barili al giorno. Il leader supremo del Paese, l’Ayatollah Ali Khamenei, ha parlato della dipendenza dal petrolio paragonandola ad ""una trappola"" che risale al periodo precedente la rivoluzione islamica iraniana del 1979 e dalla quale il Paese si dovrebbe liberare."\nDomanda: Secondo il passaggio, chi ha ammesso gli effetti delle sanzioni sull\'economia iraniana?\nOpzioni:\na. Autorevoli fonti\nb. OPEC\nc. Ayatollah Ali Khamenei\nd. L\'Unione Europea",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Testo: Il dottor Lee si è detto preoccupato anche in merito ai rapporti che rivelano che i bambini in Turchia ora sono stati contagiati dal virus dell'influenza aviaria A(H5N1) senza ammalarsi. Ha sottolineato che secondo alcuni studi la malattia diventerà meno mortale prima che possa causare un'epidemia globale. Si teme che se permangono sintomi influenzali di lieve entità, i pazienti possano continuare a contagiare più persone durante la loro routine quotidiana.\nDomanda: Secondo il brano, cosa dovrebbe accadere alla malattia prima di causare un'epidemia globale?\nOpzioni:\na. Deve diventare meno letale\nb. I sintomi devono rimanere lievi\nc. Occorre che più pazienti vengano infettati\nd. I bambini devono manifestare i sintomi",
+  "label": "a"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Le seguenti sono domande a scelta multipla (con relative risposte).
+  ```
+- Base prompt template:
+  ```
+  Domanda: {text}
+  Opzioni:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Risposta: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Domanda: {text}
+  Opzioni:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Rispondete alla domanda precedente con 'a', 'b', 'c' o 'd', e nient'altro.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-it
+```
+
+
 ## Knowledge
 
 ### MMLU-it
diff --git a/docs/datasets/norwegian.md b/docs/datasets/norwegian.md
index 7a704d76a..094a4445c 100644
--- a/docs/datasets/norwegian.md
+++ b/docs/datasets/norwegian.md
@@ -644,6 +644,70 @@ $ euroeval --model <model-id> --dataset norglm-multi-qa
 ```
 
 
+### Unofficial: BeleBele-no
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Tekst: Det kinesiske nyhetsbyrået Xinhua meldte tidligere fra om at et fly var kapret. Det ble senere rapportert at flyet fikk en bombetrussel og ble veiledet mot retur til Afghanistan med landing i Kandahar. Ifølge de første rapportene ble flyet dirigert tilbake til Afghanistan etter å ha blitt nektet nødlanding i Ürümqi.\nSpørsmål: Hva ble ikke sagt i den nyeste rapporten fra nyhetsbyrået Xinhua?\nSvaralternativer:\na. Flyet fikk en bombetrussel\nb. Flyet landet i Ürümqi\nc. Flyet ble dirigert til Afghanistan\nd. Flyet landet i Kandahar",
+  "label": "b"
+}
+```
+```json
+{
+  "text": "Tekst: Tyskland begynte å gjøre seg klare til å invadere Storbritannia da kampen om Frankrike var over. Tyskland gav angrepet kodenavnet «operasjon sjøløve». Mesteparten av den britiske hærens tunge våpen og forsyninger hadde gått tapt da den flyktet fra Dunkirk, så de var svært sårbar. Den britiske marinen var imidlertid fremdeles mye kraftigere enn den tyske («Kriegsmarine») og kunne ha ødelagt en eventuell invasjonsflåte sendt over den engelske kanal. Det var likevel svært få skip fra Royal Navy som ble stasjonert nær de sannsynlige invasjonsrutene siden admiralene var engstelige for at de kom til å bli senket av tyske luftangrep.\nSpørsmål: Hva kalte Tyskland angrepet på Storbritannia?\nSvaralternativer:\na. Dunkirk\nb. Operasjon sjøløve\nc. Kriegsmarine\nd. Royal Navy",
+  "label": "b"
+}
+```
+```json
+{
+  "text": "Tekst: Det italienske og det tyske landslaget er de nest beste lagene i verden og var FIFA World Cup-mestere i 2006. Fotball, basketball, volleyball, vannpolo, fekting, rugby, sykling, ishockey, rullehockey og Formel-1 bilsport er godt likte sportsgrener. Vintersport er mest populært i nordlige områder, der italienere deltar i internasjonale konkurranser og OL-arrangementer.\nSpørsmål: Hvilke av følgende sporter vant et verdensmesterskap for Italia, basert på informasjonen i avsnittet?\nSvaralternativer:\na. Fotball\nb. Vannpolo\nc. Basketball\nd. Sykling",
+  "label": "a"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Følgende er flervalgsspørsmål (med svar).
+  ```
+- Base prompt template:
+  ```
+  Spørsmål: {text}
+  Svaralternativer:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Svar: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Spørsmål: {text}
+  Svaralternativer:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Besvar følgende spørsmål med 'a', 'b', 'c' eller 'd', og ikke noe annet.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-no
+```
+
+
 ## Knowledge
 
 ### NRK Quiz QA
diff --git a/docs/datasets/spanish.md b/docs/datasets/spanish.md
index e84351aad..e41e3e057 100644
--- a/docs/datasets/spanish.md
+++ b/docs/datasets/spanish.md
@@ -330,6 +330,71 @@ You can evaluate this dataset directly as follows:
 $ euroeval --model <model-id> --dataset xquad-es
 ```
 
+
+### Unofficial: BeleBele-es
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Texto: Beba alcohol con moderación. Este afecta a cada persona de manera diferente y conocer sus propios límites es sumamente importante. La ingesta excesiva de alcohol puede causar problemas de salud crónicos como daño hepático e incluso ceguera y muerte. El peligro potencial se incrementa con el consumo de alcohol elaborado de forma ilegal. En las bebidas alcohólicas ilegales puede haber varias impurezas amenazantes, como el metanol, capaz de provocar ceguera o incluso la muerte, aun cuando se ingiera poca cantidad.\nPregunta: Según el fragmento, ¿cuál de los siguientes sentidos puede verse afectado por el consumo excesivo de alcohol?\nOpciones:\na. Audición\nb. Vista\nc. Gusto\nd. Olfato",
+  "label": "b"
+}
+```
+```json
+{
+  "text": "Texto: Leslie Aun, vocero de la Fundación Komen, informó que rige una nueva normativa en la organización conforme la cual no procederá el otorgamiento de subvenciones o fondos en favor de entidades que sean objeto de investigación oficial. La política de Komen desacreditó a Planned Parenthood a raíz de una investigación en curso que dirige el representante Cliff Stearns sobre la forma en la que esta organización informa y utiliza sus fondos. En su rol de director del Subcomité de Supervisión e Investigación, que se encuentra bajo el paraguas del Comité de Energía y Comercio, Stearns conduce una investigación para determinar si los impuestos se usan para financiar interrupciones de embarazos a través de Paternidad Planificada.\nPregunta: ¿Qué comité preside Cliff Stearns?\nOpciones:\na. Comité de Energía y Comercio de la Cámara de Representantes\nb. La Fundación Komen\nc. Planned Parenthood\nd. El Subcomité de Supervisión e Investigación",
+  "label": "d"
+}
+```
+```json
+{
+  "text": "Texto: El elemento del determinismo cultural se encontraba muy presente en el romanticismo, según estudiosos como Goether, Fichte y Schlegel. En el contexto del Romanticismo, la geografía moldeó a las personas y, con el transcurso del tiempo, se desarrollaron costumbres y culturas relacionadas con esa geografía que, al estar en armonía con la localización de esa sociedad, eran preferibles a leyes que se impusieran de forma arbitraria.\nPregunta: De acuerdo con el texto, ¿qué moldeó a las personas durante el período del Romanticismo?\nOpciones:\na. Leyes\nb. Geografía\nc. Costumbres\nd. Cultura",
+  "label": "b"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Las siguientes son preguntas de opción múltiple (con respuestas).
+  ```
+- Base prompt template:
+  ```
+  Pregunta: {text}
+  Opciones:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Respuesta: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Pregunta: {text}
+  Opciones:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Responda la pregunta anterior usando solo 'a', 'b', 'c' o 'd', y nada más.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-es
+```
+
+
 ## Knowledge
 
 ### MMLU-es
diff --git a/docs/datasets/swedish.md b/docs/datasets/swedish.md
index f5a29a8fb..dd5835f29 100644
--- a/docs/datasets/swedish.md
+++ b/docs/datasets/swedish.md
@@ -299,6 +299,70 @@ $ euroeval --model <model-id> --dataset scandiqa-sv
 ```
 
 
+### Unofficial: BeleBele-sv
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Sundarbans är det största kustmangrovebältet i världen och sträcker sig 80 km in i Bangladesh och det indiska inlandet från kusten. Sundarbans har antagits på Unescos världsarvslista. Den del av skogen som ligger på indiskt territorium kallas Sundarbans National Park. Skogarna är dock inte bara mangroveträsk — de innehåller några av de sista kvarvarande ståndorterna av de stora djunglerna som en gång täckte Gangesslätten. Sundarban täcker ett område på 3 850 km², varav ungefär en tredjedel utgörs av våtmarker. Sedan 1966 har Sundarbans varit ett reservat för vilda djur, och det uppskattas att det nu finns 400 bengaliska tigrar och omkring 30 000 axishjortar i området.\nFråga: Vilken del av skogen ligger på indiskt territorium?\nSvarsalternativ:\na. Sundarbans National Park\nb. Reservatet för vilda djur\nc. Världsarvet\nd. Gangesslätten",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Italiens nationella fotboll, tillsammans med det tyska fotbollslaget, är världens näst mest framgångsrika lag och var mästare i FIFA-världscupen år 2006. Populära sporter inkluderar fotboll, basket, volleyboll, vattenpolo, fäktning, rugby, cykel, ishockey, rullskridskohockey och Formel 1. Vintersporter är mest populära i de norra regionerna, där italienare tävlar i internationella tävlingar och olympiska evenemang.\nFråga: I vilken av följande sporter vann Italien en världscup enligt avsnittet?\nSvarsalternativ:\na. Fotboll\nb. Vattenpolo\nc. Basket\nd. Cykel",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Bokning i förväg ger resenären trygghet och en försäkran om att de kommer att ha någonstans att sova när de anländer till sin destination. Resebyråer har ofta avtal med särskilda hotell, men det kan vara möjligt att boka andra typer av boenden, såsom campingplatser, genom en resebyrå. Resebyråer erbjuder ofta paket som inkluderar frukost, transfer till och från flygplatsen, och till och med paketresor som kombinerar flyg och hotell. De kan också reservera din bokning åt dig om du behöver tid att tänka över erbjudandet eller skaffa fram ytterligare dokument som krävs för din destination (t.ex. visering). Alla ändringar och förfrågningar ska dock gå genom resebyrån först, och inte direkt till hotellet.\nFråga: Vilken typ av resenär kommer sannolikt inte att dra nytta av att använda sig av tjänster från en resebyrå, enligt det som står i texten?\nSvarsalternativ:\na. En obeslutsam resenär\nb. En resenär som är spontan\nc. En resenär som inte har skaffat visum än\nd. En resenär som föredra att boka paketerbjudanden",
+  "label": "b"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Följande är flervalsfrågor (med svar).
+  ```
+- Base prompt template:
+  ```
+  Fråga: {text}
+  Svarsalternativ:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+  Svar: {label}
+  ```
+- Instruction-tuned prompt template:
+  ```
+  Fråga: {text}
+  Svarsalternativ:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_d}
+
+  Besvara följande fråga med 'a', 'b', 'c' eller 'd', och inget annat.
+  ```
+
+You can evaluate this dataset directly as follows:
+
+```bash
+$ euroeval --model <model-id> --dataset belebele-sv
+```
+
+
 ## Knowledge
 
 ### MMLU-sv
diff --git a/src/euroeval/dataset_configs/english.py b/src/euroeval/dataset_configs/english.py
index 5ed376655..68c29aefe 100644
--- a/src/euroeval/dataset_configs/english.py
+++ b/src/euroeval/dataset_configs/english.py
@@ -79,7 +79,7 @@
 )
 
 BELEBELE_CONFIG = DatasetConfig(
-    name="belebele",
+    name="belebele-en",
     pretty_name="the English multiple choice reading comprehension dataset BeleBele",
     huggingface_id="EuroEval/belebele-mini",
     task=MCRC,


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

This PR adds the BeleBele dataset documentation to EuroEval to provide detailed examples and evaluation setups in multiple languages.  
- Documents added for Norwegian, Icelandic, German, French, English, Dutch, and Danish  
- Each file now includes sample JSON entries and prompt templates for model evaluation

### Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

<details>
<summary>Show a summary per file</summary>

| File                  | Description                                                                   |
| --------------------- | ----------------------------------------------------------------------------- |
| docs/datasets/norwegian.md | Added BeleBele dataset documentation and examples in Norwegian               |
| docs/datasets/icelandic.md | Added BeleBele dataset documentation and examples in Icelandic               |
| docs/datasets/german.md    | Added BeleBele dataset documentation and evaluation instructions in German     |
| docs/datasets/french.md    | Added BeleBele dataset documentation in French with prompt templates           |
| docs/datasets/english.md   | Added two sections documenting BeleBele examples for reading comprehension     |
| docs/datasets/dutch.md     | Added BeleBele dataset documentation and examples in Dutch                    |
| docs/datasets/danish.md    | Added BeleBele dataset documentation and examples in Danish                   |
</details>





state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-03T06:54:20Z
user: login: saattrupdan
body: Thanks! A few small changes and it's ready to merge 🙂
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-05-03T07:04:25Z
user: login: oliverkinch
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-03T07:13:22Z
user: login: saattrupdan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-03T09:40:14Z
user: login: saattrupdan
body: Small change regarding English Belebele. Can you also change the name in the dataset config to "belebele-en"?
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-05-04T07:21:16Z
user: login: saattrupdan
body: LGTM!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-04T11:34:27Z

Review comments:
diff_hunk: @@ -370,6 +370,70 @@ $ euroeval --model <model-id> --dataset mmlu-fr
 ```
 
 
+### Unofficial: BeleBele
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Texte: Lorsqu’un petit groupe d’êtres vivants (une petite population) est séparé de la population principale dont il est issu (par exemple, s’il se déplace au-dessus d’une chaîne de montagnes ou d’une rivière, ou s’il se déplace vers une nouvelle île de sorte qu’il ne peut pas facilement revenir en arrière), il se retrouve souvent dans un environnement différent de celui dans lequel il était auparavant. Ce nouvel environnement a des ressources et des concurrents différents, de sorte que la nouvelle population aura besoin de caractéristiques ou d'adaptations nouvelles pour être un concurrent puissant par rapport à ce dont elle avait besoin auparavant. La population d'origine n'a pas changé du tout,\xa0elle a toujours besoin des mêmes adaptations. Au fil du temps, à mesure que la nouvelle population s'adapte à son nouvel environnement, elle commence à ressembler de moins en moins à l'autre population. Enfin, après des milliers ou même des millions d'années, les deux populations paraîtront tellement différentes qu'elles ne pourront plus être considérées comme appartenant à la même espèce. Nous appelons ce processus «\u2009spéciation\u2009», ce qui signifie simplement la formation de nouvelles espèces. La spéciation est une conséquence inévitable et une partie très importante de l’évolution.\nQuestion: D’après l’extrait et parmi les exemples ci-dessous, qu’est-ce qui gênerait le processus d’évolution\xa0?\nChoix:\na. La difficulté pour un petit groupe à s’épanouir dans un nouvel endroit\nb. La migration d’une portion d’une population vers un nouvel environnement\nc. L’ajustement par une population de son adaptation à un nouvel environnement\nd. Le fait qu’une population finisse par devenir deux populations distinctes",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Texte: Le pillage généralisé se serait poursuivi pendant la nuit, les forces de l'ordre n'étant pas présentes dans les rues de Bichkek. Un observateur a décrit Bichkek comme étant en train de sombrer dans un état d’« anarchie », tandis que la population se déplaçait en bandes dans les rues et pillait les magasins de biens de consommation. Plusieurs habitants de Bichkek ont reproché les manifestants du sud d'être responsables de l'anarchie.\nQuestion: Qui a accusé les manifestants du sud de pillage\xa0?\nChoix:\na. Des habitants de Bichkek\nb. Les forces de l’ordre\nc. Les anarchistes\nd. Des bandes de personnes",
+  "label": "a"
+}
+```
+```json
+{
+  "text": "Texte: Dans de nombreuses régions du monde, faire un signe de la main est un geste amical signifiant «\u2009bonjour\u2009». En revanche, en Malaisie, du moins chez les Malais des zones rurales, cela signifie « viens par ici », comme le fait de plier l'index vers soi, geste utilisé dans certains pays occidentaux, et il ne devrait être utilisé qu'en ce sens. De même, un voyageur britannique en Espagne pourrait confondre un signe d'adieu fait par une personne qui tourne la paume de sa main vers elle-même (plutôt que vers la personne à qui elle adresse le signe) avec une invitation à revenir.\nQuestion: Dans les zones rurales de la Malaisie, quel geste signifie « viens par ici » ?\nChoix:\na. Plier l’index\nb. Faire un signe de la main\nc. Faire un « high five »\nd. Lever le pouce",
+  "label": "b"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Les questions suivantes sont des questions à choix multiples (avec réponses).
+  ```
+- Base prompt template:
+  ```
+  Question: {text}
+  Choix:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_c}
path: docs/datasets/french.md
user: login: Copilot
body: In the base prompt template, the fourth option is incorrectly set to {option_c} instead of {option_d}. Please update it accordingly.
```suggestion
  d. {option_d}
```
created_at: 2025-05-03T06:54:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -624,6 +624,70 @@ When evaluating generative models, we use the following setup (see the
   ```
 
 
+### Unofficial: BeleBele
+
+This dataset was published in [this paper](https://aclanthology.org/2024.acl-long.44/) and features multiple-choice reading comprehension questions across 122 languages.
+
+The original dataset contains 900 unique multiple-choice reading comprehension passages and questions. From these, we use a 256 / 64 / 580 split for training, validation and testing, respectively.
+
+Here are a few examples from the training split:
+
+```json
+{
+  "text": "Tekst: Prognoserne siger, at stormen, der er omkring 645 mil (1040 km) vest for Kap Verde-øerne, sandsynligvis vil forsvinde, før den truer nogen landområder. Fred har i øjeblikket vinde på 165 km/t og bevæger sig mod nordvest. Fred er den heftigste tropiske cyklon, der nogensinde er blevet registreret så sydligt og østligt i Atlanterhavet, siden man begyndte at bruge satellitbilleder, og kun den tredje store orkan, der er registreret øst for 35°V.\nSpørgsmål: Da Fred befandt sig nær Kap Verde-øerne, hvilken retning bevægede den sig så mod?\nSvarmuligheder:\na. Vest\nb. Syd\nc. Øst\nd. Nordvest",
+  "label": "d"
+}
+```
+```json
+{
+  "text": "Tekst: "Siden Pakistan i 1947 blev uafhængigt af det britiske styre, har den pakistanske præsident udpeget ""politiske agenter"", som styrer FATA, og som har næsten fuldstændig kontrol over områderne. Disse agenter er ansvarlige for at levere regerings- og retstjenester i henhold til artikel 247 i den pakistanske forfatning."\nSpørgsmål: Hvem leverer retslige tjenester til FATA?\nSvarmuligheder:\na. Den pakistanske regering\nb. Politiske agenter\nc. Pakistans præsident\nd. Den britiske regering",
+  "label": "b"
+}
+```
+```json
+{
+  "text": "Tekst: Alle er en del af samfundet og benytter transportsystemerne. Næsten alle klager over transportsystemerne. I udviklede lande hører du sjældent ligeså mange klager over vandkvalitet eller broer, der styrter sammen. Hvorfor giver transportsystemerne anledning til sådanne klager, hvorfor svigter de på daglig basis? Er transportingeniører blot inkompetente? Eller foregår der noget mere fundamentalt?\nSpørgsmål: Hvilken offentlig service siges at skabe størst utilfredshed i udviklede lande?\nSvarmuligheder:\na. Vandkvalitet\nb. Brobyggelse\nc. Offentlig transport\nd. Uddannelse",
+  "label": "c"
+}
+```
+
+When evaluating generative models, we use the following setup (see the
+[methodology](/methodology) for more information on how these are used):
+
+- Number of few-shot examples: 5
+- Prefix prompt:
+  ```
+  Følgende er multiple choice spørgsmål (med svar).
+  ```
+- Base prompt template:
+  ```
+  Spørgsmål: {text}
+  Svarmuligheder:
+  a. {option_a}
+  b. {option_b}
+  c. {option_c}
+  d. {option_c}
path: docs/datasets/danish.md
user: login: Copilot
body: The base prompt template for Danish incorrectly uses {option_c} for the fourth option; it should be {option_d}.
```suggestion
  d. {option_d}
```
created_at: 2025-05-03T06:54:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -624,6 +624,70 @@ When evaluating generative models, we use the following setup (see the
   ```
 
 
+### Unofficial: BeleBele
path: docs/datasets/danish.md
user: login: saattrupdan
body: Need the country code in the name, so that there's a correspondence between the title and the command one needs to run to evaluate a model on it:

```suggestion
### Unofficial: BeleBele-da
```

Same thing with the others. 

created_at: 2025-05-03T07:01:03Z
author_association: MEMBER
diff_hunk: @@ -624,6 +624,70 @@ When evaluating generative models, we use the following setup (see the
   ```
 
 
+### Unofficial: BeleBele
path: docs/datasets/danish.md
user: login: saattrupdan
body: The BeleBele dataset is a reading comprehension dataset, phrased as multiple choice, so we should move this to that section. 
created_at: 2025-05-03T07:03:22Z
author_association: MEMBER
diff_hunk: @@ -624,6 +624,70 @@ When evaluating generative models, we use the following setup (see the
   ```
 
 
+### Unofficial: BeleBele
path: docs/datasets/danish.md
user: login: oliverkinch
body: I just found the MC base promp template to be a better fit for BeleBele, but we should instead use the RC base promp template? E.g. the following one

- Number of few-shot examples: 4
- Prefix prompt:
  ```
  Følgende er tekster med tilhørende spørgsmål og svar.
  ```
- Base prompt template:
  ```
  Tekst: {text}
  Spørgsmål: {question}
  Svar med maks. 3 ord: {label}
  ```
- Instruction-tuned prompt template:
  ```
  Tekst: {text}

  Besvar følgende spørgsmål om teksten ovenfor med maks. 3 ord.

  Spørgsmål: {question}
  ```


created_at: 2025-05-03T07:13:22Z
author_association: COLLABORATOR
diff_hunk: @@ -624,6 +624,70 @@ When evaluating generative models, we use the following setup (see the
   ```
 
 
+### Unofficial: BeleBele
path: docs/datasets/danish.md
user: login: saattrupdan
body: I definitely agree that we should use the multiple choice prompt templates, I just meant that it should be moved into the "Reading Comprehension" section of the docs, as that's what the dataset is measuring 🙂 
created_at: 2025-05-03T09:40:14Z
author_association: MEMBER
diff_hunk: @@ -350,38 +350,30 @@ $ euroeval --model <model-id> --dataset belebele
 ```
 
 
-## Knowledge
-
-### MMLU
+### Unofficial: BeleBele
path: docs/datasets/english.md
user: login: saattrupdan
body: English one should also have an "-en" ending, as it was a natively multilingual dataset, rather than an English dataset that was translated.

I know that it is currently set to just "belebele" for English, but that was my mistake 🙈 

```suggestion
### Unofficial: BeleBele-en
```
created_at: 2025-05-04T07:19:33Z
author_association: MEMBER
diff_hunk: @@ -418,37 +410,41 @@ When evaluating generative models, we use the following setup (see the
 You can evaluate this dataset directly as follows:
 
 ```bash
-$ euroeval --model <model-id> --dataset mmlu
+$ euroeval --model <model-id> --dataset belebele
path: docs/datasets/english.md
user: login: saattrupdan
body: ```suggestion
$ euroeval --model <model-id> --dataset belebele-en
```
created_at: 2025-05-04T07:19:42Z
author_association: MEMBER

PR comments:
user: login: oliverkinch
created_at: 2025-05-03T06:54:44Z
body: We are currently not supporting BeleBele in Italian and Finnish even though BeleBele is also available in these languages (see table 4 in [the paper](https://aclanthology.org/2024.acl-long.44.pdf)). Should I add Italian and Finnish to `src/scripts/create_belebele.py` or is there a reason why these are not already added? @saattrupdan 
author_association: COLLABORATOR
user: login: saattrupdan
created_at: 2025-05-03T07:05:23Z
body: > We are currently not supporting BeleBele in Italian and Finnish even though BeleBele is also available in these languages (see table 4 in [the paper](https://aclanthology.org/2024.acl-long.44.pdf)). Should I add Italian and Finnish to `src/scripts/create_belebele.py` or is there a reason why these are not already added? @saattrupdan 

Please add these as well yeah, it was purely because we didn't support these languages at the time. 
author_association: MEMBER
user: login: oliverkinch
created_at: 2025-05-03T07:16:20Z
body: Note: I forgot the Swedish BeleBele, and I should also add the Spanish BeleBele
author_association: COLLABORATOR
user: login: oliverkinch
created_at: 2025-05-03T07:42:44Z
body: > > We are currently not supporting BeleBele in Italian and Finnish even though BeleBele is also available in these languages (see table 4 in [the paper](https://aclanthology.org/2024.acl-long.44.pdf)). Should I add Italian and Finnish to `src/scripts/create_belebele.py` or is there a reason why these are not already added? @saattrupdan
> 
> Please add these as well yeah, it was purely because we didn't support these languages at the time.

I have added them here https://github.com/EuroEval/EuroEval/pull/972
author_association: COLLABORATOR
user: login: oliverkinch
created_at: 2025-05-04T06:42:08Z
body: I have now added documentation for all languages except Faroese.
author_association: COLLABORATOR
user: login: oliverkinch
created_at: 2025-05-04T07:31:33Z
body: > Small change regarding English Belebele. Can you also change the name in the dataset config to "belebele-en"?

Done.

Also, for some reason I had included the documentation for the English BeleBele twice - not sure how that happened, but it is fixed now
author_association: COLLABORATOR