INJONGO: A Multicultural Intent Detection and Slot-filling Dataset for 16 African Languages
Authors: Hao Yu, Jesujoba Oluwadara Alabi, Andiswa Bukula, Jian Yun Zhuang, En-Shiun Annie Lee, Tadesse Kebede Guge, Israel Abebe Azime, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi KALIPE, Jonathan Mukiibi, Salomon KABONGO KABENAMUALU, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Shamsuddeen Hassan Muhammad, Salomey Osei, Sokhar Samb, Juliet, W. M, Dietrich Klakow, David Ifeoluwa Adelani
Accepted at: acl25

INJONGO: A Multicultural Intent Detection and Slot-filling Dataset for 16 African Languages

Hao Yu1, Jesujoba O. Alabi3,*, Andiswa Bukula4,*, Jian Yun Zhuang5, En-Shiun Annie Lee6, Tadesse Kebede Guge*, Israel Abebe Azime3, Happy Buzaaba7,*, Blessing Kudzaishe Sibanda*, Godson K. Kalipe*, Jonathan Mukiihi8,*, Salomon Kabongo Kabenamualu9,*, Mmasibidi Setaka4, Lolwethu Ndolela*, Nkiruka Odu*, Rooweither Mabuya4,*, Shamsuddeen Hassan Muhammad10,*, Salomey Osei11, Sokhar Samb12, Juliet W. Murage*, Dietrich Klakow3, David Ifeoluwa Adelani1,2,*

*Masakhane NLP, 1Mila - Quebec AI Institute & McGill University, Canada, 2Canada CIFAR AI Chair, 3Saarland University, Germany, 4SADiLaR, South Africa, 5University of Toronto, Canada, 6OntarioTech University, Canada, 7Princeton University, USA, 8Makerere University, Uganda, 9L3S Research Center, Germany, 10Imperial College London, United Kingdom, 11Universidad de Deusto, Spain, 12DAUST, Senegal.

Abstract

Slot-filling and intent detection are well-established tasks in Conversational AI. However, current large-scale benchmarks for these tasks often exclude evaluations of low-resource languages and rely on translations from English benchmarks, thereby predominantly reflecting Western-centric concepts. In this paper, we introduce INJONGO—a multicultural, open-source benchmark dataset for 16 African languages with utterances generated by native speakers across diverse domains, including banking, travel, home, and dining. Through extensive experiments, we benchmark the fine-tuning multilingual transformer models and the prompting large language models (LLMs), and show the advantage of leveraging African-cultural utterances over Western-centric utterances for improving cross-lingual transfer from the English language. Experimental results reveal that current LLMs struggle with the slot-filling task, with GPT-4o achieving an average performance of 26% F1-score. In contrast, intent detection performance is notably better, with an average accuracy of 70.6%, though it still falls behind the fine-tuning baselines. When compared to the English language, GPT-4o and fine-tuning baselines perform similarly on intent detection, achieving an accuracy of approximately 81%. Our findings suggest that the performance of LLMs is still behind for many low-resource African languages, and more work is needed to further improve their downstream performance.

1 Introduction

Intent detection and slot-filling are crucial components of the natural language understanding module in task-oriented dialogue systems (Hemphill et al., 1990; Coucke et al., 2018; Gupta et al., 2018). They map a user’s request to a predefined semantic category recognized by the dialogue manager, along with extracting specific entities (known as slots). This process facilitates generating an appropriate response for the end user. Despite their importance, only a few languages have labeled datasets available for these tasks across multiple domains (Larson and Leach, 2022).

Several efforts have been made to make datasets multilingual through human translation into other languages (Xu et al., 2020; Li et al., 2021; van der Goot et al., 2021; Ruder et al., 2023). However, these efforts face two key challenges: (1) the translationese effect, which makes utterances sound less natural in the target languages (Vanmassenhove et al., 2021; Bizzoni et al., 2020), and (2) the creation of utterances that are less culturally relevant. The Massive dataset (FitzGerald et al., 2023), which covers 51 languages, addresses the second challenge by encouraging translators to “localize”, “translate”, or “keep the slot unchanged”. Despite improvements in the utterance generation process, MASSIVE includes only three African languages (Amharic, Afrikaans and Swahili), and many utterances remain culturally irrelevant to the target language communities.

In this paper, we develop INJONGO—the first large-scale multicultural intent detection and slot-filling dataset covering 16 African languages, and English language. We cover the following five domains: banking, home, travel, utility, and kitchen & dining. The data construction process starts with providing an annotator with sentences from the CLINC dataset (Larson et al., 2019a) with a specified intent type, and they are to come up with culturally-relevant similar sentences and relevant slot entities (see Figure 1). The utterance genera-
tion process is followed by slots annotation. INJONGO dataset covers 5 domains, 40 intents, 23 slots, and 3,200 instances per African language.

We performed several supervised fine-tuning experiments with multilingual encoders and prompting of Large Language Models (LLMs), both using INJONGO. Our result shows that fine-tuning baselines could reach an accuracy of 93.7% and F1-score of 85.6 for intent detection and slot-filling tasks respectively. While the best prompting of LLMs results (GPT-4o) drops by ~28% accuracy point and ~52.6 F1 score. While slot-filling and named entity recognition tasks are often challenging for LLMs even for English (Yu et al., 2023), intent detection performance in English is similar performance whether we use fine-tuning baselines or prompt GPT-4o. Our findings suggest that LLMs performance is still behind for many low-resource African languages, and more work is needed to further improve their downstream performance. For reproducibility, we open-source our code1 and dataset 2 on GitHub. Dataset is released under CC BY 4.0 license. Fintuned best performance encoder models and decoder models are released on the HuggingFace3.

2 Related Work

African Benchmarks  Limited available labeled datasets are one of the major challenges of AfricaNLP. Since 2021, there have been many grassroots efforts to create large-scale datasets for African languages covering several tasks such as machine translation (Adelani et al., 2022; Alabi et al., 2025), named entity recognition (Adelani et al., 2021, 2022), sentiment classification (Muhammad et al., 2023), hate speech (Muhammad et al., 2025), question answering (Ogundepo et al., 2023), topic classification (Adelani et al., 2023b,a) covering 10 to 57 languages. The closest benchmark to our task of slot-filling is MasakhaNER (Adelani et al., 2021, 2022) that covers 20 African languages but they focus on four entity types “personal names”, “organization”, “location”, and “dates”, which are not fine-grained and well adapted to several domains such as banking and travel that we cover in INJONGO.

Intent and Slot-filling Benchmarks  Most of the existing benchmarks for intent detection and slot-filling tasks are English-only. There are a few efforts to make them multilingual in two ways: (1) human generating the utterances in a particular domain, followed by intent and slot filling annotation. (2) through human translation of annotated data from English to other languages which introduces some cultural bias since Western entities are being propagated. While the first approach is the most ideal methodology, it is very cost-intensive when scaling to many languages. The Facebook dataset (Schuster et al., 2019) followed the first approach by creating labeled data in three domains (alarm, reminder and weather) for three languages: English, Spanish and Thai. However, most other approaches make use of the second approach, where English data are translated to other languages (Xu et al., 2020; van der Goot et al., 2021; Li et al., 2021; Winkler et al., 2024), however, they often do not include African languages. XTREME-UP benchmark expanded the MTOP dataset (Li et al., 2021) to five African languages (Amharic, Hausa, Yoruba, Swahili and Zulu), while MASSIVE (FitzGerald et al., 2023) perform human translation to 50 languages including three African languages (Afrikaans, Amharic, and Swahili). MASSIVE benchmark partially addresses this Western cultural bias by encouraging translators to replace entities with more culturally relevant ones, but Western entities are still prevalent in the dataset. Table 1 summarizes all existing related works. In our paper, we introduce INJONGO which is the largest intent detection and slot-filling dataset covering 16 African languages, and we ensured that the slot entities are more culturally relevant in the respective countries the languages are from.

3 Introducing INJONGO Dataset

INJONGO4 is a joint intent detection and slot-filling dataset (ID-SF) for typologically diverse Sub-Saharan African languages and English. The selected languages represent diverse linguistic families and are widely spoken in Africa. These languages come from the two dominant language families in Africa: 13 from Niger-Congo and three from Afro-Asiatic. The languages covered are spoken by a large population in Africa, ranging from Swahili with 98M speakers to Wolof with 5M speakers,

1Code: McGill-NLP/Injongo
2Dataset: Masakhane-NLU
3Model: McGill-NLP /INJONGO Collection
4INJONGO means intent in isiXhosa language.
<table>
  <tr>
    <th>Dataset</th>
    <th># Domains</th>
    <th># Intents</th>
    <th># Slots</th>
    <th># utterances</th>
    <th># Languages</th>
    <th># African languages</th>
    <th>Multicultural?</th>
  </tr>
  <tr>
    <td>CLINC (Larson et al., 2019b)</td>
    <td>10</td>
    <td>150</td>
    <td>0</td>
    <td>23,700</td>
    <td>1</td>
    <td>0</td>
    <td>yes</td>
  </tr>
  <tr>
    <td>Facebook (Schuster et al., 2019)</td>
    <td>3</td>
    <td>12</td>
    <td>11</td>
    <td>57,000</td>
    <td>3</td>
    <td>0</td>
    <td>yes</td>
  </tr>
  <tr>
    <td>MultiATIS (Xu et al., 2020)</td>
    <td>11</td>
    <td>26</td>
    <td>140</td>
    <td>44,943</td>
    <td>9</td>
    <td>0</td>
    <td>no</td>
  </tr>
  <tr>
    <td>sSID (van der Goot et al., 2021)</td>
    <td>7</td>
    <td>16</td>
    <td>33</td>
    <td>10,000</td>
    <td>13</td>
    <td>0</td>
    <td>no</td>
  </tr>
  <tr>
    <td>MTOP (Li et al., 2021)</td>
    <td>11</td>
    <td>117</td>
    <td>78</td>
    <td>100,000</td>
    <td>6</td>
    <td>0</td>
    <td>no</td>
  </tr>
  <tr>
    <td>MTOP++ (Ruder et al., 2023)</td>
    <td>11</td>
    <td>117</td>
    <td>78</td>
    <td>144,243</td>
    <td>20</td>
    <td>5 (amh, hau, yor, swa, zul)</td>
    <td>no</td>
  </tr>
  <tr>
    <td>MASSIVE (FitzGerald et al., 2023)</td>
    <td>18</td>
    <td>60</td>
    <td>55</td>
    <td>995,571</td>
    <td>51</td>
    <td>3 (afr, amh, swa)</td>
    <td>partial</td>
  </tr>
  <tr>
    <td>INJONGO (Ours)</td>
    <td>5</td>
    <td>40</td>
    <td>23</td>
    <td>52,979</td>
    <td>17</td>
    <td>16</td>
    <td>yes</td>
  </tr>
</table>

Table 1: Overview of important related works that intent detection and slot-filling tasks. We included the number of domains, intents, slots, languages, African languages and how multicultural are the utterances.

![A table showing the overview of important related works that intent detection and slot-filling tasks.](page_120_120_1208_246.png)

Figure 1: Task description for INJONGO dataset. An example from one of the five domains. It shows the semantic-similar sentences along with intent and slot-filling labels.

<table>
  <tr>
    <th>Language</th>
    <th>Code</th>
    <th>Language Family</th>
    <th>No. of Speakers</th>
  </tr>
  <tr>
    <td>Amharic</td>
    <td>amh</td>
    <td>Afro-Asiatic/Semitic</td>
    <td>60M</td>
  </tr>
  <tr>
    <td>Ewe</td>
    <td>ewe</td>
    <td>Niger-Congo/Kwa</td>
    <td>7M</td>
  </tr>
  <tr>
    <td>Hausa</td>
    <td>hau</td>
    <td>Afro-Asiatic/Chadic</td>
    <td>63M</td>
  </tr>
  <tr>
    <td>Igbo</td>
    <td>ibo</td>
    <td>Niger-Congo/Volta-Niger</td>
    <td>27M</td>
  </tr>
  <tr>
    <td>Kinyarwanda</td>
    <td>kin</td>
    <td>Niger-Congo/Bantu</td>
    <td>10M</td>
  </tr>
  <tr>
    <td>Lingala</td>
    <td>lin</td>
    <td>Niger-Congo/Bantu</td>
    <td>41M</td>
  </tr>
  <tr>
    <td>Luganda</td>
    <td>lug</td>
    <td>Niger-Congo/Bantu</td>
    <td>7M</td>
  </tr>
  <tr>
    <td>Oromo</td>
    <td>orm</td>
    <td>Afro-Asiatic/Cushitic</td>
    <td>46M</td>
  </tr>
  <tr>
    <td>Shona</td>
    <td>sna</td>
    <td>Niger-Congo/Bantu</td>
    <td>12M</td>
  </tr>
  <tr>
    <td>Sesotho</td>
    <td>sot</td>
    <td>Niger-Congo/Bantu</td>
    <td>7M</td>
  </tr>
  <tr>
    <td>Swahili</td>
    <td>swa</td>
    <td>Niger-Congo/Bantu</td>
    <td>98M</td>
  </tr>
  <tr>
    <td>Twi</td>
    <td>twi</td>
    <td>Niger-Congo/Kwa</td>
    <td>9M</td>
  </tr>
  <tr>
    <td>Wolof</td>
    <td>wol</td>
    <td>Niger-Congo/Senegambia</td>
    <td>5M</td>
  </tr>
  <tr>
    <td>Xhosa</td>
    <td>xho</td>
    <td>Niger-Congo/Bantu</td>
    <td>9M</td>
  </tr>
  <tr>
    <td>Yoruba</td>
    <td>yor</td>
    <td>Niger-Congo/Volta-Niger</td>
    <td>42M</td>
  </tr>
  <tr>
    <td>Zulu</td>
    <td>zul</td>
    <td>Niger-Congo/Bantu</td>
    <td>27M</td>
  </tr>
</table>

Table 2: Overview of languages in the INJONGO dataset, including ISO 639-3 language codes, language families, and approximate number of speakers.

making the dataset particularly valuable for over 400 million African population. Table 2 shows the languages covered, their language family, and the number of speakers of the languages.

3.1 Data source and collection

Typical ID-SF data collection often requires large crowd-sourcing efforts to collect utterances, with additional labeling of intents and slots in various domains. Developing such a large crowd-sourcing effort is time-consuming and costly for several low-resource languages. To simplify the process while making the dataset cultural, we provide each annotator with sample sentences from the CLINC dataset (Larson et al., 2019a) with a specified intent type, say “transfer”. Then, the dataset construction follows two stages: (1) Utterance elicitation in an African language and (2) Slot-filling annotation of the generated utterance.

Figure 1 shows an example of an English utterance from the CLINC dataset in the banking domain: “please send ten dollars from bank of america to capital one”. The corresponding intent label is “transfer”, and the entities of slot filling are the amount of [money] (ten dollars), the source [bank] (bank of america), and the destination [bank] (capital one). A Xhosa annotator was asked to generate another utterance belonging to the same intent type but capturing the South African context where the language is spoken. Thus, the annotator used the R200 as “money” with currency Rand (R), and more familiar South African banks such as “FNB” and “Absa” for “bank name” slot. We provide more information about the two stages of data construction below.

Utterance generation The source data for our multilingual benchmark is from the CLINC English dataset—an intent detection with 150 intent classes across 10 domains (but without slot annotation) 5, we extracted 40 intents from five most suitable domains to the African contexts: Banking (e.g. “transfer”, “pay bill”), Home (e.g. “play music”, “calendar update”), Kitchen and Dining (e.g. “recipe”, “confirm reservation”), Travel (e.g. “exchange rate”, “book flight”), and Utility (e.g.

5The domains are: banking, work, meta, auto & commute, travel, home, utility, kitchen & dining, small talk, and credit cards
<table>
  <tr>
    <th>Lang.</th>
    <th>Total</th>
    <th>Avg.</th>
    <th>Un. Fleiss’\( \kappa \)</th>
    <th>Fleiss’\( \kappa \)</th>
    <th>Δ</th>
  </tr>
  <tr>
    <td>anh</td>
    <td>10555</td>
    <td>3.30</td>
    <td>0.850</td>
    <td>0.935</td>
    <td>+0.085</td>
  </tr>
  <tr>
    <td>ewe</td>
    <td>11181</td>
    <td>3.49</td>
    <td>0.875</td>
    <td>1.000</td>
    <td>+0.125</td>
  </tr>
  <tr>
    <td>hau</td>
    <td>11491</td>
    <td>3.59</td>
    <td>0.892</td>
    <td>0.997</td>
    <td>+0.105</td>
  </tr>
  <tr>
    <td>ibo</td>
    <td>12246</td>
    <td>3.82</td>
    <td>0.812</td>
    <td>0.973</td>
    <td>+0.161</td>
  </tr>
  <tr>
    <td>kin</td>
    <td>10112</td>
    <td>3.16</td>
    <td>0.740</td>
    <td>0.963</td>
    <td>+0.224</td>
  </tr>
  <tr>
    <td>lin</td>
    <td>11025</td>
    <td>3.44</td>
    <td>0.823</td>
    <td>0.990</td>
    <td>+0.168</td>
  </tr>
  <tr>
    <td>lug</td>
    <td>11769</td>
    <td>3.67</td>
    <td>0.888</td>
    <td>0.990</td>
    <td>+0.102</td>
  </tr>
  <tr>
    <td>orm</td>
    <td>11958</td>
    <td>3.74</td>
    <td>0.849</td>
    <td>0.992</td>
    <td>+0.143</td>
  </tr>
  <tr>
    <td>sna</td>
    <td>15222</td>
    <td>4.76</td>
    <td>0.935</td>
    <td>0.976</td>
    <td>+0.041</td>
  </tr>
  <tr>
    <td>sot</td>
    <td>6468</td>
    <td>2.02</td>
    <td>0.694</td>
    <td>0.997</td>
    <td>+0.303</td>
  </tr>
  <tr>
    <td>swa</td>
    <td>14217</td>
    <td>4.44</td>
    <td>0.878</td>
    <td>0.986</td>
    <td>+0.107</td>
  </tr>
  <tr>
    <td>twi</td>
    <td>14325</td>
    <td>4.48</td>
    <td>0.916</td>
    <td>0.986</td>
    <td>+0.070</td>
  </tr>
  <tr>
    <td>wol</td>
    <td>10942</td>
    <td>3.42</td>
    <td>0.728</td>
    <td>0.942</td>
    <td>+0.213</td>
  </tr>
  <tr>
    <td>xho</td>
    <td>12475</td>
    <td>3.90</td>
    <td>0.825</td>
    <td>0.938</td>
    <td>+0.113</td>
  </tr>
  <tr>
    <td>yor</td>
    <td>13620</td>
    <td>4.26</td>
    <td>0.862</td>
    <td>0.988</td>
    <td>+0.126</td>
  </tr>
  <tr>
    <td>zul</td>
    <td>11911</td>
    <td>3.73</td>
    <td>0.640</td>
    <td>0.913</td>
    <td>+0.273</td>
  </tr>
</table>

Table 3: Statistics of slot entity annotations across languages. For each language, we show the total number of annotated entities, average entities per sentence, and inter-annotator agreement measured by Fleiss’ Kappa (\( \kappa \)) before (Un.) and after review. Δ shows the improvement in agreement after the review process.

“alarm”, “make call” ). Next, we conducted the tutorial on the utterance generation task and a practice session and asked every annotator to generate a sample English utterance per intent that culturally aligns with the African contexts (e.g. food type or language name). Per language, we recruited three annotators, and they generated 120 utterances (40 per annotator and intent). We aggregated the practice data as the INJONGO English dataset. Finally, for the full data collection, we asked the same three annotators to generate 80 utterances per intent, given a sample sentence from CLiC . Each annotator worked on different intents. In total, we collected 3,200 utterances with a balanced number of intent types. Appendix A.1 contains all the 40 intent types selected.

Slot-filling annotation Similar to the utterance generation phase, we first conducted a practice session in English to train annotators followed by the full data annotation. We manually analyzed each generated utterance to come up with the most relevant slot entities (about 26). However, after the practice session, annotators recommended the addition of new slots such as “airline”, “airport name”, “car type”, and “supermarket name”, which we adopted. After the practice session, we gave detailed feedback on the issues with the annotation, and annotators discussed with their language coordinator how to resolve issues. Finally, we asked them to annotate the slot entities for the 3,200 utterances. Each utterance was annotated by three annotators so that we could check for agreement in the slot annotations. The annotation followed the named entity recognition setup on LabelStudio platform 6. Appendix A.2 contains all 34 slot types selected.

For both utterance elicitation and slot-filling annotation, all recruited participants received an appropriate remuneration based on the per-country rate decided by our logistics company in Kenya.7

3.2 Quality Control for Slot-filling

To ensure annotation quality and consistency, we follow a rigorous quality control process using a majority voting system with a minimum of three annotators per sentence to resolve disagreements. The annotation quality was evaluated using Fleiss’ Kappa score (Fleiss, 1971), with scores presented in Table 3 comparing agreement levels before and after the review process. Initial Fleiss’ Kappa scores revealed substantial variation across languages, ranging from 0.618 (Zulu) to 0.934 (Shona), indicating significant inter-annotator disagreement. Following the review process, agreement scores improved markedly across all languages, reaching 0.912-1.00. Notable improvements were observed in Sesotho (+0.327) and Zulu (+0.294), with other languages showing average improvements of approximately 0.1 in their Fless’ Kappa scores.

3.3 Slot-filling label merging

On completion of the final annotation, we found that some slot entities are rarely used. We performed an analysis of entity frequency distribution across all languages. Figure 2 shows the result of our analysis, we decided to exclude slot entities appearing less than 500 times across all languages (after MUSIC GENRE in the figure). Consequently, nine infrequent slots from NATIONALITY through PLUG TYPE were eliminated. Examination of annotator feedback and comparative analysis between unreviewed and reviewed versions indicated that ambiguous slot types significantly impacted annotation quality and introduced unnecessary complexity. To enhance annotation clarity and maintain consistency, the following merging strategy was

6https://labelstud.io/
7Utterance elicitation rate ranges from $1,555 to $2,838 in USD per language depending on country rate, and slot-filling annotation ranges from $388 to $709 in USD.
Figure 2: The distribution of slot entities appearances of all 16 African languages with Unreviewed and Reviewed versions. The slot entities are sorted from left to right by frequency in descending order.

<table>
  <tr>
    <th>split</th>
    <th colspan="2">INJONGO</th>
    <th colspan="2">CLINC</th>
  </tr>
  <tr>
    <th></th>
    <th>African</th>
    <th>English</th>
    <th>English</th>
    <th>English</th>
  </tr>
  <tr>
    <td>TRAIN</td>
    <td>2,240 (56 per intent)</td>
    <td>1,047</td>
    <td>4,000 (100 per intent)</td>
    <td></td>
  </tr>
  <tr>
    <td>DEV</td>
    <td>320 (8 per intent)</td>
    <td>110</td>
    <td>800 (20 per intent)</td>
    <td></td>
  </tr>
  <tr>
    <td>TEST</td>
    <td>640 (16 per intent)</td>
    <td>622</td>
    <td>1,200 (30 per intent)</td>
    <td></td>
  </tr>
</table>

Table 4: INJONGO dataset split. The African data have an equal number of samples per intent while the English samples per intent vary.

implemented:

• Geographic entities: STATE OR PROVINCE and CITY NAME were consolidated into a unified CITY OR PROVINCE category to ensure consistent handling of geographic references.
• Food-Related Labels: DISH NAME and FOOD ITEM were unified under DISH OR FOOD to eliminate classification ambiguity.

This merging process resulted in a reduction from 34 to 23 slot types. The complete enumeration of original and consolidated labels, along with unmerged entity Fleiss’ kappa scores, is provided in Appendix A.3.

3.4 Data split

Our final annotation resulted in 3,200 annotated utterances, with 80 utterances per intent for each of the 16 African languages. The dataset is partitioned following ratios of 70%, 10%, and 20% for train, dev, and test splits respectively, stratified by intent for each language. Additionally, we aggregated the practice utterances generated and the practice slot annotations as the English dataset, leading to 17 annotated languages. In total, the English consist of 1779 utterances.8 Finally, we sampled 4000 CLINC intent-only dataset to compare western-centric English dataset to our curated INJONGO dataset that captures the African contexts. Table 4 provides the comprehensive dataset statistics of the African languages and English splits.

4 Experiments Setup

4.1 Fine-tuning Multilingual Models

We evaluate three categories of models: (1) encoder-only models such as XLM-RoBERTa Large (Conneau et al., 2019), AfroXLMR (Alabi et al., 2022), AfroXLMR-76L (Adelani et al., 2023a), AfriBERTa V2 (Oladipo, 2024), (2) encoder-decoder models such as mT5-Large (Xue et al., 2020), AfriTeVa V2 Large (Oladipo et al., 2023), and (3) a multilingual variant of LLM2Vec model (BehnamGhader et al., 2024) i.e. NLLB-LLM2Vec (Schmidt et al., 2024) that stack NLLB-encoder (NLLB Team et al., 2022) model with LLaMa 3.1 8B (Grattafiori et al., 2024) to develop a multilingual sentence transformer model. These models are fine-tuned using the AdamW optimizer for 20 epochs with early stopping. All results are averaged over five seeds. Learning rates are calibrated for each architecture and task as detailed in Appendix B.2. The languages covered in each pre-trained model are available in Appendix B.3.

4.2 LLM Prompting

First, we conduct zero-shot prompting using the following widely used LLMs for evaluation: GPT-4o,9 Gemini 1.5 Pro (Reid et al., 2024), Gemma 2 9B/27B Instruct (Team et al., 2024), Llama 3.1 8B/3.3 70B Instruct (Grattafiori et al., 2024), and Aya-101 (Üstün et al., 2024). We make use of five different prompts for each LLM. Second,

8Ideally, if each language completes 120 utterance generation, we ought to have 1920 utterances, however, some languages only did 80 in the practice, leading to a slightly lower English portion.

9https://platform.openai.com/docs/models#gpt-4o
<table>
  <tr>
    <th>Task</th>
    <th>Model</th>
    <th colspan="16">eng amh ewe hau ibo kin lin lug orm sna sot swa twi wol xho yor zul AVG</th>
  </tr>
  <tr>
    <td rowspan="2">Intent Detection</td>
    <td>In-language training<br>mT5-Large<br>AfriTeVa V2 (TS)</td>
    <td>80.5</td><td>91.5</td><td>77.3</td><td>94.6</td><td>92.9</td><td>83.7</td><td>91.3</td><td>83.3</td><td>73.3</td><td>92.6</td><td>80.2</td><td>95.8</td><td>85.3</td><td>91.6</td><td>95.8</td><td>90.9</td><td>82.4</td><td>87.7<sub>±1.1</sub></td>
  </tr>
  <tr>
    <td>NLLB LLm2Vec<br>XLM-RoBERTa<br>AfriBERTA V2<br>AfroXLMR<br>AfroXLMR-76L</td>
    <td>81.6</td><td>93.2</td><td>84.4</td><td>98.9</td><td>95.7</td><td>87.8</td><td>91.6</td><td>86.8</td><td>86.6</td><td>94.6</td><td>85.7</td><td>96.8</td><td>87.1</td><td>94.0</td><td>97.3</td><td>97.0</td><td>89.2</td><td>91.7<sub>±2.7</sub></td>
  </tr>
  <tr>
    <td rowspan="2">Slot Filling</td>
    <td>In-language training<br>mT5-Large<br>AfriTeVa V2 (TS)</td>
    <td>73.7</td><td>80.9</td><td>71.6</td><td>89.4</td><td>80.5</td><td>74.2</td><td>82.6</td><td>78.9</td><td>72.1</td><td>81.1</td><td>74.7</td><td>88.1</td><td>79.0</td><td>76.9</td><td>88.4</td><td>78.9</td><td>68.3</td><td>79.1<sub>±1.7</sub></td>
  </tr>
  <tr>
    <td>NLLB LLm2Vec<br>XLM-RoBERTa<br>AfriBERTA V2<br>AfroXLMR<br>AfroXLMR-76L</td>
    <td>73.6</td><td>80.9</td><td>74.5</td><td>93.8</td><td>79.6</td><td>76.6</td><td>87.1</td><td>85.2</td><td>79.0</td><td>82.1</td><td>77.5</td><td>88.9</td><td>84.0</td><td>79.0</td><td>90.0</td><td>87.2</td><td>71.2</td><td>82.3<sub>±3.7</sub></td>
  </tr>
  <tr>
    <td colspan="2">Multi-lingual training<br>AfroXLMR-76L</td>
    <td>89.0</td><td>96.0</td><td>92.6</td><td>99.2</td><td>96.6</td><td>87.7</td><td>95.9</td><td>92.3</td><td>92.9</td><td>96.5</td><td>87.6</td><td>97.8</td><td>94.2</td><td>97.1</td><td>97.3</td><td>97.9</td><td>89.2</td><td>94.4<sub>±2.0</sub></td>
  </tr>
  <tr>
    <td colspan="2">Multi-lingual training<br>AfroXLMR-76L</td>
    <td>82.4</td><td>88.2</td><td>87.0</td><td>96.3</td><td>84.0</td><td>79.3</td><td>90.3</td><td>89.2</td><td>87.2</td><td>86.1</td><td>80.4</td><td>90.5</td><td>90.3</td><td>83.3</td><td>91.8</td><td>90.2</td><td>83.3</td><td>87.3<sub>±2.4</sub></td>
  </tr>
</table>

Table 5: Intent detection and slot-filling results for supervised fine-tuned Small LMs on INJONGO. Models are ranked by accuracy for intent detection and F1-score for slot-filling. The average performance and standard deviation across 16 African languages are reported. The best models are highlighted in Gray and Cyan colours.

we perform few-shot evaluation using the best-performing zero-shot template for each task (see Appendix C). We employ two few-shot strategies (1) 5-examples: prompting with any 5 samples from different domains (see Figure 1) i.e. one intent type is covered by domain (2) One-shot intent-type prompting i.e. one sample per intent type or 40 samples from different intent types. We used the same samples for both tasks. Finally, we extend to 4 shots —acceptable maximum context length (CL) for Gemma 2, Aya-101 was excluded for small CL.

Finally, as an additional strong baseline for LLMs, we performed supervised fine-tuning (SFT) on Gemma 2 9B for 5 epochs using learning rates of \(2 \times 10^{-5}/2.5 \times 10^{-5}\) for intent detection and slot filling. The dataset of SFT was obtained by aggregating all the training samples of the 17 languages in INJONGO i.e. “Combined INJONGO”, with randomly sampled prompts from a pool of 5. The evaluations of LLMs use 5 different prompting templates and a temperature of 0.5. We provide all the prompts used in Appendix C.

4.3 Cross-lingual Transfer Analysis

To investigate how well our dataset facilitates cross-lingual learning and transfer capabilities across languages, we tested two settings (1) zero-shot transfer from the English split of INJONGO, and evaluated on African languages. (2) Translate-Test where we evaluate on the machine-translated sentence test sets from an African language to English. We leveraged the NLLB-200-3.3B (NLLB Team et al., 2022) machine translation model for the Translate-test setting. We compare the results with LLM prompting.

Hyper-parameters and Prompts used Experiments of the baselines and cross-lingual transfer runs make use of five fixed random seeds. Detailed experiments setup, training configuration and prompts are in Appendix B.

5 Results

5.1 Fine-tuned Multilingual Encoders

Table 5 summarizes the results of the multilingual encoders fine-tuned INJONGO dataset. Overall, AfroXLMR-76L achieves the best performance on both ID-SF tasks, with an average accuracy of 93.7% and an F1 score of 85.6%, respectively. We attribute the success of this model to the coverage of all languages in INJONGO during its pre-training (see Appendix Table 11). AfroXLMR, the earlier version of AfroXLMR-76L, follows closely with an average accuracy of 92.2% and an F1 score of 85.2%. However, this model was not pre-trained on some of the languages such as ewe, twi, lin, and wol leading to a significant drop in performance of −5.8, −4.8, −1.3, −0.4 for intent detection when compared to AfroXLMR-76L. This shows that multilingual encoders for African languages can significantly improve the performance over massively multilingual encoders covering more than 100 languages such as XLM-R and NLLB LLm2Vec. While NLLB LLm2Vec covers all languages in
<table>
  <tr>
    <th>Task</th>
    <th>Model</th>
    <th colspan="14">eng anth ewu hau ibo kin lin lug orm ana sot swa twi wol xho yor zul AVG</th>
  </tr>
  <tr>
    <td rowspan="6">INTENT DETECTION</td>
    <td>Llama 3.1 8B</td>
    <td>27.6</td><td>1.9</td><td>2.1</td><td>4.8</td><td>5.5</td><td>3.3</td><td>5.3</td><td>2.4</td><td>1.6</td><td>2.8</td><td>2.9</td><td>14.1</td><td>2.6</td><td>4.0</td><td>3.2</td><td>3.5</td><td>2.8</td><td>3.9<sup>±2.4</sup></td>
  </tr>
  <tr>
    <td>Gemma 2 9B</td>
    <td>77.6</td><td>49.2</td><td>6.1</td><td>40.8</td><td>31.5</td><td>23.8</td><td>22.2</td><td>23.2</td><td>7.7</td><td>29.7</td><td>19.9</td><td>70.0</td><td>21.0</td><td>13.8</td><td>40.1</td><td>32.2</td><td>36.3</td><td>29.2<sup>±8.7</sup></td>
  </tr>
  <tr>
    <td>Aya-101 13B</td>
    <td>65.3</td><td>62.9</td><td>13.4</td><td>57.8</td><td>56.9</td><td>40.4</td><td>27.8</td><td>33.9</td><td>20.8</td><td>51.2</td><td>43.9</td><td>65.9</td><td>27.2</td><td>19.7</td><td>58.1</td><td>45.9</td><td>53.2</td><td>42.4<sup>±9.1</sup></td>
  </tr>
  <tr>
    <td>Gemma 2 27B</td>
    <td>79.5</td><td>74.2</td><td>6.3</td><td>46.5</td><td>36.9</td><td>26.7</td><td>27.5</td><td>26.1</td><td>5.8</td><td>36.7</td><td>25.6</td><td>75.5</td><td>21.2</td><td>14.6</td><td>50.2</td><td>34.8</td><td>44.3</td><td>33.0<sup>±9.6</sup></td>
  </tr>
  <tr>
    <td>Llama 3.3 70B</td>
    <td>81.1</td><td>56.2</td><td>9.5</td><td>52.3</td><td>52.4</td><td>35.0</td><td>37.5</td><td>37.7</td><td>12.4</td><td>32.3</td><td>30.5</td><td>80.6</td><td>29.3</td><td>20.9</td><td>43.5</td><td>41.4</td><td>43.9</td><td>38.5<sup>±9.5</sup></td>
  </tr>
  <tr>
    <td>Gemini 1.5 Pro</td>
    <td><b>81.8</b></td><td><b>77.9</b></td><td><b>24.3</b></td><td><b>74.8</b></td><td><b>65.4</b></td><td><b>61.5</b></td><td><b>54.6</b></td><td><b>59.3</b></td><td><b>39.3</b></td><td><b>68.6</b></td><td><b>51.6</b></td><td><b>83.2</b></td><td><b>47.2</b></td><td><b>25.6</b></td><td><b>76.2</b></td><td><b>66.8</b></td><td><b>68.7</b></td><td><b>59.1<sup>±9.6</sup></b></td>
  </tr>
  <tr>
    <td>GPT-4o (Aug)</td>
    <td>80.9</td><td>76.0</td><td>15.1</td><td>80.7</td><td>71.8</td><td>64.7</td><td><b>56.4</b></td><td><b>68.2</b></td><td><b>59.3</b></td><td><b>75.5</b></td><td><b>59.7</b></td><td><b>84.5</b></td><td><b>58.6</b></td><b>43.7</b><td><b>79.6</b></td><td><b>77.0</b></td><td><b>71.2</b></td><td><b>65.1<sup>±9.3</sup></b></td>
  </tr>
  <tr>
    <td rowspan="6">SLOT FILLING</td>
    <td>Llama 3.1 8B</td>
    <td>25.0</td><td>3.7</td><td>5.6</td><td>11.1</td><td>12.6</td><td>8.5</td><td>9.1</td><td>10.1</td><td>2.8</td><td>9.9</td><td>11.5</td><td>17.3</td><td>11.2</td><td>9.2</td><td>2.6</td><td>11.0</td><td>9.0</td><td>9.1<sup>±2.2</sup></td>
  </tr>
  <tr>
    <td>Gemma 2 IT 9B</td>
    <td>34.1</td><td>4.5</td><td>0.3</td><td>7.4</td><td>10.6</td><td>5.0</td><td>6.0</td><td>5.6</td><td>0.1</td><td>7.3</td><td>10.8</td><td>21.2</td><td>2.4</td><td>2.6</td><td>2.2</td><td>5.2</td><td>8.2</td><td>6.2<sup>±2.9</sup></td>
  </tr>
  <tr>
    <td>Aya-101 13B</td>
    <td>21.4</td><td>8.2</td><td>7.9</td><td>11.8</td><td>14.6</td><td>12.2</td><td>9.4</td><td>15.5</td><td>3.6</td><td>15.0</td><td>17.0</td><td>16.2</td><td>13.8</td><td>14.0</td><td>2.8</td><td>9.6</td><td>10.6</td><td>11.4<sup>±2.4</sup></td>
  </tr>
  <tr>
    <td>Gemma 2 IT 27B</td>
    <td>49.8</td><td>15.7</td><td>9.5</td><td>24.1</td><td>25.2</td><td>21.7</td><td>15.2</td><td>28.4</td><td>2.6</td><td>29.8</td><td>28.0</td><td>40.2</td><td>24.3</td><td>23.3</td><td>4.5</td><td>28.1</td><td>31.0</td><td>22.0<sup>±5.5</sup></td>
  </tr>
  <tr>
    <td>Llama 3.3 70B Instruct</td>
    <td>52.6</td><td><b>26.3</b></td><td><b>22.0</b></td><td>29.5</td><td>35.0</td><td>31.4</td><td>25.0</td><td>30.4</td><td>9.3</td><td>29.3</td><td>36.4</td><td>40.7</td><td>35.6</td><td>36.4</td><td>6.9</td><td>34.2</td><td>31.9</td><td>28.8<sup>±5.2</sup></td>
  </tr>
  <tr>
    <td>Gemini 1.5 Pro</td>
    <td>52.8</td><td>15.2</td><td>18.7</td><td>31.9</td><td>35.8</td><td>34.4</td><td><b>34.9</b></td><td>34.4</td><td>12.2</td><td>36.8</td><td><b>43.0</b></td><td>37.5</td><td>34.5</td><td>34.2</td><td>6.9</td><td>33.2</td><td><b>38.6</b></td><td>30.1<sup>±6.1</sup></td>
  </tr>
  <tr>
    <td>GPT-4o (Aug)</td>
    <td><b>55.4</b></td><td>22.8</td><td>19.4</td><td>37.8</td><td><b>38.9</b></td><td><b>36.4</b></td><td>33.5</td><td><b>35.3</b></td><td>13.0</td><td>40.2</td><td>40.9</td><td><b>46.5</b></td><td><b>40.1</b></td><td><b>37.9</b></td><td><b>10.0</b></td><td><b>42.4</b></td><td><b>37.6</b></td><td><b>33.3<sup>±6.0</sup></b></td>
  </tr>
</table>

Table 6: Zero-Shot performance of LLMs on Intent Detection (ID) and Slot Filling (SF). Evaluation is based on accuracy and F1-score for ID and SF tasks. Average computed on five templates, and on only African languages.

our dataset and is very effective for intent detection, it leads to −3.5 on slot-filling when compared to the performance of AfroXLMR-76L. In general, T5-based models such as mT5 and AfriTeVa V2 performed worse on both tasks compared to the BERT-based models, however, we still observe better performance of the African-centric T5-model, AfriTeVa V2 which gave decent results comparable to other models except AfroXLMR (-76L) models.

Finally, we find that multilingual training of AfroXLMR-76L over all languages gave better overall performance than in-language training leading to +0.7 and +1.7 boost in performance on intent detection and slot-filling tasks respectively. This highlights the additional benefit of joint training of several languages, resulting in a single checkpoint and better overall performance because they benefited from cross-lingual transfer learning among the languages. The languages that benefited the most are Oromo (orm) and English (eng) with +4.6 and +4.5 improvement respectively for intent detection. The large boost for English is because the training data is twice smaller than the remaining African languages (1,047 vs. 2,240). Similarly, for slot-filling, the benefit of multilingual training is more obvious since all languages consistently improved in performance. We see that joint training benefited both high-resourced and low-resourced languages.

5.2 LLMs Prompting Results

Table 6 shows the zero-shot LLM evaluation of five open models and two closed models. Our key findings are below:

Slot-filling task is difficult for all LLMs including on English The highest average performance achieved by the LLMs is 33.3% for GPT-4o, although much better than the open model at 28.8. We attribute this to the difficulty of LLMs on the named entity recognition task as reported by other researchers (Yu et al., 2023; Ojo et al., 2023). In comparison to the best-finetuned model, there is a large drop in performance of −53.2. This shows that having training data is still relevant for this task even in the LLM era, especially for low-resource languages.

Large gap in the performance of closed and open models For intent detection, we find that all open models achieved below 50% on the relatively easy task of intent detection. The poor performance may be attributed to either a lack of exposure to many African languages or the large label space (i.e. 40) for the classification task. The closed models result are better, with GPT-4o and Gemini 1.5 Pro achieving more than +20 points than the best open model, Llama 3.3 70B. However, if we compare the results in the English language, open models such as Gemma 2 27B and Llama 3.3 70B are competitive with closed models. This shows that open models are more biased toward high-resource languages. This implies that there is a continuous need to keep improving the capabilities of models for low-resource languages.

Intent detection performance varies by languages The performance of some African languages is often higher than others, this is probably connected to the amount of monolingual data available on the web. For example, Swahili (swa) with over 1 billion monolingual data (Kudugunta et al., 2023) has 80.6 accuracy point that is comparable performance to English performance (81.1) with Llama 3.3 70B, while other languages have much lower performance. Similarly, GPT-4o has more
Figure 3: Performance of cross-lingual transfer across different shot settings and supervised fine-tuning (SFT) on the merged 17 languages INJONGO dataset.

than 70 accuracy points for Amharic, Hausa, Igbo, Shona, Swahili, Xhosa, Yorùbá, and Zulu. These languages also have larger monolingual data on the web than the ones with lower than 70% accuracy.

5.3 Few-shot Performance

Figure 3 shows the result of the various few-shot setups: 5-examples, 1-shot (40 examples, one from each intent type), and 4-shots (160 examples). Our result shows a big boost in performance with only 5-examples, especially for the slot-filling task and some LLMs: GPT-4o and Gemini 1.5 Pro improved the most by more than +19 points. Similarly, Gemma 2 9B improved from 2.4 to 33.5 matching the performance of Llama 3.3 70B (with 5-examples). Additional samples from 1-shot and 4-shot consistently improved performance for all models except Llama 3.3 70B. Similarly, for intent detection, there is consistent improvement in performance with more examples used for few-shot evaluations. We find Gemini 1.5 Pro, Gemma 2 9B and Gemma 2 27B to benefit the most from 5-examples, with an accuracy boost of +14.3, +15.7, and +21.8 respectively. Interestingly, while the zero-shot performance of Gemini 1.5 Pro is worse than GPT-4o, the few-shot performance exceeds that of GPT-4o with +1.8 and +2.3 improvement in 5-examples and 1-shot. Our result shows the effectiveness of LLMs in adapting quickly to a new task in low-resource settings. We provide the results of individual languages in Appendix B.7.

Would Few-shot performance match Supervised fine-tuning (SFT)? While all LLMs improve performance with more shots, there is still a large gap with SFT. We performed SFT on Gemma 2 9B with all training samples and prompt templates, we found a large performance gap of +15.8 and +34.1 for intent detection and slot-filling respectively if we compare SFT (52k samples) to 4-shots (160 examples). However, for closed models, the gap of SFT on Gemma 2 9B to Gemini 1.5 Pro and GPT-4o is much smaller, especially for intent detection. In general, few-shots of LLMs are still worse than SFT but are very crucial and effective when training data are scarce.

Figure 4: Cross-lingual transfer results from CLINC and INJONGO English data on intent detection task with AfroXLMR-76L.

5.4 Cross-lingual Transfer Results

Figure 4 shows our final experiments that compare cross-lingual transfer results from two English datasets: CLINC (Western-centric) and INJONGO (African-centric) on the intent detection task. At 5-shots, in both in-language and translate-test settings, the accuracy of all settings is quite similar, however as we increase the number of instances to 10-shots (400 examples), we find that the INJONGO in-language performance was better than the CLINC (16.1 vs. 4.0) that is more Western-centric. Similarly, in translate-test setting, the gain in performance is much larger (+29), which implies that in a low-resource setting, leveraging a multicultural dataset with the African context is effective. However, with more samples (25-shots), there is no significant difference in whether the samples are Western-centric or not, and training data size seems to be more important.
6 Conclusion

We present INJONGO, a new benchmark dataset for evaluating intent detection and slot-filling for 16 African languages. INJONGO represents the first large-scale multicultural dataset focused on African language Conversation AI. Our experiments reveal that while fine-tuned multilingual models such as AfroXLMR-76L achieved strong performance LLMs still struggle with African languages, particularly in slot filling tasks. We hope INJONGO will accelerate the development of more effective and culturally-aware conversational AI systems for African languages.

Acknowledgment

We gratefully acknowledge the support of the Lacuna Fund, a collaborative initiative co-founded by The Rockefeller Foundation, Google.org, and Canada’s International Development Research Centre (IDRC). We also extend our appreciation to IVADO and the Canada First Research Excellence Fund for their generous funding support that made this research possible. Finally, we are very grateful to OpenAI for granting API credits through their Researcher Access API program, which enabled the evaluation of GPT-4o. We also appreciate Google’s provision of GCP credits via the Gemma 2 Academic Program, supporting the Gemini-1.5-Pro inference.

Limitations

The scope of INJONGO is constrained by its coverage of only 5 domains and 40 intents, missing some other domains such as healthcare and education that are essential for real-world applications. Our language selection, while substantial, still represents only a fraction of Africa’s linguistic diversity, particularly lacking representation from other language families such as Nilo-Saharan languages. The annotation process revealed inherent challenges in entity classification across languages, requiring two rounds of review to achieve consistent quality. Although significant for low-resource languages, the dataset size of 3,200 examples per language remains modest compared to high-resource benchmarks, potentially limiting model performance. Additionally, the fixed distribution of examples across intents may not accurately reflect the natural frequency of these interactions in real-world conversations.

References

David Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba Alabi, Shamsudeen Muhammad, Peter Nabende, Cheikh M. Bamba Dione, Andiswa Bukula, Roowither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukibiib, Godson Kalipe, Derguene Mbaye, Amelia Taylor, Fatoumata Kabore, Chris Chinenye Emezue, Anouluwapo Aremu, Perez Ogayo, Catherine Gitau, Edwin Munkoh-Buabeng, Victoire Memdjojkan Koagne, Allahserra Auguste Tapo, Tegbo Macucwa, Vukosi Marivate, Mboning Tchiazie Elvis, Tajudddeen Gwadabe, Tosin Adewumi, Oreagohene Ahia, and Joyce Nakatumba-Nabende. 2022. MasakhaNER 2.0: Africa-centric transfer learning for named entity recognition. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4488–4508, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’Souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsudeen H. Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Aremu Anouluwapo, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Rabiu Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukibiib, Verrah Otiente, Iroto Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwu Adeyemi, Gerald Muruki, Emmanuel Anebi, Chiamaka Chukwunweke, Nikiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Batesea, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayo-dele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Oreagohene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahim DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, and Salomey Osei. 2021. MasakhaNER: Named entity recognition for African languages. Transactions of the Association for Computational Linguistics, 9:1116–1131.

David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and Annie En-Shiu Lee. 2023a. Sib-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects. Preprint, arXiv:2309.07445.

David Ifeoluwa Adelani, Marek Masiak, Israel Abebe Azime, Jesujoba Alabi, Atnafu Lambebo Tonja, Christine Mwase, Odunayo Ogundepo, Bonaventure F. P. Dossou, Akintunde Oladipo, Doreen Nixdorf, Chris Chinenye Emezue, Sana Al-azzawi, Blessing Sibanda, Davis David, Lolwethu Ndolela, Jonathan
Mukiibi, Tunde Ajayi, Tatiana Motue, Brian Odhiambo, Abraham Owodunni, Nnaemeka Obiefuna, Muhidin Mohamed, Shamsuddeen Hassan Muhammad, Teshome Mulugeta Ababu, Saheed Abdullahi Salahudeen, Mesay Gameda Yigezu, Tajudeen Gwadabe, Idris Abdulmumin, Mahlet Taye, Oluwabusayo Awoyomi, Iyanuoluwa Shode, Tolulope Adelani, Habiba Abdulkaniyu, Abdul-Hakeem Omotayo, Adetola Adeeko, Abeeb Afolabi, Anuoluwapo Aremu, Olanrewaju Samuel, Clemencia Siro, Wangari Kimotho, Onyekachi Ogu, Chinedu Mbonu, Chiamaka Chukwuneke, Samuel Fanijo, Jessica Ojo, Oyinkansola Awosan, Tadesse Kebede, Toadoum Sari Sakayo, Pamela Nyatise, Freedmore Sidume, Oreon Yousuf, Mardiyah Oduwole, Kanda Tshinu, Ussen Kimanuka, Thina Diko, Siyanda Nxakama, Sinodos Nigusse, Abdulmejid Johar, Shafie Mohamed, Fuad Mire Hassan, Moges Ahmed Mehamed, Evrad Ngabire, Jules Jules, Ivan Ssenkungu, and Pontus Tenstorp. 2023b. MasakhaNEWS: News topic classification for African languages. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 144–159, Nusa Dua, Bali. Association for Computational Linguistics.

Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022. Adapting pre-trained language models to African languages via multilingual adaptive fine-tuning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4336–4349, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.

Jesujoba O. Alabi, Israel Abebe Azime, Miaoran Zhang, Cristina España-Bonet, Rachel Bawden, Dawei Zhu, David Ifeoluwa Adelani, Clement Oyeleke Odjoje, Idris Akinade, Ifiat Maab, Davis David, Shamsuddeen Hassan Muhammad, Neo Putini, David O. Ademuyiwa, Andrew Caines, and Dietrich Klakow. 2025. AfriDoc-mt: Document-level mt corpus for african languages.

Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2Vec: Large language models are secretly powerful text encoders. In First Conference on Language Modeling.

Yuri Bizzoni, Tom S Juzek, Cristina España-Bonet, Koel Dutta Chowdhury, Josef van Genabith, and Elke Teich. 2020. How human is machine translationese? comparing human and machine translations of text and speech. In Proceedings of the 17th International Conference on Spoken Language Translation, pages 280–290, Online. Association for Computational Linguistics.

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale.

Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, Maël Primet, and Joseph Dureau. 2018. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. ArXiv, abs/1805.10190.

Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishes Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuw, Gokhan Tur, and Prem Natarajan. 2023. MASSIVE: A 1M-example multilingual natural language understanding dataset with 51 typologically-diverse languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4277–4302, Toronto, Canada. Association for Computational Linguistics.

Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378.

Aaron Grattafiore, Abhimanyu Dubey, Abhinav Jauhari, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, and ... Goyal. 2024. The llama 3 herd of models.

Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018. Semantic parsing for task oriented dialog using hierarchical representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2787–2792, Brussels, Belgium. Association for Computational Linguistics.

Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990. The ATIS spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990.

Sneha Kudugunta, Isaac Rayburn Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. MADLAD-400: A multilingual and document-level large audited dataset. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
Stefan Larson and Kevin Leach. 2022. A survey of intent classification and slot-filling datasets for task-oriented dialog. ArXiv, abs/2207.13211.

Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019a. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1311–1316, Hong Kong, China. Association for Computational Linguistics.

Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019b. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

Haoran Li, Abhinav Arora, Shuhui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. 2021. MTOP: A comprehensive multilingual task-oriented semantic parsing benchmark. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2950–2962, Online. Association for Computational Linguistics.

Shamsuddeen Muhammad, Idris Abdulmumin, Abinew Ayele, Nedjma Ousidhoum, David Adelani, Seid Yimam, Ibrahim Ahmad, Meriem Beloucif, Saif Mohammad, Sebastian Ruder, Ouamima Hourrane, Alipio Jorge, Pavel Brazdil, Felermino Ali-Davis David, Salomey Osei, Bello Shehu-Bello, Falalu Lawan, Tajudeen Gwadabe, Samuel Rutunda, Tadesse Destaw Belay, Wendimu Messelle, Hailu Balcha, Sisay Chala, Hagos Gebremichael, Bernard Opoku, and Stephen Arthur. 2023. AfriSent: A Twitter sentiment analysis benchmark for African languages. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13968–13981, Singapore. Association for Computational Linguistics.

Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, David Ifeoluwa Adelani, Ibrahim Said Ahmad, Saminu Mohammad Aliyu, Nelson Odhiambo Onyango, Lilian D. A. Wanzare, Samuel Rutunda, Lukman Jibril Aliyu, Esubahale Alemneh, Ouamima Hourrane, Hagos Tesfahun Gebremichael, Elyas Abdi Ismail, Meriem Beloucif, Ebrahim Chekol Jibril, Andisius Bukula, Rooewither Mabuya, Salomey Osei, Abigail Oppong, Tadesse Destaw Belay, Tadesse Kebede Guge, Tesfa Tegegne Asfaw, Chiamaka Ijeoma Chukwuneke, Paul Rottger, Seid Muhie Yimam, and Nedjma Djouhra Ousidhoum. 2025. Afrihate: A multilingual collection of hate speech and abusive language datasets for african languages.

NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hefernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejía Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation.

Odunayo Ogundepo, Tajudeen Gwadabe, Clara Rivera, Jonathan Clark, Sebastian Ruder, David Adelani, Bonaventure Dossou, Abdou Diop, Claytone Sikatosse, Gilles Hacheme, Happy Buzaaab, Ignatius Ezeani, Rooewither Mabuya, Salomey Osei, Chris Emezue, Albert Kahira, Shamsuddeen Muhammad, Akintunde Oladipo, Abraham Owodunni, Atnafu Tonja, Iyanuoluwu Shode, Akari Asai, Anuoluwapo Aremu, Ayodele Awokoya, Bernard Opoku, Chiamaka Chukwuneke, Christine Mwase, Clemencia Siro, Stephen Arthur, Tunde Ajayi, Verrah Otiente, Andre Rubungo, Boyd Sinkala, Daniel Ajisafe, Emeka Onwuegbuza, Falalu Lawan, Ibrahim Ahmad, Jesujoba Alabi, Chinedu Mboun, Mofetolua Adeyemi, Mofya Phiri, Orevaoghene Ahia, Ruqayya Iro, and Sonia Adhiambo. 2023. Cross-lingual open-retrieval question answering for African languages. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14957–14972, Singapore. Association for Computational Linguistics.

Jessica Ojo, Kelechi Ogueji, Pontus Stenetorp, and David Ifeoluwa Adelani. 2023. How good are large language models on african languages? ArXiv, abs/2311.07978.

Akintunde Oladipo. 2024. Scaling pre-training data and language models for african languages.

Akintunde Oladipo, Mofetolua Adeyemi, Orevaoghene Ahia, Abraham Owodunni, Odunayo Ogundepo, David Adelani, and Jimmy Lin. 2023. Better quality pre-training data and 15 models for African languages. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 158–168, Singapore. Association for Computational Linguistics.

Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap, Jean-Baptiste Alayrac, Radu Soriciut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, and et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. ArXiv, abs/2403.05530.
Sebastian Ruder, Jonathan Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia, Shruti Rijhwani, Parker Riley, Jean-Michel Sarr, Xinyi Wang, John Wieting, Nitish Gupta, Anna Katanova, Christo Kirov, Dana Dickinson, Brian Roark, Bidisha Samanta, Connie Tao, David Adelani, Vera Axelrod, Isaac Caswell, Colin Cherry, Dan Garrette, Reeve Ingle, Melvin Johnson, Dmitry Panteleev, and Partha Talukdar. 2023. XTREME-UP: A user-centric scarce-data benchmark for under-represented languages. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1856–1884, Singapore. Association for Computational Linguistics.

Fabian David Schmidt, Philipp Borchert, Ivan Vulić, and Goran Glavaš. 2024. Self-distillation for model stacking unlocks cross-lingual nlu in 200+ languages.

Sebastian Schuster, Sonal Gupta, Rushin Shah, and Mike Lewis. 2019. Cross-lingual transfer learning for multilingual task oriented dialog. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3795–3805, Minneapolis, Minnesota. Association for Computational Linguistics.

Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118.

Rob van der Goot, Ibrahim Sharaf, Aizhan Imankulova, Ahmet Üstün, Marija Stepanović, Alan Ramponi, Siti Oryza Khairunnisa, Mamoru Komachi, and Barbara Plank. 2021. From masked language modeling to translation: Non-English auxiliary tasks improve zero-shot spoken language understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2479–2497, Online. Association for Computational Linguistics.

Eva Vanmassenhove, Dimitar Shterionov, and Matthew Gwilliam. 2021. Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2203–2213, Online. Association for Computational Linguistics.

Miriam Winkler, Virginija Juozapaityte, Rob van der Goot, and Barbara Plank. 2024. Slot and intent detection resources for Bavarian and Lithuanian: Assessing translations vs natural queries to digital assistants. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 14898–14915, Torino, Italia. ELRA and ICCL.

Weijia Xu, Batool Haider, and Saab Mansour. 2020. End-to-end slot alignment and recognition for cross-lingual NLU. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5052–5063, Online. Association for Computational Linguistics.

Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mt5: A massively multilingual pre-trained text-to-text transformer.

Hao Yu, Zachary Yang, Kellin Pelrine, Jean François Godbout, and Reihaneh Rabbany. 2023. Open, closed, or small language models for text classification? ArXiv, abs/2308.10092.

Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D’souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Anr Kayid, Freddie Vargas, Phil Blunsom, Shayne Longpre, Niklas Muenninghoff, Marzieh Fadaee, Julia Kreutzer, and Sara Hooker. 2024. Aya model: An instruction finetuned open-access multilingual language model. arXiv preprint arXiv:2402.07827.
A INJONGO Dataset

A.1 Categories of Intent Detection
The following are the intent labels used in the INJONGO dataset. These are a total of 40 categories across 5 domains (Banking, Kitchen and Dining, Travel, Utility, and Home).

<table>
  <tr>
    <th>Domain</th>
    <th>Intent</th>
  </tr>
  <tr>
    <td>Banking</td>
    <td>freeze_account, pin_change, pay_bill, interest_rate, min_payment, bill_balance, balance, spending_history, transactions, transfer</td>
  </tr>
  <tr>
    <td>Kitchen and Dining</td>
    <td>food_list, confirm_reservation, ingredients_list, cook_time, restaurant_reviews, meal_suggestion, restaurant_suggestion, restaurant_reservation, cancel_reservation, recipe</td>
  </tr>
  <tr>
    <td>Home</td>
    <td>play_music, calendar_update, update_playlist, shopping_list_update</td>
  </tr>
  <tr>
    <td>Travel</td>
    <td>plug_type, travel_notification, translate, international, visa, exchange_rate, travel_suggestion, book_flight, book_hotel, car_rental</td>
  </tr>
  <tr>
    <td>Utility</td>
    <td>weather, alarm, share_location, make_call, time, text</td>
  </tr>
</table>

Table 7: Grouped intents categories by five domains.

A.2 Categories of Slot Filling
Table 9 shows the original slot types and their final status after merging similar or low-frequency types during preprocessing. The “Original Slot Type” are used during the dataset annotation phase, which contained 34 slot types. After merging similar or low-frequency types during data preprocessing in Section 3.3, it was reduced to 23 distinct slot types as shown in the “Final Merged Type” column.

<table>
  <tr>
    <th>Original Slot Type</th>
    <th>Status</th>
    <th>Final Merged Type</th>
  </tr>
  <tr>
    <td>account type</td>
    <td>kept</td>
    <td>account type</td>
  </tr>
  <tr>
    <td>artist name</td>
    <td>kept</td>
    <td>artist name</td>
  </tr>
  <tr>
    <td>bank name</td>
    <td>kept</td>
    <td>bank name</td>
  </tr>
  <tr>
    <td>bill type</td>
    <td>kept</td>
    <td>bill type</td>
  </tr>
  <tr>
    <td>calendar event</td>
    <td>kept</td>
    <td>calendar event</td>
  </tr>
  <tr>
    <td>country</td>
    <td>kept</td>
    <td>country</td>
  </tr>
  <tr>
    <td>currency</td>
    <td>kept</td>
    <td>currency</td>
  </tr>
  <tr>
    <td>date</td>
    <td>kept</td>
    <td>date</td>
  </tr>
  <tr>
    <td>hotel name</td>
    <td>kept</td>
    <td>hotel name</td>
  </tr>
  <tr>
    <td>language name</td>
    <td>kept</td>
    <td>language name</td>
  </tr>
  <tr>
    <td>meal period</td>
    <td>kept</td>
    <td>meal period</td>
  </tr>
  <tr>
    <td>money</td>
    <td>kept</td>
    <td>money</td>
  </tr>
  <tr>
    <td>music genre</td>
    <td>kept</td>
    <td>music genre</td>
  </tr>
  <tr>
    <td>number</td>
    <td>kept</td>
    <td>number</td>
  </tr>
  <tr>
    <td>payment company</td>
    <td>kept</td>
    <td>payment company</td>
  </tr>
  <tr>
    <td>personal name</td>
    <td>kept</td>
    <td>personal name</td>
  </tr>
  <tr>
    <td>place name</td>
    <td>kept</td>
    <td>place name</td>
  </tr>
  <tr>
    <td>restaurant name</td>
    <td>kept</td>
    <td>restaurant name</td>
  </tr>
  <tr>
    <td>shopping item</td>
    <td>kept</td>
    <td>shopping item</td>
  </tr>
  <tr>
    <td>song name</td>
    <td>kept</td>
    <td>song name</td>
  </tr>
  <tr>
    <td>time</td>
    <td>kept</td>
    <td>time</td>
  </tr>
  <tr>
    <td>airline</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>airport name</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>car rental company</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>car type</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>continent</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>nationality</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>plug type</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>supermarket name</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>timezone</td>
    <td>deleted</td>
    <td>–</td>
  </tr>
  <tr>
    <td>city name</td>
    <td>merged</td>
    <td>city or province</td>
  </tr>
  <tr>
    <td>state or province</td>
    <td>merged</td>
    <td>city or province</td>
  </tr>
  <tr>
    <td>dish name</td>
    <td>merged</td>
    <td>dish or food</td>
  </tr>
  <tr>
    <td>food item</td>
    <td>merged</td>
    <td>dish or food</td>
  </tr>
</table>

Table 9: Original and final slot types in the INJONGO dataset. “kept” indicates the slot type was retained, while “deleted” indicates the slot type was removed. “merged” indicates the slot type was combined with another similar type.

A.3 Statistics of Corpus and Slot Entities

<table>
  <tr>
    <th>Language Code</th>
    <th>Name</th>
    <th>Corpus Statistics in Token</th>
    <th colspan="2">Slot Entities</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>Total</th>
    <th>Avg.</th>
    <th>Unique</th>
  </tr>
  <tr>
    <td>amh</td>
    <td>Afar</td>
    <td>24233</td>
    <td>7.573</td>
    <td>5279</td>
  </tr>
  <tr>
    <td>ces</td>
    <td>Ewe</td>
    <td>32210</td>
    <td>10.378</td>
    <td>4422</td>
  </tr>
  <tr>
    <td>haa</td>
    <td>Hausa</td>
    <td>32350</td>
    <td>10.103</td>
    <td>1896</td>
  </tr>
  <tr>
    <td>hau</td>
    <td>Hausa</td>
    <td>32350</td>
    <td>10.103</td>
    <td>1896</td>
  </tr>
  <tr>
    <td>kin</td>
    <td>Kinyarwanda</td>
    <td>30216</td>
    <td>9.443</td>
    <td>6112</td>
  </tr>
  <tr>
    <td>lnv</td>
    <td>Lingala</td>
    <td>29771</td>
    <td>9.218</td>
    <td>2542</td>
  </tr>
  <tr>
    <td>lvi</td>
    <td>Livonian</td>
    <td>29429</td>
    <td>9.197</td>
    <td>5706</td>
  </tr>
  <tr>
    <td>omn</td>
    <td>Oromo</td>
    <td>29429</td>
    <td>9.197</td>
    <td>5706</td>
  </tr>
  <tr>
    <td>soo</td>
    <td>Sotho</td>
    <td>29155</td>
    <td>9.223</td>
    <td>3323</td>
  </tr>
  <tr>
    <td>swa</td>
    <td>Swahili</td>
    <td>38822</td>
    <td>12.132</td>
    <td>4603</td>
  </tr>
  <tr>
    <td>taa</td>
    <td>Taa</td>
    <td>37120</td>
    <td>11.600</td>
    <td>3460</td>
  </tr>
  <tr>
    <td>wol</td>
    <td>Wolof</td>
    <td>37120</td>
    <td>11.600</td>
    <td>3460</td>
  </tr>
  <tr>
    <td>xho</td>
    <td>Xhosa</td>
    <td>26118</td>
    <td>8.162</td>
    <td>5086</td>
  </tr>
  <tr>
    <td>zul</td>
    <td>Zulu</td>
    <td>26496</td>
    <td>8.285</td>
    <td>7742</td>
  </tr>
  <tr>
    <td>eng</td>
    <td>English</td>
    <td>20566</td>
    <td>10.861</td>
    <td>3097</td>
  </tr>
</table>

<table>
  <tr>
    <th>Slot Entities</th>
    <th>Total</th>
    <th>Avg.</th>
    <th>Unique</th>
    <th>1st Floor*</th>
    <th>Floor**</th>
  </tr>
  <tr>
    <td>amh</td>
    <td>10748</td>
    <td>7.66</td>
    <td>31</td>
    <td>0.886</td>
    <td>0.951</td>
  </tr>
  <tr>
    <td>ces</td>
    <td>11563</td>
    <td>3.61</td>
    <td>34</td>
    <td>0.854</td>
    <td>1.000</td>
  </tr>
  <tr>
    <td>haa</td>
    <td>11792</td>
    <td>3.69</td>
    <td>33</td>
    <td>0.863</td>
    <td>0.996</td>
  </tr>
  <tr>
    <td>hau</td>
    <td>11792</td>
    <td>3.69</td>
    <td>33</td>
    <td>0.863</td>
    <td>0.996</td>
  </tr>
  <tr>
    <td>kin</td>
    <td>10753</td>
    <td>3.36</td>
    <td>34</td>
    <td>0.712</td>
    <td>0.959</td>
  </tr>
  <tr>
    <td>lnv</td>
    <td>11406</td>
    <td>3.66</td>
    <td>36</td>
    <td>0.786</td>
    <td>0.999</td>
  </tr>
  <tr>
    <td>lvi</td>
    <td>12570</td>
    <td>3.93</td>
    <td>33</td>
    <td>0.864</td>
    <td>0.990</td>
  </tr>
  <tr>
    <td>omn</td>
    <td>12570</td>
    <td>3.93</td>
    <td>33</td>
    <td>0.844</td>
    <td>0.992</td>
  </tr>
  <tr>
    <td>soo</td>
    <td>6699</td>
    <td>2.09</td>
    <td>44</td>
    <td>0.864</td>
    <td>0.985</td>
  </tr>
  <tr>
    <td>swa</td>
    <td>14750</td>
    <td>4.61</td>
    <td>44</td>
    <td>0.864</td>
    <td>0.985</td>
  </tr>
  <tr>
    <td>taa</td>
    <td>11265</td>
    <td>3.53</td>
    <td>33</td>
    <td>0.726</td>
    <td>0.943</td>
  </tr>
  <tr>
    <td>wol</td>
    <td>12673</td>
    <td>3.96</td>
    <td>33</td>
    <td>0.804</td>
    <td>0.936</td>
  </tr>
  <tr>
    <td>xho</td>
    <td>12310</td>
    <td>4.86</td>
    <td>34</td>
    <td>0.848</td>
    <td>0.991</td>
  </tr>
  <tr>
    <td>zul</td>
    <td>12310</td>
    <td>4.86</td>
    <td>34</td>
    <td>0.848</td>
    <td>0.991</td>
  </tr>
  <tr>
    <td>eng</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>0.618</td>
    <td>0.912</td>
  </tr>
</table>

Table 8: Statistics of the INJONGO dataset across 17 languages, including corpus statistics (token counts and distributions) and slot entity analysis (entity counts, averages, and inter-annotator agreement measures) with unmerged slot types.

B Experiments Setup

B.1 Training Configuration
To ensure equitable comparison across architectures, we implement a standardized training protocol. All SLMs are finetuned using the AdamW optimizer in 20 epochs with a learning rate schedule incorporating 10% linear warmup steps followed by linear decay. Early stopping (patience=5) is adopted, and the dev set performance is monitored. Learning rates are carefully calibrated for each architecture type as detailed in Table 10. Our empirical investigations demonstrate that slot filling tasks consistently require higher learning rates compared to intent detection tasks specifically, encoder-only models utilize \( 1 \times 10^{-5}/3 \times 10^{-5} \) for intent detection/slot filling respectively, while encoder-decoder architectures necessitate elevated rates of \( 5 \times 10^{-5}/1 \times 10^{-4} \).

Given the computational constraints of finetuning LLMs, Fully Supervised Fine-Tuning (FSFT) is
exclusively performed on the Gemma 2 9B model with 5 epochs. Based on established SFT practices and task-specific requirements, we use learning rates of \(2 \times 10^{-5}\) and \(2.5 \times 10^{-5}\) for intent detection and slot filling respectively. Training data is constructed from the combined train splits of INJONGO dataset across all 17 languages, with prompts randomly sampled from a pool of 5 predefined templates.

All experiments are conducted using full precision (FP32) on NVIDIA H100/L40S GPUs with a consistent batch size of 32, achieved through gradient accumulation when necessary.

B.2 Learning Rate Choice

Before the final model training, we conducted a comprehensive analysis of learning rate variations to understand their effect on model performance across Intent Detection and Slot Filling tasks. This investigation helped determine optimal learning rates for different model architectures. Table 12 presents detailed results, extending the findings from Table 10.

<table>
  <tr>
    <th>Task</th>
    <th>Encoder Only</th>
    <th>Encoder-Decoder</th>
    <th>NLLB LLM2Vec</th>
  </tr>
  <tr>
    <td>Intent Detection</td>
    <td>\(1 \times 10^{-5}\)</td>
    <td>\(5 \times 10^{-5}\)</td>
    <td>\(1 \times 10^{-4}\)</td>
  </tr>
  <tr>
    <td>Slot Filling</td>
    <td>\(3 \times 10^{-5}\)</td>
    <td>\(1 \times 10^{-4}\)</td>
    <td>\(3 \times 10^{-4}\)</td>
  </tr>
</table>

Table 10: Selected learning rates for different architectures of both tasks of intent detection and slot filling.

B.3 Language Coverage of Baselines

The table below briefly introduces the baseline models along with the languages they were trained on in the INJONGO dataset.

<table>
  <tr>
    <th>Model</th>
    <th>Languages</th>
  </tr>
  <tr>
    <td>AfroXLMR Large (550M)</td>
    <td>amh, hau, ibo, kin, omn, sma, sot, owa, sho, yor, zul</td>
  </tr>
  <tr>
    <td>AfroXLMR Large 76L (550M)</td>
    <td>amh, ewe, hau, ibo, kin, lin, lug, urm, sma, sot, owa, twi, wol, sho, yor, zul</td>
  </tr>
  <tr>
    <td>XLM-Roberta Large (550M)</td>
    <td>amh, omn, owa, sho</td>
  </tr>
  <tr>
    <td>AfroBERTa V2 Large (387M)</td>
    <td>amh, hau, ibo, sma, sot, owa, sho, yor</td>
  </tr>
  <tr>
    <td>AfroTeVa V2 Large (1.2B)</td>
    <td>amh, hau, ibo, sma, sot, owa, sho, yor</td>
  </tr>
  <tr>
    <td>mT5-Large (1.2B)</td>
    <td>amh, hau, ibo, sot, owa, yor, zul</td>
  </tr>
</table>

Table 11: Baseline models with their corresponding language coverage in INJONGO.

B.4 Results of Multi-lingual Training

We selected the 4 top-performing models from the in-language training phase and evaluated them on the INJONGO test set, comparing the performance of the models when trained on individual languages and when trained on the combined dataset. The results are shown in Table 13.

B.5 Results of Cross-lingual Transfer

This section provides additional commentary on Table 14 which reports the cross-lingual transfer performance of AfroXLMR-76L on the Intent Detection task under different shot conditions. The table compares two datasets (CLINC and Injongo) in both their original in-language and translate-test settings. For each dataset, results are presented at multiple shot levels (e.g., 5, 10, 25, 50, and 100 shots), with the average performance and corresponding standard deviation indicated. Notably, the results illustrate how performance progressively improves as the number of shots increases, and how the transfer capability is affected by the linguistic diversity of the datasets.

B.6 Inference Setup of LLMs

For closed-source models (GPT-4o and Gemini 1.5 Pro), we utilize the API provided by the respective vendor for inference. For open-source models, inference is performed using vLLM (Kwon et al., 2023), except for Aya-101, where Text Generation Inference (TGI)\(^{10}\) is employed.

B.7 Results of LLMs prompting

Across 5 LLMs, we evaluated the performance of zero-shot and few-shot learning on the Intent Detection and Slot Filling tasks. The complete results are presented in Table 15. We only evaluate the performance of the models on the best prompt for each task. The 2nd prompt for Intent Detection and the 3rd prompt for Slot Filling are used for evaluation.

\(^{10}\)Text Generation Inference
<table>
  <tr>
    <th rowspan="2">Task</th>
    <th rowspan="2">Model Type</th>
    <th rowspan="2">Model</th>
    <th colspan="10">Learning Rate</th>
  </tr>
  <tr>
    <th>1 × 10<sup>-5</sup></th>
    <th>2 × 10<sup>-5</sup></th>
    <th>3 × 10<sup>-5</sup></th>
    <th>5 × 10<sup>-5</sup></th>
    <th>1 × 10<sup>-4</sup></th>
    <th>2 × 10<sup>-4</sup></th>
    <th>3 × 10<sup>-4</sup></th>
    <th>5 × 10<sup>-4</sup></th>
  </tr>
  <tr>
    <td rowspan="6">Intent Detection</td>
    <td rowspan="4">Encoder</td>
    <td>AfriBERTa V2 Large</td>
    <td>97.50</td><td>98.13</td><td>98.13</td><td>98.13</td><td>97.81</td><td>97.81</td><td>95.00</td><td>2.50</td>
  </tr>
  <tr>
    <td>AfroXLMR-large</td>
    <td>98.13</td><td>98.13</td><td>98.75</td><td>2.50</td><td>2.50</td><td>2.50</td><td>2.50</td><td>2.50</td>
  </tr>
  <tr>
    <td>AfroXLMR-large 76L</td>
    <td>98.75</td><td>98.75</td><td>99.06</td><td>98.75</td><td>2.50</td><td>2.50</td><td>2.50</td><td>2.50</td><td>2.50</td>
  </tr>
  <tr>
    <td>XLM-RoBERTa Large</td>
    <td>98.75</td><td>2.50</td><td>13.44</td><td>6.25</td><td>2.50</td><td>2.50</td><td>4.06</td><td>2.50</td><td>2.50</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>98.28</td><td>74.38</td><td>77.34</td><td>51.41</td><td>26.33</td><td>26.33</td><td>26.02</td><td>2.50</td><td>2.50</td>
  </tr>
  <tr>
    <td rowspan="3">Encoder-Decoder</td>
    <td>AfriTeVa V2 Large</td>
    <td>0.00</td><td>0.00</td><td>96.88</td><td>97.81</td><td>97.19</td><td>97.81</td><td>96.36</td><td>97.50</td>
  </tr>
  <tr>
    <td>mT5-Large</td>
    <td>0.00</td><td>95.51</td><td>95.94</td><td>97.50</td><td>97.50</td><td>97.50</td><td>98.13</td><td>97.50</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>0.00</td><td>47.66</td><td>96.41</td><td>97.66</td><td>97.34</td><td>97.66</td><td>97.34</td><td>97.50</td>
  </tr>
  <tr>
    <td rowspan="2">Other</td>
    <td>NLLB LLM2Vec</td>
    <td>97.19</td><td>97.50</td><td>96.88</td><td>95.94</td><td>98.44</td><td>97.81</td><td>98.44</td><td>97.19</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>97.19</td><td>97.50</td><td>96.88</td><td>95.94</td><td>98.44</td><td>97.81</td><td>98.44</td><td>97.19</td>
  </tr>
  <tr>
    <td rowspan="6">Slot Filling</td>
    <td rowspan="4">Encoder</td>
    <td>AfriBERTa V2 Large</td>
    <td>86.12</td><td>89.70</td><td>90.21</td><td>90.74</td><td>91.22</td><td>90.45</td><td>88.24</td><td>0.00</td>
  </tr>
  <tr>
    <td>AfroXLMR-large</td>
    <td>89.95</td><td>90.13</td><td>91.04</td><td>89.87</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td>AfroXLMR-large 76L</td>
    <td>90.04</td><td>90.91</td><td>90.96</td><td>90.58</td><td>91.28</td><td>0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td>XLM-RoBERTa Large</td>
    <td>88.55</td><td>89.72</td><td>91.63</td><td>89.52</td><td>88.10</td><td>0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>88.67</td><td>90.11</td><td>90.96</td><td>90.18</td><td>67.65</td><td>22.61</td><td>22.06</td><td>0.00</td>
  </tr>
  <tr>
    <td rowspan="3">Encoder-Decoder</td>
    <td>AfriTeVa V2 Large</td>
    <td>39.07</td><td>83.47</td><td>83.47</td><td>89.63</td><td>90.51</td><td>81.59</td><td>88.11</td><td>88.44</td>
  </tr>
  <tr>
    <td>mT5-Large</td>
    <td>22.31</td><td>59.61</td><td>89.16</td><td>82.71</td><td>88.54</td><td>89.67</td><td>89.16</td><td>90.40</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>30.69</td><td>71.54</td><td>86.32</td><td>86.17</td><td>89.53</td><td>85.63</td><td>88.64</td><td>89.42</td>
  </tr>
  <tr>
    <td rowspan="2">Other</td>
    <td>NLLB LLM2Vec</td>
    <td>81.13</td><td>84.84</td><td>85.33</td><td>85.69</td><td>85.57</td><td>84.44</td><td>87.02</td><td>86.12</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>81.13</td><td>84.84</td><td>85.33</td><td>85.69</td><td>85.57</td><td>84.44</td><td>87.02</td><td>86.12</td>
  </tr>
</table>

Table 12: Comparative analysis of model performance across different learning rates for Intent Detection and Slot Filling tasks. Results are shown for various model architectures including Encoder-only, Encoder-Decoder, and other approaches. Bold values indicate the best performance for each model type.

<table>
  <tr>
    <th>Task</th>
    <th>Model</th>
    <th>amh</th>
    <th>ewe</th>
    <th>hau</th>
    <th>ibo</th>
    <th>kin</th>
    <th>lin</th>
    <th>lug</th>
    <th>orm</th>
    <th>sna</th>
    <th>sot</th>
    <th>swa</th>
    <th>twi</th>
    <th>wol</th>
    <th>xho</th>
    <th>yor</th>
    <th>zul</th>
    <th>eng</th>
    <th>AVG</th>
  </tr>
  <tr>
    <td rowspan="4">Intent Detection</td>
    <td>AfroXLMR-large 76L</td>
    <td>96.0</td><td>92.6</td><td>99.2</td><td>96.6</td><td>87.7</td><td>95.9</td><td>92.3</td><td>92.9</td><td>96.5</td><td>87.6</td><td>97.8</td><td>94.2</td><td>97.1</td><td>97.3</td><td>97.9</td><td>89.2</td><td>89.0</td><td>94.4<sub>±0.6</sub></td>
  </tr>
  <tr>
    <td>AfroXLMR-large</td>
    <td>96.1</td><td>90.3</td><td>99.3</td><td>96.5</td><td>86.4</td><td>94.2</td><td>91.6</td><td>92.2</td><td>96.0</td><td>87.1</td><td>97.9</td><td>91.6</td><td>96.1</td><td>97.4</td><td>88.6</td><td>89.7</td><td>93.7<sub>±0.9</sub></td>
  </tr>
  <tr>
    <td>NLLB LLM2Vec</td>
    <td>95.8</td><td>90.2</td><td>98.7</td><td>96.5</td><td>86.2</td><td>95.4</td><td>92.6</td><td>87.9</td><td>96.9</td><td>86.8</td><td>97.3</td><td>93.9</td><td>95.5</td><td>96.9</td><td>97.2</td><td>88.6</td><td>89.1</td><td>93.5<sub>±1.1</sub></td>
  </tr>
  <tr>
    <td>AfriTeVa V2 Large</td>
    <td>94.6</td><td>85.8</td><td>99.2</td><td>96.5</td><td>87.3</td><td>93.6</td><td>90.8</td><td>88.6</td><td>95.9</td><td>85.3</td><td>98.0</td><td>89.6</td><td>94.7</td><td>97.3</td><td>97.6</td><td>88.3</td><td>89.7</td><td>92.7<sub>±1.6</sub></td>
  </tr>
  <tr>
    <td rowspan="4">Slot Filling</td>
    <td>AfroXLMR-large 76L</td>
    <td>88.2</td><td>87.0</td><td>96.3</td><td>84.0</td><td>79.3</td><td>90.3</td><td>89.2</td><td>87.2</td><td>86.1</td><td>80.4</td><td>90.5</td><td>90.1</td><td>83.3</td><td>91.8</td><td>90.2</td><td>83.3</td><td>82.4</td><td>87.3<sub>±1.4</sub></td>
  </tr>
  <tr>
    <td>AfroXLMR-large</td>
    <td>87.9</td><td>84.0</td><td>96.4</td><td>83.6</td><td>80.4</td><td>89.5</td><td>88.4</td><td>88.2</td><td>87.0</td><td>82.0</td><td>91.5</td><td>87.7</td><td>81.9</td><td>91.7</td><td>90.4</td><td>84.2</td><td>82.8</td><td>87.2<sub>±2.2</sub></td>
  </tr>
  <tr>
    <td>NLLB LLM2Vec</td>
    <td>84.3</td><td>82.0</td><td>94.6</td><td>80.3</td><td>72.3</td><td>86.9</td><td>85.1</td><td>81.9</td><td>82.0</td><td>77.2</td><td>87.3</td><td>85.8</td><td>80.0</td><td>90.4</td><td>87.1</td><td>79.9</td><td>80.8</td><td>83.6<sub>±5.2</sub></td>
  </tr>
  <tr>
    <td>AfriTeVa V2 Large</td>
    <td>78.9</td><td>72.6</td><td>92.0</td><td>80.0</td><td>75.7</td><td>85.3</td><td>81.8</td><td>76.0</td><td>79.8</td><td>77.0</td><td>88.2</td><td>81.7</td><td>76.5</td><td>86.7</td><td>86.3</td><td>66.7</td><td>78.9</td><td>80.3<sub>±6.3</sub></td>
  </tr>
</table>

Table 13: Multilingual Training: 4 model performance on Intent Detection and Slot Filling tasks across languages.

C Prompts for Large Language Models

We provide the prompts in Jinja format 11 used for the Intent Detection and Slot Filling tasks in the zero-shot and few-shot learning experiments. The prompts are designed to guide the model to perform the specific task on the given input text. The prompts are language-specific and tailored to the task requirements.

The variables in the prompts are replaced with the actual input text during the model evaluation. Here is the list of variables used in the prompts:

• shot_count: The number of examples provided to the model, if shot_count is 0 zero, means zero-shot.

• examples: A list of examples provided to the model for few-shot learning.

• text: The sentence for which the model needs to predict the intent or slot.

11Jinja: A fast, expressive, extensible templating engine.

C.1 Intent Detection

Prompt I

Classify the given sentence by
→ identifying its intent and
→ selecting the most appropriate
→ category from the provided list.

# Steps
1. Analyze the sentence to understand
→ its primary intention or purpose.
2. Compare the identified intention
→ against the possible intent
→ categories.
3. Select the category that best
→ matches the sentence’s intent.

# Output Format
- Return the only one matching intent
→ category from the list above.
- No additional text or punctuation
→ should be included in the output.
<table>
  <tr>
    <th># of Shots</th>
    <th>amh</th>
    <th>ewe</th>
    <th>hau</th>
    <th>ibo</th>
    <th>kin</th>
    <th>lin</th>
    <th>lug</th>
    <th>orm</th>
    <th>sna</th>
    <th>sot</th>
    <th>swa</th>
    <th>twi</th>
    <th>wol</th>
    <th>xho</th>
    <th>yor</th>
    <th>zul</th>
    <th>Avg</th>
  </tr>
  <tr>
    <th colspan="18">CLINC Dataset</th>
  </tr>
  <tr>
    <td>5</td><td>3.1</td><td>2.7</td><td>2.5</td><td>2.6</td><td>2.5</td><td>3.2</td><td>2.6</td><td>2.4</td><td>2.9</td><td>2.9</td><td>3.6</td><td>1.9</td><td>2.8</td><td>2.3</td><td>2.7</td><td>3.0</td><td>2.7±0.5</td>
  </tr>
  <tr>
    <td>10</td><td>6.7</td><td>2.8</td><td>5.5</td><td>4.1</td><td>3.5</td><td>5.1</td><td>3.2</td><td>3.2</td><td>3.1</td><td>4.2</td><td>7.0</td><td>2.7</td><td>3.1</td><td>2.6</td><td>3.4</td><td>3.6</td><td>4.0±1.0</td>
  </tr>
  <tr>
    <td>25</td><td>80.3</td><td>24.4</td><td>69.6</td><td>51.9</td><td>49.9</td><td>57.2</td><td>37.6</td><td>29.8</td><td>53.0</td><td>42.9</td><td>78.0</td><td>38.4</td><td>26.4</td><td>59.6</td><td>35.6</td><td>55.1</td><td>49.4±9.4</td>
  </tr>
  <tr>
    <td>50</td><td>83.8</td><td>36.1</td><td>78.4</td><td>61.9</td><td>55.3</td><td>63.4</td><td>45.7</td><td>39.4</td><td>59.8</td><td>50.3</td><td>82.6</td><td>47.1</td><td>34.9</td><td>65.3</td><td>48.7</td><td>60.0</td><td>57.1±8.4</td>
  </tr>
  <tr>
    <td>100</td><td>84.7</td><td>37.8</td><td>80.9</td><td>62.6</td><td>55.7</td><td>65.2</td><td>47.2</td><td>39.6</td><td>63.2</td><td>50.9</td><td>85.2</td><td>48.8</td><td>36.0</td><td>66.7</td><td>52.5</td><td>62.1</td><td>58.7±8.5</td>
  </tr>
  <tr>
    <th colspan="18">Injongo Dataset</th>
  </tr>
  <tr>
    <td>5</td><td>3.6</td><td>2.6</td><td>3.3</td><td>2.6</td><td>3.1</td><td>3.0</td><td>2.8</td><td>2.3</td><td>3.2</td><td>3.3</td><td>3.9</td><td>2.2</td><td>2.8</td><td>2.8</td><td>2.5</td><td>2.8</td><td>2.9±0.5</td>
  </tr>
  <tr>
    <td>10</td><td>29.7</td><td>7.3</td><td>27.0</td><td>15.3</td><td>13.9</td><td>19.4</td><td>9.3</td><td>6.3</td><td>13.4</td><td>16.2</td><td>36.7</td><td>9.4</td><td>7.1</td><td>16.5</td><td>10.0</td><td>20.3</td><td>16.1±5.1</td>
  </tr>
  <tr>
    <td>25</td><td>76.1</td><td>24.2</td><td>70.2</td><td>53.9</td><td>50.1</td><td>55.2</td><td>41.2</td><td>28.4</td><td>53.9</td><td>45.1</td><td>78.5</td><td>41.1</td><td>27.8</td><td>59.7</td><td>38.8</td><td>55.1</td><td>50.0±8.9</td>
  </tr>
  <tr>
    <th colspan="18">CLINC Dataset (translate test)</th>
  </tr>
  <tr>
    <td>5</td><td>4.2</td><td>3.6</td><td>3.5</td><td>4.0</td><td>3.9</td><td>4.0</td><td>3.6</td><td>3.5</td><td>3.7</td><td>3.8</td><td>4.3</td><td>3.2</td><td>3.6</td><td>3.9</td><td>3.6</td><td>4.0</td><td>3.8±0.5</td>
  </tr>
  <tr>
    <td>10</td><td>14.9</td><td>11.2</td><td>15.9</td><td>13.7</td><td>11.9</td><td>15.3</td><td>11.0</td><td>4.8</td><td>10.2</td><td>12.4</td><td>13.8</td><td>9.4</td><td>10.2</td><td>12.4</td><td>10.3</td><td>12.5</td><td>11.9±1.9</td>
  </tr>
  <tr>
    <td>25</td><td>83.2</td><td>58.8</td><td>85.7</td><td>78.3</td><td>67.6</td><td>76.7</td><td>71.4</td><td>32.1</td><td>80.2</td><td>67.4</td><td>86.4</td><td>67.8</td><td>56.8</td><td>82.8</td><td>75.3</td><td>71.0</td><td>71.2±7.4</td>
  </tr>
  <tr>
    <td>50</td><td>86.2</td><td>61.7</td><td>89.9</td><td>81.9</td><td>69.1</td><td>78.8</td><td>74.2</td><td>32.2</td><td>82.7</td><td>71.7</td><td>85.8</td><td>70.1</td><td>58.1</td><td>86.7</td><td>79.2</td><td>74.4</td><td>74.0±7.7</td>
  </tr>
  <tr>
    <td>100</td><td>86.7</td><td>62.5</td><td>91.0</td><td>83.3</td><td>69.5</td><td>80.5</td><td>75.4</td><td>34.4</td><td>84.9</td><td>72.2</td><td>87.7</td><td>71.1</td><td>59.1</td><td>86.2</td><td>81.2</td><td>76.6</td><td>75.1±7.7</td>
  </tr>
  <tr>
    <th colspan="18">Injongo Dataset (translate test)</th>
  </tr>
  <tr>
    <td>5</td><td>4.4</td><td>4.7</td><td>5.3</td><td>4.4</td><td>3.7</td><td>5.0</td><td>4.4</td><td>2.7</td><td>4.5</td><td>4.5</td><td>4.7</td><td>3.8</td><td>4.2</td><td>5.1</td><td>3.6</td><td>4.4±0.6</td>
  </tr>
  <tr>
    <td>10</td><td>45.4</td><td>32.2</td><td>52.0</td><td>44.7</td><td>39.7</td><td>43.9</td><td>42.8</td><td>16.3</td><td>44.2</td><td>38.6</td><td>50.6</td><td>37.1</td><td>32.7</td><td>48.0</td><td>45.3</td><td>40.8</td><td>40.9±4.9</td>
  </tr>
  <tr>
    <td>25</td><td>80.5</td><td>59.0</td><td>86.0</td><td>77.5</td><td>66.4</td><td>74.2</td><td>72.7</td><td>31.7</td><td>77.9</td><td>68.1</td><td>84.1</td><td>66.6</td><td>55.9</td><td>82.4</td><td>76.8</td><td>69.7</td><td>70.6±7.3</td>
  </tr>
</table>

Table 14: AfroXLMR-76L Intent Detection performance under varying shot conditions.

<table>
  <tr>
    <th>Task</th>
    <th>Model</th>
    <th colspan="18">Setup</th>
    <th>Avg</th>
  </tr>
  <tr>
    <td rowspan="5">INTENT DETECTION</td>
    <td>GPT-4o</td>
    <td colspan="18">0 shot<br>81.2<br>81.8<br>75.9<br>82.6<br>83.0<br>82.8<br>83.3<br>85.2<br>84.5<br>91.7<br>86.7<br>79.4<br>85.0<br>85.3<br>83.3<br>89.7<br>75.0<br>87.8<br>82.1<br>87.2<br>79.7<br>82.2<br>83.2±4.4</td>
    <td></td>
  </tr>
  <tr>
    <td>Gemini 1 Pro</td>
    <td colspan="18">0 shot<br>82.3<br>81.8<br>80.2<br>76.9<br>78.8<br>69.5<br>66.2<br>58.1<br>64.4<br>72.8<br>71.4<br>55.5<br>85.0<br>50.5<br>27.9<br>79.4<br>71.6<br>71.9<br>62.5±12.7</td>
    <td></td>
  </tr>
  <tr>
    <td>Gemini 2 IT 9B</td>
    <td colspan="18">0 shot<br>78.9<br>78.4<br>71.0<br>73.1<br>33.4<br>28.1<br>24.5<br>25.6<br>8.6<br>30.5<br>21.7<br>73.1<br>23.7<br>18.4<br>38.4<br>40.5<br>31.2±12.2</td>
    <td></td>
  </tr>
  <tr>
    <td>Gemini 2 IT 27B</td>
    <td colspan="18">0 shot<br>80.2<br>80.8<br>75.9<br>80.0<br>70.0<br>52.2<br>55.5<br>52.5<br>22.5<br>68.3<br>55.3<br>84.7<br>54.5<br>37.0<br>73.9<br>63.3<br>64.2<br>56.7±19.5</td>
    <td></td>
  </tr>
  <tr>
    <td>Llama 3 70B</td>
    <td colspan="18">0 shot<br>80.9<br>80.5<br>75.3<br>80.5<br>73.0<br>53.3<br>53.5<br>38.0<br>39.7<br>13.8<br>32.8<br>31.7<br>81.4<br>31.7<br>21.0<br>44.7<br>41.6<br>44.8<br>39.4±18.8</td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="5">SLOT FILLING</td>
    <td>GPT-4o</td>
    <td colspan="18">0 shot<br>55.1<br>53.8<br>23.8<br>38.8<br>38.9<br>37.3<br>33.6<br>37.9<br>12.7<br>41.4<br>42.6<br>44.3<br>39.1<br>41.7<br>36.9<br>37.3±18.7</td>
    <td></td>
  </tr>
  <tr>
    <td>Gemini 1 Pro</td>
    <td colspan="18">0 shot<br>63.9<br>63.5<br>49.3<br>60.8<br>59.8<br>46.8<br>61.0<br>51.0<br>36.6<br>60.6<br>58.2<br>62.4<br>61.8<br>58.5<br>50.7<br>59.4<br>40.3<br>53.3±8.8</td>
    <td></td>
  </tr>
  <tr>
    <td>Gemini 2 IT 9B</td>
    <td colspan="18">0 shot<br>53.3<br>53.3<br>56.2<br>56.9<br>54.3<br>61.3<br>60.5<br>56.4<br>56.7<br>61.5<br>62.6<br>61.8<br>56.2<br>61.8<br>56.2<br>56.9±9.6</td>
    <td></td>
  </tr>
  <tr>
    <td>Gemini 2 IT 27B</td>
    <td colspan="18">0 shot<br>64.7<br>64.7<br>62.4<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3<br>61.3±9.5</td>
    <td></td>
  </tr>
  <tr>
    <td>Llama 3 70B</td>
    <td colspan="18">0 shot<br>59.6<br>59.6<br>67.6<br>72.2<br>73.4<br>66.8<br>77.4<br>77.4<br>66.6<br>77.0<br>65.9<br>79.9<br>77.2<br>69.8<br>80.0<br>81.0<br>57.4<br>72.1±6.7</td>
    <td></td>
  </tr>
</table>

Table 15: Zero-shot and few-shot performance comparison across languages on tasks. For Intent Detection, shots refer to examples per domain 5 examples, per (1 shot), and 4 examples per (4 shots). For Slot Filling, shots refer to examples per domain 5 examples, per slot type (1 shot), and 4 examples per slot type (4 shots).
Prompt II

Identify the intent of the provided text by selecting the most suitable category from the list of available options.

# Steps
1. Analyze the sentence to determine its primary purpose or intention.
2. Match the identified intention with the available intent categories.
3. Choose the category that best aligns with the sentence's intent.

# Output Format
- Return the selected intent category from the list above.
- Do not include any additional text or punctuation in the response.

Prompt III

Determine the intent of the provided text by selecting the most appropriate category from the given options.

# Steps
1. **Read the Text**: Carefully read the provided text to understand the context and main message.
2. **Identify Key Elements**: Identify the main action, subject, and any relevant details that indicate the overall purpose of the text.
3. **Consider Categories**: Review the list of available categories and consider which category best matches the text's intent.
4. **Reasoning**: Consider why you believe the text fits a certain category by assessing how the identified key elements align with the category's definition.
5. **Selection**: Select the category that most accurately represents the intent of the text.

# Output Format
- Provide the selected category as a plain text response.
- Don't include any justification.

Prompt IV

Identify the intent of the provided text by selecting the most suitable category from the list of available options.

# Steps
1. Analyze the text to understand its primary purpose and context.
2. Consider the range of possible intents that the text might express, such as inquiry, statement, request, etc.
3. Match the text with the most appropriate category based on its content and purpose.

# Output Format
Provide the resulting intent category as a short, concise phrase or word that best represents the text's purpose from the available options.

# Notes
- Carefully evaluate any subtleties in the language to determine the intent accurately.
- Consider edge cases where texts might have multiple overlapping intents, and choose the most dominant one.

Prompt V

Identify the intent of the provided text by selecting the most suitable category from the list of available options.

Consider the subtleties in language and any overlapping intents to determine the most dominant intent category.

# Steps
1. **Analyze the Text**: Thoroughly read and understand the text to grasp its primary purpose and context.
2. **Consider Possible Intents**: Reflect on the range of potential intents the text could express, such as inquiry, statement, or request.
3. **Match with Category**: Align the text with the most appropriate category based on content, language subtleties, and dominant purpose.

# Output Format

Provide the resulting intent category as a short, concise phrase or word.

# Notes

- Pay attention to context and subtleties in the text.
- Evaluate texts with multiple intents, prioritizing the most dominant one.

Suffix for Zero-shot and Few-shot

# Intent Categories
alarm, balance, bill_balance,
book_flight, book_hotel,
calendar_update, cancel_reservation,
car_rental, confirm_reservation,
cook_time, exchange_rate, food_last,
freeze_account, ingredients_list,
interest_rate, international_visa,
make_call, meal_suggestion,
min_payment, pay_bill, pin_change,
play_music, plug_type, recipe,
restaurant_reservation,
restaurant_reviews,
restaurant_suggestion,
share_location,
shopping_list_update,
spending_history, text, time,
timezone, transactions, transfer,
translate, travel_notification,
travel_suggestion, update_playlist,
weather

{% if shot_count == 0 -%}

{# Zero-shot Suffix #}
# Format Example:
Sentence: Can you tell me the weather forecast for today?
Output: weather
{% else %}
{# Few-shot Suffix #}
{% for example in examples -%}
Sentence: {{ example.text }}
Output: {{ example.intent }}
{% endfor %}
Based on the example, consider the following:
{% endif %}
Sentence: {{ text }}
Output:

C.2 Slot Filling

Prompt I

Identify all named entities in the sentence provided according to the available entity types. Use `$$` as a separator between each pair of identified named entity types and corresponding content from the sentence. Only return the listed named entities without providing any additional commentary.

# Output Format
- List all the named entities found in the passage provided by the user.
- Separate the paired named entities types and text using a `$$` symbol.
- Only return the entity list, without any prefix or explanation.

Prompt II

Identify and extract named entities from the provided sentence. Each identified entity pair (including entity type and content from the sentence) should be separated from their content using the "$$" delimiter.

# Steps
1. Analyze the sentence to identify named entities.
2. Extract each identified named entity and its content.
3. Concatenate the named entity type and its content with space as one pair.
4. Join all pairs of named entities using "$$" as a delimiter.

Prompt III

Extract named entities from the provided text and format the output by placing $$ between each entity type and its respective content.
Ensure the output contains only the extracted entities and their labels, with no additional commentary or information.

# Steps
1. Analyze the provided text and identify named entities.
2. Categorize each identified entity by its correct type, careful to match the entity with the appropriate label.
3. Format the output by placing the entity type and its corresponding content, separated by $$.

Prompt IV

Identify named entities from the provided text. Format each entity and its content using $$ as a separator.

# Steps
1. Parse the input text to identify all named entities. This includes proper nouns like names of people, places, organizations, dates, etc.
2. For each identified entity, extract the specific text corresponding to the entity.
3. Concatenate the name of the entity type and the associated text using space.
4. Compile these formatted entries into a list with the $$ as a separator.

# Output Format
- A string joined by a " $$ " for each pair of the entity type and content, formatted as `EntityType EntityContent`.

Prompt V

Detect named entities in the supplied sentence. Use $$ as a separator between entities and their corresponding parts of the sentence. Limit the response strictly to the formatted list.

# Output Format
- Entities and their parts separated by $$
- Return a plain list with no additional context
- If no entities are present, return $$

Suffix for Zero-shot and Few-shot

# Named Entities Types to Identify
ACCOUNT_TYPE, ARTIST_NAME, BANK_NAME,
BILL_TYPE, CALENDAR_EVENT,
CITY_OR_PROVINCE, COUNTRY, CURRENCY,
DATE, DISH_OR_FOOD, HOTEL_NAME,
LANGUAGE_NAME, MEAL_PERIOD, MONEY,
MUSIC_GENRE, NUMBER,
PAYMENT_COMPANY, PERSONAL_NAME,
PLACE_NAME, RESTAURANT_NAME,
SHOPPING_ITEM, SONG_NAME, TIME

{% if shot_count == 0 -%}
{# Zero-shot Suffix #}
Please ensure that the entities match the listed types and that unstated entities should not be included in the response if no entities are found, return `$$` only.

# Format Example:
Sentence: John went to Paris and paid 100 dollars at an Awater restaurant.
Output: PERSONAL_NAME John $$ CITY_OR_PROVINCE Paris $$ MONEY 100 $$ RESTAURANT_NAME Awater
{% else %}
{# Few-shot Suffix #}
# Output Examples (Do not include in the response):
{% for example in examples -%}
Sentence: {{ example.text }}
Output: {{ example.slot }}
{% endfor %}
Based on the example, consider the following:
{% endif %}
Sentence: {{ text }}
Output:

D Instruction for Annotators

This section provides the complete annotation guide and instruction for annotators working for labeling all slots types.

A Slot Filling task is a natural language processing (NLP) task that involves extracting specific pieces of information (slots) from a given text. This task is commonly used in dialogue systems and information extraction applications where the goal is to identify and fill predefined categories or slots with relevant information from user inputs or text data.

### LANGUAGE_NAME
1. Spanish: A Romance language that originated in the Iberian Peninsula and is now the primary language of Spain and most Latin American countries.
2. Luganda: A Bantu language spoken primarily in Uganda, particularly by the Ganda people.
3. French: A Romance language spoken as a first language in France, parts of Belgium, and Switzerland, and in various communities worldwide.

### ACCOUNT_TYPE
1. Savings Account: A bank account that earns interest over time, typically used for long-term savings.
2. Checking Account: A bank account used for everyday transactions, such as deposits and withdrawals.
3. Student Account: A bank account designed for students, often with no monthly fees and special benefits.
Not to be confused with payment company. A credit card is NOT an account type.

### MONEY
1. $500: Five hundred dollars, often used to signify a substantial amount of money in various contexts.
2. 5 dollars: A small amount of money, typically used for minor purchases or expenses.
3. $1,000: One thousand dollars, indicating a significant sum, commonly used in transactions or savings.

### CURRENCY
1. Dollar: The currency of several countries, including the United States, Canada, and Australia.
2. Euro: The official currency of the Eurozone, used by 19 of the 27 European Union member states.
3. Yen: The official currency of Japan.

### CITY_NAME
1. London: The capital city of the United Kingdom, known for its historical landmarks and cultural diversity.
2. Kampala: The capital city of Uganda, known for its bustling markets and vibrant cultural scene.
3. New York: A major city in the United States, known for its skyscrapers and as a global financial and cultural center.
If you are not sure if a place is a City name (Town name) State/Province or Village name, please refer to a search engine for clarification.

### FOOD_ITEM
1. Sugar: A sweet substance commonly used in baking and cooking.
2. Orange: A citrus fruit known for its
   → sweet and tangy flavor and high
   → vitamin C content.
Not to be confused with Shopping item or
   → Dish name.

### BANK_NAME
1. Ecobank: A pan-African banking
   → conglomerate with operations in 36
   → African countries.
2. Wells Fargo: An American
   → multinational financial services
   → company headquartered in San
   → Francisco, California.
3. HSBC: A British multinational banking
   → and financial services organization
   → with global operations.
When annotating Bank names, you do not
   → need to include “bank” unless it is
   → attached to the bank name, like seen
   → above, with Ecobank.

### RESTAURANT_NAME
1. KFC: An American fast-food restaurant
   → chain known for its fried chicken.
2. McDonald’s: An American fast-food
   → company famous for its hamburgers,
   → fries, and other quick-serve meals.
3. Subway: An American fast-food
   → franchise known for its submarine
   → sandwiches (subs) and salads.

### DISH_NAME
1. Jollof Rice: A popular West African
   → dish made with rice, tomatoes,
   → onions, and various spices.
2. Paella: A Spanish rice dish
   → originally from Valencia, featuring
   → saffron, meat, seafood, and
   → vegetables.
3. Sushi: A Japanese dish consisting of
   → vinegared rice accompanied by
   → various ingredients such as raw fish
   → and vegetables.

### TIME
1. 2pm: A specific time in the
   → afternoon.
2. Morning: The period from sunrise
   → until noon.
3. Evening: The period of the day from
   → the end of the afternoon to the
   → beginning of night.
Anything that is less than one day
   → should be annotated as TIME and not
   → DATE, as seen in the above examples.

### TIMEZONE
1. Pacific Time (PT): A time zone
   → covering parts of western Canada,
   → the western United States, and
   → western Mexico.
2. West Africa Time (WAT): A time zone
   → used by countries in West Africa,
   → one hour ahead of Coordinated
   → Universal Time (UTC+1).
3. Eastern Standard Time (EST): A time
   → zone covering parts of the eastern
   → United States and parts of Canada,
   → five hours behind Coordinated
   → Universal Time (UTC-5).

### DATE
1. January: The first month of the year
   → in the Gregorian calendar.
2. 2024: A specific year.
3. October: The tenth month of the year
   → in the Gregorian calendar.
Anything that is more than one day must
   → be annotated as DATE and not time,
   → as seen above

### BILL_TYPE
1. Internet Fees: Charges for the
   → provision of internet services.
2. School Fees: Costs associated with
   → attending an educational
   → institution.
3. Electricity Bill: Charges for the
   → consumption of electrical power.
4. Water Bill:
You include “bill” as part of the
   → annotation.

### PLUG_TYPE
1. Type A: A two-pronged plug commonly
   → used in North America and Japan.
2. Type C: A two-pin plug used in
   → Europe, South America, and Asia.
3. Type G: A three-pronged plug used in
   → the United Kingdom and other
   → countries.
Internet cable, extension cord are NOT plug types.

### COUNTRY
1. Germany: A country in Central Europe known for its rich history and economic strength.
2. Nigeria: A country in West Africa, known for its diverse cultures and large population.
3. Japan: An island nation in East Asia known for its technology and rich cultural heritage.

### PERSONAL_NAME
1. Dave: A common given name.
2. Maria: A common given name, often used in Spanish and Portuguese-speaking countries.
3. Akiko: A common Japanese given name.
4. Don’t annotate titles as personal names e.g Mr., Dr., Mrs. Mom, dad, aunt, sister is NOT a personal name

### MUSIC_GENRE
1. Fuji: A popular Nigerian musical genre that originated from the Yoruba people.
2. Gospel: A genre of Christian music.
3. Rock: A broad genre of popular music that originated as “rock and roll” in the United States in the late 1940s and early 1950s.
Old songs are not genres- Do not annotate them

### ARTIST_NAME
1. Fela: Refers to Fela Kuti, a Nigerian multi-instrumentalist and pioneer of Afrobeat music.
2. Beyoncé: An American singer, songwriter, and actress.
3. Mozart: Wolfgang Amadeus Mozart, an influential classical composer from Austria.

### HOTEL_NAME
1. Radisson: A global hotel chain known for its upscale accommodations and services.
2. Marriott: A worldwide hospitality company with a broad range of hotels and related services.
3. Hilton: A global brand of full-service hotels and resorts. You can annotate Radisson Hotel as a whole.

### MEAL_PERIOD
1. Breakfast: The first meal of the day, typically eaten in the morning.
2. Lunch: A meal eaten around midday.
3. Dinner: The main meal of the day, usually eaten in the evening.

### PAYMENT_COMPANY
1. Paypal: An American company operating a worldwide online payments system.
2. Stripe: An Irish-American financial services and software as a service (SaaS) company.
3. Visa: A multinational financial services corporation known for its credit and debit cards.
Not to be confused with account type.

### CONTINENT
1. Africa: The second-largest and second-most-populous continent on Earth.
2. Europe: A continent located entirely in the Northern Hemisphere and mostly in the Eastern Hemisphere.
3. Asia: The largest and most populous continent, located primarily in the Eastern and Northern Hemispheres.

### AIRPORT_NAME
1. Bole Addis Ababa International Airport: The main international gateway to Addis Ababa, Ethiopia.
2. Heathrow Airport: A major international airport in London, United Kingdom.
3. John F. Kennedy International Airport: A major international airport in New York City, United States.

### SUPERMARKET
1. Shoprite: A leading food retailer in Africa with stores in several countries.
2. Walmart: A large multinational retail corporation operating a chain of hypermarkets.
3. Tesco: A British multinational groceries and general merchandise retailer.

### STATE/PROVINCE
1. Quebec Province: A province in eastern Canada, the largest in area and second-largest in population.
2. Ogun State: A state in southwestern Nigeria.
3. California: A state in the western United States, known for its diverse geography and large economy.

### NUMBER
1. 10: A numerical value, often used to denote quantity or ranking.
2. 20: A numerical value, commonly used to signify quantity or sequence.
3. Fifty-four: non-numeric should be annotated as a number.

### NATIONALITY
1. Nigerian: Pertaining to Nigeria or its people.
2. Kenyan: Pertaining to Kenya or its people.
3. American: Pertaining to the United States of America or its people.

### CALENDAR_EVENT
1. Football Match: A scheduled competitive game of football (soccer).
2. Concert: A live music performance.
3. Wedding: A ceremony where two people are united in marriage. Christmas, Valentines day, birthdays, etc

### SHOPPING_ITEM
1. Shoe: A covering for the foot, typically made of leather, having a sturdy sole and not reaching above the ankle.
2. Shirt: A piece of clothing worn on the upper body, typically with sleeves and a collar.
3. Laptop: A portable personal computer with a screen and alphanumeric keyboard.
Not to be confused with Food items

### SONG_NAME
1. African Queen: A popular song by Nigerian artist 2Baba.
2. Thriller: A song by Michael Jackson from his album of the same name.
3. Shape of You: A song by Ed Sheeran.

### CAR_TYPE
1. BMW: A German multinational company that produces luxury vehicles and motorcycles.
2. Sedan: A passenger car in a three-box configuration with separate compartments for the engine, passenger, and cargo.
3. SUV: A sport utility vehicle, typically equipped with four-wheel drive for on- or off-road ability. Ambulance, Fire truck are not car types.

### PLACE
1. Tourist Attractions: Places of interest that draw visitors due to their cultural, historical, natural, or recreational significance. Examples include the Eiffel Tower in Paris, a global cultural icon of France, and the Grand Canyon in Arizona, known for its immense size and its intricate and colorful landscape.
2. Museums: Institutions that collect, preserve, and display objects of historical, cultural, artistic, or scientific importance. Examples include the Louvre Museum in Paris, which houses a vast collection of art, and the Smithsonian National Museum of Natural History in Washington, D.C., known for its exhibits on natural history and anthropology.
3. Mall: A large indoor shopping complex featuring a variety of retail stores, restaurants, and entertainment facilities. Examples include the Mall of America in Minnesota, which is one of the largest malls in the United States, and the Dubai Mall in the UAE, known for its luxury shops and attractions like the Dubai Aquarium and Underwater Zoo.

4. Park: A public area set aside for recreation and enjoyment, often featuring green spaces, playgrounds, and walking paths. Examples include Central Park in New York City, a vast urban park offering numerous recreational activities, and Hyde Park in London, known for its historical significance and open-air concerts.

With this entity, only annotate if entity is named explicitly, e.g Name of airport, museum or mall is not and nt just “mall”, “airport” etc

PS: Do not skip any annotations, if there is nothing to annotate, submit and go to the next one.

Metareview score: 4
Metareview: The paper introduces Injongo, a slot-filling and intent detection benchmark dataset covering 16 African languages (i.e., Amharic, Ewe, Hausa, Igbo, Kinyarwanda, Lingala, Luganda, Oromo, Shona, Sesotho, Swahili, Twi, Wolof, Xhosa, Yoruba, Zulu) across 5 domains, 23 slots, and 40 intents. 

The dataset is carefully designed to reflect locally relevant entities and topics, and the annotation is performed by three native speakers for each language. The authors included a number of experiments using Injongo with fine-tuned encoder models, encoder-decoder models, and a LLM2Vec model, and with LLMs in a zero-shot, few-shot, and fine-tuning settings.
Summary of reasons to publish: The paper contributes to increase the representation of African languages in the NLP community, and it does so by presenting a carefully designed benchmark dataset backed up by a detailed description of the dataset creation process. Reviewers i) agree that the work is central to enable research on those languages, and recognize ii) the quality of annotation (IAA and process), and iii) the attention to culturally-relevant entities/slots in the work. The dataset will be also released under CC BY 4.0 license. The experiments are comprehensive (across model types and regimens) and all reviewers refer to this as a strength of the work.