BIG-C: a Multimodal Multi-Purpose Dataset for Bemba
Year: 2023

Abstract:
We present BIG-C (Bemba Image Grounded Conversations), a large multimodal dataset for Bemba. While Bemba is the most populous language of Zambia, it exhibits a dearth of resources which render the development of language technologies or language processing research almost impossible. The dataset is comprised of multi-turn dialogues between Bemba speakers based on images, transcribed and translated into English. There are more than 92,000 utterances/sentences, amounting to more than 180 hours of audio data with corresponding transcriptions and English translations. We also provide baselines on speech recognition (ASR), machine translation (MT) and speech translation (ST) tasks, and sketch out other potential future multimodal uses of our dataset. We hope that by making the dataset available to the research community,1 this work will foster research and encourage collaboration across the language, speech, and vision communities especially for languages outside the "traditionally" used high-resourced ones.

heading: 1 Introduction
text: The Bemba language, spoken by over 10 million people in Zambia and other parts of Africa, is a rich and vibrant language with a unique cultural heritage. However, despite its significance, Bemba is a dramatically under-resourced language, lacking in high-quality language data and resources for natural language processing (NLP) experiments and for the development of language technologies. With this work, we address this issue by creating a new multimodal dataset for Bemba. Our goal is to improve the accuracy and effectiveness of NLP systems for speakers of Bemba and support research in this under-served language.
While most datasets are constructed with a specific task in mind and tailored to its characteris-
1All data and code are publicly available: https:// github.com/csikasote/bigc.
tics, we aim to provide a path towards building multi-purpose datasets. Under a limited budget, we hypothesize that the ideal scenario is to create datasets that can be useful for developing multiple language technologies for both practical applications and also facilitate cutting-edge NLP research on many dimensions. Our hope is that such datasets will aid in bridging the ever-widening language divide both in terms of data availability (Joshi et al., 2020) and NLP research (Blasi et al., 2022), and make language technologies more accessible for speakers of Bemba.
In this work, we present our methodology and results of creating a new multimodal dataset for Bemba, and demonstrate the potential of this dataset to develop NLP systems and support NLP research. Our dataset will fill multiple roles: enable development of fundamental tools such as speech recognition, speech and text translation systems for Bemba; serve as a benchmark for academic and in-
2062
dustry research even as NLP for low-resource and under-represented African languages gets developed; facilitate research in language grounding and multimodal model development, or building context-based dialogue agents, among other possible use cases. To our knowledge this is the first such dataset of its kind for any Zambian and possibly African language. We hope that it will provide an example of how to create a multi-purpose dataset in an under-served language to facilitate its coverage by multiple technologies.
The rest of the paper is structured as follows: in Section 2, we briefly introduce the Bemba language discussing any currently available resources. In Section 3, we summarise work related to multimodal tasks and existing datasets. In Section 4, we provide a description of the BIG-C dataset and the methodology used, and in Section 5, we provide baseline experiments for some NLP tasks.

heading: 2 The Bemba Language
text: Bemba, also known as IciBemba or Cibemba, is a Bantu language native to Luapula, Muchinga and Northern provinces of Zambia. It is also spoken in other urban parts of the country like Copperbelt, Central and Lusaka provinces. It is estimated that Bemba is spoken by over 30% of the population of Zambia as either the first or second language, making it the language with the most speakers in the country (Kapambwe, 2018). A map of Bemba usage in Zambia is provided in Appendix Figure 3.
The Bemba language has a number of dialects and the main varieties are: Standard Bemba also Central Bemba, Aushi, Bisa, Chishinga, Lamba,
Lala, Luunda, Ngumbo, Swaka, Tabwa and Unga. These dialects show minor differences in phonology, morphology and vocabulary(Spitulnik and Kashoki, 2001; Spitulnik and Kashoki., 2014). In this work, we focus on the Standard Bemba dialect, i.e., the one spoken in urban centers around the country.
Datasets for Bemba For ASR, to the best of our knowledge, there is only a single dataset publicly available for Bemba, BembaSpeech (Sikasote and Anastasopoulos, 2022). It contains 24 hours of read-styled speech data recorded from text mainly sourced from various source but mainly literature books. The low resource nature of the BembaSpeech (Sikasote and Anastasopoulos, 2022) dataset makes it difficult to build usable ASR system for Bemba. For machine translation (textto-text), there is not a single dedicated dataset for Bemba. However, there exist some parallel text-to-text data in multilingual datasets such as JW300 (Željko Agic and Vulic, 2020) and in evaluation benchmarks such as NTREX-128 (Federmann et al., 2022) and FLORES-200 (NLLB Team et al., 2022). The text in the JW300 (Željko Agic and Vulic, 2020) is mostly religious as it is derived from the Bible text. For speech translation (speechto-text; ST), to our knowledge, no prior work or Bemba dataset exists. This essentially renders it impossible to build a ST system where Bemba is a source or target language. The same is true for multimodal and dialogue datasets: there is no multimodal or dialogue-related dataset for any Zambian language that would enable development of multimodal systems. Our work aims to fill these gaps.

heading: 3 Related Work
text: In the recent years, NLP, speech processing (SP) and computer vision (CV) fields have rapidly advanced, with computational models’ performance achieving new heights on a wide range of downstream tasks. This, to some degree, can be attributed to factors such as the emergence of pre-trained models leveraging self-supervised learning, the availability of large-scale datasets, and increased large-scale computational infrastructure (Hirschberg and Manning, 2015). In NLP, language models like BERT (Devlin et al., 2019), T5 (Raffel et al., 2020), GPT3 (Brown et al., 2020) and XLM-R (Conneau et al., 2020), pretrained on massive text datasets such as C4 (Raffel et al., 2020), mC4 (Xue et al., 2021) and BooksCorpus (Zhu et al., 2015) among others, have lead to significant performance improvements on several language understanding and generation downstream tasks. Likewise, for speech processing, the unsupervised pretraining of models like wav2vec2.0 (Baevski et al., 2020) or XLS-R (Babu et al., 2021) – having been pretrained on publicly available speech datasets such as VoxPopuli (Wang et al., 2021), MLS (Pratap et al., 2020), Commonvoice (Ardila et al., 2020), BABEL (Punnakkal et al., 2021) among others, have led to advances on speech downstream tasks like ASR (Babu et al., 2021) and ST. In computer vision, deep learning models like DeepCNN (Simonyan and Zisserman, 2015; He et al., 2016) have become the de facto solution for standard vision problems like object recognition (He et al., 2016), image classification (Krizhevsky et al., 2017), or semantic segmentation (Shelhamer et al., 2017).
Since these neural models are conceptually (and architecturally) quite similar they have also enabled the integration of multiple modalities, with models such as ViLBERT (Lu et al., 2019), UNITER (Chen et al., 2020), Unicoder-VL (Huang et al., 2019) able to jointly model the relationship between text and image modalities resulting into breakthroughs across a myriad of tasks such as imagetext retrieval/search (Frome et al., 2013; Huang et al., 2020), image or video captioning (Biten et al., 2019), and vision-question answering (VQA; Agrawal et al., 2017; Nam et al., 2017). A crucial necessary component for all of the above, of course, is the availability of relevant datasets. Below we discuss works that go beyond the collection of raw datasets that are used for self-supervised learning.
Dialogue In the recent past, a lot of work has been focused on dialogue datasets. On one hand there exist goal-oriented dialogue datasets, such as the case of the Ubuntu dialogue corpus (Lowe et al., 2015), the largest corpus of dialogues (almost 1 million mainly 3-turn dialogues in English) for the specific topic of troubleshooting Ubuntu problems. On the other hand, open ended conversations, such as those on the CALLHOME/CALLFRIEND (Canavan et al., 1997) or Fisher corpora (Cieri et al., 2004), often leads to uninteresting conversations. Grounding the dialogue to event-centric images and potentially a specific scenario constrains the topic of conversation to event-rich and contentful utterances.
Multimodality Multimodal works combining visual and language information typically focus on image captioning and visual question answering (Antol et al., 2015). For example, the IAPR TC-12 dataset (Grubinger et al., 2006) provides images with titles and descriptions (mostly in English, German, and Spanish), as do commonly used datasets like MSCOCO (Lin et al., 2015) and Flickr30K (Plummer et al., 2015). Flickr8K Audio (Harwath and Glass, 2016) extended a subset of the Flickr images with audio, by crowdsourcing readings of the English captions, while Multi30K (Elliott et al., 2016) further extended Flickr30K with German translations and annotations. Wikipedia-based Image Text (WIT) Dataset (Srinivasan et al., 2021) provided large multilingual coverage (108 languages) based on 11.5M images and captions from Wikipedia. More recent, Hausa Visual Genome (HaVG; Abdulmumin et al., 2022) provided over 30K parallel descriptions in English and Hausa of images from the Hindi Visual Genome (HVG; Parida et al., 2019). The dataset was created by automatically translating the English descriptions of the images in the HVG to Hausa using Google Translate2 and postedited by crowd-sourced Hausa volunteers. Similarly, BAN-Cap (Khan et al., 2022) provides over 40K human-annotated parallel English-Bangla image description pairs based on 8,091 images from Flickr8K (Harwath and Glass, 2016). Lastly, the Bloom Library (Leong et al., 2022) provides a set of multilingual datasets for language modeling, image captioning and visual-story telling tasks containing more than 110K image captions for over 90K images in 351 languages. It also provides a
2https://translate.google.com/
speech dataset with 428 hours of speech data for speech synthesis/recognition tasks covering 56 languages.
Beyond captioning tasks, the dialog component was first explored by Das et al. (2017), who extended the VQA scenario collecting sequential questions grounded on images. Mostafazadeh et al. (2017) went beyond goal-oriented dialogue to collect image-grounded conversations (contrasting this to open-ended dialogue research). More recently, the Image-Chat dataset (Shuster et al., 2020) collected open-ended conversations grounded in images with a focus on engagement, by assigning desired style traits to the speaker.
Discussion There are notable limitations with most publicly available multimodal datasets. To make comparisons easy, we outline most relevant works in Table 1. While the list shown there is non-exhaustive, these limitations can be grouped in terms of language coverage, modality composition, tasks supported i.e., single-purpose or multipurpose tasks. To give more context to this categorization:
• In terms of languages, they cover only a handful of high-resourced languages like English. • In terms of modality composition, the majority only contain image and text modalities, ignoring the audio component. • With regards to tasks, the majority are meant for a single-purpose task such as image captioning.3
In contrast, our work presents a multimodal but also multi-purpose dataset for Bemba. Our aim is for BIG-C to be the first-of-its-kind dataset for an under-served language that can simultaneously serve as:
• a monolingual dataset for Bemba e.g., to be used for training language models on this under-served language; • a parallel dataset to allow for building and evaluating machine translation solutions; • an image captioning dataset with image descriptions in Bemba; • an image-grounded dialogue dataset; • a benchmark for any combination between
the above modalities e.g., one could use our dataset to evaluate image-grounded dialogue translation systems.
3An exception to this is the Bloom Library (Leong et al., 2022). But note that it lacks representation of any Zambian language among the covered languages.
We achieve this through careful instructions and data collection practices, outlined in Section §4.

heading: 4 Dataset Description
text: Description The dataset consists of a parallel corpus of speech and transcriptions of image-grounded dialogues between Bemba speakers and their corresponding English translations. It contains 92,117 spoken utterances (complete and incomplete dialogues), amounting to 187 hours of speech data grounded on 16,229 unique images. There are 16,697 complete 5-turn unique dialogues grounded on 14,551 unique images. Of the total 16,697 complete dialogues, 2,146 are unique dialogues grounded on duplicated images, each recorded by unique pairs of speakers. A second set of dialogues is comprised of 2,314 incomplete dialogues missing one or more utterances as a result of the preprocessing step that involved removing all audio files that are silent and corrupted. The sum of utterances that make up the incomplete dialogues is 8,632 of the total 92,117 utterances. All audio files are encoded in Waveform Audio File format (WAVE) with a single track (mono) and sample rate of 16kHz. In Table 2, we provide basic dataset statistics.
Source of images We randomly selected images from the Flickr30K (Plummer et al., 2015) dataset, a publicly available multimodal dataset for vision and language that has become a standard benchmark for sentence-based image descriptions.
Speakers To record conversations, we recruited 86 speakers of the Bemba language; 60% male and 40% female, based on their competency to speak, read and write the language. Based on the metadata information supplied by participants, we summarise the characteristics of our speakers as follows: • Age: the majority of the speakers (98%) were
youth whose age falls between 20 and 35 years old with the 2% being over 35 years old. • Education: all speakers had some form of secondary education; 90% of the participant were either pursuing or recently graduated with a college/university degree; and the rest 8% had only completed high school. • Language(s): all speakers were bilingual; with 90% indicating Bemba as their first language and Nyanja as the majority non-English second language. • Regions: in terms of regional representations, over 90% of the speakers were drawn from Lusaka, Central, and Copperbelt regions; with small representations from Muchinga and Northen provinces. This in effect indicates that the dataset is composed of the current ’urban’ Bemba variety. • Racial diversity: the composition of our participants lacks racial diversity, as all speakers are identified as black.
Recording The speakers were randomly paired with gender-balancing in mind. Each pair was allocated 250 images to create 5 sentence-turn conversation per image for each recording session. There was no restriction to what each pair would converse about on an image. The participants were encouraged to be creative. However, the conversation starter (speaker 1) was instructed to first describe the image, so as to give context to the conversation (and essentially provide data for the image captioning component of our dataset). We provide the sample instructions that were given to the annotators in Appendix A. All recordings were conducted in minimally controlled conditions. The pairs recorded as per their comfort, we therefore expect that some spoken utterances have background noise. All participants used the LIG-AIKUMA (Gauthier et al., 2016) mobile application, using the ‘elicitation by image’ mode to record spoken utterances.
Transcribers To transcribe the audio data generated from the image-grounded conversations, we re-
cruited 93 participants, who in their majority were students of the University of Zambia. All were competent Bemba speakers and writers. As shown in Table 2, 92,117 spoken utterances were transcribed representing 187 hours of Bemba speech data.
Translators To translate a subset of the transcriptions to English, we recruited 115 participants with experience in translating Bemba text to English or vice versa. Public education in Zambia is conducted in English, hence we are confident in a minimum translation quality.
Splitting We have split the dataset into training, validation and testing sets following the original splits in the Flickr30K (Plummer et al., 2015) dataset according to the images. See Table 3 for more details.
Data quality Several measures were set up during the data collection process to ensure quality submissions from project participants; speakers, transcribers and translators. First, at recruitment stage for audio recording, we considered only competent Bemba speakers with ability to speak, read and write in Bemba. All the speakers underwent a training exercise to make sure they understood and followed instructions of how to go about the task of creating and recording multi-turn conversations using the Lig-Aikuma (Gauthier et al., 2016) mobile application. For the transcriptions, we retained good number of the speakers - over 50% to also participate in transcribing the audio files at transcribing stage. In addition, we recruited validators, who together with the authors of this study checked and verified manually every submission made by the participants at every stage of the process. All audio files that were deemed to be of low quality i.e., silent, corrupted and inaudible due to background noise, were removed as part of data pre-processing at the quality assurance and validation stage.
Last, during the translation stage, besides the ability to speak, read and write, we recruited participant who had experience with translating Bemba text to English as translators. Most of the participants had prior experience as professional or volunteer translators.
Availability The dataset is made available to the research community licensed under the Creative Commons BY-NC-ND 4.0 license and can be ac-
cessed at our Github repository.4 We do plan to keep a small held-out portion unpublished, to be used in future shared tasks or as part of leaderboards that require hidden test sets to ensure a fair measure of task progress.

heading: 5 Baseline Experiments
text: In this section, we detail some baseline experiments carried out to demonstrate the potential of the dataset. We provide unimodal baselines using the train-validation-test splits in Table 3 on the following tasks: ASR for Bemba, MT and ST of Bemba utterances to English text.
Data preprocessing For ASR and ST, similar to Wang et al. (2020a), all text i.e., transcriptions and translations, we lower the cases and remove punctuation except for apostrophes, and build 1K unigram character vocabularies with 100% coverage of all the characters using SentencePiece (Kudo and Richardson, 2018) without pre-tokenization. We extract 80-dimensional log-mel scale filterbank features from Bemba utterances using a 25ms window size and 10ms window shift using torchaudio.5 The features are normalized to 0 mean and 1.0 standard deviation. All models are trained without an auxillary language model.
Model Architecture We use the small Transformer (Vaswani et al., 2017) base architecture with 71 M parameters, s2t_transformer_s, having 12- layers encoder, 6-layers decoder, and hidden dimension D=256 to train end-to-end (E2E) ASR and ST models using FAIRSEQ S2T Toolkit (Ott et al., 2019; Wang et al., 2020b). Models are trained on a single NVIDIA Tesla P100 GPU using the Google Colab+ platform.
4https://github.com/csikasote/bigc 5https://github.com/pytorchaudio

heading: 5.1 Automatic Speech Recognition
text: For the ASR baseline model for Bemba, we trained the model for 500 epochs using the Adam optimiser (Kingma and Ba, 2015) with 10K warm up steps. The model is optimised to minimise the label_smooth_cross_entropy criterion function using the learning rate coefficient of 2e-3. For decoding, we use the beam search algorithm with a beam size of 5. We use the average of the last 5 checkpoints for evaluation. In Table 4, we report the model performance on the Test set using word error rate (WER) metric.

heading: 5.2 Speech Translation
text: For speech to text translation of Bemba spoken utterances to English text, we use the same model architecture as ASR. The model is trained with same configuration as the ASR model except we use the learning rate coefficient of 3e-4. Similarly, we use the beam search algorithm with beam size of 5 for decoding. We use the best checkpoint to evaluate the model on the test set. We report the detokenised case-sensitive BLEU (Papineni et al., 2002) using sacreBLEU (Post, 2018) in Table 4.
Evaluation We use beam search with a beam size of 5 for decoding. We use the average of the last 5 checkpoints to evaluate both ASR and the best checkpoint saved for ST model. We report the results in Table 4. For ST, we report detokenised case-sensitive BLEU (Papineni et al., 2002) using sacreBLEU (Post, 2018) and word error rate (WER) for ASR.
Results discussion For both ASR and ST, we consider the results obtained decent for the size of our dataset and the basic training configurations of our baseline models, which are without auxillary models, and mostly relied on default settings in the FAIRSEQ S2T implementation. We believe the results can be improved upon, and we leave
the full exploration of the best configurations to future work. We encourage the community to improve upon these baselines, for instance, by exploring cross-lingual transfer learning by leveraging large scale multilingual pretrained models like XLS-R (Babu et al., 2021) and Whisper (Radford et al., 2022).

heading: 5.3 Machine (Text) Translation
text: For Machine Translation we rely on the results of the WMT Shared Task on Large Scale Machine Translation Evaluation for African Languages (Adelani et al., 2022). In particular, we use the same system and approach as Alam and Anastasopoulos (2022), which ranked third in the Shared Task.6 These models are based on the DeltaLM (Ma et al., 2021) pre-trained model, which is the adapted through fine-tuning on 24 African languages (note that Bemba is not included), as well as English and French. The adaptation happens using adapter units (Pfeiffer et al., 2020) organized in a hierarchy following language typology (Faisal and Anastasopoulos, 2022) so that similar languages share similar "family" adapters. We also compare against a baseline that finetunes the whole DeltaLM model on our training set.
Here, we only use our training data to fine-tune the adapter units for Bemba, and evaluate on both our test set as well as on the publicly available FLORES-200 devtest (NLLB Team et al., 2022). The results are presented in Table 5, where we report sentencepiece-BLEU (NLLB Team et al., 2022) with the FLORES-200 tokenizer. In general, translating into English seems to perform well, especially for the phylogeny-based model.
The difference between the performance in the two test sets can be explained by the difference of domains. All BIG-C training data are from dia-
6We note that this is the best-performing system that is publicly available – to our knowledge, the first two performing systems were industry submissions without publicly released models or code.
logues, while the FLORES-200 evaluation dataset is comprised of translated Wikipedia articles. Of course, larger and more diverse data collection in the future should help mitigate these issues and allow us to build general translation systems capable of handling various domains adequately.

heading: 5.4 Other Tasks
text: The authors of this study unfortunately lack the financial and compute resources, as well as required expertise, to provide baseline results for additional multimodal tasks. Nevertheless, we devote this subsection to outlining some other potential downstream uses of BIG-C. • Image Captioning The dataset could be used
directly for image captioning in Bemba (or English), by pairing the images with the first utterance of the conversation, which will largely function as a caption by design. • Multimodal Language Modeling Similarly, the corpus could be used for language and vision pre-training, and particularly to study multilingual approaches (in a field that has largely focused solely on English). • Multimodal Dialogue Modeling Similar to other image-grounded tasks (see §3), one could use to BIG-C to study dialogue, with a focus on open-ended but still grounded conversation. One could also use our dialogues as (pre-)training data for chatbots in Bemba, which could then potentially be adapted to handle specific goals or domains with fewer in-domain data. • Multimodal Translation In the experiments above we did not take advantage of the image when translating. One could explore whether multimodal machine translation approaches (Barrault et al., 2018, ; inter alia) could improve downstream performance in these resourcescarce settings. • Cross-Cultural NLP A major limitation of our dataset (also discussed in the relevant Limitations section) is that most of the images that we use are not particularly relevant to the Zambian or sub-Saharan African context. We plan to mitigate this issue by collecting an addendum to BIG-C with images crowd-sourced in Zambia. Nevertheless, this limitation is simultaneously an opportunity to study cross-cultural understanding as well as the priors/assumptions/biases that speakers with a certain background exhibit. To highlight this potential, we show some additional
interesting examples from BIG-C in Figure 2. In the top-left example, the first speaker’s utterances reveal several assumptions: that the musicians are “Indian" (likely correct, since this image is located in India); that they “are on a roof" (correct); that they “sing religious songs" (unsupported); or that “it’s time to congregate and pray" (unsupported). In the example in the top-right, the first speakers assumes the image is “by the riverside", and not e.g., by the seaside or lakeside.7

heading: 6 Conclusion
text: In this paper, we presented a multimodal corpus comprised of multi-turn dialogues between speakers of the Zambian language, Bemba, grounded on images, transcribed and translated into English. It contains over 92,000 utterances/sentences, 180 hours of speech grounded over 16,000 images. The dataset aims to fill multiple roles: enable development of fundamental tools like speech recognition, machine translation and speech-to-text translation systems between Bemba and English; serve as a benchmark for academic and industry research; and to facilitate research in language grounding and multimodal model development towards building context-based dialogue agents, among other potential use cases. We have also provided baseline for ASR, MT and ST task.
In future work, we plan to conduct multimodal baseline experiments, as well as attempt to mitigate the image diversity limitation by collecting an addendum to BiG-C using images taken locally in Zambia. In addition, we plan to further expand to other Zambian languages such as Tonga, Tumbuka, Chewa, or Lozi, by translating the existing dataset (creating an n-way parallel corpus for Zambian languages) and by direct data collection. Further down the roan we plan to study the dialectal varieties of Bemba and the other languages, by collecting contrastive datasets from different regions of the country.
7Note that Zambia is a land-locked country.
Limitations
We observe the following limitations with the dataset:
• Language Diversity: In terms of number of languages, the presented dataset only covers two languages; Bemba and English. • Image Diversity All the images used in this dataset were obtained from Flickr30K image dataset. Therefore, in terms image composition, our dataset is limited to the image diversity in the Flickr30K dataset. It mostly lacks images that could be considered as "culturally relevant" ones for the Zambian or generally sub-Saharan African context. We plan to mitigate this in future work.
Ethics Statement
We make the following declarations for the ethics statement:
• Research: This work was carried out mostly in Zambia, and most authors are native speakers of Bemba who also worked as validators for the data collection process. • Participants: All project participants; transcribers, translators and speakers/recorders were informed about the goals of the project and they signed consent forms to participate. All participants were monetarily compensated at around $20/h for all their work. • Personal Identifiable Information: All information that can potentially be regarded as PII such as names of participants, IDs have been removed for anonymity and will not be released with the dataset. • Copyright: There is no potential copyright matters associated with the data contained in this dataset. We are publicly releasing the dataset under the Creative Commons BY-NCND 4.0 license.

heading: Acknowledgements
text: We would like to thank all the participants that were involved at different stages of the dataset creation
process. We would also like to thank Desmond Elliott and Graham Neubig for insightful conversations and constructive feedback at earlier stages of our project. This project would not have been possible without generous funding by the LacunaFund. Antonios Anastasopoulos is also supported by NSF-NEH grant BCS-2109578.

heading: A Language Map of Zambia

heading: A Participant Training Exercise
text: The following instructional steps depict the participants exercise/tutorial during a training exercise session before actual recording. The instructions were given to a pair of participant. The objective was to create a text conversations for 5 sample images in a specified image folder using Google Sheets. The recording session followed the same process, except with additional instructions involving the use of the LIG-Aikuma (Gauthier et al., 2016) app.
• STEP 1: Open the first image in your image folders. If you are P16, for example, Go to P1_Session_01 > Image7501 > Speaker_01 [If you are Speaker 1] or Speaker_02 [If you are Speaker 2]. Open any of the images in the folder.
• STEP 2: While you are able to view the image, open the spreadsheet. Now that you have both image and spreadsheet opened.
• STEP 3: Speaker 1 should enter the image number (in this case, 7501) in cell A3.
• STEP 4: Speaker 1 should describe what is in the image by a single sentence in cell B3. The description should be a single sentence giving a clear mental picture of what is in the image.
• STEP 5 : Speaker 2 should be able to respond to Speaker 1 by entering their response in C3. The response can be a question, a statement or an addition to what Speaker 1 said. As long as it’s a sentence in Bemba. Remember this is a conversation and it should be able to naturally flow.
• STEP 6: Speaker 1 should complete cell D3 with a sentence in response to what Speaker 2 texted in cell C3.
• STEP 7: Speaker 2 should put a response in cell E3 in response to what Speaker 1 texted in cell D3.
• STEP 8: Speaker 1 closes the conversation with a sentence, however it may be in cell F3.
• STEP 9: If you have successfully generated the conversation/dialogue in the spreadsheet for the first image, then go ahead and do so for the next 4 images.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: After Section 6
3 A2. Did you discuss any potential risks of your work? 6
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? 5
3 B1. Did you cite the creators of artifacts you used? 5
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? 1,5,6

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 1,5,6

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? 6

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? 4
7 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 4
C 3 Did you run computational experiments? 5
7 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Using default parameters and recipes
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 5, No hyperparam search
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Not applicable. Left blank.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 5
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? 4
7 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? In Bemba

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? 4

heading: 3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to
text: crowdworkers explain how the data would be used? 6

heading: 3 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board?
text: 4,6

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: 4

heading: Future Work
text: In future work, we plan to conduct multimodal baseline experiments, as well as attempt to mitigate the image diversity limitation by collecting an addendum to BiG-C using images taken locally in Zambia. In addition, we plan to further expand to other Zambian languages such as Tonga, Tumbuka, Chewa, or Lozi, by translating the existing dataset (creating an n-way parallel corpus for Zambian languages) and by direct data collection. Further down the roan we plan to study the dialectal varieties of Bemba and the other languages, by collecting contrastive datasets from different regions of the country. 7Note that Zambia is a land-locked country. Limitations
We observe the following limitations with the dataset:
• Language Diversity: In terms of number of languages, the presented dataset only covers two languages; Bemba and English. • Image Diversity All the images used in this dataset were obtained from Flickr30K image dataset. Therefore, in terms image composition, our dataset is limited to the image diversity in the Flickr30K dataset. It mostly lacks images that could be considered as "culturally relevant" ones for the Zambian or generally sub-Saharan African context. We plan to mitigate this in future work