S peech R eporting Corpus: annotated corpora of W est A frican traditional narratives (2023)

Abstract: This paper describes the SpeechReporting database, an online collection of corpora annotated for a range of discourse phenomena. The corpora contain folktales from 7 lesser-studied West African languages. Apart from its value for theoretical linguistics, especially for the study of reported speech, the database is an important resource for the preservation of intangible cultural heritage of minority languages and the development and testing of cross-linguistically applicable computational tools.

## introduction recent decades have seen an upsurge of interest in issues of language extinction, leading to increased efforts to describe and document the world's endangered languages. the major adverse effects of language endangerment are also associated with loss of different forms of traditional knowledge @xcite . the speechreporting corpus @xcite explores the relationship between specific discourse practices that represent the nucleus of the transmission of traditional knowledge and the linguistic strategies associated with it, centering on one particular problem: discourse reporting in traditional oral storytelling in west africa, the "oral continent par excellence" and the homeland of a rich and vibrant oral tradition @xcite . the article is structured as follows: in section 2 we discuss why the speechreporting database is particularly relevant for west africa. section 3 is dedicated to database composition. our workflow and tools are described in section 4, while section 5 shows some basic principles of annotation of reported discourse. in section 6, we show how the online interface of the corpus works. in section 7, we illustrate how the corpus can be used for dissemination among linguistic communities in order to archive and help preserve intangible cultural heritage. section 8 concludes the paper. ## west african storytelling traditions in traditional rural societies of west africa, the acute feeling of loss is related to the diminishing role played by culturally significant discourse practices: even in communities that retain traditional social organization and economy, modern western cultural practices seep into daily life with new forms of entertainment (television, radio broadcasts) and education (compulsory western-style schooling). under the pressure from these new practices, traditional forms of knowledge transmission -including techniques of storytelling and instruction -become unimportant, and may eventually go out of use. in many local communities across africa, storytelling is more than a favorite pastime. viewed as a vital part of cultural heritage, it serves as a central medium for the transmission of cultural knowledge. storytelling traditions have accumulated special linguistic techniques that respond to the needs of specific practices of textual production and performance. as storytelling traditions vanish with older generations of speakers, they take along with them an array of linguistic tools on which such specialized techniques relied @xcite . while oral traditions of west africa have received considerable attention from anthropologists @xcite , their linguistic aspects have not been subject to systematic investigation. our knowledge of the special ways in which language is used in traditional genres is largely limited to observations of frequent use of special vocabulary and opening/closing formulae @xcite , singing @xcite @xcite , and various forms of repetition @xcite . the goal of our database is to start filling this gap using an interdisciplinary approach combining rigorous analysis of primary data with meticulous attention to genre characteristics and culture-specific contexts of textual production. ## database composition the speechreporting corpus contains multiple sub-corpora of traditional folk stories, annotated for a number of discourse phenomena using the elan-corpa software and tools @xcite . it is updated regularly with newly available data, including data from new languages. the project currently involves 11 different languages, of which 7 are spoken in west africa. all texts are transcribed, glossed, translated, and annotated. ## workflow and tools the project unites multiple collaborators that work in different frameworks and use different tools for data documentation and analysis. as a result, we had two basic workflows. in one, segmentation and transcription are done in saymore @xcite or @xcite . segmented and transcribed texts are glossed in toolbox or flex and then imported to elan in order to add annotations of reported discourse. the other workflow allows researchers to use only one software product, elan-corpa for segmentation, transcription, glossing and annotation of reported discourse. 2 annotated files are checked manually and by using elan tools @xcite , a collection of scripts that checks the consistency of labels and the structure of the elan files. the manual checking consists of, among other things, proofreading the free translations and looking out for irregularities in the glosses and the morphemic analysis. doublechecked files are uploaded to tsakorpus. collaborators could contribute to the project in various ways. since 2019, we have had 4 postdocs, 2 phd students, 6 research assistants and 8 non-contractual academic visitors working on the corpus. ## annotation of reported discourse annotation of reported discourse consists of four levels: the function of the construction's elements; the construction's syntactic type; the semantic type of the discourse report; and the encoding of participants within the discourse report. they correspond to four additional elan tiers (in our template, qt, rp, typ and par, respectively). figure 1 is an example of our annotation of a gizey sentence in an elan file. explained below are the basic principles of annotation that are relevant to searching in the corpus interface. 3 a reported speech construction consists of different elements; for example, in john said: hello! the reported utterance (hello!) is introduced by a clause describing the speech event (john said). in the gizey example in figure 1 , "she says" is expressed by a quotative, while "give millet; this red mare of mine..." is a discourse report. the semantic type of the discourse report is command. 4 different syntactic types of reported discourse constructions are visually represented by different frames. the types are defined by the elements the construction consists of. the syntactic type in the gizey example is quotative + discourse report. the elements referring to participants in the current or reported speech event are annotated in the participant tier. the gizey example contains a reference to the reported speaker (rs). in the tsakorpus interface, these annotations are reflected by background colors, frames, and pop-up windows. this is illustrated in figure 2 for a sentence in bandial (also known as jóola eegima), where the reported segment is in green and the speech event is in red. ## searching in tsakorpus equipping the annotated corpora with a web-based search interface makes them more accessible both to linguists and to language communities. we made our corpora available online with the help of the tsakorpus platform. 5 the platform was mostly developed independently of the project. however, a number of features were added specifically to accommodate the needs of the speechreporting database. search queries in the online interface are formed by clicking on buttons and filling out text fields. a single-word query can include constraints on the word, its lemma, its part of speech and/or its glosses. all fields can handle boolean functions (, for and, | for or, ~for not). word and lemma search can include regular expressions and provide instant suggestions when the user starts typing. multi-word queries consist of several single-word queries with additional distance constraints. when clicking "search sentences", the user gets randomly ordered search hits, split into pages. the sound associated with a particular search hit, if any, can be played by clicking on that hit. one limitation of tsakorpus is that its basic search unit is a sentence (or any sentence-like segment of text). it is not possible to search for units that are either larger than a "sentence", or smaller than a "sentence" but larger than a word. elan segments (which normally represent intonational units) were reinterpreted as "sentences" in tsakorpus. however, our discourse annotation often consists of multi-word spans that are either smaller than a sentence or transcend the segment sentence. in order to make them searchable, we add values of all discourse annotations that appear anywhere within a sentence as sentence-level metadata. this way, a query like "quotative and a word glossed as say" will return all sentences that contain both a quotative span and a word glossed as "say", but they will not necessarily overlap. this option was added to tsakorpus in the course of the project. nevertheless, the exact spans inside sentences that have discourse annotations are highlighted with figure 2: example of search within tsakorpus, a sentence in bandial (jóola eegima) different colors in the search hits. the online corpus contains all languages present in the discourse reporting database. when searching in the corpora, the user can choose between selecting a specific language and searching in all language subcorpora at once. in the latter case, the search query must include annotation that is uniform across the subcorpora. this includes annotation of reported discourse (see section 5) and part-of-speech tags in the ud format @xcite . currently, the corpus interface is available in english and russian. the french interface is under construction. ## dissemination target users of the speechreporting corpus are linguists and anthropologists who are interested in traditional narratives. besides academic uses, this corpus is a valu-able source of materials for language communities to keep their languages and linguistic traditions alive; first of all, by simply having online access to recorded narration sessions of some of their folktales. in addition, we make materials, such as storybooks, that the communities can use for educational purposes. moreover, our project has received additional funding from the humanités numériques et science ouverte program of the sorbonne nouvelle in paris for producing animated youtube videos of the recorded folktales and spreading them among the linguistic communities and wider audiences. furthermore, the availability of open-access annotated linguistic data of minority african languages is important for the development of machine learning based technologies, which currently under-represent these languages.