Semi-automatic discourse annotation in a low-resource language: Developing a connective lexicon for N igerian P idgin (2021)

Abstract: Cross-linguistic research on discourse structure and coherence marking requires discourse-annotated corpora and connective lexicons in a large number of languages. However, the availability of such resources is limited, especially for languages for which linguistic resources are scarce in general, such as Nigerian Pidgin. In this study, we demonstrate how a semi-automatic approach can be used to source connectives and their relation senses and develop a discourse-annotated corpus in a low-resource language. Connectives and their relation senses were extracted from a parallel corpus combining automatic (PDTB end-to-end parser) and manual annotations. This resulted in Naija-Lex, a lexicon of discourse connectives in Nigerian Pidgin with English translations. The lexicon shows that the majority of Nigerian Pidgin connectives are borrowed from its English lexifier, but that there are also some connectives that are unique to Nigerian Pidgin.

## introduction resources such as discourse-annotated corpora (e.g. pdtb, @xcite and connective lexicons (e.g. dimlex, @xcite have proven highly valuable in investigating discourse structure and coherence marking, but they are only available for a limited number of languages. one promising way to expand these resources is to project annotations from languages for which such resources are available to underresourced languages. the current paper sets out to investigate how automatic discourse parsing can be combined with manual annotation to construct a connective lexicon in a low-resource language. we apply this method to create naija-lex, a lexicon of nigerian pidgin, but the method can also be adapted to apply to other low-resource languages. pidgin and creole languages, such as nigerian pidgin (commonly referred to as 'pidgin' or 'naija'), tend to be less complex on a syntactic and morphological level than their lexifiers @xcite . this raises interesting questions with respect to their discourse complexity, such as: how is discourse structure realized in contact languages? and how does this affect discourse marking? in addition, because pidgins are still in an evolutionary stage, they are a testing ground for examining how connectives emerge. however, the scarcity of linguistic resources, such as discourseannotated corpora, for these languages makes it difficult to answer these questions. another contribution of this paper is the creation of a discourseannotated layer for a parallel corpus of nigerian pidgin, which can be used to study these questions. we adopt a semi-automatic approach using automatic connective identification and annotation projection, combined with manual annotation, to initialize a list of discourse connective candidates in nigerian pidgin. this allows us to evaluate how existing tools can be adopted to investigate lowresource languages. more specifically, we assess the usefulness of the pdtb end-to-end parser @xcite in identifying connectives in informal spoken english translated text, to source connectives in the target language. the main contributions of the present research are the following: section 2 discusses earlier work on automatic discourse annotation and lexicon construction, after which nigerian pidgin is introduced in section 3. section 4 describes the method used to identify the connectives in pidgin. we elaborate on the naija-lex lexicon that we created using this method in section 5 and evaluate our method in section 6. ## nigerian pidgin nigerian pidgin is a contact language spoken all over nigeria. although officially a pidgin, it is also often considered a creole (e.g., @xcite , as it is a relatively stable language with a fullydeveloped vocabulary and grammar. in addition to approximately 5 million native speakers, there are estimated to be almost 100 million speakers of nigerian pidgin as a second language. it is often also referred to simply as 'pidgin' by nigerian pidgin speakers themselves. english is one of the main lexifiers of nigerian pidgin, with many words having a similar form and meaning as the english origin, as can be seen in example (1). nigerian pidgin is also influenced by other european languages (e.g. portuguese, illustrated by sabi in the example above) as well as various indigenous languages it has been in contact with, such as hausa, igbo and yoruba. nigerian pidgin is characterized by focus constructions (see e.g. me in (1)), which are often indicated with the focus particle na @xcite . in addition, serial verb constructions are typical for nigerian pidgin. as a result, coordinating conjunctions are less frequent in nigerian pidgin than in english @xcite . an example of such a serial verb construction is provided in example (2), which has been translated in english with the coordinating conjunction and. this suggests that discourse relations might be expressed differently in nigerian pidgin. then we hustled, got money and went to give it to her. linguistic research and resources on nigerian pidgin are limited, although this has shifted in recent years. as part of a larger project studying the syntactic and prosodic structure of nigerian pidgin, @xcite are in the process of creating a 500,000 word corpus of spoken data, part of which is already available and will be used in the current study. we also note related work from the nlp community in studying pidgin to explore new approaches for low-resource natural language generation @xcite and translation @xcite . however, no previous research has examined the discourse structure of nigerian pidgin. ## naija-lex: a lexicon of nigerian pidgin connectives in total, 4,186 nigerian pidgin connective tokens were identified. they were retrieved based on 40 english connective types in the translation text. the nigerian pidgin connective tokens consisted of 77 types, which were categorized in 57 entries in the lexicon. naija-lex is made available online. 2 for each connective entry, the lexicon contains information on its frequency, alternative forms, syntactic category(/ies), english translation equivalents and non-connective usage. in addition, the various relation senses that the connective can signal are included, together with an example of the nigerian pidgin connective in every sense and the relation sense distribution. ## evaluation of the method 6.1 automatic connective identification in tt to evaluate accuracy of the first step in our pipeline we manually annotated the tt in the development set for presence of connectives, according to the criteria set by @xcite . connectives were identified in the english tt by the pdtb end-to-end parser with relatively high reliability (@xmath0.2%, @xmath1.6%, @xmath2.4%). this is lower than the parser's performance on the test set of the pdtb (i.e. 95.76%), but that was expected, because the parser was trained on text from the written newspaper domain. a closer examination of the performance per connective shows that precision is especially low on so, so that, also and later (<= 50%). in spoken language, so is often used as a discourse marker rather than a connective. 3 for so that, the majority of mistakes were caused by the parser identifying so with a demonstrative that as a connective. low recall was reached for afterwards and yet, which occurred infrequently and were not detected by the parser. the inaccuracy of automatic connective identification is thus mainly due to a small number of connectives. manual examination of these instances can improve the accuracy significantly. ## discussion the present research shows how extending discourse research to contact languages provides more insight in how discourse structure is realized crosslinguistically. we find a similar relational distribution between english and pidgin. moreover, we identified a relatively large variety of discourse markers in nigerian pidgin. taken together, this indicates that the discourse structure of pidgin is not necessarily less complex than in english, even though pidgins are generally considered to be less syntactically complex. furthermore, the analysis reveals that most of the discourse connectives in nigerian pidgin are derived from english discourse markers. however, discourse relations in pidgin are not always marked by lexical items from a limited set of syntactic categories (cf. @xcite : auxiliary verbs can also signal coherence relations in nigerian pidgin. we therefore recommend researchers working on identifying connectives in different languages to be flexible in what items might signal coherence relations. cross-linguistic variation should be taken into account when defining restrictions on what can constitute a discourse connective. in order to do so, more research on coherence marking in different languages (especially non-indo-european languages) is needed. nigerian pidgin is mainly a spoken language. this affects our analysis in several ways. firstly, the nature of spoken language influences the distribution of connectives and the relation senses they mark. a smaller number of connective types are used in spoken compared to written text @xcite . at the same time, connectives in spoken language are used in a wider variety of functions. this should therefore be taken into account when considering the frequency of connectives and their relation sense distribution in the lexicon. secondly, the performance of the parser is in-fluenced by the change in domain. a comparison with the parser's performance on a different english discourse-annotated spoken corpus @xcite revealed that the parser makes comparable mistakes in identifying connectives in dialogues by native speakers of english (see also @xcite . inaccuracies of the parser in the current study are thus for the most part due to the fact that our data is spoken. despite the fact that the process described in this paper contains a number of manual steps, we believe that this method can easily be extended to other low-resource languages. the annotations were for the most part done by the second author, who is semi-proficient in pidgin. in addition, comparing instances of connective candidates that were automatically identified in the corpus allowed a non-native speaker with little knowledge of the language (the first author) to manually verify the data. although the native speaker we consulted was a trained linguist, the additional required judgments can be also be obtained in such a way that no linguistic knowledge is required from the native speaker. furthermore, the required amount of manual annotations does not increase considerably with larger datasets. firstly, by annotating the dev set alone, only a few infrequent connectives had been missed (6 out of 77 connective variants) and annotating a small subset of the data helps to identify these cases. secondly, larger resources lead to a stronger reliability of statistical approaches. a combination of manual and automatic approaches therefore seems a fruitful way to construct resources for other under-researched languages as well. another challenge in identifying connectives in contact languages specifically, is the large amount of variation and the frequent use of code-switching. especially for an english-lexified language like nigerian pidgin, it can be difficult to distinguish between cases in which the utterance is english or nigerian pidgin. in fact, some of the connectives that we initially identified as st connectives were in fact english connectives that had been used during code-switching. frequency filters, which have also been adopted in fully-automatic approaches to cross-lingual connective identification @xcite , are therefore crucial. lexical items that only occur a few times or are only used by a small number of speakers should be checked manually. to distinguish between code-switched and infrequent connective uses, the context should be taken into account. if the connective is embedded in an english-only sentence, it is likely not a connective in the target language. the similarities between nigerian pidgin and english could also have been employed as an advantage. for example, as one reviewer suggests, levenshtein's distance can be used to find nigerian pidgin connective candidates that are similar to the english connective. however, there are a few drawbacks to such an approach. the nigerian pidgin version of an english connective sometimes diverged strongly from its english cognate (e.g. because -cos) and some of the nigerian pidgin connectives with an english origin were not cognates of the english connective, but of a synonym (e.g. the pidgin meaning-equivalent of instead is rader). levenshtein's distance might therefore not always be a reliable measure. more importantly, our goal was to establish the usefulness of existing resources to research discourse structure in low-resource languages in general, not english-lexified contact languages only. in addition, in searching for nigerian pidgin connectives, we did not want to create a bias towards english connective cognates, as one of our research goals was to examine how discourse connectives emerge and to establish which connectives would not originate from english. ## conclusion this study presents naija-lex, a lexicon of connectives in nigerian pidgin, which was constructed by combining automatic discourse parsing with manual annotation. in doing so, we showed that automatic discourse parsing is a valuable resource in constructing a discourse-annotated database to extract connectives. however, construction of a seed set based on a subset of the data and manual annotation of selected connectives and relation senses is necessary to obtain a reliable database. in addition, we made a first step to building a nigerian-pidgin discourse-annotated corpus by annotating st connectives that were explicit in the tt. we will extend this corpus with implicit relations in further work.