How to Parse a Creole: When Martinican Creole Meets F rench (2022)

Abstract: We investigate methods to develop a parser for Martinican Creole, a highly under-resourced language, using a French treebank. We compare transfer learning and multi-task learning models and examine different input features and strategies to handle the massive size imbalance between the treebanks. Surprisingly, we find that a simple concatenated (French + Martinican Creole) baseline yields optimal results even though it has access to only 80 Martinican Creole sentences. POS embeddings work better than lexical ones, but they suffer from negative transfer.

## introduction syntactic analysis is an essential task for language documentation and language revitalization, as it allows a deeper understanding of languages. underresourced languages often suffer from the lack of annotated gold standard data available to develop and offer nlp solutions for the communities speaking the language. moreover, there is a low number of researchers trained in formal linguistics and/or linguistic annotations which causes additional challenges in the creation of language resources for such languages. in recent years, research on parsing has developed a focus on underresourced languages @xcite @xcite , but creoles have received less attention. in this study, we develop a dependency parsing model for martinican creole (mc), a french-based creole, mostly spoken in martinique, a french island in the caribbean. being a french territory, french and mc coexist in an unbalanced manner. the diglossic situation makes french the dominant language in many contexts, although in the past decades martinican creole has seen an expansion of its communicative contexts @xcite . this is due to the codification and standardization processes the language underwent, especially by the gerec @xcite , and to the linguistic policies developed in an effort to safeguard and revitalize this language. however, one aspect of this revitalization process that is currently missing is the expansion of nlp tools and resources for mc (and other creole languages). creole languages based on the same lexifier language (in our case french) are extremely similar and in many cases, mutually intelligible. thus, developing nlp solutions for one creole language provides a basis to transfer knowledge to other related creole languages. the goal of this project is to investigate the best methods for developing a parser for an extremely low resource language when this language is a creole language. in our case, the creole is martinican creole. the main question here is whether the lexifier language, i.e., french, is similar enough to serve as basis for training a parser without further modification. ## research questions our overarching research question is the following: can we leverage a french treebank using transfer learning or multi-task learning approaches to create a parser model for martinican creole given that we only have a very small treebank ? can we leverage the similarity between the creole and its lexifier language, french? to answer this question, we need to answer the following questions: given the differences in spelling, are character embeddings, pos embeddings, or bert embeddings the best representation of the input sentence? (we will not consider multilingual bert models since the closest language is french, and we have access to large french embeddings models.) 2. in a transfer learning setting, how do we best use the very limited martinican creole data? is it worth the effort to annotate data for optimizing the parser, or can we optimize it on french? is there enough structural and lexical similarity between french and the creole to make this possible? creoles are still under-researched in nlp. noticeable work includes language model comparisons by @xcite between haitian creole, nigerian pidgin english, and singaporean colloquial english, trained with empirical risk minimization, against language models with distributionally robust ones, finding that the former performed better for creoles. one reason postulated may be the absence of drift due to the relative stability of creoles. regarding french-based creoles, haitian creole was the subject of an extensive collaboration in machine translation led by microsoft research @xcite following the 2010 earthquake. millour and fort (2018) led a project of crowdsourcing of pos tags for guadelupean creole in which they describe the necessary steps and methodology to crowdsource a language for pos tagging. they were able to collect a corpus of nearly 2,500 tokens pos tagged and create a pos tagger reaching 84% accuracy. the lack of available creole treebanks, with nigerian pidgin english @xcite the only publicly available universal dependency treebank, means that best parsing strategies for creoles are still being developed. given the lack of available data, parsing creoles can be viewed as similar to the need to leverage related treebanks to try and increase performance on the target treebank. a common approach is to concatenate available treebanks and optimize towards the target treebank. this has demonstrated gains in both monolingual @xcite and cross-lingual @xcite experiments. another successful technique is to instead train a model on a source treebank and then fine-tune on the target treebank @xcite . the most directly related works to ours are @xcite since they parse singlish, an english-based creole, by leveraging its lexifier language, english, to boost performance. @xcite propose a neural stacking architecture which yielded promising results which were further investigated by @xcite . they tripled the size of their original singlish treebank by web scraping and annotating more data and performed additional multi-task experiments for integrating english syntactic knowledge. while multi-task models showed some success, neural stacking methods were still better, as was simply concatenating english and singlish treebanks in some experiments. such neural stacking architectures with additional pos information also have helped in the related task of parsing hindi-english code-switching data @xcite . as far as we know, we are the first to approach the task of dependency parsing a frenchbased creole. ## properties of martinican creole martinican creole (mc) is a french-based creole and part of the atlantic creoles language family. syntactically, mc is an svo language and is closely related to french, other creoles such as guadeloupean, marie galante, st. barth, saint lucian creoles, and haitian creole, and to a lesser degree to african languages. the differences between mc and the closely related antillean creoles are mostly lexical, they share very similar syntactic structures. while mc originates from french, both languages show noticeable syntactic differences, especially wrt. the word order in noun phrases. despite these differences in morpheme order, it is still relatively easy to see the direct parallels between both languages. this makes french a good candidate for a transfer learning approach to parsing mc. mc is considered a morphologically reduced language (hazaël-massieux, 2002): tense, mood and aspect features are expressed as separate morphemes/markers instead of inflections on the verbal element. there is also no morphological gender/number marking on nouns and adjective. asiparé apparently yo 3pl té ké pst.fut vann sell prop own frè-yo brothers-3pl épi and sè-yo. sisters-3pl a ce qu'il parait, ils vendraient leurs propres frères et leurs soeurs. "apparently, they would sell their own siblings" in example (2), we see that the conditional is expressed in mc by a morpheme combination of the past/perfective marker té and the future/irrealis marker ké whereas in french, the conditional is expressed synthetically by the affix -raient attached to the end of the verb vendre. we also see that general plural nouns like frè-yo and sè-yo are not morphologically marked in mc, and neither is their accompanying adjective prop, whereas in french frères and soeurs and propres are all morphologically marked for gender and number. finally, while mc uses a different spelling system from french, the mc pronunciation is much closer to its spelling than in french. mc acquired most of its lexicon from french. lexical transfer was either phonetically transparent or underwent reanalysis via several phono-lexical processes (such as agglutination (see example (3)), apheresis (see example (4)), syncope (see example ( 5 )), etc.). in both cases, while the lexical transfer can easily be identified at the phonetic level, it is a more difficult to identify at the orthographic level, since there are significant differences in the respective spelling systems. because of the amount of differences, it is possible that french embeddings may not be useful, since there may not be enough lexical overlap between french and mc, even on the subword level. ## analysis all the experiments described above tell us that the best method to parse martinican creole given a very small treebank is to concatenate the two treebanks. it is unclear why first training on french and then fine-tuning on mc does not result in a similar performance. and it is equally unclear why the pos embeddings are successful in transfer learning but not in a multi-task learning setting. we assume that the two are related and will analyze the data to shed light on these questions.