# Better Word Representation Vectors Using Syllabic Alphabet: A Case Study of Swahili
## 1 Introduction  
Natural language processing (NLP) relies on word embeddings as input for machine learning or deep learning algorithms. For decades, NLP solutions were restricted to machine learning approaches that trained on handcrafted, high dimensional and sparse features [1]. Nowadays, the trend is neural networks [2], which use dense vector representations. Hence, the superior results on NLP tasks is attributed to word embeddings [3, 4] and deep learning methods [5]. Therefore, as observed by the authors of [6, 7, 8, 9], improved performance of downstream NLP tasks is achieved by learning vector representation of words in language models. Quality word vectors are expected to capture syntactic and semantic similarities among words by addressing the similarities in surface form of words and the context [8]. This has motivated the transition from the conventional one-hot word representation to word representation [10] based on words and sub-word information (characters and morphemes). Despite [PERSON] et al. 's [4] contribution of distributed word representation, the urge for even better word representation has led to composition of word embeddings from sub-word information such as characters [8, 11, 12], character n-gram [13] and morphemes [7, 14, 15]. However, with all these developments, deep learning is yet to be utilized in processing low resource languages [16; 17], particularly the syllabic based languages, such as Swahili, Xhosa, Luhya, Shona, Kikuyu and Mijikenda. In fact, the East Africa integration initiative by the respective countries has encountered language barrier as a challenge in the quest for a common language [18], a problem that could be overcome by automated language systems. A probable solution is automating Swahili using deep learning models that can learn its syllabic alphabet to effectively represent the Swahili words, which are highly agglutinative [19]. To the best of our knowledge, no study has considered learning word representation from constituent syllables of words in syllabic-based languages. We are inspired by the Swahili language teaching methodology, which first introduces syllables, then two-syllable words, and lastly complex words and sentences [20]. For this reason, we propose syllabic based word embeddings (WEFSE) to match Swahili's complex word morphology, as opposed to using characters or morphemes. This study generated word embeddings from syllable embeddings (WEFSE) but differently from [PERSON] et al. [21], who used an external hyphenator to segment the words into syllables. We hypothesize that learning word representations from syllabic alphabets captures both semantic meaning of words and handles new words. We attribute this to the fact that the syllables in Swahili are the smallest meaningful semantic unit and are a subset of the morphemes [22]. Consider the following Swahili verbs:  
1. anapojitakia (when he/she wants it for himself/herself);
2. aliandika (he wrote); and
3. atakikamata (he will catch it). From these examples we note that Swahili is highly agglutinative, as it starts with a root word and creates a new word by adding syllables such as \"a\", \"na\", \"po\", \"ji\", \"ki\", \"li\", \"ndi\", \"ka\", \"ta\" and \"ma\". This indicates that the ultimate meaning of the verb _anapjitakia_ is a culmination of the subject prefix (\"a\"), tense prefix (\"na\"), relative prefix (\"po\"), object prefix (\"ji\"), root (\"taka\") and extension (\"ia\"). The position of each syllable in the word also bears syntactic and semantic meaning. For example, the second syllable \"na\", \"li\" and \"ta\" in Examples 1, 2 and 3 connote present, past and future tenses, respectively. Intuitively, words with the same syllables have a similar context. The study's objective was to effectively represent Swahili words by capturing the compositional and contextual aspects. There is a trend in NLP of optimizing performance of downstream tasks through high quality vectors for word representation [6; 23].