Corpus Based Amharic Sentiment Lexicon Generation

keywords : Amharic Sentiment lexicon , Amharic Sentiment Classification , Seed words 1 INTRODUCTION . Most of sentiment mining research papers are associated to English languages . Linguistic computational resources in languages other than English are limited . Amharic is one of resource limited languages . Due to the advancement of World Wide Web , Amharic opinionated texts is increasing in size . To manage prediction of sentiment orientation towards a particular object or service is crucial for business intelligence , government intelligence , market intelligence , or support decision making . For carrying out Amharic sentiment classification , the availability of sentiment lexicons is crucial . To-date , there are two generated Amharic sentiment lexicons . These are manually generated lexicon ( 1000 ) ( Gebremeskel , 2010 ) and dictionary based Amharic SWN and SOCAL lexicons ( Neshir Alemneh et al. , 2019 ) . However , dictionary based generated lexicons has short-comings in that it has difficulty in capturing cultural connotation and language specific features of the language . For example , Amharic words which are spoken culturally and used to express opinions will not be obtained from dictionary based sentiment lexicons . The word ጉርሻ/ '' feed in other people with hands which expresses love and live in harmony with others '' / in the Amharic text : `` እንደ ጉርሻ ግን የሚያግባባን የለም . . . ጉርሻ እኮ አንዱ ለሌላው የማጉረስ ተግባር ብቻ አይደለም፤ በተጠቀለለው እንጀራ ውስጥ ፍቅር አለ፣ መተሳሰብ አለ፣ አክብሮት አለ። '' has positive connotation or positive sentiment . But the dictionary meaning of the word ጉርሻ is `` bonus '' . This is far away from the cultural connotation that it is intended to represent and express . We assumed that such kind of culture ( or language specific ) words are found in a collection of Amharic texts . However , dictionary based lexicons has short comings to capture sentiment terms which has strong ties to language and culture specific connotations of Amharic . Thus , this work builds corpus based algorithm to handle language and culture specific words in the lexicons . However , it could probably be impossible to handle all the words in the language as the corpus is a limited resource in almost all less resourced languages like Amharic . But still it is possible to build sentiment lexicons in particular domain where large amount of Amharic corpus is available . Due to this reason , the lexicon built using this approach is usually used for lexicon based sentiment analysis in the same domain from which it is built . The research questions to be addressed utilizing this approach are : ( 1 ) How can we build an approach to generate Amharic Sentiment Lexicon from corpus ? ( 2 ) How do we evaluate the validity and quality of the generated lexicon ? In this work , we set this approach to build Amharic polarity lexicons in automatic way relying on Amharic corpora which is mentioned shortly . The corpora are collected from different local news media organizations and also from facebook news ' comments and you tube video comments to extend and enhance corpus size to capture sentiment terms into the generated PPMI based lexicon . 2 RELATED WORKS . In this part , we will present the key papers addressing corpus- based Sentiment Lexicon generation . In ( Velikovich et al. , 2010 ) , large polarity lexicon is developed semiautomatically from the web by applying graph propagation method . A set of positive and negative sentences are prepared from the web for providing clue to expansion of lexicon . The method assigns a higher positive value if a given seed phrase contains multiple positive seed words , otherwise it is assigned negative value . The polarity p of seed phrase i is given by : pi = p + i − βp − i , where β is the factor that is responsible for preserving the overall semantic orientations between positive and negative flow over the graph . Both quantitatively and qualitatively , the performance of the web generated lexicon is outperforming the other lexicons generated from other manually annotate lexical resources like WordNet . The authors in ( Hamilton et al. , 2016 ) developed two domain specific sentiment lexicons ( historical and online community specific ) from historical corpus of 150 years and online community data using word embedding with label propagation algorithm to expand small list of seed terms . It achieves competitive performance with approaches relying on hand curated lexicons . This revealed that there is sentiment change of words either positively to negatively or vice-versa through time . Lexical graph is constructed using PPMI matrix computed from word embedding . To fill the edges of two nodes ( wi , wj ) , cosine similarity is computed . To propagate sentiment from seeds in lexical graph , random walk algorithm is adapted . That says , the polarity score of a seed set is proportional to probability of random walk from the seed set hitting that word . The generated lexicon from domain specific embedding outperforms very well when compared with the baseline and other variants . Our work is closely associated to the work of Passaro et al . ( 2015 ) . Passaro et al . ( 2015 ) generated emotion based lexicon by bootstrapping corpus using word distributional semantics ( i.e . using PPMI ) . Our approach is different from their work in that we generated sentiment lexicon rather than emotion lexicon . The other thing is that the approach of propagating sentiment to expand the seeds is also different . We used cosine similarity of the mean vector of seed words to the corresponding word vectors in the vocabulary of the PPMI matrix . Besides , the threshold selection , the seed words part of speech are different from language to language . For example , Amharic has few adverb classes unlike Italian . Thus , our seed words do not contain adverbs . 3 PROPOSED CORPUS BASED APPROACHES . There are variety of corpus based strategies that include count based ( e.g.PPMI ) and predictive based ( e.g . word embedding ) approaches . In this part , we present the proposed count based approach to generate Amharic Sentiment lexicon from a corpus . In Figure 1 , we present the proposed framework of corpus based approach to generate Amharic Sentiment lexicon . The framework has four components : ( Amharic News ) Corpus Collections , Preprocessing Module , PPMI Matix of Word-Context , Algorithm to generate ( Amharic ) Sentiment Lexicon resulting in the Generated ( Amharic ) Sentiment Lexicon . The algorithm and the seeds in figure 1 are briefly described as follows . To generate Amharic Sentiment lexicon , we follow four major steps : 1 . Prepare a set of seed lists which are strongly negatively and positively polarized Adjectives , Nouns and Verbs ( Note : Amharic language contains few adverbs ( Passaro et al. , 2015 ) , adverbs are not taken as seed word ) . We will select at least seven most polarized seed words for each of aforementioned part-of-speech classes ( Yimam , 2000 ) . Selection of seed words is the most critical that affects the performance of bootstrapping algorithm ( Waegel , 2003 ) . Most authors choose the most frequently occurring words in the corpus as seed list . This is assumed to ensure the greatest amount of contextual information to learn from , however , we are not sure about the quality of the contexts . We adapt and follow seed selection guidelines of Turney & Littman ( 2003 ) . After we tried seed selection based on Turney & Littman ( 2003 ) , we update the original seed words . Sample summary of seeds are presented in Table 1 . 2 . Build semantic space word-context matrix ( Potts , 2013 ; Turney & Pantel , 2010 ) using the number of occurrences ( frequency ) of target word with its context words of window size±2 . Word-Context matrix is selected as it is dense and good for building word rich representations ( i.e . similarity of words ) unlike word-document design which is sparse and computationally expensive ( Potts , 2013 ; Turney & Pantel , 2010 ) . Initially , let F be word-context raw frequency matrix with nr rows and nc columns formed from Amharic text corpora . Next , we apply weighting functions to select word semantic similarity descriminant features . There are variety of weighting functions to get meaningful semantic similarity between a word and its context . The most popular one is Point-wise Mutual Information ( PMI ) ( Turney & Littman , 2003 ) . In our case we use positive PMI by assigning 0 if it is less than 0 ( Bullinaria & Levy , 2007 ) . Then , let X be new PMI based matrix that will be obtained by applying Positive PMI ( PPMI ) to matrix F. Matrix X will have the same number of rows and columns matrix F. The value of an element fij is the number of times that word wi occurs in the context cj in matrix F. Then , the corresponding element xij in the new matrix X would be defined as follows : xij = { PMI ( wi , cj ) if PMI ( wi , cj ) > 0 0 if PMI ( wi , cj ) ≤ 0 ( 1 ) Where , PMI ( wi , cj ) is the Point-wise Mutual Information that measures the estimated co-occurrence of word wi and its context cj and is given as : PMI ( wi , cj ) = log P ( wi , cj ) P ( wi ) P ( cj ) ( 2 ) Where , P ( wi , cj ) is the estimated probability that the word wi occurs in the context of cj , P ( wi ) is the estimated probability of wi and P ( cj ) is the estimated probability of ci are defined in terms of frequency fij . 3 . Compute the cosine distance between target term and centroid of seed lists ( e.g . centroid for positive adjective seeds , −−→ µ+adj ) . To find the cosine distance of a new word from seed list , first we compute the centroids of seed lists of respective POS classes ; for example , centroids for positive seeds S+ and negative seeds S- , for adjective class is given by : −−→ µ+adj ( S + ) = Σw∈S+ ~w |S+| & −−→ µ−adj ( S − ) = Σw∈S− ~w |S−| ( 3 ) Similarly , centroids of the other seed classes will be found . Then , the cosine distances of target word from positive and negative adjective seeds of centroids , −−→ µ+adj and −−→ µ−adj is given by : cosine ( ~wi , −−→ µ+adj ) = ~wi . −−→ µ+adj || ~wi|||| −−→ µ+adj || & cosine ( ~wi , −−→ µ−adj ) = ~wi . −−→ µ−adj || ~wi|||| −−→ µ−adj || ( 4 ) As word-context matrix x is vector space model , the cosine of the angle between two words vectors is the same as the inner product of the normalized unit word vectors . After we have cosine distances between word wi and seed with centroid ~wi , ~µ+adj , the similarity measure can be found using either : Sim ( ~wi , −−→ µ+adj ) = 1 cosine ( ~wi , −−→ µ+adj ) or Sim ( ~wi , −−→ µ+adj ) = 1− cosine ( ~wi , −−→ µ+adj ) ( 5 ) Similarly , the similarity score , Sim ( ~wi , −−→ µ−adj ) can also be computed . This similarity score for each target word is mapped and scaled to appropriate real number . A target word whose sentiment score is below or above a particular threshold can be added to that sentiment dictionary in ranked order based on PMI based cosine distances . We choose positive PMI with cosine measure as it is performed consistently better than the other combination features with similarity metrics : Hellinger , Kullback-Leibler , City Block , Bhattacharya and Euclidean ( Bullinaria & Levy , 2007 ) . 4 . Repeat from step 3 for the next target term in the matrix to expand lexicon dictionary . Stop after a number of iterations defined by a threshold acquired experimental testing . The detail algorithm for generating Amharic sentiment lexicon from PPMI is presented in algorithm 1 Algorithm description : The algorithm 1 reads the seed words and generates the merge of expanded seed words using PPMI . Line 1 loads the seed words and assigns to their corresponding category of seed words . Similarly , from line 2 to 6 loads the necessary lexical resources such as PPMI matrix , vocabulary list , Amharic-English , AmharicAmharic , Amharic-Sentiment SWN and in line 7 , the output Amharic Sentiment Lex . by PPMI is initialized to Null . From line 8 to 22 , it iterates for each seed words polarity and categories . That is , line 9 to 11 checks that each seed term is found in the corpus vocabulary . Line 12 initializes the threshold by a selected trial number ( in our case 100,200,1000 , etc. ) . From line 13 to 22 , iterates from i=0 to threshold in order to perform a set of operations . That is , line 16 computes the mean of the seed lexicon based on equation 3 specified in the previous section . Line 17 computes the similarity between the mean vector and the PPMI word-word co occurrence matrix and returns the top i most closest terms to the mean vector based on equation 5 . Lines 18-19 , it removes top closest items which has different part-of-speech to the seed words . Line 20-21 check the top i closest terms are has different polarity to the seed lexicon . Line 22 updates the PPMI lexicon by inserting the newly obtained sentiment terms . Line 23 returns the generated Amharic Sentiment lexicon by PPMI . Using corpus based approach , Amharic sentiment lexicon are built where it allows Input : PPMI : Word-Context PPMI matrix Output : AM_Lexicon_by_PPMI : Generated Amharic Sentiment Lexicon 1 seed_noun+ , seed_noun− , seed_adj+ , seed_adj− , seed_verb+ , seed_verb− ← LoadCorespondingSeedCatagoryfile PPMI ← LoadPPMIMatrixfile V ocab ← LoadV ocabularyfile AmharicEnglishDic ← LoadAmharicEnglishDictionaryfile AmharicAmharicDic ← LoadAmharicAmharicDictionaryfile AmharicSWN ← LoadAmharicSentimentSWNfile Amharic_Sentiment_Lex_by_PPMI ← Null foreach seed_lexicon ∈ seed_noun+ , seed_noun− , seed_adj+ , seed_adj− , seed_verb+ , seed_verb− : do 2 foreach seed ∈ seed_lexicon : do 3 if seed ∈ Vocab then 4 Remove Seed from Seed_Lexicon 5 Threshold←Number of iterations foreach i← 0 to Threshold : do 6 mean_vector← compute_mean ( seed_lexicon ) by equation 3 top_ten_closest_terms← compute_similarity ( mean_vector , PPMI ) by equation 4 if term ∈ top_ten_closest_terms and term ∈ seed_lexicon : then 7 Remove the term from top_ten_closest_terms list as it is duplicate 8 if Any term in top_ten_closest_terms has different part_speech from seed_lexicon then 9 Remove the term from top_ten_closest_terms list 10 if Any term in top_ten_closest_terms has different polarity from Amharic SWN lexicon then 11 Remove the term from top_ten_closest_terms list 12 Update seed_lexicon by inserting top_ten_closest_terms list 13 AM_Lexicon_by_PPMI←AM_Lexicon_by_PPMI + seed_lexicon ; algorithm 1 : Amharic Sentiment Lexicon Generation Algorithm Using PPMI finding domain dependent opinions which might not be possible by sentiment lexicon generated using dictionary based approach . The quality of this lexicon will be evaluated using similar techniques used in dictionary based approaches ( Neshir Alemneh et al. , 2019 ) . However , this approach may not probably produce sentiment lexicon with large coverage as the corpus size may be insufficient to include all polarity words in Amharic language . To reduce the level of this issue , we combine the lexicons generated in both dictionary based and corpus based approaches for Amharic Sentiment classification .

Summary: This paper introduces a corpus-based approach to build sentiment lexicon for Amharic. In order to save time and costs for the resource-limited language, the lexicon is generated from an Amharic news corpus by the following steps: manually preparing polarized seed words lists (strongly positive and strongly negative), calculating the co-occurrence of target word in its context via Positive Point-wise Mutual Information (PPMI) method, measuring the similarity between target words and seed words by cosine distance, iterating with the threshold 100 and 200. The PPMI lexicon is stemmed and evaluated from aspects of subjectivity detection, coverage, agreement and sentiment classification. Three other lexicons: Manual developed by manual, SOCAL and SWN developed by bilingual dictionary, are used as benchmark to compare with the PPMI lexicon. In sentiment classification experiment the PPMI lexicon did not show a superior performance. All the four lexicons have similar accuracy, between 42.16% ~ 48.87%.  Only when the four are combined together the result is improved to 83.51%.