Corpus Based Amharic Sentiment Lexicon Generation
Authors: Girma Neshir
Andeas Rauber
and Solomon Atnafu
Venue:  (2020)
Keywords: Amharic sentiment lexicon
Amharic sentiment classification
seed words

Abstract: Sentiment classification is an active research area with several applications including analysis of political opinions, classifying comments, movie reviews, news reviews and product reviews. To employ rule based sentiment classification, we require sentiment lexicons. However, manual construction of sentiment lexicon is time consuming and costly for resource-limited languages. To bypass manual development time and costs, we tried to build Amharic Sentiment Lexicons relying on corpus based approach. The intention of this approach is to handle sentiment terms specific to Amharic language from Amharic Corpus. Small set of seed terms are manually prepared from three parts of speech such as noun, adjective and verb. We developed algorithms for constructing Amharic sentiment lexicons automatically from Amharic news corpus. Corpus based approach is proposed relying on the word co-occurrence distributional embedding including frequency based embedding (i.e. Positive Point-wise Mutual Information PPMI). First we build word-context unigram frequency count matrix and transform it to point-wise mutual Information matrix. Using this matrix, we computed the cosine distance of mean vector of seed lists and each word in the corpus vocabulary. Based on the threshold value, the top closest words to the mean vector of seed list are added to the lexicon. Then the mean vector of the new sentiment seed list is updated and process is repeated until we get sufficient terms in the lexicon. Using PPMI with threshold value of 100 and 200, we got corpus based Amharic Sentiment lexicons of size 1811 and 3794 respectively by expanding 519 seeds. Finally, the lexicon generated in corpus based approach is evaluated.


Under review as a conference paper at ICLR 2020
CORPUS BASED AMHARIC SENTIMENT LEXICON GENERA- TION
Anonymous authors
Paper under double-blind review
ABSTRACT
Sentiment classification an active research area with several applications in- cluding analysis of political opinions, classifying comments, movie reviews, news reviews and product reviews. To employ rule based sentiment classification, we require sentiment lexicons. However, manual construction of sentiment lexicon is time consuming and costly for resource- limited languages. To bypass manual de- velopment time and costs, we tried build Amharic Sentiment Lexicons relying corpus based approach. The intention of this approach is to handle sentiment terms specific to Amharic language from Amharic Corpus. Small set of seed terms are manually prepared from three parts of speech such as noun, adjective and verb. We developed algorithms for constructing Amharic sentiment lexicons automati- cally from Amharic news corpus. Corpus based approach is proposed relying the word co-occurrence distributional embedding including frequency based em- bedding (i.c. Positive Point-wise Mutual Information PPMI). Using PPMI with threshold value of 100 and 200, we got corpus based Amharic Sentiment lexicons of size 1811 and 3794 respectively by expanding 519 seeds. Finally, the lexicon generated in corpus based approach is evaluated.
keywords: Amharic Sentiment lexicon, Amharic Sentiment Classification, Seed words
INTRODUCTION
Most of sentiment mining research papers are associated to English languages. Linguistic compu- tational resources in languages other than English limited. Amharic is of resource limited languages. Due to the advancement of World Wide Web, Amharic opinionated texts is increas- ing size. To manage prediction of sentiment orientation towards a particular object or service is crucial for business intelligence, government intelligence, market intelligence, or support decision making. For carrying out Amharic sentiment classification, the availability of sentiment lexicons is crucial. To-date, there are two generated Amharic sentiment lexicons. These are manually gener- ated lexicon( (0000) (Gebremeskel)) 2010) and dictionary based Amharic SWN and SOCAL lexicons (Neshir Alemneh et al. 2019)). However, dictionary based generated lexicons has short-comings in that it has difficulty in capturing cultural connotation and language specific features of the language. For example, Amharic words which are spoken culturally and used to express opinions will not be obtained from dictionary based sentiment lexicons. The word TCTT/"feed in other people with hands which expresses love and live in harmony with others"/ in the Amharic text: "A7& TC. 9 por, PΛg".
7C AP h78. AdA0 por7-ch 4-9C A3F ARRATT; (+ 2776 whhr F7C kA: ----1 KΛ: Kh-lcc+- hΛ=" has positive connotation or positive sentiment. But the dictionary meaning of the word 7CT is "bonus". This is far away from the cultural connotation that it is intended to represent and express. We assumed that such kind of culture (or language specifie) words are found in collection of Amharic texts. However, dictionary based lexicons has short comings capture sentiment terms which has strong ties to language and culture specific connotations of Amharic. Thus, this work builds corpus based algorithm to handle language and culture specific words in the lexicons. However, it could probably be impossible to handle all the words in the language as the corpus a limited resource in almost all less resourced languages like Amharic. But still it is possible to build sentiment lexicons in particular domain where large amount of Amharic corpus available. Due to this reason, the lexicon built using this approach is usually used for lexicon based

---

Under review as a conference paper at ICLR 2020
sentiment analysis in the same domain from which it is built.
The research questions to be addressed utilizing this approach are: (1) How can we build an approach to generate Amharic Sentiment Lexicon from corpus? (2)How do we evaluate the validity and quality of the generated lexicon? In this work, we set this approach to build Amharic polarity lexicons in automatic way relying on Amharic corpora which is mentioned shortly. The сorpora are collected from different local news media organizations and also from facebook news' comments and you tube video comments to extend and enhance corpus size to capture sentiment terms into the generated PPMI based lexicon.
2 RELATED WORKS
In this part, we will present the key papers addressing corpus- based Sentiment Lexicon generation. In (Velikovich automatically from the web by applying graph propagation method. A set of positive and negative sentences are prepared from the web for providing clue to expansion of lexicon. The method assigns a higher positive value if a given seed phrase contains multiple positive seed words, otherwise it is assigned negative value. The polarity p of seed phrase i is given by: Pt - P By, , where 3 is the factor that is responsible for preserving the overall semantic orientations between positive and negative flow over the graph. Both quantitatively and qualitatively, the performance of the web generated lexicon is outperforming the other lexicons generated from other manually annotate lexical resources like WordNet. The authors in (Hamilton et al., 2016) developed two domain specific sentiment lexicons (historical and online community specific) from historical corpus of 150 years and online community data using word embedding with label propagation algorithm to expand small list of seed terms. It achieves competitive performance with approaches relying on hand curated lexicons. This revealed that there is sentiment change of words either positively to negatively or vice-versa through time. Lexical graph is constructed using PPMI matrix computed from word embedding. To fill the edges of two nodes (wi, Wy), cosine similarity is computed. To propagate sentiment from seeds in lexical graph, random walk algorithm is adapted. That says, the polarity score of seed set is proportional to probability of random walk from the seed set hitting that word. The generated lexicon from domain specific embedding outperforms very well when compared with the baseline and other variants.
al., 2010), large polarity lexicon is developed semi-
Our work is closely associated to the work of Passaro al. (2015). Passaro et al. (2055) generated emotion based lexicon by bootstrapping corpus using word distributional semantics (i.e. using PPMI) Our approach is different from their work in that we generated sentiment lexicon rather than emotion lexicon. The other thing is that the approach of propagating sentiment cosine similarity of the mean vector of seed words to the corresponding word vectors in the vocabulary of the PPMI matrix. Besides, the threshold selection, the seed words part of speech are different from language to language. For example, Amharic has few adverb classes unlike Italian. Thus, our seed words do not contain adverbs.
expand the seeds is also different. We used
3 PROPOSED CORPUS BASED APPROACHES
There are variety of corpus based strategies that include count based(e-g. PPMI) and predictive based(e-g. word embedding) approaches.
this part, we present the proposed count based approach to generate Amharic Sentiment lexicon from a corpus. In Figure [T, we present the proposed framework of corpus based approach to generate Amharic Sentiment lexicon. The framework has four components: (Amharic News) Corpus Collections, Preprocessing Module, PPMI Matix Word-Context, Algorithm to generate (Amharic) Sentiment Lexicon resulting in the Generated (Amharic) Sentiment Lexicon.
2

---

Under review as a conference paper at ICLR 2020
Seed Words
Preproceesing
Amharic News Collection
Punctuation Removal Normalization Speiling Corrcction Light Stemming, etc.
Word-Word PPMI Matrix
Lexccn Gen. Algaritm
Amharic Sentimant Lexizan
Figure 1:
Corpus based Amharic Sentiment Lexicon Building Framework
The algorithm and the seeds in figure 1 are briefly described as follows. To generate Amharic Sentiment lexicon, we follow four major steps:
1. Prepare a set of seed lists which are strongly negatively and positively polarized Adjectives, Nouns and Verbs (Note: Amharic language contains few adverbs (Passaro et al., 2015), adverbs are not taken as seed word). We will select at least seven most polarized seed words for each of aforementioned part-of-speech classes (Yimam, 2000). Selection of seed words is the most critical that affects the performance of bootstrapping algorithm (Waegel, 2003). Most authors choose the most frequently occurring words in the corpus as seed list. This is assumed to ensure the greatest amount of contextual information to learn from, however, we not sure about the quality of the contexts. We adapt and follow seed selection guidelines of Turrny & Littman (2003). After tried seed selection based [Turney & Littman (2003), update the original seed words. Sample summary of seeds are presented in Table 1
2. Build semantic space word-context matrix (Potts) 2013 Turney & Pantel 2010) using the number of occurrences(frequency) of target word with its context words of window si Word-Context matrix is selected as it is dense and good for building word rich representations (i.e. similarity of words) unlike word-document design which is sparse and computationally expensive (Potts 2013 Turney Pantel (2010).
Initially, let F be word-context raw frequency matrix with rows and ne columns formed from Amharic text corpora. Next, we apply weighting functions to select word semantic similarity descriminant features. There are variety of weighting functions to get meaningful semantic similarity between a word and its context. The most popular one is Point-wise Mutual Information (PMI) (Turney & Littman, 2003). In our case we use positive PMI by assigning 0 is less than 0 (Buliinaria & Levy, 2007). Then, let X be PMI based matrix that will be obtained by applying Positive PMI(PMM) to matrix F. Matrix X will have the same number of rows and columns matrix F. The value of element fi is the number of times that word occurs in the context in matrix F. Then, the corresponding element Σi in the new matrix X would be defined as follows:
PMI 0j) if PMI(W,, c,) >
Fij Oif PMI(W, 0₁) <
Where, PMI(W, e,) is the Point-wise Mutual Information that measures the estimated co-occurrence of word and its context сj and is given as:
P(w;, Cz) PMI(wi, log P(w:) P(gz)
(2)
Where, P(w,, is the estimated probability that the word w, occurs in the context of oj P(w:) is the estimated probability of w; and P(c;) is the estimated probability of e; are defined in terms of frequency fij-
3

---

Under review as a conference paper at ICLR 2020
3. Compute the cosine distance between target term and centroid of seed lists (e.g. centroid for positive adjective seeds, ) To find the cosine distance of new word from seed list, first compute the centroids of seed lists of respective POS classes; for example, centroids for positive seeds S and negative seeds s-, for adjective class is given by:
Pady Pat Swes+ & Hady (S-) EwES-W (s+)
[S+
|
Similarly, centroids of the other seed classes will be found. Then, the cosine distances of target word from positive and negative adjective seeds of centroids,
-+' Pady and Mady is given by:
Wi-Padj
Wi Wi-Haah
Dosine(W; pads)
& cosine (Ts, Madj)
(4)
Mayll
|| Tuis|lllad
As word-context matrix x is vector space model, the cosine of the angle between two words vectors is the same as the inner product of the normalized unit word vectors. After we have cosine distances between word w, and seed with centroid W7, Pady the similarity measure can be found using either:
Sim Mody))
Sim(Wr Hadf)) cossnn((ii Peay)
(5)
cosine(w, Mads))
Similarly, the similarity score, Sim (Maa)) can also be computed. This similarity score for each target word is mapped and scaled to appropriate real number. A target word whose sentiment score is below threshold can be added to that sentiment dictionary in ranked order based on PMI based cosine distances. We choose positive PMI with cosine measure as it is performed consistently better than the other combination features with similarity metrics: Hellinger, Kullback-Leibler City Block, Bhattacharya and Euclidean (Bulliaria & Levy, 2007
above a particular
4. Repeat from step 3 for the next target term in the matrix to expand lexicon dictionary. Stop after a number of iterations defined by a threshold acquired experimental testing. The detail algorithm for generating Amharic sentiment lexicon from PPMI is presented in algorithm [
Algorithm description: The algorithm reads the seed words and generates the merge of expanded seed words using PPMI. Line loads the seed words and assigns to their corresponding category of seed words. Similarly, from line 2 to 6 loads the necessary lexical resources such as PPMI matrix, vocabulary list, Amharic-English, Amharic- Amharic, Amharic-Sentiment SWN and in line 7, the output Amharic Sentiment Lex. by PPMI is initialized to Null. From line 8 to 22, it iterates for each seed words polarity and categories. That is, line 9 to 11 checks that each seed term is found in the corpus vocabulary. Line 12 initializes the threshold by a selected trial number(in our case 100,200,1000, etc.). From line 13 22, iterates from i=0 threshold in order to perform set of operations. That is, line 16 computes the mean of the seed lexicon based on equation 3 specified in the previous section. Line 17 computes the similarity between the mean vector and the PPMI word-word co occurrence matrix and returns the top most closest terms to the mean vector based on equation Lines 18-19, it removes top closest items which has different part-of-speech to the seed words. Line 20-21 check the top closest terms are has different polarity to the seed lexicon. Line 22 updates the PPMI lexicon by inserting the newly obtained sentiment terms. Line 23 returns the generated Amharic Sentiment lexicon by PPMI. Using corpus based approach, Amharic sentiment lexicon are built where it allows
4

---

Under review as a conference paper at ICLR 2020
Input: PPMI: Word-Context PPMI matrix
Output: AM Lexicon by PPMI: Generated Amharic Sentiment Lexicon
seed noun*, seed noun seed adj*, ,de*d adj- sccd verb+ seed werb-
SeaitorrepponnnngeeCaaeaeee ffle PPMI + LoudPPMTMMariitf Vocab file
LoadV ocabulary. file. AmharicEnglishhDD. LoadAmhart www.rmaaaaaaaeeeeeeeea ach seed lericon seed noun" seed seed_noun noun seed _add__djj" ssed _eeed_ajj" seed verb* _erb_
Load.amlaarcnngissiDiccoooo AmharicSW N f file Amharic Sentiment Anhhrri_Sentimment Lex by__bMP_ Null fore-
& www.aaaarriimmaaaiiiiiiiiiiiaaaa
do
foreach seed E seed, Iericon: do
seed E Vocab then
Remove Seed from Seed Lexicon
Threshold N-uumeer of iterations foreach 0 to Threshold: do mcan vector + compute mean( ssed lexicon) by equation 3
top_ten_ closest terms compute similarity(mean vector,PPMI) by equation 4 if term E top_ ten _hit_sssssst__ terms and term E seed lexicon: then
Remove the term from top_tep ten seen_clossst terms list as it is duplicate
if Any term in top ten closest terms has different part_speect speech from seed lexicon then
Remove the term from top ten closest terms list
10
Any term in top len closest terms has different polarity from Amharic SWN lexicon then Remove the term from top ten tes_tee___cooee see_closest___ terms list
12
Update seed lexicon by inserting top_ten ten sien_clossstttteeeeeeeeeeeesss_e terms list
13
AM Lexicon by PPMI LAM_Lexicon BPy_ seed lexicon:
algorithm I: Amharic Sentiment Lexicon Generation Algorithm Using PPMI
finding domain dependent opinions which might be possible by sentiment lexicon generated using dictionary based approach. The quality of this lexicon will be evaluated using similar techniques used dictionary based approaches (Neshir Alemneh et al., 2019). However, this approach may not probably produce sentiment lexicon with large coverage as the corpus size may be insufficient to include all polarity words Amharic language. To reduce the level of this issue, we combine the lexicons generated in both dictionary based and corpus based approaches for Amharic Sentiment classification.
4 RESULTS AND DISCUSSIONS
Using corpus based approach, Amharic sentiment lexicon is built where it allows finding domain dependent opinions which might not be possible by sentiment lexicon generated using dictionary based approach. In this section, have attempted to develop new approaches to bootstrapping relying on word-context semantic space representation of large Amharic corpora.
4.1 SEED WORDS
We have manually prepared Amharic opinion words with highest sentimental strength either positively or negatively from three parts- of -speech categories: Adjectives, Nouns and Verbs. We expanded each seed category from which the snapshot seed words are presented in Table
Table 1: Selected Seeds for Corpus based Amharic Sentiment Lexicon Expansion
Part of speech
Negative
Positive
Adjectives
99
79
Nouns
145
120
Verbs
40
4.2 STEMMING
Amharic is highly morphological language both inflectional and derivational morphology is complex. Thus, without applying stemming, it is easy to do computational

---

Under review as a conference paper at ICLR 2020
linguistic tasks like sentiment classification. Amharic roots are very few thousands from which other words are derived. The level of stemming determines whether stemmed terms preserve the semantics of original word or not. So, in this case, we developed light stemmer to reduce surface words into corresponding stems with minimal lose of semantic information. A sample of stemmed words with thier coresponding surface words are presented in Table
5
EVALUATION
5.1 CORPUS AND LEXICAL RESOURCES
The corpus and data sets used in this research are presented as follows:
Amharic Corpus: The size of this corpus is 20 milion tokens (teams from Addis Ababa University et al.). This corpus is used to build PPMI matrix and also to evaluate the coverage of PPMI based lexicon.
ii. Facebook Users' Comment This data set is used to build PPMI matrix and also to evaluate subjectivity detection, lexicon coverage and lexicon based sentiment classifica- tion of the generated Amharic Sentiment lexicon. The data set is manually annotated by Government Office Affairs CommunicationIGOAC) professional and it is labeled ei- ther positive and negative.
iii. Amharic Sentiment Lexicons: The Amharic sentiment lexicons includes manual (1000) (Gebremeskel, 2010), Amharic SentiWordNer/SWWW133377 (Neshir Alemneh et al.) 2019) and Amharic Semantic Orientation Calculator(SOcAL) (5683) (Neshir Alem- nnhh al.) 2019). These lexicons are used benchmark to compare the performance of PPMI based lexicons.
5.2 EXPERIMENTAL SETTINGS
Using the aforementioned corpus, Amharic News Corpus with 11433 documents and 2800 Facebook News post and Users Comments are used to build word-context PPMI. First, we tried to clean the data set. After cleansing, we tokenized and word-Context count dictionary is built. Relying on this dictionary and in turn, it is transformed into PPMI Sparse matrices. This matrix is saved in file for further tasks of Amharic Sentiment lexicon generation. The total vocabulary size is 953,406. After stop words are removed and stemming is applied, the vocabulary size is reduced to 231,270, Thus, the size of this matrix is (231270, 231270). Based on this word-word information in this matrix, the point-wise mutual information (PMI) of word-word is computed as in equation 2 The PMI is input to further computation of our corpus based lexicon expansion algorithm [ Finally, we generated Amharic sentiment lexicons by expanding the seeds relying on the PPMI matrix of the corpus by implementing this algorithm 1 at two threshold iteration values of 100 and 200. With these iterations, we got corpus based Amharic Sentiment lexicons of size 1811 and 3794 respectively. We think, iterations >20 is better. Thus, our further discussion on evaluation of the approach is based on the lexicon generated at 200 iteration(i. e. Lexicon size of 3794). This lexicon saved with entries containing stemmed word, part of speech, polarity strength and polarity sign. Sample of this lexicon is presented in Table 2
Table 2: Sample list of terms in the generated Amharic sentiment lexicon using PPMI with their corresponding surface words
Stem
POS
Sentiment
Sample Surfice Worls
wn-h
/cock
Wiir///att/, mitb/fakee, tun-h/take/, hove5/trom take maker/, alat and/, fune9//for falee maker/, nott laiz/, pltspuesseee, swtt//, fakee//akeness.. ubD33/fkee/ hun-raa/froo the lalr/, /hrte//alt/,, oben/lair/. tundd/from lair/, undrs/fake and/, humee/from the falee/ on.5 maae/pueauure and/, uadrs/the pleasuze/. uhit/fake/, nobi5/to take and/, vabt/, runds/for fake and/, notit/to fake/, fake//ett....
/
up
'act
Noun
-0.81
/ack// We/lact/ w89t/hones/ uP3/the trath/, hod/Trom fact/, wk/the truth,, w8 muth/, suel//the one who is honest/, wes/the muth/, wets/the wuth and,, oke/truth and/, hoke/tom the truth/, o47e-/tbst /ftt// hwet//rom facts/ ubsa//ruthnees/, obv"/truth/, tod/tor truch/, weg/tact and/ nobr//ooe truth and/, nok-ffor truth/, rubrit/for truthiness/, sa/facc,, wwe/hon- esty,, wt//, nAuR/for truth,, oe who bonest/
veracity//
bonest/,
We will evaluate in three ways: external to lexicon and internal to lexicon. External to lexicon is to test the usefulness and the correctness of each of the lexicon to find

---

Under review as a conference paper at ICLR 2020
sentiment score of sentiment labeled Amharic comments corpus. Internal evaluation is compute the degree to which each of the generated lexicons are overlapped( or agreed ) with manual, SOCAL and SWN(Amharic) Sentiment lexicons.
5.3 SUBIECTVVITY DETECTION
In this part evaluate the accuracy of the subjectivity detection rate of generated PPMI based Amharic lexicon on 2800 facebook annotated comments. The procedures for aggregating sentiment score is done by summing up the positive and negative sentiment values of opinion words found in the comment if those opinion words are also found in the sentiment lexicon. If the sum of positive score greater less than sum of negative score, then the comment is said to be subjective comment, otherwise the comment is either neutral mixed. Based this technique, the subjectivity detection rate is presented in Table 3
Table Comparison of the PPMI Lexicon' subjectivity detection rate (in%) on facebook comments Sentiment Lexicons
Detection withNoStem
Detection withStem
Amharic (Basellle
43.23
93.56
Amharic PPMI(Ours)
97.29
Amharic SOCAL-
31.28
96.65
Amharic SWN
75.83
99.33
Discussion: As subjectivity detection rate of the PPMI lexicon and others are depicted in Table 3, the detection rate of PPMI lexicon performs better than the baseline(manual lexicon). Where as Lexicon from SWN outperforms the PPMI based Lexicon with 2% accuracy.
5.4 GENERATED LEXICON'S COVERAGE
This is evaluate the coverage of the generated lexicon externally by using the aforementioned Amharic corpus(both facebook comments and general corpus). That is, the coverage of PPMI based Amharic Sentiment Lexicon on facebook comments and a general Amharic corpus is computed by counting the occurrence tokens of the corpus the generated sentiment lexicons and both positive and negative counts are computed in percent and it presented in Table 4
Table 4: Comparison of PPMI Lexicons' coverage (positive/negative count and in percent)
2500 Amharic Comments
20 millions Amharic tokens
Lexicons
coverage(+,)) count
in%
coverage(+t-) count
in %
Manual
[4399, 2995]
31.95
2161501
25.01
SOCAL
[5738, 3953]
41.87
339922,
38,88
SWN
[9447, 4803]
61.57
(6645592, 4006072)
54.77
PPMI
[5130, 4928]
45.07
Discussion: Table 4 depicted that the coverage of PPMI based Amharic sentiment lexicon is better than the manual lexicon and SOCAL. However, it has less coverage than SWN. Unlike SWN, PPMI based lexicon is generated from corpus. Due to this reason its coverage work on general domain is limited. also demonstrated that the positive and negative count in almost all lexicons seems to have balanced and uniform distribution of sentiment polarity terms the corpus.
5.5 LEXICONS' AGREEMENT
In this part, we will evaluate to what extent the generated PPMI based Lexicon agreed or overlapped with the other lexicons. This type of evaluation (internal) which validates by stating the percentage
entries in the lexicons are available common. The more percentage means the better agreement or overlap of lexicons. The agreement of PPMI based in percentage is presented in Table 5

---

Under review as a conference paper at ICLR 2020
Table 5: The Agreement (in percent) between PPMI based lexicon and other Lexicons
Lexicons Combinations
Agreement(in %)
1
PMI and SOCAL
67.34
2
PMI and SWN
97.93
3
PMI and Manual
74.96
Discussion: Table presents the extent which the PPMI based lexicon is agreed with other lexicons.PPMI based lexicon has got the highest agreement rate (overlap) with SWN lexicon than the other lexicons.
5.6 LEXICON BASED AMHARIC SENTIMENT CLASSIFICATIO
In this part, Table 6 presents the lexicon based sentiment classification performance of generated PPMI based Amharic lexicon on 2821 annotated Amharic Facebook Com- ments. The classification accuracy of generated PPMI based Lexicon and other lexicons are compared.
Table The Accuracy (in percent) PPMI based Lexicons for Sentiment Classification
Accrracy3%)
Amharie Lesicons
NoSion-NoNee.
MenrrNoNe
Stom-Neg
Manusii bascline)
*16.7
"225"
12.16
SOCAL
"14."
*403"
47.2
SWN
"30
"50T"
18.77
+SWN
"44377
Manual SOCAL ISWN
$753*
78.19
SW M-mmall
"
Discussion: Besides the other evaluations of the generated PPMI based lexicon, the usefulness of this lexicon is tested on actual lexicon based Amharic sentiment classifi- cation. As depicted in Table 6 The accuracy of PPMI based lexicon for lexicon based sentiment classification is better than the manual benchmark lexicon. As discussed on dictionary based lexicons (Neshir Alemneh al.) (2019) for lexicon based sentiment classification in earlier section, using stemming and negation handling are far im- proving the performance lexicon based classification. Besides combination of lexicons outperforms better than the individual lexicon.
CONCLUSIONS
Creating a sentiment lexicon generation not an objective process. The generated lexicon is dependent on the task it is applied. Thus, in this work we have seen that it possible create Sentiment lexicon for low resourced languages from corpus. This captures the language specific features and connotations related to the culture where the language spoken. This can not be handled using dictionary based approach that propagates labels from resource rich languages. To the best of our knowledge, the the PPMI based approach to generate Amharic Sentiment lexicon form corpus is performed for first time for Amharic language with almost minimal costs and time. Thus, the generated lexicons can be used in combination with other sentiment lexicons to enhance the performance of sentiment classifications in Amharic language. The approach is generic approach which can be adapted to other resource limited languages to reduce cost of human annotation and the time it takes annotated sentiment lexicons. Though the PPMI based Amharic Sentiment lexicon outperforms the manual lexicon, prediction (word embedding) based approach recommended generate sentiment lexicon for Amharic language to handle context sensitive terms.
REFERENCES
Alessia, Fernando Ferri, Patrizia Grifoni, and Tiziana Guzzo. Approaches, tools and applications for sentiment analysis implementation. International Journal of Computer Applications, 125(3), 2015.
John A Bullinaria and Joseph P Levy. Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior research methods, 39(3):
8

---

Under review as a conference paper at ICLR 2020
510--526, 2007.
S. Gebremeskel. Sentiment mining model for opinionated amharic texts. Unpublished Masters Thesis and Department of Computer Science and Addis Ababa University and Addis Ababa, 2010.
William L. Hamilton, Kevin Clark, Jure Leskovec, and Dan Jurafsky. Inducing domain- specific sentiment lexicons from unlabeled corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2016, pp. 595. NIH Public Access, 2016.
Girma Neshir Alemneh, Andreas Rauber, and Solomon Atnafu. Dictionary Based Amharic Sentiment Lexicon Generation, pp. 311--326. 08 2019. ISBN 978-3-030- 26629-5. doi: 10.1007/979.3-00022663---1 27.
Lucia Passaro, Laura Pollacci, and Alessandro Lenci. Item: A vector space model to bootstrap an italian emotive lexicon. Second Italian Conference on Computational Linguistics CLiC-it 2015, pp. 215--220. Academia University Press, 2015.
Chris Potts. Distributional approaches to word meanings. 1atfloutt0--0-vsmmppppo 2013. Ling
https://web.ste.
claas/ wttp:iitt26/mmattiallll/iggggi/o ford. eedccllsss///////e////./s
236/Psych 236c: Representations of meaning, Spring 2013.
Project teams from Addis Ababa University, Masarykova univerzita, Norges teknisk- naturvitenskapelige universitet, The University of Oslo, and Hawassa University. 7f14047 habit harvesting big text data for under-resourced languages. http://habit-
Peter D Turney and Michael I. Littman. Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems (TOIS), 21(4):315-336, 2003.
Peter D Turney and Patrick Pantel. From frequency to meaning: Vector space models of semantics. Journal of artificial intelligence research, 37:141--188 2010.
Leonid Velikovich, Sasha Blair-Goldensohn, Kerry Hannan, and Ryan McDonald. The viability of web-derived polarity lexicons. Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 777--785. Association for Computational Linguistics, 2010.
Daniel Waegel. A survey of bootstrapping techniques in natural language processing. 2003.
Baye Yimam. (PA*ICT-nPnaayyammart säwasäw. Educational Materials Production and Distribution Enterprise(EMPDE) 2000.


Reviews:
[{"review_id": "SklXBpOQcB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a domain-specific corpus-based approach for generating semantic lexicons for the low-resource Amharic language. Manual construction of lexicons is especially hard and expensive for low-resource languages. More importantly, the paper points out that existing dictionaries and lexicons do not capture cultural connotations and language specific features, which is rather important for tasks like sentiment classification. Instead, this work proposes to automatically generate a semantic lexicon using distributional semantics from a corpus.\n\nThe proposed approach starts from a seed list of sentiment words in 3 pre-determined POS classes. This is followed by deriving a PPMI matrix from the word-context co-occurence matrix (context size= 2). Now given a word, the cosine distance is computed from centroid of each seed classes and words which are similar than a given threshold, are added to the original list. This process is repeated for a pre-specified number of iterations. They apply the generated lexicons to subjectivity detection and sentiment classification task in a small annotated Amharic corpus of facebook posts\n\nStrengths:\n\n1. NLP tasks on low-resource language is very challenging. This paper proposes a efficient, unsupervised way of gathering semantic lexicons that perform reasonably well on a downstream task.\n\nWeaknesses:\n\n1. Related work: The paper needs to cite and mention a much more broader literature. Currently, it mentions just 3 related work, and contrasts itself with one. But given there is so much work on distributional semantics (in English and other languages), they deserve mentioning. Few example of ones are 1. The distributional inclusion hypotheses and lexical entailment (Geffet and Dagan 2005), Distributed representations of words and phrases and their compositionality (Mikolov et al., 2013), Improving hypernymy detection with an integrated path-based and distributional method (Shwartz et al., 2017), Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection (Chang et al., 2018). However, this is by no means a complete list and you should refer to these papers to get a list of other relevant work.\n2. Novelty of work: The novelty of the work is rather limited and the paper should try more low-dimensional embedding based approaches which has been proven to be very effective for a wide variety of tasks. In section 3, the paper mentions there are primarily two kinds of approaches (count based vs embedding based) — The paper should motivate why it chose one over another.\n3. Baselines: The paper would benefit from having some comparison with learned baselines trained from some distantly supervised data from, for example the SWN and SOCAL lexicons. \n4. Organization of the paper: The writing and the organization of the paper needs to be better. For example, equation 3, 4, 5 are rather simple and can be condensed to one equation (no need to show averaging of the seed embeddings). Also in equation 3, would \\vec{w} would be \\vec{x}?. The results and discussions did not have either and over-all there are many grammatical errors. \n5. I am not sure what the subjectivity detection task is in Table 3. Is it a standard task -- if not, the paper should define it first.\n\nOverall, the paper is a nice effort but in its current form it is not ready for ICLR and I hope the comments will help make the paper better for upcoming NLP workshops and conferences.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "Hkec2hOaYB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a corpus-based approach to build sentiment lexicon for Amharic. In order to save time and costs for the resource-limited language, the lexicon is generated from an Amharic news corpus by the following steps: manually preparing polarized seed words lists (strongly positive and strongly negative), calculating the co-occurrence of target word in its context via Positive Point-wise Mutual Information (PPMI) method, measuring the similarity between target words and seed words by cosine distance, iterating with the threshold 100 and 200. The PPMI lexicon is stemmed and evaluated from aspects of subjectivity detection, coverage, agreement and sentiment classification. Three other lexicons: Manual developed by manual, SOCAL and SWN developed by bilingual dictionary, are used as benchmark to compare with the PPMI lexicon. In sentiment classification experiment the PPMI lexicon did not show a superior performance. All the four lexicons have similar accuracy, between 42.16% ~ 48.87%.  Only when the four are combined together the result is improved to 83.51%. \n\nWhile this paper presents an economical and practical method to generate a sentiment lexicon for resource-limited language it is not acceptable in it's current state to ICRL. The following points should be improved or clarified. \n(1) The generalizability to any language needs to be shown. What are the lessons learned for any resource-limited language?\n(2) The fit to ICLR is not perfect. I would expect a stronger focus on representation learning.\n(3)\tThe conclusion is not well proved. Especially about the claim that their method is “with almost minimal costs and time”. PPMI lexicon might cost less than Manual lexicon, but SWN and SOCAL (Neshir Alemneh et al., 2019) are automatically generated by English-Amharic dictionary, not manually. Their generation costs and time are not told and can not be compared here. \n(4)\tThe paper is incorrectly structured. The part “4 RESULTS AND DISCUSSION” presents neither results nor discussion. Part 4.1 shows seed words which should belongs to part 3. Part 4.2 shows stemming which should belong either to part 3 or to part 5. Therefore, it is better to make a restructure. \n(5)\tThe contents following “Discussion:” in part 5 are not discussion.  Moreover, there is almost no discussion in this paper. \n(6)\tIn part 5.4 Table 4, why there is result of PPMI in column “2500 Amharic Comments” but no result in column “20 million Amharic tokens”? Are the 2500 comments stemmed? \n(7)\tIn part 5.6, why negation handling is adopted in part 5.6?\n\nMinor comments:\n(1)\tGrammatical problems such as: Page 3 last paragraph: “Amharic is highly morphological language both inflectional and derivational morphology is complex.” \n(2)\tPage 6 last paragraph: “We will evaluate in three ways: external to lexicon and internal to lexicon.” It should be two ways. \n(3)\tThe number of facebook comments varies: it is 2800 in part 5.3, 2500 in 5.4 including Table 4, and 2821 in 5.6.  \n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJgnKrVaFH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposes a corpus based approach of mining lexicon for a low resource language focusing on Amharic sentiment. The proposed approach is a classic one already presented elsewhere, i.e., PMI to compute the similarity of terms using the co-occurrence statistics and thresholding to control the precision of lexicon. \n\nI have strong concerns to this work.\n\n- The proposed method already appeared in many literatures under different tasks, e.g., translation lexicon generation, or languages, e.g., Chinese, and the proposal is merely a yet another application of PMI.\n\n- The condition of the proposed method is not clear. I initially assumed an existence of a sentiment dictionary for Amharic, but section 3 introduced other resources, e.g., English/Amharic dictionary, which are not clearly stated in the introductory nor in the beginning of section 3. I'd strongly suggest the author to make it clear what kind of assumptions are made, e.g., existence of bilingual dictionary, or not. Otherwise, it is very hard to replicate the experiments.\n\n- Baselines sound not appropriate. Authors should conduct additional studies, e.g., graph propagation algorithm, to make sure that the proposed heuristic based approach is superior or not.\n\nI think this work should have been submitted to other conferences, workshops or journals, focusing more on low resource languages.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This papers addresses the problem of creating sentiment lexicon for a resource limited language (Amharic). This task is time consuming and requires skilled annotators. Hence the authors propose a method for constructing this automatically from News corpora. They start with a seed list of sentiment bearing words and then add new words to this list based on their PPMO scores with existing words. 

While the reviewers agreed that this work is of practical importance, they had a few objections which I have summarised below: 
1) Lack of novelty: The work has very few new ideas
2) Lack of comparison with existing work: Several missing citations have been pointed out by the reviewers
3) Weak experiments: The experimental section needs to be strengthened with more comparisons to existing work as well as proving the results for at least one more language. 
4) Organisation of the paper: The paper needs to be restructured for better presentation. In particular,  the Results and Discussions section does not really contain any discussions.
5) Grammatical errors: Please proofread the paper thoroughly and fix all grammatical and typo errors.

Based on the reviewer comments and lack of any response from the authors, I recommend that the paper in it current form cannot be accepted. 

Decision: Reject