Adapted Sentiment Similarity Seed Words For F rench Tweets’ Polarity Classification (2018)

Abstract: We present, in this paper, our contribution in DEFT 2018 task 2 : “Global polarity”, determining the overall polarity (Positive, Negative, Neutral or MixPosNeg) of tweets regarding public transport, in French language. Our system is based on a list of sentiment seed-words adapted for French public transport tweets. These seed-words are extracted from DEFT’s training annotated dataset, and the sentiment relations between seed-words and other terms are captured by cosine measure of their word embeddings representations, using a French language word embeddings model of 683k words. Our semi-supervised system achieved an F1-measure equals to 0.64.

## introduction sentiment analysis aims to obtain feelings expressed as positive, negative, neutral, or even expressed with different strength or intensity levels. one of the well known extracting sentiment approaches is the lexicon-based approach. a sentiment lexicon is a list of words and phrases, such as excellent and awf ul, each is being assigned with a sentiment polarity. using sentiment lexicon can provide rich sentiment information what make it the foundation of many sentiment analysis systems @xcite . in our previous work @xcite , @xcite , we used tweet's adapted seed-words in english and arabic languages as sentiment lexicon, and then we extracted a score for our test sentences based on the cosine similarity measure between their word embeddings vectors and the sentiment seed-words word embeddings vectors. based on that score, the sentences were classified positive, negative or neutral. to participate in deft 2018 challenge @xcite , we had to adapt our method to deft dataset as language, adapted seed-words and number of classes since deft classify tweets as positive, negative, neutral and m ixedp osn eg. the detailed description of the system and the results of our participation in deft 2018 are presented in the sections below. ## related work the seed-words were the base of many sentiment analysis experiments, some used the concept of seedwords with supervised or semi-supervised methods. for example, ju et al. @xcite worked on a semi-supervised method for sentiment classification that aims to train a classifier with a small number of labeled data (called seed data). some other experiments used the concept with unsupervised methods which reduces the need of annotated training data. for example turney @xcite , which used statistical measures, such as point wise mutual information (pmi), to calculate the similarities between words and its list of 14 sentiment seed-words (good, nice, excellent, positive, f ortunate, correct, superior, bad, nasty, poor, negative, unf ortunate, wrong, inf erior). in our previous work @xcite , new lists of adapted seed-words were extracted in english language. the most frequent words in sentiment140 @xcite were retrieved and then manually filtered to eliminate the irrelevant words. the tests in @xcite showed the efficiency of the new seed-words over turney's 14 seed-words in sentiment polarity and sentiment intensity detection. also, they showed that using cosine similarity measure of word embeddings representations (word2vec) yields better results than using statistical measures like pmi to calculate the similarities between words. for the new system in french language, a similar method of our previous work is followed. a list of sentiment seed-words adapted to tweets regarding french public transport is created, and the fourth type of sentiment classification (mixposneg) is added to our system. ## adapted seed-words the seed-words are words with strong semantic orientation, chosen for their lack of sensitivity to the context. they are used as paradigms of positive and negative semantic orientation. adapted seed-words are seed-words with the characteristic of being used in a certain context or subject. in our previous work @xcite , @xcite , the extracted seed-words were adapted to micro-blogs in general. for example, the word cool is an adjective that refers to a moderately low temperature and has no strong sentiment orientation, but it is often used in micro-blogs as an expression of admiration or approval. therefore, in micro-blogs, the word cool is considered a positive seed-word and the tweet including it is mostly considered a positive tweet. for deft 2018 challenge, and since the tweets are about the french public transport, the extracted seed-words are adapted to this subject. for example the word retard (as late in french) is considered a negative seed-word since the tweets are about transport, and a late train or bus usually provoke negative feelings. the procedure of extracting seed-words is done in three steps : the list of french seed-words adapted to public transport tweets is as shown in positive negative mdr, bien, merci, bon, mdrr, bonne, mdrrr, rigole, retard, merde, grve, grave, ratp, putain, problème, ptdr, juste, aime, beau, cool, super, coucou,respect, trafic, problme, pute, puent, problèmes, odeur, heureusement, rire, adore, bravo, mdrrrr, jolie, coup, panne, mal, flemme, fdp, marre, problmes, belle, blague, ok, gagner, ptdrrr, ptdrrrrr, gueule, bordel, accident, rater, con, chier, ptdrr, génial, gnial, ouais, meilleur, bisous, couilles, retards, perdu, pue, rat, casse, courage, vive, offre, joie, haha, sourire, pu, grves, graves, bah, taper, bizarre, louper, ptdrrrr, tranquille, gentil, parfait, mddr, loup, franais, travaux, galre, galère, fou, bonheur, magnifique, jéme, jme, prfre, chou, chiant, gnant, incident, galérer, peine, ptn, mignon, gratuit, amour, bons, content, remercie chelou, perdre, foutre, morte, tard, horrible, ahah, ouf, direct, trql, heureux, mdrrrrr mauvais, loin, manque, connard honte, tape, for further tests, we extended the lists of adapted seed-words using normafe foot_1 , a tool that creates dictionaries for micro-blogs normalization, in a form of pairs of misspelled word with its standard-form word, in the languages : arabic, french and english. using normafe, we extract the misspellings of our seed-words and we add them to the original list. for example some of the misspellings of the word magnif ique (as magnif icent in french) are : magnf ique, magnif , magnif i, magnif iiiiique, magnif iiique, magnif iique, magnif ik, magnif ike, magnif iq, etc. the result of this procedure is a 1358 positive seed-word and 1330 negative seed-words. ## system of sentiment classification the system is based on sentiment similarity cosine measure with word embeddings representations (word2vec). for this purpose, the word embeddings model foot_2 of 48m french tweets and 683k words by @xcite is used. to predict the tweets polarity, first, each tweet is cleaned by removing links, user names, stop words, numeric tokens and characters. also most emoticons and emojis, like : " :-)" or , are replaced by positive_emoji and negative_emoji, since these expressions replace most emoticons and emojis in the word embeddings model. after that, the tweet is segmented into tokens or words. the similarity between each word with positive seed-words and negative seed-words is calculated using gensim tool 3 with the previously mentioned word2vec model. having the sentiment score of each word in a tweet, we aggregate by sum to combine these values. the final score specify the tweet's polarity as positive, negative or neutral. the deft 2018 task 2 requires determining the overall polarity of a tweet as positive, negative, neutral or mixposneg. therefore, we need to detect the tweets of mixed polarity as mixposneg, which is not covered by the current system. by observing the training dataset, we notice that a large number of mixed polarity tweets are a combination of a text with a certain polarity and an emoji with an opposite polarity, like the following example where the person is mostly complaining about negative events and then he adds a smiley at the end of his tweet which shows sarcasm and the expression of mixed sentiment : " ... des vieux types mn clc dans le métro et un pigeon s'est lacher sur ma veste ... ". based on the previous observation, we decide to consider the tweets of a certain polarity with an emoji of opposite polarity as a mixedposneg tweets. ## results for deft 2018 task 2 challenge, four runs are sent to predict the polarity of 7816 public transportation tweets in french language : ## conclusion this paper presents our contribution in deft 2018 task 2 : "global polarity". the system used is based on a list of sentiment seed-words adapted for french public transport tweets. these seed-words are extracted from deft's training annotated dataset, and the sentiment relations between seed-words and other terms are captured by cosine measure of their word embeddings representations, using a word embeddings model of 683k french words. we participated at the deft challenge with four runs, and our best run achieved an f1-measure equals to 0.64 as shown in even though our results were not the best, but they are promising results since our best results are achieved predicting only three classes : positive, negative and neutral, in a challenge where the prediction of four classes is required (positive, negative, neutral and mixedposneg). unfortunately, our method to predict the fourth class m ixedp osn eg decreased the f1-measure to 0.63 in run_4 and to 0.61 in run_2. therefore, and as a future work, we are seeking on a new method to predict the fourth class m ixedp osn eg, by predicting the polarities of tweet segments and detecting opposite polarities in the same tweet.