LITL at SMM 4 H : An Old-school Feature-based Classifier for Identifying Adverse Effects in Tweets (2020)

Abstract: This paper describes our participation to the SMM4H shared task 2. We designed a rule-based classifier that estimates whether a tweet mentions an adverse effect associated to a medication. Our system addresses English and French, and is based on a number of specific word lists and features. These cues were mostly obtained through an extensive corpus analysis of the provided training data. Different weighting schemes were tested (manually tuned or based on a logistic regression), the best one achieving a F1 score of 0.31 for English and 0.15 for French.

## overview this article describes the participation of the students of the litl master and their teachers to the social media mining for health (smm4h) shared task 2 @xcite . litl (stands for linguistique, informatique, technologies du langage, i.e. linguistics, it, language technologies) is a master's program at the university of toulouse, france that is mainly aimed at linguistics and humanities students. the shared task is a binary classification of twitter messages in different languages, indicating whether the message contains a mention of medication adverse effects. participation to this task was part of the first year students' curriculum. at this stage, their computer skills were still limited to corpus processing and simple programs, so it was decided that the system's architecture would be a traditional linear classifier based on ad-hoc features. this approach was also deemed justified given the heavily biased distribution of data (known to be an issue for most machine learning techniques). however, the students were encouraged to apply and hone their corpus linguistics skills, and to perform some feature engineering. the approach was the following: ## technical details for observation and actual processing in both languages the tweets were preprocessed as follows: fifteen word lists were compiled for each language, each one targeting a specific aspect of the tweets content. those word lists contain keywords extracted from the target tweets and non target tweets using the txm corpus analysis tool @xcite . the lists were extended with existing lexical resources such as sentiment lexicons (e.g. the socialsent lexicon @xcite and the feel -french expanded emotion lexicon @xcite ) and biomedical domain language resources @xcite . three additional non-lexical features were also used: number of hash tags, number of urls and number of twitter user names (i.e. sacha, cf. supra). each feature was assigned a weight proportional to its relative importance in the decision process. for runs 2 and 3 the weights were individually fixed based on the frequency ratios in target (vs non target) tweets in the training data, and then manually adjusted based on the scores obtained on the validation sets. the best weights were found by progressively increasing the weight of each feature independently of each other until the best f1 score is reached. for run 2, we adopted a principle of equality between features. for run 3, some features were considered as more important than others on the basis of manual observations. run 1 used a standard logistic regression classifier trained on training data. ## results and discussion