Analysis of the Formality of Text and its Impact on Pharmacovigilance Systems

Digital Content and Media Sciences Research Division School of Multidisciplinary Sciences Doctor of Philosophy Analysis of the Formality of Text and its Impact on Pharmacovigilance Systems by Néstor Álvaro Gradillas This thesis aims to answer the question of whether drug use reports obtained from formal and informal sources have noticeable differences in their formality, and also assess whether these differences in the formality can provide gains to pharmacovigilance systems. Working towards these goals we made three clear contributions that are the development of new resources, i.e. corpora, to perform our studies, a linguistic analysis of the differences between formal and informal pharmacovigilance reports, and an assessment on the impact of the register-related features in pharmacovigilance systems. The first contribution is motivated after finding that there is no repository meeting our requirements, this is, a data set composed of sentences retrieved from academic texts and from social media messages that contain reports on the use of a closed set of drugs that could be used in our linguistic study to provide our second contribution. Our second contribution focuses on the linguistic register where we explore it by using a well established method (Multidimensional analysis) from the area of linguistics that is known for being used to evaluate differences in the formality of texts. By using the multidimensional analysis (MD) proposed by Biber we are able to study the register from a more inclusive point of view as this method does not only account for a set of traits, but it also characterizes the texts using different combinations of the studied features so that other elements such as the “abstractness” or the “narrativeness” of the texts are assessed. The assessment on the differences between generic tweets and drug-related tweets shows that even if both of them belong in the same type of register Biber’s schema is able to capture the differences and helps in telling apart the drug use reports due to their higher level of informativeness. Similarly, an analysis comparing the set of drug-related tweets and the corpus of PubMed sentences shows that the academic texts have more traits of “Information Productions”, proving that the MD analysis can capture those characteristics. For our third contribution we use a set of features able to capture differences in the register and explore the power of those features in drug safety systems. For that we prepare four different binary classifiers for the tasks of detecting sentences containing first-hand experience reports on the drug use, sentences containing beneficial outcomes, sentences containing negative outcomes and sentences containing any type of outcome (positive or negative) related to the drug use. We also build a named entity recognition (NER) system to detect the mentions of drugs and diseases and symptoms. Those systems also explore additional set of features that MD analysis did not assess, but which are known to carry important formality information. Those experiments show that for the set of assessed classifiers using PubMed texts the use of Biber features, the use of custom expansion to those features, nor the use of our set of Politeness features can beat the other configurations of the classifiers that do not include such registerrelated information. We can see, though, that the two of the classifiers for Twitter data do benefit from the use of our custom set of Biber features and from the use of our set of Politeness features as those two sets of features provide significant gains to the baseline system. On the other hand, when evaluating NER systems targeting at the identification of drugs and symptoms and diseases we saw that the set of Politeness features combined with the baseline and word2vec features scored the best result in both PubMed and Twitter data sets, showing that these features can contribute to pharmacovigilance systems.