Information retrieval for animal disease surveillance: a pattern-based approach. (2020)

Abstract: Animal diseases-related news articles are richin information useful for risk assessment. In this paper, we explore a method to automatically retrieve sentence-level epidemiological information. Our method is an incremental approach to create and expand patterns at both lexical and syntactic levels. Expert knowledge input are used at different steps of the approach. Distributed vector representations (word embedding) were used to expand the patterns at the lexical level, thus alleviating manual curation. We showed that expert validation was crucial to improve the precision of automatically generated patterns.

## introduction the ability to rapidly recognise emerging and reemerging animal infectious diseases is a critical global health priority. early warning is crucial for the quick implementation of effective control strategies at global and local levels @xcite . in recent decades, several outbreaks have highlighted the limitations of conventional disease surveillance, which is hampered by delayed detection and latency of the communication channels @xcite . the growing availability of digital data represents an unprecedented source of real-time disease information. online news, social media and electronic health records are among the so-called informal sources that have proven to be valuable sources of disease information @xcite . their mainstreaming into surveillance systems, via the epidemic intelligence (ei) concept, has been a game-changer for disease surveillance and control. ei integrates two components in a single surveillance system: indicatorbased surveillance (collection of structured data through traditional surveillance systems) and eventbased surveillance (collection of unstructured data from informal sources, such as online news articles) @xcite . event-based surveillance (ebs) systems increasingly marshal text-mining methods to alleviate the amount of manual curation of the continuous flow of free text from informal sources @xcite . animal-health online news articles are rich in different types of epidemiological information. for instance, news articles that report an outbreak often also describe outbreak control measures or economic impacts, share information about the outbreak source or draw attention to a given area at risk. those elements may be of relevance to ei teams to assess risks associated with the occurrence of an outbreak. in this paper, our objective is to value new types of epidemiological information contained in news. we describe a pattern-based method to automatically retrieve sentence-level epidemiological information. empirically, sentence-level seems more homogeneous in terms of the topic than the whole news content. our method is an incremental approach to create and expand patterns at both lexical and syntactic levels, with expert knowledge input. the learnt patterns can be further integrated into ebs systems. ## conclusion in this paper, we presented an incremental approach to generate patterns for sentence-based retrieval, in the context of animal disease surveillance. the role of the expert was crucial to pinpoint the irrelevant terms generated by the word embedding model. besides, the time cost of manual curation was minimal, as it mostly consisted of validating or adding terms. this time depends directly on the threshold chosen for top k retrieved terms. the use of syntactic dependency was easy to implement, thus alleviating the cost and bias of wildcard fine-tuning. contrary to the supervised approach, the patternbased method is not hampered by so-called "black box" problems and can be easily enhanced by expert knowledge. besides, even though it was not evaluated in this study, it is suitable for multi-label sentences as a sentence can match patterns from several classes. in future work, we aim at evaluating the pattern-based method on a new and larger corpus of news. besides, it would be interesting to compare the pattern-based results with supervised sentence-based classifiers. we also think that the advantages of each method could be synergistic. a promising perspective could thus be to evaluate how to jointly take full advantage the strength of both methods. @xcite proposed an interesting approach to combine both supervised learning and manually built patterns and rules. they applied heuristic-based regular expression when the prediction confidence of the supervised classifier confidence was less than a specific threshold. such an approach may help enhance the performance of information retrieval in the event-based surveillance context.