Improved Topic Representations of Medical Documents to Assist COVID -19 Literature Exploration (2020)

Abstract: Efficient discovery and exploration of biomedical literature has grown in importance in the context of the COVID-19 pandemic, and topic-based methods such as latent Dirichlet allocation (LDA) are a useful tool for this purpose. In this study we compare traditional topic models based on word tokens with topic models based on medical concepts, and propose several ways to improve topic coherence and specificity.

## introduction as the coronavirus 2019 (covid-19) pandemic has presented unprecedented challenges to wellbeing and safety, the medical community has responded by rapidly conducting and publishing a vast amount of related research. this in turn has made it difficult for medical professionals and researchers to keep abreast of the latest evidence. combined with the need to explore legacy literature on related coronaviruses such as sars or mers, there is a need for tools supporting efficient knowledge discovery and exploratory search that goes beyond simple text retrieval @xcite . in this context, representing and visualising the content of documents to allow the user to quickly identify relevant studies is becoming critically important. latent dirichlet allocation (lda: @xcite ) is probably the most commonly used method for topic-based analysis of documents. it was applied by many systems in a recent kaggle challenge over coronavirus literature, 1 and is used in search and exploration tools recently developed for covid-19 research, 2 including our own covid-see system 3 @xcite . however, while conventional lda models work well 1 kaggle.com/allen-institute-for-ai/cord-19-research-challenge 2 discovid.ai, strategicfutures.org/topicmaps/covid-19/ 3 covid-see.com for topically-diverse document collections, they are less informative in narrow, knowledge-rich domains such as medicine, especially when the corpus consists of documents related to one broad topic, such as in the coronavirus-related literature. an ideal topic model should capture more specific, discriminating topics rather than generic topics made up of terms occurring in the majority of documents. in this paper, we compare topic models based on medical concepts with traditional models based on words, and examine the nature of the inferred topics in terms of genericness and coherence. ## related work topic modelling has been applied in biomedical domain to cluster documents @xcite , improve document retrieval @xcite and discover biological relationships @xcite or similar drugs @xcite . in practice, however, lda is most commonly used to discover salient topics in a document collection (see, for example, @xcite ), including for topical representation of covid-19 related literature @xcite . several attempts have been made to improve biomedical topic models by extracting a controlled set of biomedical entities @xcite or using mesh headers @xcite ; our approach differs in that we do not use a set of known relevant terms but rather filter out noninformative words to allow for more unrestricted knowledge discovery. in terms of topic quality, @xcite introduced the notion of "junk" (incoherent) and "background" (generic) topics, which are uniformly distributed over words and documents, respectively. however, though their method allows to rank topics based on their usefulness and quality, the authors do not experiment with improving them in terms of their specificity and coherence. for our experiments we use the cord-19 dataset, which is currently the most extensive coronavirus literature corpus publicly available @xcite . the dataset includes covid-19 and coronavirus-related publications from various sources, such as pubmed central open access corpus, research articles from a corpus maintained by the who, and biorxiv and medrxiv pre-prints. for our dataset we use abstracts of the papers in the corpus, or the first two paragraphs of the full text if no abstract is available. we remove documents in languages other than english using the cld2 library. 4 the resulting dataset consists of 103955 documents with the average length of 156 words. ## experiments incoherent [[1], avian, [2], [3], [4]; lung, acute, or, pulmonary, blood]]; [or, other, infectious, diseases, viral]; [=, ace2, p, 95%, enzyme]; [al, dogs, canine, 2003, milk]; [[1], [3], [2], [4], water]; [r, e, o, n, d]; [will, research, information, or, many]; [t, calves, bovine, immune, mucosal]; [network, (1), (2), s, or]; [after, days, or, group, higher] generic [against, vaccines, drug, activity, development]; [care, medical, health, or, healthcare]; [respiratory, syndrome, coronavirus, middle, east]; [health, public, global, infectious, world]; [analysis, methods, method, biological, sequencing]; [covid-19, sars-cov-2, coronavirus, 2019, case]; [hospital, surgery, emergency, surgical, university] specific [detection, diagnostic, pcr, assay, testing]; [cell, immune, expression, viral, cellular]; [cell, antibodies, antibody, against, immune]; [protein, rna, viral, gene, sequence]; [porcine, strains, diarrhea, pedv, strain]; [risk, mortality, severe, cancer, therapy]; [respiratory, children, infections, viral, tract] ## conclusions and future work in this paper, we have shown that coherence and specificity of topics can be improved through strategies that emphasise domain-targeted conceptual representations of texts. we compared word-based models to models based on medical concepts and showed that the latter, especially when only nongeneric concepts are used, help to induce more informative and useful topics. we also showed that subdividing the documents by their major topic incoherent @xcite ; @xcite ; @xcite generic @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite specific @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; [mortality, associated with, severe, outcomes, ards]; [gene, sequence, evolution, strains, coronavirus]; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; [antibiotic, antibiotics, cap, antimicrobial, respiratory viral infections]; @xcite loglikelihood @xcite ; @xcite ; @xcite ; @xcite ; @xcite