Tracking the Evolution of Covid-19 Symptoms through Clinical Conversations (2023)

Abstract: The Coronavirus pandemic has heightened the demand for technological solutions capable of gathering and monitoring data automatically, quickly, and securely. To achieve this need, the Plantão Coronavirus chatbot has been made available to the population of Ceará State in Brazil. This chatbot employs automated symptom detection technology through Natural Language Processing (NLP). The proposal of this work is a symptom tracker, which is a neural network that processes texts and captures symptoms in messages exchanged between citizens of the state and the Plantão Coronavirus nurse/doctor, i.e., clinical conversations. The model has the ability to recognize new patterns and has identified a high incidence of altered psychological behaviors, including anguish, anxiety, and sadness, among users who tested positive or negative for Covid-19. As a result, the tool has emphasized the importance of expanding coverage through community mental health services in the state.

## introduction the covid-19 pandemic required efficient and agile measures from governments to mitigate the effects caused by the disease. plantão coronavírus (pc) is a chatbot, and one of the solutions developed around april 2020 in ceará state, brazil, to hold back the pandemic. pc is an automated tool to converse with patients via text and provide them guidelines on how to proceed based on their level of emergency. the plantão coronavirus project was specifically developed by íris lab of innovation and data in ceará state, brazil, in collaboration with the health secretary of ceará state. its purpose is to create a chatbot with artificial intelligence capabilities. the platform incorporates chatbot technology, enabling users to interact with an artificial intelligence system, and also offers the option to redirect to a virtual service manned by healthcare professionals such as doctors or nurses. the interactions between patients and healthcare professionals through the pc generated a lot of clinical conversation data that needed to be mined, analyzed, and transformed into valuable information. the health secretary was required to track the signs of the disease, and it was not feasible to perform this task manually by reading thousands of texts. in this way, an automated and intelligent solution to classify the covid-19 symptoms was essential. especially at the beginning of the pandemic, when little was known about covid-19. this work proposes a solution to address the issue of screening the symptoms reported by users in a chat dialogue box. ultimately, the chatbot classifies the user's health status as mild, moderate, or severe. based on this classification, the system recommends various services to the user, including medical appointments or covid-19 tests. our solution approaches the identification of symptoms in text as a named entity recognition (ner) problem. ner identifies named entities in documents and categorizes them into predefined classes based on the type of entity @xcite . typically, a neural network is utilized for entity recognition. in this study, the named entity is a symptom. however, in order to train a ner model for extracting symptoms and diseases from clinical conversations in plantão coronavírus, automatic annotation was necessary due to the impracticality of manual annotation given the large size of the dataset. additionally, there was no publicly available ner model for brazilian portuguese that could extract symptoms and diseases. therefore, we utilized scispacy @xcite , a disease-focused ner model trained on englishlanguage texts, and employed transfer learning to build our model. during the training process, we initially translated the portuguese texts into english and used the scispacy model to analyze each english text, extracting the symptoms identified and translating them back into portuguese. the training set comprised the original text and the portuguese symptoms extracted by the scispacy model. the intelligence developed in this study was crucial in identifying patterns of disease indicators, moreover, new or rare symptoms that had not yet been documented by researchers and health professionals in the state of ceará. this enabled the tracking of the evolution of covid-19 findings over time. the entity recognition process was performed automatically. related works as @xcite achieved an f1-score of 78.5% and @xcite reached 84.94% of f1-score for the symptom/disease discovery model. in contrast, this work achieved f1-score equal to 85.66%. the f1-score is a metric that quantifies the harmonic mean of precision and recall. in our case, we report the f1score for the test set. our approach has made significant advances in tackling the disease in ceará, given the possibility of virus mutations and the consequent appearance of new symptoms. with the capability to recognize new patterns, our model identified a high frequency of altered psychological behaviors, such as anxiety, anguish, and sadness, in both covid-19 positive and negative users. as a result, our tool highlighted the need for the state to expand its mental health care services to the population through various channels. our work has enabled the government to develop a public policy to address this need. moreover, less research is available for clinical texts in low resource languages as brazilian portuguese @xcite . one can argue whether or not brazilian portuguese is an low resource language, but we say that, at least for some tasks, there are fewer resources compared to english or other languages, as discussed by @xcite and @xcite . ## data and methods the symptom capture mechanism is the primary contribution of this work and is integrated into the entire triage process, from the chatbot to the teleservice with health professionals. our solution is a technology that leverages plantão coronavírus data to process and identify symptoms contained in natural language texts. in the following sections, we describe the pipeline for building a tracker that monitors the evolution of covid-19 symptoms. note that the sentences from plantão coronavírus were not annotated with symptoms. the main challenge at this point was to build a training set using this data. the pipeline from data collection to implementation of the ner model is shown in figure 1 and described as follows. ## experimental results figure 2 demonstrates the evolution of symptoms in a time series. not all symptoms could be shown in the figure's legend. world health organization (who), at the beginning of the pandemic, stated a set of symptoms commonly reported by people who got positive for covid-19 very related to flu, like nasal flaring, runny nose, malaise, fever, cough, sore throat, diarrhea, headache, and no appetite. however, new symptoms came along. through figure 2 , it is possible to identify the detection of a new symptom of loss of smell on the 8th of may. this symptom appeared and became quite characteristic of covid-19 after a certain period. still, on the time series, it is possible to see that the frequency of each symptom is seasonal during the analyzed period. diabetes is not tecnically a symptom, however our ner model considered as a symptom, possibly due to contextual factors or biases introduced by the scispacy model. however, it is worth mentioning that there has been an observed rise in hyperglycemic conditions associated with covid-19, particularly in patients with diabetes and those receiving steroid treatment @xcite . nevertheless, this topic falls outside the scope of the current paper. another interesting point is that our ner model could capture some symptoms related to altered psychological behaviors, such as anxiety, mental confusion, neurological disorder, and disorientation among the symptoms, as highlighted in figure 3 . figures 2 and 3 do not represent the entire population as a whole, but rather the number of dialogues on the plantão coronavírus platform that report specific symptoms. as already reported in the previous session, our covid-19 symptom tracker achieved an f1-score of 85.66, which is competitive compared to the scispacy english model, which has an f1-score of 85.02. to mitigate catastrophic forgetting of old knowledge as we update our ner model, we kept including new sentences from plantão coronavírus with the following symptoms annotated, such as breathing difficulty, mental confusion, loss of smell, loss of taste, tiredness, anxiety, anosmia, neurological disorder, and disorientation, so the model could learn not only from the frequent symptoms identified by en_ner_bc5cdr_md from scispacy and the symptoms commonly reported by people who got positive to covid-19. ## conclusion and future works this research provides a ner model to recognize covid-19 symptoms in portuguese textual conversations. at the start of the pandemic, no model could automatically identify the symptoms in a text written in brazilian portuguese; instead, we utilized scispacy, an english-language ner model for diseases, which through transfer learning, trained our ner model. the texts were initially translated from por-tuguese into english as part of the training procedure. the scispacy model then processes each english-language input text, and its identified symptoms are subsequently translated from english to portuguese. the original text and the portuguese symptoms determined by the scispacy model comprise the training set. on the plantão coronavírus dataset, our ner model achieved an f1-score of 85.66, which is competitive with the english model of scispacy, which has an f1-score of 85.02. the ner model has brought to light the necessity for the state to increase its coverage of mental health services through the community mental health channel. as a future research direction, we intend to extend the ner model to other diseases prevalent in brazil, such as influenza, and explore various neural architectures. another future work is to investigate the translation performance. one alternative might be manually translate a small sample of our dataset and then compute the bleu score of the automatic translations on this sample for a more accurate estimate. as mentioned earlier, we selected data from april and may 2020, which corresponds to the early stages of the pandemic when plantão coronavírus followed the same protocol. as a future study, we can also examine the evolution of "altered physiological behaviors" in the dialogues that occurred after this time period.