# Employing a Chatbot for News Dissemination during Crisis: Design, Implementation and Evaluation
## 5 Evaluation of COVINFO Reporter Chatbot: Findings and Analysis  
Drawing from earlier chatbot evaluation studies [6; 10; 11], the COVINFO Reporter chatbot was assessed in terms of the following characteristics: (a) performance, (b) reliability, (c) functionality, (d) personalization, (e) interactivity, (f) ethics and behavior, and (g) accessibility. Although the two groups of participants did not interact with each other, both came to the same conclusions and offered similar assessments; therefore, their comments are jointly presented based on the earlier mentioned categorization of the chatbot's characteristics and not on the basis of two separate focus groups. Figure 4: Screenshots of contents (a), available discussion topics (b) and articles (c) in the COVINFO Reporter chatbot. Figure 3: The flow diagram of the COVINFO Reporter chatbot. The _performance_ of the COVINFO Reporter chatbot was examined in regard to its ability to respond in a timely and efficient manner, both via a larger screen (tablet or laptop) and a smaller one (mobile phone). All participants deemed it efficient, and most indicated that, in times of crisis, the chatbot can save time when looking for crucial information: \"It helped me save time while looking for much needed information, for example guidelines on medical facilities\", stated [PERSON]. (female, 20 years old, Cyprus). The vast majority of participants stated that there were no differences between the two screen categories used. However, they found the mobile phone screen to be more efficient, perhaps because they were more familiar with it: \"All the information is there; either you use it via a computer or via a smartphone, but my opinion is that COVINFO Reporter was constructed for a smartphone\" ([PERSON], female, 22 years old, Cyprus). Several problems were initially detected by participants of both focus groups, focusing mainly on the chatbot's technical performance. \"I detected some 'technical problems' in regard to the chatbot's ability to return to the main screen that need to be fixed; otherwise it would be tiring to navigate through it\" ([PERSON], male, 22 years old, Cyprus); \"When using it through a computer, it does not seem to be able to use the full-screen mode, only part of the screen\" ([PERSON], male, 23 years old, Cyprus); \"I agree, this could entail further problems with people with visual disabilities and older users\" ([PERSON], female, 21 years old, Cyprus). _Functionality_ was tested in terms of linguistic accuracy and knowledgeable information offered. Most of the participants agreed that the language used is simple and accurate, and the information offered is useful for everybody during the pandemic crisis: \"I found everything I was looking for--the information was filtered in a useful way and the news categories were expressed with simple words\" ([PERSON], male, 23 years old, Greece). Although some participants indicated that they enjoyed the fact that the COVINFO Reporter offers the basic information needed, others argued that more information could be added: \"At least another information category has to be added regarding information for the pandemic outside the UK, for people who want to know what is happening in other countries\" ([PERSON], male, 20 years old, Greece; [PERSON], 21 years, female, Cyprus). An important point was made by several participants regarding the colors used: \"the use of red color could eventually make us tired while using the chatbot for [a] longer duration of time\" ([PERSON], female, 22 years old, Greece); \"the partial use of white color should be avoided in a news chatbot because it can make users feel bored\" ([PERSON], male, 20 years old, Greece); \"definitely the use of white color for the fonts should be avoided in the chatbot responses; it makes it difficult to read the information offered\" ([PERSON], female, 21 years old, Greece). _Reliability_ was measured in regard both to the content offered and to the chatbot's proper function.