## 5 Smartphone App Development and Evaluation
### Smartphone App Evaluation  
Due to the COVID-19 lockdown measures and difficulties to test the proposed smartphone app in the wild, a preliminary remote testing and evaluation regarding the user experience (UX) was carried out. A total of 33 volunteer users participated in the evaluation process, including ordinary visitors, two of the State Gallery's personnel, as well as other professionals in the field of cultural heritage. Users were asked to download the app on their personal smartphones, use it and familiarize themselves with it, and then follow an online questionnaire created for the evaluation. The online questionnaire was structured into four parts: (A) demographics, (B) using the app, (C) evaluating the app's recognition accuracy and (D)UX evaluation. Five demographic questions were included in Part A concerning the gender, age, familiarity with smartphones, frequency of museum visits and educational background and/or work experience in museology and cultural heritage. Fifteen female and eighteen male participants aged between 16 and 60 years old, who were moderately to extremely familiar with smartphone devices, were involved in the evaluation procedure. One participant stated that they visited a museum one or more times per month, five participants stated that they visited once every 3 months, seven participants stated that they visited once every 6 months, twelve stated that they visited once a year, while the remaining eight stated that they visited a museum less than once a year. Finally, six participants answered positively to the question of whether they had relevant experience or were considered as experts in the evaluation process. The second part of the questionnaire (Part B) asked the participants to explore the app and use it to identify artworks as they would use it during their visit to the State Gallery. More specifically, they were able to select any artwork, from the eight randomly selected, for display on their computer screen and then use the app for its automatic identification and provision of auxiliary information. Participants could repeat the process without restrictions before proceeding to Part C for evaluating recognition accuracy. At this part, the participants were asked to select the name given by the app during automatic recognition in three randomly selected artworks. The app recognized artworks with a high accuracy giving 32 correct answers to the first and second artwork (96.73%), and 31 correct answers to the third one (93.43%). Figure 11: The process used for artwork identification: while the user is pointing at an artwork, the embedded CNN is used to continuously analyze individual frames from the phone’s camera stream. The app considers a sliding window of 5 CNN predictions and picks the artwork with the highest mean probability in the window as the correct artwork. Part D was concerned with the UX evaluation by utilizing a shortened version of the User Experience Questionnaire (UEQ) [34]. Two UEQ questions for each one of the following six scales, attractiveness, dependability, efficiency, novelty, perspicuity and stimulation, were included in the questionnaire (Table 5). The scores depicted in Figure 12 show that the app was positively evaluated, achieving an _Excellent_ evaluation for all scales. Concerning the differences between demographic factors, the results reveal only a significant difference (\(p\)-value = 0.02) between experts and non-experts in the way they evaluate the efficiency of the app. All experts evaluated the app as very fast and practical to very practical. The last part of the questionnaire focused on collecting users' feedback on their overall experience and suggestions for possible improvements. All participants were asked: \"What is the likelihood that the app will positively enhance the experience of a visitor at the Gallery?\". About 44% of participants answered \"Very likely\", 35.5% answered \"Likely\", 17.5% answered \"Moderately likely\", and 3% answered \"Neither likely nor unlikely\".