# Moodle LMS Integration with Amazon Alexa: A Practical Experience
## 4 Usability Evaluation  
We confronted several issues while developing our Alexa skill. One of the conscious decisions we made was to hard-code part of the events' information on a file, as opposed to get the information programmatically from Moodle instance directly. This decision meant that, while any user with an account in Moodle would be able to access and interact with the Alexa skill, only those who were participants on the particular courses we had chosen and which events' information was included in the file. However, as mentioned, any user with an account on Moodle would be able to access and interact with the Alexa skill, and that would include the introduction to the use of the skill, as well as the account linking process and other available paths of interactions. Once our skill was fully developed, deployed, and publicly available, we began a usability evaluation to understand the performance and impact that the skill could have on the students. It is of importance to note that we did not recruit any student to participate in the evaluation of the skill. The students were told about the Alexa skill on the course presentation day and the goals of the project were explained to them. As part of the evaluation we aim to measure what would be the acceptance level of the Alexa skill and how students would adopt such innovation, understanding if students would interact with the skill willingly on their own. To increase the amounts of data measured over the evaluation period, we decided to boost the interest of students for using the skill on their own by offering an Alexa Echo Flex device as reward from a drawing between those students who would interact with the skill. Totally, there were 61 students accounted to be part of the course to be used for the usability evaluation. Prior the beginning of this study, both the authorization of the Bioethics Committee of the University of Burgos and the informed consent of all participants were obtained in writing (see the Ethics Statement section at the end of this article). The student group was composed by 56 female and 5 male students, with further statistical data to be seen on Table 2.  
The evaluation was conducted over the period of two months, from 1 February 2020 until 31 March 2020. Information regarding the usage of the skill was taken from the Analytics section found on the Alexa Developer Console (See Figure 8). This tool makes it possible to monitor many aspects on how the skill is being utilized by the users and how it is performing. These measurements will allow us to study if, and how, the technical performance of the skill might have a positive or negative impact on its usability, and detect those areas where problems might arise during the interaction of the users with the skill. A short non-mandatory survey was given to students at the beginning of the course to better understand their familiarity with Amazon Alexa technology. This survey was composed of one question with dichotomous reply options (yes/no) and a second one with multi-choice question (see Table 3). The answers from this survey offered us information to create participant profiles, which could later on help us understand better the results obtained by the usage of the skill and the usability evaluation (see Figure 9). From the 26 students who participated in the survey, 12 of them (46.2%) answered that they had used a voice-activated device before this evaluation. Only one of the participants answered to use the device on a weekly basis, and ten of them answered to use it seldom. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Participant Gender** & **Count** & **Mean Age** & **Standard Deviation Age** \\ \hline Male & 5 & 22.00 & 2.24 \\ Female & 56 & 23.54 & 6.30 \\ \hline Total & 61 & 23.36 & 6.01 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Description of the evaluation sample.