## 7 System Implemented and Real-World Evaluation  
### System Implemented  
DynDevice is the e-learning platform of Mega Italia Media; it is entirely written in PHP and students can use the chat system to send requests to the tutors. The chatbot system is designed to minimize the changes that have to be made to the existing system. It needs to be able to be easily deactivated in order to redirect the conversation to the human tutor if necessary. Before the introduction of the virtual assistant, the student had to write the request in the chat interface, then the request was sent to the human tutor who answered through the same chat interface. This flow is shown in Figure 11 highlighted in the red dots box. The new system follows a new execution flow, highlighted in Figure 11 with blue dashes, and each step is enumerated:  
1. The chat is opened and the system sends a signal to the conversation manager. Figure 10: Results of Experiment (D) in terms of Error Rate at the increase of the spelling errors. \begin{table}
\begin{tabular}{c l l l l l} \hline \hline
**Experiment** & **Requirement** & **[PERSON]** & **Dialogflow** & **[PERSON]** & **[PERSON]** \\ \hline \hline (A) & F-Score greater than 0.8 & ✓ & ✓ & ✓ & \(\times\) \\ \hline \hline (B) & \begin{tabular}{l} Good performance with few \\ training instances \\ \end{tabular} & ✓ & ✓ & \(\times\) & \(\times\) \\ \hline (C) & \
\begin{tabular}{l} Response times always under \\ 0.5 s \\ \end{tabular} & ✓ & ✓ & ✓ & ✓ \\ \hline (D) &
\begin{tabular}{l} Spelling errors robustness \\ \end{tabular} & ✓ & \(\times\) & \(\times\) & \(\times\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Overall comparison of the platforms tested considering the Experiments (A), (B), (C) and (D). 2. The conversation manager creates a thread representing an instance of a conversation between a student and the NLU engine. Therefore, for each student who opens the chat, the conversation manager creates a dedicated conversation instance; this allows the system to manage multiple conversations simultaneously. Each instance is independent from the others. 3. The student enters the text into the chat interface. The request is sent to the conversation instance. 4. The conversation instance receives the student's request and checks the integrity of the sentence. 5. The conversation instance sends the student's request to the NLU engine using a specific API. 6. The NLU engine returns a message containing one or more intentions recognized with their confidence scores to the conversation instance. The NLU engine often predicts more than one intent, but the confidence allows distinguishing which intention is more likely than another. 7. The conversation instance scans all the intentions recognized and selects the one which has the highest confidence and builds the corresponding answer. If the student's request was unclear for the NLU engine, it is possible to have more intentions with similar confidence values, in this case, a disambiguation message is generated. A disambiguation message is a special request that the conversation instance sends to the student asking to specify, from a list of possible intentions previously selected, the correct one. Whether the intention of the student's request was evident, or the student responds to a disambiguation message, the conversation instance selects the correct answer and sends it back to the student. 8. The student receives the answer and (s)he can enter a new request into the chat interface and send it to the conversation instance. The chatbot may be unable to answer for various reasons; for example, the NLU engine cannot recognize the underlying intention of a request because it was not trained to recognize that specific intent, the student sent an unclear sentence, or the student asked to contact a human tutor. To guarantee the student's support, if the system implemented cannot provide help to the student, it deactivates the flow described, eliminates the conversation instance and redirects the student's conversation to the human tutor.