# Using Chatbots as AI Conversational Partners in Language Learning
## 5 Results and Discussion
After considering the participants' responses, it became necessary to add two new codes to our analysis: intentional meaning (pragmatics) and data privacy (design). Quantitative results seemed to be confirmed by the qualitative findings. Table 10 shows some comments selected from the template analysis (TA). Customizing options, semantic coherence and lexical richness became key factors for the teacher candidates' satisfaction. The three conversational partners selected had not been specifically programmed for language learning, so error detection and correction was not necessarily one of their functionalities. However, chatbot-human communication was not interrupted despite the presence of intentional mistakes or the use of abbreviations (P23 in Table 10). Still, some participants believed this lack of error correction could pose a problem for learners with a lower English level (P102), as already pointed out in previous works [47]. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Gender** & **[PERSON]–Whitney U** & **Z** & \begin{tabular}{c} **Sig.** \\ **(2-Tailed)** \\ \end{tabular} & **\
\begin{tabular}{c} **Educational** \\ **Setting** \\ \end{tabular} & **Mann–Whitney U** & **Z** &
\begin{tabular}{c} **Sig.** \\ **(2-Tailed)** \\ \end{tabular} \\ \hline TAM2 & 13,848,500 & \(-\)1.699 & 0.089 & TAM2 & 6,822,000 & \(-\)0.573 & 0.567 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Effect of educational setting and gender on the TAM2 results. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Chatbot** & **Category** & **Code (CHISM Item/s)** & **Freq.** \\ \hline Replika & Benefits & 1. natural conversation (semantic coherence \& response interval) & 75.5\% \\  & & 2. avatar can be customized (design) & 75.0\% \\  & & 3. use of inclusive language and design, politeness (design \& lexical richness) & 74.0\% \\  & & 4. rich vocabulary, colloquialisms (lexical richness) & 73.0\% \\  & & 5. storage capacity, memory, keeps a diary (design \& interface) & 72.5\% \\  & Limitations & 6. privacy issues, asks for personal information (data privacy) & 54.0\% \\  & & 7. chatbot did not correct mistakes (error correction) & 42.5\% \\  & & 8. chatbot made some grammar mistakes (grammatical accuracy) & 21.0\% \\ \hline [PERSON] & Benefits & 1. wide vocabulary options, colloquial expressions (lexical richness) & 75.0\% \\  & & 2. coherent responses, no off topics (semantic coherence) & 64.5\% \\  & & 3. a good number of social media options (multimedia content) & 64.0\% \\  & & 4. rich variety of girls, memes and emojis (non-verbal language) & 63.0\% \\  & & 5. immediate response (response interval) & 52.5\% \\  & Limitations & 6. bad sense of humor, very sarcastic, offensive (intentional meaning) & 54.0\% \\  & & 7. insistence on ungradable pay options (design and data privacy) & 52.0\% \\  & & 8. some of topics and misunderstandings (semantic coherence) & 21.0\% \\ \hline Wysa & Benefits & 1. innovative and helpful tips about mental issues (design) & 66.5\% \\  & & 2. friendly avatar, penguin (design) & 45.0\% \\  & & 3. user can exercise and play games or learn new things (multimedia) & 38.5\% \\  & & 4. no error correction but does not hinder communication (error correction) & 35.0\% \\  & & 5. positive use of pics and emojis (non-verbal language) & 34.5\% \\ \cline{2-5} Limitations & & 6. some responses based on preset options, predictable (natural interaction) & 59.5\% \\  & & 7. poor voice technology, canned (R\& S technologies) & 45.0\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Benefits and limitations of each conversational agent (code frequency). Similarly, the use of non-verbal language in the conversation, such as emojis and memes, and multimedia content, for example videos, enhanced the participants' engagement (P23), which confirms the multimodal nature of modern computer-mediated communication (instant messaging, social networking).