# Predictive Evaluation of Artificial Intelligence Functionalities of Software: A Study of Apps for Children's Learning of English Pronunciation

Mengqi Fang, Mary Webb

► To cite this version:

Mengqi Fang, Mary Webb. Predictive Evaluation of Artificial Intelligence Functionalities of Software: A Study of Apps for Children's Learning of English Pronunciation. IFIP World Conference on Computers in Education (WCCE), Aug 2022, Hiroshima, Japan. pp.100-112, 10.1007/978-3-031-43393-1_11. hal-05533332

HAL Id: hal-05533332

https://inria.hal.science/hal-05533332v1

Submitted on 2 Mar 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
<figure><img src="image_3.png" /></figure>

# Predictive Evaluation of Artificial Intelligence Functionalities of Software: A Study of Apps for Children's Learning of English Pronunciation

Mengqi Fang(✉) and Mary Webb

King's College London, London, UK
{mengqi.fang, mary.webb}@kcl.ac.uk

**Abstract.** This paper is based on a study that developed and used a predictive evaluation method to evaluate educational software with embedded Artificial Intelligence (AI) functionalities for children's learning of English pronunciation. The approach built on Squires and Preece's [1] heuristics which used a social constructivist view of learning but were developed before AI functionalities were commonplace in educational software. Three AI-powered English learning apps were selected for the predictive evaluation. The evaluation enabled a comparison between the AI-powered English-learning apps based on their potential for learning. The study is the first to predictively evaluate pedagogical values of AI functionalities for improving ESL pronunciation and may provide an approach to developing predictive evaluation more widely for educational software that embeds AI functionalities.

**Keywords:** AI Functionalities · Software Evaluation · Predictive Evaluation · English Learning · English Pronunciation

## 1 Introduction

AI functionalities have been incorporated into educational software for some years [2] but recent developments have greatly increased the potential of AI in education [3] and the range of AI functionalities that may be incorporated into software. Earlier explanations [2] of the potential of AI to support learning focused on: 1) the system being knowledgeable in the domain to be taught; 2) the modelled expertise enabling the system to conduct interactions; 3) natural language processing enabling improved interaction; 4) planning for the learner or inferring learners' plans from their behaviour; 5) self-improving systems that learn from the effects of their behaviour. These functionalities are still prevalent in AI-enabled educational software but they have been further improved by developments especially in machine learning [3]. Machine learning (ML) is the underlying driver of most recent AI applications. ML uses a range of different algorithms that can “learn” from data and therefore they learn and adapt to students’ behaviour in real-time. Furthermore, significant research has focused on embedding AI functionalities into intelligent
agents (bots) that can perform various roles in educational software including tutor, mentor or co-learner. AI functionalities currently being embedded in educational software include: automatic evaluation of students' skills and understanding; personalised feedback; expert systems; intelligent human-computer interaction including identifying students' emotional responses, visualisation, intelligent agents, teachable agents, adaptive instruction and tracking learning activities [3, 4].

The rapid development and incorporation of AI into education have led to concerns about insufficient teacher preparation and teachers' limited understanding of AI and its potential for learning [5]. In order to make effective decisions about the use of software with students, all educators including parents as well as teachers need to be able to predictively evaluate [6] the pedagogical potential of software. However, approaches to predictive evaluation of educational software were developed prior to incorporation of significant AI functionalities [1] and little recent research exists in this area. Therefore, there is a need to examine how software with AI functionalities can be evaluated and whether previous approaches can be applied or adapted.

The study on which this paper is based focused on AI functionalities in English language learning software for young Chinese learners to develop their pronunciation. This context was chosen because AI is becoming used extensively in such software for two main reasons. First the Chinese government has made a massive investment, issued tax breaks and policies to facilitate the use of AI in education [7]. Secondly as English has become a global lingua franca, speaking English is increasingly important and pronunciation is a basic element in building speaking skills and enhancing intelligible communication [8]. However, young Chinese ESL learners inevitably face challenges of learning pronunciation owing to the lack of exposure to English and limited professional and personal guidance for English pronunciation in China [9, 10]. These actions and concerns have been an impetus for the design of AI-powered language learning tools (such as English Liulishuo, Tencent English, Open Language, Zebra English, Squirrel AI learning, Whales Love, etc.), and stimulated tens of millions of students to use AI-powered language learning applications [7]. AI functionalities currently available in such software typically include automatic evaluation of pronunciation, personalised feedback, intelligent human-computer interaction, adaptive instruction and tracking of learning activities [11]. However, the support for using AI-powered English learning apps seems to have been driven by development of technologies rather than considerations of learning [12]. Limited evidence about the effectiveness of AI in supporting language learning, particularly the pedagogical value of AI functionalities for children's learning of English pronunciation has been provided.

In this paper we begin with a review of developments and challenges of predictive evaluation especially social constructivist approaches which are particularly relevant to language learning but are also relevant to most areas of learning. Based on this review, we adopt Squires and Preece's [1] social constructivist evaluation approach. We explain our procedure for identifying and selecting AI-powered English-learning apps for the evaluation. We then discuss our adaptation of Squires and Preece's approach for predictive evaluation and present the results of our evaluation of three selected apps. We conclude with implications for practice and further research.
# 2 Literature Review

## 2.1 What Is Predictive Evaluation?

Predictive evaluation is an assessment of the quality and potential of a software application performed before its intended use with students [6]. Teachers typically conduct such an evaluation when planning lessons or purchasing educational software [6] and parents also need to be able to make informed choices. There are two types of predictive evaluation: informal and formal [1]. Informal predictive evaluation depends on previous experiences to make a judgement concerning the potential and quality of an application before using it [1]. However, such an evaluation could be challenging when teachers and parents have limited experience and face new and innovative educational software applications. Therefore, a formal approach that enables educators to review software and make initial decisions about which software to use is needed [1]. The typical approach of formal predictive evaluation is to use a checklist which provides a set of questions addressing both educational and usability issues. Usability typically refers to the degree of ease and effectiveness of use [13]. However, checklists have been criticized by Squires and Preece [6] who argued that typically checklists fail to: 1) encompass a consideration of learning issues [1] and 2) consider the interaction between learning and usability in relation to how, for example, technologies are used for pedagogical purposes or how usability issues relate to learning outcomes. Similar problems were also identified more recently in Dimitrijević and Devedžić's study [14] which systematically reviewed 45 papers regarding methods used to select educational technologies. McDougall and Squires [15] maintained that the problems associated with using checklists are symptomatic of the failure to adopt a situated perspective on the evaluation of educational software. Since learning is situated in a specific context and influenced by environments, people and artefacts, evaluations of educational software need to be conducted according to the educational settings in which they will be used [15]. Therefore, scholars have developed social constructivist guidelines with consideration of both learning and contexts in which learners exist.

## 2.2 A Social Constructivist Approach to Predictive Evaluation of Software

Social constructivism is often used to explain the nature of learning and is frequently applied to language learning [16]. According to Vygotsky [17], the learning process is affected and mediated by external factors such as cultural, historical and social interactions. Vygotsky [17] believed that children first obtain knowledge and develop their skills in a social context and then internalise and use those skills as individuals. Therefore, the construction of their knowledge is not independent of the social and cultural contexts in which learners exist.

Vygotsky [17] further proposed that a child's learning occurs in their zone of proximal development (ZPD), which is defined as 'the distance between the child's actual developmental level as determined by independent problem-solving and the level of potential development as determined through problem-solving under adult guidance, or in collaboration with more capable peers' [17]. Thus, a learner is better able to learn through social interaction with more capable people, such as teachers, tutors and peers
who can act as knowledgeable others and help the learner by providing guidance, suit-
able tasks, feedback and support [18]. Traditional collaborative learning happens in small
human groups. However, recently collaboration has been extended to involve non-human
agents such as virtual agents or robots with AI technologies [19].

In Vygotskian theory, artefacts are also important for learners to construct knowledge or accomplish their goals [17]. Artefacts can be physical (pen and paper), symbolic (music, numbers, arithmetic systems and languages) or cultural (technologies, portfolios and tasks) tools which can be used for learning a second language (L2). Vygotsky [17] argued that humans do not act on the world directly but rather rely on artefacts and activities to mediate or regulate their relationships with others and themselves. Lantolf [16] who pioneered the application of Vygotsky's theories in second language learning agreed that, through mediational tools, learners can strengthen their control over their behaviours externally and regulate their minds internally.

An AI-powered English-learning app can be regarded as a cultural artefact that mediates learning of ESL pronunciation. Learners can initially use the AI functionalities to accomplish activities with the assistance of intelligent tutors, interact with others and co-construct knowledge. They then internalise that knowledge individually.

Squires and Preece [1] claimed that adopting a socio-constructivist approach can help provide insights into the interaction between usability and learning in the context in which the software is used. From the social constructivist perspective, constructing knowledge is both a social and cognitive process [20]. Squires and Preece [1] identified the importance of learning being authentic and described cognitive and contextual authenticity—as key perspectives for predictively evaluating educational software.

According to Squires and Preece [1], cognitive authenticity consists of three con-
cepts: credibility, complexity and ownership. Credibility refers to the plausible oppor-
tunities that the environment (educational software) offers to learners for exploration
and learning. The environment featured in the software should be a simulation of a real
learning context; learners should be provided with a channel to express their opinions or
ideas and to actively construct their knowledge. In an authentic learning environment,
learners are provided with feedback about their actions, reflecting the influence of their
actions on the system, artefacts or environment. The feedback can also allow users to
correct their responses, test their ideas and attempt different solutions to their problems.
In addition, credibility refers to the activities offered by an environment that support mul-
tiple knowledge representations, thus allowing learners to gain multiple perspectives and
experience various contexts.

Complexity refers to the intricacy of learners and the environment [1]. As suggested by social constructivism, diversity in terms of learners’ prior knowledge or experience can account for their different learning goals or processes, thus influencing their construction of knowledge. To manage such complexity, learners need help or scaffolding. Educational software, which functions as a kind of cultural artefact, is expected to offer this help by addressing learners’ differences and specific problems.

As learning is supposed to be actively constructed and internalised by learners in a
social constructivist learning environment, a sense of ownership is important. The soft-
ware can provide strategies that encourage learners to take control of and be responsible
for their individual learning. Therefore, strategies that promote intentional learning and motivation are important to consider when evaluating educational software.

In line with a situated perspective and social constructivism, Squires and Preece [1] also characterised contextual authenticity, which concerns two components of the learning environment: the real curriculum and collaboration. Contextual authenticity emphasises that the curriculum and skills provided in an app need to meet learners' goals and needs. Furthermore, the quality of a curriculum's content is also critical; for example, determining whether the content relates to learners' prior knowledge. Finally, regarding social interactions, collaborative learning such as peer discussions and group work must be considered when evaluating software.

The feasibility of applying the concepts of cognitive and contextual authenticity proposed by Squires and Preece [1] in predictive evaluation of language learning software is supported by drawing parallels with more recent studies that used social constructivism to evaluate traditional language learning software. Simina and Hamel [21] presented the characteristics of an ideal socio-constructivist language learning software environment for second language acquisition. According to Simina and Hamel [21], the characteristics of the language learning software environment should be centred on learners and provide a space in which learners are free to form their own interpretations, which corresponds to the credibility concept proposed by Squires and Preece [1]. Moreover, the software environment should promote the authenticity of learning through providing context-related activities for learners to connect both new and prior knowledge, which corresponds to the contextual authenticity proposed by Squires and Preece [1]. Simina and Hamel [21] added that the ideal language learning software environment should embed assistance for learners in constructing knowledge and learners should be able to interact with others and share multiple representations to reflect and monitor their learning progress. The opportunities for learners to interact with others and share multiple representations are in line with the collaboration and credibility concepts proposed by Squires and Preece [1].

Overall, the elements of the ideal socio-constructivist language learning software proposed by Simina and Hamel [21] closely correspond with Squires and Preece's [1] emphasis on cognitive and contextual authenticity thus lending support for evaluation of language learning software that focuses on these elements.

While Squires and Preece [1] developed a tentative set of heuristics incorporating learning and usability considerations for evaluating the types of software available at that time, our analysis suggested that the nature of AI-functionalities was changing usability considerations. Therefore, we developed a framework that characterised the intersection between the elements of Squires and Preece's [1] analysis of cognitive and contextual authenticity and AI functionalities. First we identified the AI functionalities of English language apps for ESL pronunciation.
# 3 Predictive Evaluation of the AI Functionalities of English Learning Apps for ESL Pronunciation

## 3.1 Selection of AI-Powered English Learning Apps for Pronunciation

To identify the AI-powered English-learning apps available to Chinese children, we searched for the keywords 'ESL pronunciation' and 'AI' in the App Store. We then filtered these apps according to their reviews and ratings and chose apps designed for ESL learners aged under 16. To ensure the apps had embedded AI features, we reviewed the apps' websites and explored the apps. We selected the three most well-known AI-powered English-learning apps which feature free content that is facilitated by AI functionalities for learning ESL pronunciation (see Table 1).

<table><caption>**Table 1.** Selected Apps</caption><thead><tr><th>App name</th><th>English<br>Liulishuo</th><th>Open Language</th><th>Tencent English</th></tr></thead><tbody><tr><td>URL</td><td>https://www.liulishuo.com/liulishuo.html</td><td>https://www.opeanlanguage.com/p/index/</td><td>https://study.qq.com</td></tr><tr><td>App store rating</td><td>4.5; 130,000<br>ratings</td><td>4.9; 69,000<br>ratings</td><td>4.6; 25,000<br>ratings</td></tr><tr><td>Target age group</td><td>Age over 12</td><td>Age over 12</td><td>Age over 4</td></tr><tr><td rowspan="4">AI functionalities</td><td>Automatic<br>evaluation</td><td>Mark<br>pronunciation<br>using a number<br>score and a<br>colour</td><td>Mark<br>pronunciation<br>using a number<br>score and a<br>colour</td></tr><tr><td>Personalised<br>feedback</td><td>Implicit<br>feedback; Audio<br>of model<br>pronunciation<br>accompanied by<br>a score</td><td>Implicit<br>feedback; Audio<br>of model<br>pronunciation<br>accompanied by<br>a score</td></tr><tr><td>Intelligent<br>communication</td><td>A casual chat<br>with a humanoid<br>chatbot</td><td>Fixed<br>conversation<br>with a chatbot<br>that has no<br>appearance</td></tr><tr><td>Adaptive<br>instruction</td><td>Adaptive<br>feedback<br>provided by the<br>chatbot on<br>learners' output</td><td>Test learners'<br>competence and<br>promote courses<br>based on<br>individual levels</td></tr></tbody></table>

(continued)
<table><caption>**Table 1.** (continued)</caption><thead><tr><th>App name</th><th>English<br/>Liulishuo</th><th>Open Language</th><th>Tencent English</th></tr></thead><tbody><tr><td>Tracking<br/>learning<br/>activities</td><td>Record the<br/>number of<br/>learning days in<br/>a week, average<br/>scores and<br/>average duration<br/>of learning</td><td>Record the<br/>courses that have<br/>been followed</td><td>Record the<br/>duration of<br/>engagement, the<br/>number of<br/>vocabulary words<br/>and sentences<br/>learned, and the<br/>learning score</td></tr></tbody></table>

Note: all data about ratings and downloads were obtained on 15 December 2021

## 3.2 AI Functionalities for English Pronunciation

The selected AI-powered learning apps have multiple AI functionalities that can support learning ESL pronunciation. We analysed the AI functionalities of the selected learning apps to determine how they support social constructivist heuristics (see Table 2 for a summary).

<table><caption>**Table 2.** The Relationships Between AI Functionalities and Social Constructivist Heuristics</caption><thead><tr><th rowspan="2">AI<br/>functionalities</th><th colspan="3">Cognitive authenticity</th><th colspan="2">Contextual authenticity</th></tr><tr><th>Credibility</th><th>Complexity</th><th>Ownership</th><th>Collaboration</th><th>Curriculum</th></tr></thead><tbody><tr><td>Automatic<br/>evaluation</td><td>Help learners<br/>monitor and<br/>test ideas;<br/>Guide learners<br/>to solve<br/>problems</td><td></td><td></td><td></td><td></td></tr><tr><td>Personalized<br/>feedback</td><td>Provide<br/>multiple forms<br/>of feedback<br/>and multiple<br/>ways for<br/>learners to<br/>learn</td><td></td><td></td><td></td><td></td></tr></tbody></table>

(continued)
<table><caption>**Table 2.** (continued)</caption><thead><tr><th rowspan="2">AI functionalities</th><th colspan="3">Cognitive authenticity</th><th colspan="2">Contextual authenticity</th></tr><tr><th>Credibility</th><th>Complexity</th><th>Ownership</th><th>Collaboration</th><th>Curriculum</th></tr></thead><tbody><tr><td>Intelligent communication</td><td>Act as knowledgeable others and simulate teacher-student interaction</td><td></td><td>Motivate learners to control and be responsible for individual learning</td><td>Act as peers to practise pronunciation with</td><td>Provide contextualised input and authentic learning materials</td></tr><tr><td>Adaptive instruction</td><td></td><td>Adaptive feedback or instruction that is consistent with learners' performance</td><td></td><td></td><td>Consider learners' ZPD; promote courses related to a learners' individual level</td></tr><tr><td>Tracking learning activities</td><td>Help learners monitor their learning process and make improvement</td><td></td><td></td><td>Share learning reports with others; extend authentic learning</td><td></td></tr></tbody></table>

**Automatic Evaluation of Pronunciation.** All three selected learning apps feature AI-enabled automatic scoring to evaluate pronunciation quality and provide real-time scoring or feedback regarding learners' pronunciation of individual words. Open Language evaluates learners' pronunciation and provides a score based on the criteria of fluency, accuracy and integrity. Tencent English assesses learners' pronunciation based on the three abovementioned criteria but does not provide a score to mark learners' performance; instead, it rates learners' pronunciation using one to five stars, which represents implicit feedback. Similarly, English Liulishuo also assesses learners' pronunciation with regard to accuracy, fluency, integrity and intonation and gives a score for each of these elements. The fluency, accuracy, integrity (i.e. wholeness and completeness of the utterance) and intonation are all important elements that should be considered when assessing pronunciation.

The three apps all allow learners to practise their pronunciation by speaking into a microphone. The apps then record, recognise and analyse English speech by using sophisticated integrated algorithms to compute scores (in the form of values ranging from 0 to 100) or a number of stars thus providing immediate feedback. Furthermore, when learners practise their pronunciation, their performance is highlighted by a colour in all three apps—poor pronunciation is normally highlighted in red, whereas correct
pronunciation is highlighted in green. This visualisation can help learners become aware of their mispronunciation. Thus, learners can adjust their pronunciation learning plans or repeatedly practise pronunciation to achieve a higher score over time.

Automatic evaluation simulates pronunciation tutoring, in which learners can obtain an immediate evaluation of their performance and have their pronunciation automatically corrected. The intelligent evaluation function of AI is similar to an English teacher who has the patience to constantly offer feedback to learners. Learners who are provided with such immediate feedback can monitor their progress. This functionality, which simulates a real learning context, is consistent with the ideal social constructivist language learning suggested by Simina and Hamel [21]. Furthermore, in line with the notion of credibility proposed by Squires and Preece [1], the feedback can help learners test their ideas and guide learners to solve their problems and improve their performances.

**Personalised Feedback.** The three mobile learning apps provide personalised feedback based on individual learners' mispronunciations or problems. The Tencent English app offers personalised feedback for particular sounds a learner has mispronounced by highlighting their errors in red. When they press the red-coloured word or phoneme (unit of sound), the learner receives specific instructions for further practice such as making the sound, understanding words that contain the specific phonemes and studying sentences that contain the words a learner has mispronounced. The instructions are accompanied by visual or textual aids, such as a video that features a model speaker correctly pronouncing a word. Providing feedback in multiple forms enables learners to explore and learn to choose the types of feedback they prefer in order to construct their knowledge actively. In Open Language, when learners press their mispronounced words, they only receive feedback on the word in the text, accompanied by an audio of model pronunciation. Similarly, English Liulishuo only displays learners' scores and colours learners' mispronunciations with limited textual instructions concerning how learners can correct their pronunciation. Learners need to press mispronounced words to obtain hyperlinks to the model pronunciation, which they then imitate and rerecord their pronunciation. It seems that in these two apps, learners follow automatic speech evaluation-based imitation-speaking methods with immediate feedback. Such feedback provides little information about errors in pronunciation or the causes thereof, which may impede learners' attempts to correct their pronunciation. Furthermore, the representation of the feedback is limited to text and audio elements, which means that these two apps cannot provide alternative ways for learners to explore pronunciation correction and limit multiple interpretations. This more limited range of feedback could further the fossilisation of errors and might lead to frustration.

**Intelligent Communication.** Intelligent communication refers to human-computer interactions in which learners speak English to chatbots on mobile phones or computers and receive a response. This functionality is supported by natural language processing (NLP) and machine learning (ML) techniques, which create a natural language communication environment. All three apps have a chatbot functionality. In Tencent English and Open Language, learners can roleplay with a chatbot to practise speaking. The roleplay creates a supportive speaking environment for students who are reluctant to speak in front of other people. Learners are provided with various speaking tasks to complete whenever they choose. In accordance with social constructivist principles, the chatbot
can act as a peer for learners to practise and collaborate with. The chatbot also acts as a teacher or knowledgeable other for learners, as it incorporates machine learning to detect learners' pronunciation, provides personalised feedback for individual learning and promotes learners' interactions with it.

However, the chatbots in Tencent English and Open Language are not trained to respond to casual conversations. The human-chatbot communication seems to be based on drill practices that resemble a traditional pronunciation-training approach [22]. Unlike the predetermined chatbots in Tencent English and Open Language, English Liulishuo incorporates an AI foreign teacher chatbot. The chatbot can have a casual chat with learners, which creates an immersive and authentic learning environment that enables learners to explore and express their ideas actively. Furthermore, the chatbot provides contextualised input by using multimedia, such as pictures and videos, which corresponds to contextual authenticity from the perspective of social constructivism.

Previous studies have claimed that incorporating chatbots into apps could motivate learners to practise pronunciation [23, 24]. According to Zhang [25], who analysed the advantages and disadvantages of AI technology-assisted English learning from the perspectives of Krashen's second language acquisition theory, communicating with AI-powered chatbots can reduce the anxiety caused by face-to-face communication, lessen the degree of learners' emotional filtering and create a relaxing input and output environment. A relaxing environment can promote positive attitudes towards and motivation for speaking English. From the perspective of social constructivism, motivated and intentional learning can help learners to control and take responsibility for their learning.

**Adaptive Instruction.** According to Aleven [26], adaptivity is the most important feature of AI because it allows an app to study individual differences, provides different guidance for different learners and adapts to learners' needs. In order to enable adaptive instruction, Open Language assesses learners' English level following the Common European Framework of Reference (CEFR). The app identifies a learner's issues in terms of comprehension, listening, pronunciation, vocabulary, grammar, speaking and fluency and then automatically offers courses that are consistent with that learner's level. The chatbot in English Liulishuo can understand learners' output and provide adaptive feedback on their actions. The Tencent English-learning app also provides both adaptive feedback and instruction according to learners' performances and regarding particular mispronunciation to help increase awareness and promote progress. Aligned with social constructivism, the adaptive learning materials provided by the apps can manage the complexity of both learners and their environments by taking into account both differences of needs, goals and levels faced by an individual learner.

**Tracking Learning Activities.** All three apps can also supervise and track learners' learning activities and learners can view their learning records. In Tencent English, once a learner completes a training session, the app produces a learning profile for them. The profile is presented in the form of a card that displays the number of vocabulary words and sentences learned, the learner's average score, the phrases practised, the length of time the learner spent on the app and the quality of his or her performance. Open Language and Liulishuo can also record the time that a learner spends on each app and how many times a week they have engaged with the app. However, the report reveals
nothing regarding learners' pronunciation improvements or progress, which means that the apps cannot help learners monitor how their pronunciation has improved. However, learners can share the tracking results of the three apps with others via links to Wechat or QQ, which are the largest social media platforms in China. Sharing with other people can increase learners' social interaction with peers or teachers, which extends their authentic learning experiences.

## 4 Conclusion

All three of the AI-powered English-learning apps that we analysed have elements of the AI functionalities of automatic evaluation, personalised feedback, intelligent communication, adaptive instruction and tracking learning activities that can support a socio-constructivist approach to language learning. The analysis of the details of their functionalities revealed many similarities but also some differences that would be likely to affect their value for a socio-constructivist approach to language learning. In relation to automatic evaluation and adaptive instruction there are no important differences between the apps from a socio-constructivist perspective. However personalised feedback in Tencent English was more supportive of a socio-constructivist approach than the other two apps as it provides multiple forms of feedback which could allow learners to choose types of feedback that suit them best in order to construct their knowledge actively. Furthermore, Tencent English has more sophisticated tracking of learning activities which would enable learners to monitor their progress more effectively. In relation to intelligent communication, however, English Liulishuo, unlike the other two apps, provides a chatbot capable of a casual chat with learners. This casual chat capability can create a more immersive and authentic learning environment and contextual authenticity is further enhanced by its use of multimedia. Therefore, we can conclude that all of these apps have different advantages and limitations from a socio-constructive perspective at the present time. Their different strengths and limitations will affect their potential for supporting language learning across different learning contexts and for different learners. For learners who need more explicit feedback on the correction of mispronunciation, for example, Tencent English may be the most appropriate app as it provides feedback in visual, textual and audio forms that enable learners to explore and choose the types of feedback they prefer to construct their knowledge actively. Learners who have a low motivation for English speaking may find chatting with the chatbot incorporated in English Liulishuo interesting and immersive. Open Language may be more appropriate for those who want to develop their skills in relation to specific levels of English and prefer to receive instruction designed for their particular levels.

The use of social constructivist theory for the evaluation helps to determine whether the apps support the general rules of foreign-language learning in the process of assisting learning of ESL pronunciation. Such evaluations provide a means for teachers, parents and learners to make informed choices regarding the use of AI-powered technologies. While this study focused only on apps for supporting pronunciation by young Chinese learners, the approach may be valuable for other educational software that incorporates AI functionalities since social constructivist approaches to learning are applicable across many fields. However, it should be noted that for some types of software, for example
creative and problem-solving apps where more open-ended opportunities are needed, further usability considerations may be needed because AI functionalities may not yet be as effective.

Predictive evaluations are a useful first step in examining the potential of software but are insufficient to indicate the value of AI for English learning. Empirical studies of the apps in use will be needed to determine to what extent and how the AI functionalities can enable the learning of English pronunciation. Comparing the outcomes of such studies with the predictions made by the earlier evaluations are also needed to validate the quality of the predictive evaluation approach. However, studies of software in use are time-consuming and the rapid developments in AI functionalities mean that predictive evaluations will continue to be important ways for teachers, parents and learners to select software and decide how to use it.

## References

1. Squires, D., Preece, J.: Predicting quality in educational software: evaluating for learning, usability and the synergy between them. Interact. Comput. **11**(5), 467–483 (1999)

2. Dillenbourg, P.: The role of artificial intelligence techniques in training software. In: Proceedings of LEARNTEC. Karlsruhe (1994)

3. Schiff, D.: Out of the laboratory and into the classroom: the future of artificial intelligence in education. AI & Soc. **36**(1), 331–348 (2020). https://doi.org/10.1007/s00146-020-01033-8

4. Zhang, K., Aslan, A.B.: AI technologies for education: recent research & future directions. Comp. Educ. Artif. Intell. **2**, 100025 (2021)

5. Woo, J.H., Choi, H.: Systematic review for AI-based language learning tools (2021). https://arxiv.org/abs/2111.0445

6. Squires, D., Preece, J.: Usability and learning: evaluating the potential of educational software. Comp. Educ. **27**(1), 15–22 (1996)

7. Hao, K.: China has started a grand experiment in AI education. It could reshape how the world learns. MIT Technol. Rev. 1–9 (2019)

8. Mahdi, H.S., Al-Khateeb, A.A.: The effectiveness of computer-assisted pronunciation training: a meta-analysis. Rev. Educ. **7**(3), 733–753 (2019)

9. Gao, Y., Hanna, B.: Exploring optimal pronunciation teaching: integrating instructional software into intermediate-level EFL classes in China. CALICO J. **33**(2), 201–230 (2016)

10. Wang, Y., Yang, S.: A study of the design and implementation of the ASR-based iCASL system with corrective feedback to facilitate English learning. Educ. Technol. Soc. **17**(2), 219–233 (2014)

11. Biswas, G., Leelawong, K., Schwartz, D., Vye, N.: The teachable agents group at vanderbilt.: learning by teaching: a new agent paradigm for educational software. Appl. Artif. Intell. **19**(3–4), 363–392 (2005)

12. Rogerson-Revell, P.M.: Computer-Assisted pronunciation training (CAPT): current issues and future directions. RELC J. **52**(1), 1–17 (2021)

13. Shackel, B.: The concept of usability. In: Visual Display Terminals: Usability Issues and Health Concerns, pp. 45–87. Prentice Hall (1984)

14. Dimitrije, S., Devedžić, V.: Usability evaluation in selecting educational technology. In: Proceedings of the 11th Conference of Information Technology and Development of Education, pp. 208–214. ITRO, Zrenjanin (2020)

15. McDougall, A., Squires, D.: A critical examination of the checklist approach in software selection. J. Educ. Comp. Res. **12**(3), 263–274 (1995)
16. Lantolf, J.P.: Language emergence: Implications for applied linguistics - A sociocultural perspective. Appl. Linguis. **27**(4), 717–728 (2006)

17. Vygotsky, L.S.: Mind in society: The Development of Higher Psychological Process. Harvard University Press, Cambridge (1978)

18. Salomon, G., Perkins, D.N.: Individual and social aspects of learning. Rev. Res. Educ. **23**(1), 1–24 (1998)

19. Leelawong, K., Biswas, G.: Designing learning by teaching agents: the Betty's brain system. Int. J. Artif. Intell. Educ. **18**, 181–208 (2008)

20. Li, J.: English pronunciation curriculum model on reading assistant SRS: constructivism view. Educ. Sci. Theory Pract. **18**(5), 1246–1254 (2018)

21. Simina, V., Hamel, M.J.: CASLA through a social constructivist perspective: WebQuest in project-driven language learning. ReCALL **17**(2), 217–228 (2005)

22. Pokrivcakova, S.: Preparing teachers for the application of AI-powered technologies in foreign language education. J. Lang. Cult. Educ. **7**(3), 135–153 (2019)

23. Fryer, L.K., Ainley, M., Thompson, A., Gibson, A., Sherlock, Z.: Stimulating and sustaining interest in a language course: an experimental comparison of chatbot and human task partners. Comput. Hum. Behav. **75**, 461–468 (2017)

24. Fryer, L.K., Nakao, K., Thompson, A.: Chatbot learning partners: connecting learning experiences, interest and competence. Comput. Hum. Behav. **93**, 279–289 (2019)

25. Zhang, Y.: An analysis of AI technology assisted English learning from the perspective of SLA theory. In: 2019 3rd International Conference on Economics, Management Engineering and Education Technology, pp. 885–890. ICEMEET, Nantong (2019)

26. Aleven, V.: A is for adaptivity, but what is adaptivity? Re-defining the field of AIED. In: CEUR Workshop Proceedings, pp. 11–20 (2015)