###### Abstract  
General-purpose automatic speech recognition (ASR) systems have improved in quality and are being used for pronunciation assessment. However, the assessment of isolated short utterances, such as words in minimal pairs for segmental approaches, remains an important challenge, even more so for non-native speakers. In this work, we compare the performance of our own tailored ASR system (kASR) with the one of Google ASR (gASR) for the assessment of Spanish minimal pair words produced by 33 native Japanese speakers in a computer-assisted pronunciation training (CAPT) scenario. Participants in a pre/post-test training experiment spanning four weeks were split into three groups: experimental, in-classroom, and placebo. The experimental group used the CAPT tool described in the paper, which we specially designed for autonomous pronunciation training. A statistically significant improvement for the experimental and in-classroom groups was revealed, and moderate correlation values between gASR and kASR results were obtained, in addition to strong correlations between the post-test scores of both ASR systems and the CAPT application scores found at the final stages of application use. These results suggest that both ASR alternatives are valid for assessing minimal pairs in CAPT tools, in the current configuration. Discussion on possible ways to improve our system and possibilities for future research are included. automatic speech recognition (ASR); automatic assessment tools; foreign language pronunciation; pronunciation training; computer-assisted pronunciation training (CAPT); automatic pronunciation assessment; learning environments; minimal pairs +
Footnote †: journal: Applied Sciences  
[PERSON]  
1 Centre for Language and Speech Technology (CLST), Radboud University Nijmegen, P.O. Box 9103, 6500 Nijmegen, The Netherlands 2  
[PERSON]  
1 Centre for Language and Speech Technology (CLST), Radboud University Nijmegen, P.  
lack of concentration [3]. Thus, ASR systems can help in the assessment and feedback of learner production, reducing human costs [4, 5]. Although most of the scarce empirical studies which include ASR technology in CAPT tools assess sentences in large portions of either reading or spontaneous speech [6, 7], the assessment of words in isolation remains a substantial challenge [8, 9]. General-purpose off-the-shelf ASR systems such as Google ASR ([[https://cloud.google.com/speech-to-text](https://cloud.google.com/speech-to-text)]([https://cloud.google.com/speech-to-text](https://cloud.google.com/speech-to-text)), accessed on 27 June 2021) (gASR) are becoming progressively more popular each day due to their easy accessibility, scalability, and, most importantly, effectiveness [10, 11]. These services provide accurate speech-to-text capabilities to companies and academics who might not have the possibility of training, developing, and maintaining a specific-purpose ASR system. However, despite the advantages of these systems (e.g., they are trained on large datasets and span different domains), there is an obvious need for improving their performance when used on in-domain data-specific scenarios, such as segmental approaches in CAPT for non-native speakers. Concerning the existing ASR toolkits, Kaldi has shown its leading role in recent years, with its advantages of having flexible and modern code that is easy to understand, modify, and extend [12], becoming a highly matured development tool for almost any language [13, 14]. English is the most frequently addressed L2 in CAPT experiments [6] and in commercial language learning applications, such as Duolingo ([[https://www.duolingo.com/](https://www.duolingo.com/)]([https://www.duolingo.com/](https://www.duolingo.com/)), accessed on 27 June 2021) or Babbel ([[https://www.babbel.com/](https://www.babbel.com/)]([https://www.babbel.com/](https://www.babbel.com/)), accessed on 27 June 2021).