# Non-Autoregressive End-to-End Neural Modeling for Automatic Pronunciation Error Detection
## 4 Experimental Setup
### Experimental Results
(**d**) Output of the fourth head. Figure 9: Visualization of the effect of training and validation scores along with the number of epochs on different datasets using the wav2 vec 2.0 (large) feature extraction model. (**a**) L2-Arctic dataset. (**b**) SpeechOcean dataset. Figure 11 shows that PDS fills the space signal sequence length with \(<\)eos\(>\) to create the specified length L of the token sequence. The different attention heads of the PDS generate different token representations, as visualized in Figure 11; the y-axis illustrates the token sequence; and the x-axis illustrates the time frame. PDS reconstructs all tokens according to token positions in different time frames. The mean of the eight attention heads is the final score of an attention block. We clearly observe that the encoder's outputs and the relevant tokens are aligned from topmost to lower right. The alignment for the padding token \(<\)eos\(>\) is ambiguous, because there is no definite link between both the padding token and the encoder's outputs. The overall results are not very accurate because various heads in the multi-head attention have various alignments. The decoder self-attention layer calculates the interaction of the focus words with other specific words, as shown in Figure 12. The words in the source sequence are analyzed by the decoder, which utilizes them to forecast the words in the target pattern. Figure 12 visualizes the attention score of different heads of the last attention layer in the decoder module. In Figure 12a, we show that the first attention head represents the sequence relationship of the various tokens. However, some attention heads hide the representation of previous token positions, as shown in Figure 12b,c and hides the subsequent token representation as shown in Figure 12d. In the dictation model, Equation (2) is used to minimize the position-wise cross entropy loss and ultimately remove the \(<\)eos\(>\) token from the tail and calculate the maximum possible token at each position. We then used the edit distance algorithm to calculate the phoneme error rate (PER) from the generated and annotated phoneme sequences. Table 3 displays the phone error rate (%) of the different baseline models and the proposed model on the L2-Arctic dataset. The highest PER for the CNN-RNN-CTC baseline model is 27.75%. Our proposed model produces 20.3% PER with the FBANK feature extraction method and improves PER with the W2V-based feature extraction model, which is similar to CTC-ATT with a baseline model with a beam size of 10. The proposed model significantly improves the phone error rate for wav2 vec 2.0 large, which is 14.1%. Repeated misrecognition of vowels and consonants is the major cause of increased phoneme error rates. Figure 11: Graphical view of the various attention head outputs at the last attention level of the PDS block. (**a**) Output of the first head. (**b**) Output of the second head. (**c**) Output of the third head. (**d**) Output of the fourth head. Table 4 shows the frequency of incorrect vowels and consonants during the pronunciation of different words. The total number of misrecognized vowels and consonants are shown as the number of targets. Our model can identify the maximum number of misrecognized characters more effectively than the existing CNN-RNN-CTC model. The most misrecognized vowel is ah, where the CNN-RNN-CTC baseline model detects 2035 instances, and the proposed model detects 2460 instances in the complete dataset, as shown in Table 4. Similarly, the letters s and t are most often misrecognized for consonant sounds. We compared the results of different existing methods with the proposed method as shown in Table 5. The table shows results for both existing and baseline mispronunciation detection and diagnosis models built on L2-ARCTIC datasets. The mask-CTC model analyzes the results with and without the pronunciation model as shown in Table 5. The Mask-CTC model with the pronunciation model shows better F1 -scores for mispronunciation detection.