# Non-Autoregressive End-to-End Neural Modeling for Automatic Pronunciation Error Detection
## 3 The Proposed Architecture
### Dictation Model (DM)  
Various automatic speech recognition (ASR) [48] methods, such as MASK-CTC, CTC-ATT, LASO, AlignRefine, etc., are used to generate dictation models [49]. This dictation approach supports non-autoregressive end-to-end modeling to shorten the execution time of end-to-end machine translation. Transformer-based end-to-end techniques derived from ASR, such as LASO (listen attentively and spell once), have been successfully used to identify and correct pronunciation errors. In this context, we investigate a unique usage of LASO [13] to develop the dictation model instead of Mask-CTC in this study. The mechanism of the dictation method is shown in Figure 4.  
Figure 3: Step-by-step process of automatic pronunciation mistake identification. Figure 2: Workflow of the non-autoregressive model. The main principle of the dictation method is that the phoneme feature sequence comprises characteristics for both language semantics and pronunciation. Location-wise token estimation is possible if patterns for every token location are extracted from the entire auditory feature sequence. The dictation model has three modules: encoder, position-dependent summarizer (PDS), and decoder modules. The main core fundamental for the dictation model is the multi-head self-attention (MHSA) layer. For every module, there are a different number of MHSA and feed-forward network layers. The encoder module captures the acoustic characteristics and long-term dependencies. The PDS reorganizes the acoustic sequence and summarizes the semantic relationship. The decoder module captures the sequence relationship and predicts tokens. The sequence relationship is captured in a sequence by the feed-forward attention framework [50]. The feed-forward attention mechanism employs a weighted sum to combine the input sequence, in contrast to recurrent neural networks that incorporate past context into latent vectors. It can be  
Figure 4: Workflow of the dictation model for automatic pronunciation mistake identification. calculated in parallel due to its feed-forward nature. In this study, we follow [23] and employ a position-wise feed-forward network with scaled dot-product attention as the fundamental sub-module. However, for fast training, we employ the \"pre-norm\" [51]. Figure 5 shows the structure of the feed-forward attention block. Key (k), value (V), and query (Q) are the three inputs of an attention block. Some modules generate K and V as outputs, which are inputs to other modules, and some modules use attention blocks to create semantic relationships or dependencies between different tokens. The self-attention block calculates the dot product using Equation (1)  
\[Atn(Q,K,V)=Softmax(\frac{QK^{T}}{\sqrt{D_{k}}}) \tag{1}\]  
where \(Q\) denotes queries (\(Q\in\mathbb{R}(T_{q}\times D_{k})\)), \(K\) denotes keys (\(K\in\mathbb{R}(T_{k}\times D_{k})\)), and \(V\) denotes values (\(V\in\mathbb{R}(T_{k}\times D_{v})\)). \(T_{q}\) and \(T_{k}\) denote the sequence length of the query and the key, respectively, and \(D_{k}\) and \(D_{v}\) denote the key dimensionality where \(D_{v}=D_{k}\). For example, the speech signal input to the encoder block is \"[PERSON] your letter came just in time\". The multi-head attention blocks in the encoder layer generate corresponding values from queries and keys using dot products. Suppose the generated keys (K) are [0.12, 0.342, 0.453, 0.45, 0.265, 0.432, 0.653, 0.525, 0.753, 0.844, 0.854, 0.964, 0.654, 0.432, 0.455, 0.786, 0.456] and the corresponding values (V) are [Ga, ad, Gadd, Gad, you, your, let, letter, come, came, jute, just, on, in, tick, time]. The encoder output keys and values are used as inputs to the PDS block. A query in a PDS block is an input vector with a predetermined length L. Suppose L is 15 and the queries (Q) are [0.45, 0.432, 0.525, 0.844, 0.964, 0.456, 0.674] for the above example.