# An Effective Artificial Intelligence-Enabled Error Detection and Accuracy Estimation Technique for English Speech Recognition System

Authors: Lu Han; Xueqin Du; Li Yan; Jing Yu
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Error detection and accuracy estimation in automated speech recognition (ASR) systems act a vital part in the design of human-computer spoken dialogue systems, as recognition error can hamper accurate systems in understanding the end user intentions. The major aim is to identify the errors in an utterance, and therefore, the dialogue manager can provide proper clarifications to the user. Therefore, the design of accurate error detection and accuracy determination techniques becomes essential in the ASR system. With this motivation, this paper presents a novel artificial intelligence-enabled accuracy estimation and error detection technique for the English speech recognition system (AIEDAE-ESRS). The goal of the AIEDAE-ESRS technique is to perform three actions such as confidence estimation, out-of-vocabulary (OOV) word identification, and error type categorization. In addition, the AIEDAE-ESRS technique performs different levels of preprocessing including sampling of input speech signal, bandpass filtering, and noise removal. Besides, a new deep neural network with hidden Markov model- (DNN-HMM-) based speech recognition technology is designed, which also aims to estimate the accuracy and error. Finally, the hyperparameters of the DNN-HMM model can be optimally chosen by the use of flower pollination algorithm (FPA) and thereby accomplished improved recognition performance. In order to demonstrate the better performance of the AIEDAE-ESRS technique, a series of simulations were conducted and the results are examined under varying aspects. English voice recognition system’s accuracy estimation and error detection were made possible using artificial intelligence (AIEDAE-ESRS). There are three steps in the AIEDAE-ESRS method: confidence estimation; identifying out-of-vocabulary words (OOV); and categorizing mistake types. The simulation results reported the enhanced performance of the AIEDAE-ESRS methodology over current advanced approaches. Our AIEDAE-ESRS methodology outperforms existing methodologies by a factor of ten. The simulation results demonstrated that the AIEDAE-ESRS methodology outperformed previous approaches in terms of efficiency. The improved experimental results indicated that the AIEDAE-ESRS technique produced superior results across a variety of measures.


---

## Body

## 1. Introduction

The speech signal is one of the essential and common ways of communicating between people. In these communications, the speaker’s emotion performs an important role in the transfer of concept in such a way that a change in the emotions may result in distinct translations of speech [1]. Therefore, to make effective communication between man and machine, speech emotion recognition (SER) is becoming a hot research topic. In the selection of important features, together with accurate SER system, an effective way to decrease the data dimension is needed [2]. With the continued growth of science and technology, the global village is shrinking, and the usage of English has become increasingly widespread. The development of artificial intelligence computers that could understand English speech will significantly encourage the new experience and complete intelligence of human life and work eventually [3]. The speech emotion recognition (SER) system is built on CNNs and RNNs that have been trained on a database of emotional speech. Our primary objective is to offer a SER approach that is based on concatenated CNNs and RNNs and does not rely on any typical hand-crafted features. The literature on speech emotion recognition (SER) has employed a variety of approaches to extract emotions from signals, including numerous well-known speech analysis and classification techniques. Recently, deep learning approaches have been presented as a possible replacement for classic SER techniques. Language interaction and intelligent English speech recognition systems (SRS) affect their study and work life, as well as have promotion significance and extensive application in areas like language promotion, military, and education. Now, there are multiple implementation methods and system designs for SRS. There are different kinds of classification, primarily separated into specific-and nonspecific-persons SRS, continuous and isolated word SRS, embedded/server SRS, small vocabulary, and large vocabulary SRS. In everyday life, people’s natural speech is depending on the speaker’s need to break at the end of a sentence or add punctuation, and other parts could be continuously pronounced [4].In the earlier SRS, the isolated word phonetic systems were based primarily on single words or characters [5]. Depending on the way the acoustic method is developed, we could separate SRS as specific- and nonspecific-person recognition. Specific-person recognition implies that the user needs to input a massive number of pronunciations and train recognition in advance. The nonspecific-person is that afterward the scheme is developed, the user does not need to input the trained information before and could recognize directly [6]. The deep learning (DL) method has different areas of application, and several achievements have been found. Another area where DL is effectively used is automated SRS. In automated SRS, better language and acoustic methods are integrated [7]. The SRS problems involve time-series data. In several fields, such as read continuous speech where usually the speech is recorded under clean conditions, the outcomes are satisfied with an error rate under 5%. Since in another field that has high speech differences, like distant conversational speech (meeting) or video speech, the outcomes are still not satisfactory exhibiting 50% of an error rate [8].To handle these problems and improve the performances of inaccurate ASR systems, the automated correction and detection of the transcript error could be the only choice in some cases [9], especially while tuning the ASR systems by itself is impossible (for example, the system is purchased as a black box) or the manual correction is inconvenient or even impractical as in the case where the transcriptions are not the ultimate objective of the systems (for example, question answering, machine translation, and information retrieval systems). In that respect, ASR classification and error detection are also called confidence estimation [10]. The more commonly studied method is feature-based, where classification is constructed by the feature generated from distinct sources (that is, decoder and nondecoder characteristics) to differentiate the accurately from the inaccurately identified word.This paper presents a novel artificial intelligence-enabled accuracy estimation and error detection technique for the English speech recognition system (AIEDAE-ESRS). The AIEDAE-ESRS technique intends to accomplish three actions such as OOV word identification, confidence estimation, and error type categorization. Furthermore, the AIEDA-ESRS technique’s architecture incorporates a deep neural network with hidden Markov model- (DNN-HMM-) based speech recognition model. Furthermore, the flower pollination algorithm (FPA) is used to fine-tune the DNN-HMM model’s hyperparameters. Flower pollination algorithm (FPA) is a nature-inspired metaheuristic algorithm that replicates the pollination activities of blossoming plants. The implementation of several FPA variants based on tweaks, parameter adjustment, and hybridization with other algorithms is addressed in this article. The design of FPA for hyperparameter optimization of the DNN-HMM model shows the novelty of the work. The experimental result analysis of the AIEDAE-ESRS technique takes place using benchmark dataset and investigated the results under several aspects.


## 2. Literature Review

In Alhamada et al. [11], the usage of DL in SRS was examined and an appropriate DL framework has been was recognized. A technique using CNN is employed to improve the efficiency of SRS. Han et al. [12] examined the efficacy of different DL-based acoustic models for conversation telephone speech, especially CNN-bLSTM, bLSTM, and TDNN systems. They estimated this model on research test sets, like recordings, Switchboard, and CallHome from a real-time call center applications. In Blaise. O. Yenke et al., due to the large variety of applications and interfaces or computing equipment that can enable speech processing, automatic speech recognition (ASR) is a very active research subject. It is true that well-resourced languages outnumber underresourced languages in most applications. It is evident that ASR may be used to enhance illiterate people’s languages. Starting with a small vocabulary is one way to construct an ASR system for underresourced languages. Assertive speech recognition (ASR) with a limited vocabulary recognizes words or sentences in small groups.Grozdić et al. [13] extended a method for whispered SRS that is the most difficult challenge in ASR. Specifically, because of the profound variances among acoustic features of whispered and neutral speech, the efficiency of conventional ASR system trained on neutral speech greatly reduces once whisper is used. Misbullah et al. [14] investigated the efficiency of SRS for dysarthric speakers using time delay DNN. Furthermore, examine the system performances by integrating dysarthria and normal speech corpus. Lastly, well-tuned hyperparameter of DNN structure gives potential outcomes on English dysarthria and Mandarin speech.Ogawa and Hori [15] explored three kinds of ASR error detection processes, that is, OOV word recognition, error type classification (ETC), and confidence estimation, and also evaluated the detection rate from the ETC result. The simulation result shows that the DBRNN considerably outperforms conditional random field (CRF). Ogawa et al. [16] presented detection accuracy estimation method based on ETC. The ETC is an extension of confidence estimate. In ETC, all the words in the detection outcomes (detected word sequence) for the targeted speech information are categorized into three classes: insertion error (I), substitution error (S), and correct recognition (C).


## 3. Materials and Methods

In this study, an effective AIEDAE-ESRS technique has been developed for the error detection and accuracy evaluation in SRS. The AIEDAE-ESRS technique involves three major processes, namely, preprocessing, DNN-HMM-based speech signal recognition, and FPA-based hyperparameter tuning. The utilization of the FPA helps to properly alter the hyperparameter of the DNN-HMM model which assists in significantly boosting the detection performances. Figure1 demonstrates the overall working procedure of the suggested AIEDAE-ESRS technique.Figure 1
Overall process of AIEDAE-ESRS technique.


### 3.1. Level I: Speech Signal Preprocessing

The speech input is the original voice signal gathered by the voice tool; the preprocessing method chiefly consists of three factors: antialiasing bandpass filtering, eliminating the noise effect, and sampling the input original voice signal; the feature extraction method extracts the reflection in the voice. The acoustic parameter of the speaker’s key features primarily includes short-term average zero-crossing rate, cepstrum, short-term energy, and linear prediction coefficient. In the recognition phase, the speech feature parameter is attained, and the test template is made. In the test, the template is matched with the reference template as per some discriminative rules (i.e., semantic and grammar rules), later in the training phase, the feature parameter is processed for establishing a reference model, and the better reference template is attained as the detection outcome. Better matching results are closely associated with the matching template, quality of speech feature parameter, and speech technique.


### 3.2. Level II: Design of DNN-HMM-Based Speech Signal Recognitions

In traditional DNN-HMM-based recognition, the probability is modelled by GMM under the maximal probability condition. Such potential models are constrained because GMM is statistically ineffective to model information that lies on or near a nonlinear in the data space. To conquer this limitation, we proposed a DNN-HMM method for recognizing speech, in which the outputs of the DNN are given to the HMM as substitute for the GMM. GMM simulates the observed probability distribution of a feature vector in the presence of a phone. It establishes a sound foundation for determining the “distance” between a phone and the audio frame being observed. The GMM is a probabilistic model capable of simulating normally distributed subpopulations. GMM’s components are all Gaussian distributions. A statistical Markov model (HMM) is a sort of hidden Markov model. When the data is continuous, a Gaussian distribution is used to represent each hidden state.


#### 3.2.1. Overview of Hidden Markov Models

The HMM is a statistical Markov method where the algorithm that has been modelled is presumed as a Markov model using unobserved (hidden) state. An HMM, denoted by=A,B,π, contains the subsequent element:
(1)
The amount of states in the system represented asQ, the number of states represented by S=s1,s2,⋯,sQ, and qt the states at time t(2)
A=aij, the transition state likelihood distribution(1)aij=Pqt+1=sj∣qt=si,1≤i,j,≤Q(3)
B=biOt, the observation probability, in which biOt signifies the likelihood of observing Ot at state si.B is denoted as a finite mixture:(2)biOt=∑m=1McimℵOt,μim,Uim,1≤i≤QLet cim be the mixture coefficients for mth mixture in state si, as well as ℵ elliptically symmetric density or log-concave, with covariance matrix Uim and mean vector μim for mth mixture element in i state
(4)
π=πi, the first state distribution, in which πi=Pq1=si,1≤i≤QIn order to apply HMM, two issues must be resolved:(i)
Learning issue: assuming a collection of ground truthX (represent as trained set), the learning process detects the group of variables λ∗=A∗,B∗,π∗; therefore, λ∗=argmaxλPX∣λ that detects the model parameter that well fits the trained set. The forward-backward method is utilized for calculating X∣λ [17]. It finds the model parameter that best fits the training data. In order to compute X∣λ, the forward-backward approach is used(ii)
Decoding issue: assuming aλ parameter and a series of new observation O=O1,O2,⋯,OT (represent as testing set), the decoder process is determined as(3)PO∣λ=maxq1,⋯,qTπq1∏t=2Tpqt∣qt−1bqtOtIn the event of speech recognition, trainC HMM λc,c=1,⋯,C for C discrete class. For novel speech input O,
(4)c∗=argmax1≤c≤CPO∣λc,with PO∣λc estimated from the Viterbi model.


#### 3.2.2. Structure of DNN-HMM Model

The main variation among GMM-HMM and DNN-HMM is the utilizing of GMM (rather than DNN) to evaluate the observation probability. We employ the DNN for modellingpqt∣0t; the following probabilities of the parameters provided the vector 0t, i.e., feasible, while pqt is easier for estimating from a first state-level position of the trained set. Figure 2 depicts the framework of DNN technique.Figure 2
DNN structure.


### 3.3. DNN-HMM Training Process

The thoroughly trained procedure for recognition is given below:(a)
For all the classescc=1,⋯,C, a GMM-HMM λc with Q state is instructed by the training sentence of parameter c(b)
For all the sentencesO=O1,O2,⋯,OT in trained set c, the Viterbi model of GMM-HMM as per Equation (3) is executed on λc to attain an optimum state sequence q1c,⋯,qTc, and all the states qtc are allotted a label Lii∈1,⋯,C×Q [18](c)
Each training sentence and labelled state sequence is utilized as input to train a DNN, where output is the previous probability of theC×Q output unit. The DNN is executed by BP model using (i) unsupervised pretraining or (ii) the discrimination pretraining


### 3.4. DNN-HMM Recognition Process

In the detection procedure, for input sentenceO=01,02,⋯,0T, one must evaluate the likelihood pO∣λc for all the classes c and attain the last recognition results as per Equation (4). In GMM-HMM, this likelihood is attained by the Viterbi model using Equation (3).In DNN-HMM, adapt the subsequent process to estimate the likelihoodO∣λc.
(a)
The input structureO is initially inputted to DNN, obtaining the previous probability pLi∣oti=1,⋯,C×Q as output. Next, previous probabilities pqt=Skc∣Ot are attained from Li∣ot, through representing the label Li to the k and c, as follows(b)
As per the Bayesian principles, estimate the possibilitypot∣qt as(5)pOt∣qt=pqt∣OtpOtpqtIn this process, the previous probabilities of each stateqt is estimated from (occurrence of) the trained set, and pOt is allocated a constant because the feature vector is considered independent of one another [19].
(c)
For all the modelsλc, the Viterbi model is executed to estimate the prospect pO∣λc. But, the likelihood bqtOt is substituted with pot∣qt estimated by Equation (5)


### 3.5. Level III: Design of FPA-Based Hyperparameter Tuning

The FPA is used to effectively adjust the DNN-HMM model’s hyperparameter settings. The abiotic pollination is discussed as well as induced in a flower pollination approach in the optimization. Pollination challenges encompass a difficult process in plant generation theory. The pollen gamete and bloom are more likely to provide a consistent solution to the optimization challenge. The advantages of FPA are listed below. FPA, unlike GA, HS, and PSO, provides a simple floral analogy with lightweight computationally based control parameters (that is, switch conditions,p). It also provides a balanced diversity and intensity of solution through the adaptation of levy flight (random walks punctuated by larger leaps) and switch conditions, which are utilized to transition between intensive local search and global search.Flower constancy was identified as a precise solution that might be differentiated. In the case of global pollination, the pollinator transports pollen from a great distance to a more suitable location. In another example, local pollination was carried out inside a smaller region of a unique bloom in shade water [20]. Global pollination is carried out through a possibility known as switch probability. Pollination occurs all across the world when a pollinator transports pollen from vast distances to higher fitting. Global pollination is accomplished by the use of a probability known as switch probability. In a tiny area of a unique bloom, local pollination is carried out in water shade. Pollinators like bees are vital to the sexual reproduction of about ninety percent of wildflowers. Ecosystems depend on these plants to function. They provide food, shelter, and other resources for many animal species, including humans. Once the stage was removed, local pollination is substituted. In FPA technique, the following 4 rules are used (also shown in Algorithm 1):
(1)
Cross and live pollination is called global pollination as well as the carrier of pollen pollinator apples the LF(2)
Abiotic and self-pollination are represented as local pollination(3)
Pollinators are insects, i.e., able to develop flower constancy. It is determined as the possibility of two employed flowers(4)
The transmission of local and international pollination is handled by switch possibilityAlgorithm 1: Pseudocode of FPA
Input: Objective functionfxPopulation initialization: Flower/Pollen gamete with arbitrary solutions;Determine the optimal solutiong∗ in the primary population;Represent a switching possibility p ∈ [0, 1];while (t<Maximum_iterations) dofor i =1:n (everyn flower in the population) doif rand < p thenDraw a (d dimension) step vector L from a Levy delivery;Apply global pollination;ElseDrawϵ from a similar distribution in 0,1;Apply local pollination;EndDetermine new solutions;When better solutions are obtained, updating it in population;EndCompute the present optimum solutiong∗;EndTherefore, the first and second rules are given by(6)xqt+1=xpt+γ×Lλ×g∗−xpt,in which xpt is the pollen vector at iteration t; g∗ indicates a present solution from present generated outcomes; γ=a indicates the level factor to control phase size; and L denotes pollination power, which is related with a step-size of levy allocation. The LF is calculated as a collection of random computations that have the duration of all the leaps and use the levy likelihood distribution function with infinite variation. Following that, L represents a levy distribution:
(7)L∼λ×Γλ×sinπλ/2π×1S1+λS≫S00,in which Γλis the basic gamma function.In the event of local pollination, the second and third rules are formulated as(8)xpt+1=xpt+εxqt−xkt,in which xqt and xkt are 2 pollens from several blooms on the same plant, if xqt and xkt come from the same species and are chosen from a homogenous population; this is represented as local random walks and is included by a standard distribution in zero and one [21].FF acts as an important part of the optimization problem. It estimates a positive integer to represent how better the candidate solution is. In the work, classification error rate is considered as a minimalizing FF. The poorer solutions have high fitness scores and the richer solutions have less fitness scores.(9)fitnessχi=ClassifierErrorRateχi=numberofmisclassifiedinstancesToralnumberofinstances∗100.


## 3.1. Level I: Speech Signal Preprocessing

The speech input is the original voice signal gathered by the voice tool; the preprocessing method chiefly consists of three factors: antialiasing bandpass filtering, eliminating the noise effect, and sampling the input original voice signal; the feature extraction method extracts the reflection in the voice. The acoustic parameter of the speaker’s key features primarily includes short-term average zero-crossing rate, cepstrum, short-term energy, and linear prediction coefficient. In the recognition phase, the speech feature parameter is attained, and the test template is made. In the test, the template is matched with the reference template as per some discriminative rules (i.e., semantic and grammar rules), later in the training phase, the feature parameter is processed for establishing a reference model, and the better reference template is attained as the detection outcome. Better matching results are closely associated with the matching template, quality of speech feature parameter, and speech technique.


## 3.2. Level II: Design of DNN-HMM-Based Speech Signal Recognitions

In traditional DNN-HMM-based recognition, the probability is modelled by GMM under the maximal probability condition. Such potential models are constrained because GMM is statistically ineffective to model information that lies on or near a nonlinear in the data space. To conquer this limitation, we proposed a DNN-HMM method for recognizing speech, in which the outputs of the DNN are given to the HMM as substitute for the GMM. GMM simulates the observed probability distribution of a feature vector in the presence of a phone. It establishes a sound foundation for determining the “distance” between a phone and the audio frame being observed. The GMM is a probabilistic model capable of simulating normally distributed subpopulations. GMM’s components are all Gaussian distributions. A statistical Markov model (HMM) is a sort of hidden Markov model. When the data is continuous, a Gaussian distribution is used to represent each hidden state.


### 3.2.1. Overview of Hidden Markov Models

The HMM is a statistical Markov method where the algorithm that has been modelled is presumed as a Markov model using unobserved (hidden) state. An HMM, denoted by=A,B,π, contains the subsequent element:
(1)
The amount of states in the system represented asQ, the number of states represented by S=s1,s2,⋯,sQ, and qt the states at time t(2)
A=aij, the transition state likelihood distribution(1)aij=Pqt+1=sj∣qt=si,1≤i,j,≤Q(3)
B=biOt, the observation probability, in which biOt signifies the likelihood of observing Ot at state si.B is denoted as a finite mixture:(2)biOt=∑m=1McimℵOt,μim,Uim,1≤i≤QLet cim be the mixture coefficients for mth mixture in state si, as well as ℵ elliptically symmetric density or log-concave, with covariance matrix Uim and mean vector μim for mth mixture element in i state
(4)
π=πi, the first state distribution, in which πi=Pq1=si,1≤i≤QIn order to apply HMM, two issues must be resolved:(i)
Learning issue: assuming a collection of ground truthX (represent as trained set), the learning process detects the group of variables λ∗=A∗,B∗,π∗; therefore, λ∗=argmaxλPX∣λ that detects the model parameter that well fits the trained set. The forward-backward method is utilized for calculating X∣λ [17]. It finds the model parameter that best fits the training data. In order to compute X∣λ, the forward-backward approach is used(ii)
Decoding issue: assuming aλ parameter and a series of new observation O=O1,O2,⋯,OT (represent as testing set), the decoder process is determined as(3)PO∣λ=maxq1,⋯,qTπq1∏t=2Tpqt∣qt−1bqtOtIn the event of speech recognition, trainC HMM λc,c=1,⋯,C for C discrete class. For novel speech input O,
(4)c∗=argmax1≤c≤CPO∣λc,with PO∣λc estimated from the Viterbi model.


### 3.2.2. Structure of DNN-HMM Model

The main variation among GMM-HMM and DNN-HMM is the utilizing of GMM (rather than DNN) to evaluate the observation probability. We employ the DNN for modellingpqt∣0t; the following probabilities of the parameters provided the vector 0t, i.e., feasible, while pqt is easier for estimating from a first state-level position of the trained set. Figure 2 depicts the framework of DNN technique.Figure 2
DNN structure.


## 3.2.1. Overview of Hidden Markov Models

The HMM is a statistical Markov method where the algorithm that has been modelled is presumed as a Markov model using unobserved (hidden) state. An HMM, denoted by=A,B,π, contains the subsequent element:
(1)
The amount of states in the system represented asQ, the number of states represented by S=s1,s2,⋯,sQ, and qt the states at time t(2)
A=aij, the transition state likelihood distribution(1)aij=Pqt+1=sj∣qt=si,1≤i,j,≤Q(3)
B=biOt, the observation probability, in which biOt signifies the likelihood of observing Ot at state si.B is denoted as a finite mixture:(2)biOt=∑m=1McimℵOt,μim,Uim,1≤i≤QLet cim be the mixture coefficients for mth mixture in state si, as well as ℵ elliptically symmetric density or log-concave, with covariance matrix Uim and mean vector μim for mth mixture element in i state
(4)
π=πi, the first state distribution, in which πi=Pq1=si,1≤i≤QIn order to apply HMM, two issues must be resolved:(i)
Learning issue: assuming a collection of ground truthX (represent as trained set), the learning process detects the group of variables λ∗=A∗,B∗,π∗; therefore, λ∗=argmaxλPX∣λ that detects the model parameter that well fits the trained set. The forward-backward method is utilized for calculating X∣λ [17]. It finds the model parameter that best fits the training data. In order to compute X∣λ, the forward-backward approach is used(ii)
Decoding issue: assuming aλ parameter and a series of new observation O=O1,O2,⋯,OT (represent as testing set), the decoder process is determined as(3)PO∣λ=maxq1,⋯,qTπq1∏t=2Tpqt∣qt−1bqtOtIn the event of speech recognition, trainC HMM λc,c=1,⋯,C for C discrete class. For novel speech input O,
(4)c∗=argmax1≤c≤CPO∣λc,with PO∣λc estimated from the Viterbi model.


## 3.2.2. Structure of DNN-HMM Model

The main variation among GMM-HMM and DNN-HMM is the utilizing of GMM (rather than DNN) to evaluate the observation probability. We employ the DNN for modellingpqt∣0t; the following probabilities of the parameters provided the vector 0t, i.e., feasible, while pqt is easier for estimating from a first state-level position of the trained set. Figure 2 depicts the framework of DNN technique.Figure 2
DNN structure.


## 3.3. DNN-HMM Training Process

The thoroughly trained procedure for recognition is given below:(a)
For all the classescc=1,⋯,C, a GMM-HMM λc with Q state is instructed by the training sentence of parameter c(b)
For all the sentencesO=O1,O2,⋯,OT in trained set c, the Viterbi model of GMM-HMM as per Equation (3) is executed on λc to attain an optimum state sequence q1c,⋯,qTc, and all the states qtc are allotted a label Lii∈1,⋯,C×Q [18](c)
Each training sentence and labelled state sequence is utilized as input to train a DNN, where output is the previous probability of theC×Q output unit. The DNN is executed by BP model using (i) unsupervised pretraining or (ii) the discrimination pretraining


## 3.4. DNN-HMM Recognition Process

In the detection procedure, for input sentenceO=01,02,⋯,0T, one must evaluate the likelihood pO∣λc for all the classes c and attain the last recognition results as per Equation (4). In GMM-HMM, this likelihood is attained by the Viterbi model using Equation (3).In DNN-HMM, adapt the subsequent process to estimate the likelihoodO∣λc.
(a)
The input structureO is initially inputted to DNN, obtaining the previous probability pLi∣oti=1,⋯,C×Q as output. Next, previous probabilities pqt=Skc∣Ot are attained from Li∣ot, through representing the label Li to the k and c, as follows(b)
As per the Bayesian principles, estimate the possibilitypot∣qt as(5)pOt∣qt=pqt∣OtpOtpqtIn this process, the previous probabilities of each stateqt is estimated from (occurrence of) the trained set, and pOt is allocated a constant because the feature vector is considered independent of one another [19].
(c)
For all the modelsλc, the Viterbi model is executed to estimate the prospect pO∣λc. But, the likelihood bqtOt is substituted with pot∣qt estimated by Equation (5)


## 3.5. Level III: Design of FPA-Based Hyperparameter Tuning

The FPA is used to effectively adjust the DNN-HMM model’s hyperparameter settings. The abiotic pollination is discussed as well as induced in a flower pollination approach in the optimization. Pollination challenges encompass a difficult process in plant generation theory. The pollen gamete and bloom are more likely to provide a consistent solution to the optimization challenge. The advantages of FPA are listed below. FPA, unlike GA, HS, and PSO, provides a simple floral analogy with lightweight computationally based control parameters (that is, switch conditions,p). It also provides a balanced diversity and intensity of solution through the adaptation of levy flight (random walks punctuated by larger leaps) and switch conditions, which are utilized to transition between intensive local search and global search.Flower constancy was identified as a precise solution that might be differentiated. In the case of global pollination, the pollinator transports pollen from a great distance to a more suitable location. In another example, local pollination was carried out inside a smaller region of a unique bloom in shade water [20]. Global pollination is carried out through a possibility known as switch probability. Pollination occurs all across the world when a pollinator transports pollen from vast distances to higher fitting. Global pollination is accomplished by the use of a probability known as switch probability. In a tiny area of a unique bloom, local pollination is carried out in water shade. Pollinators like bees are vital to the sexual reproduction of about ninety percent of wildflowers. Ecosystems depend on these plants to function. They provide food, shelter, and other resources for many animal species, including humans. Once the stage was removed, local pollination is substituted. In FPA technique, the following 4 rules are used (also shown in Algorithm 1):
(1)
Cross and live pollination is called global pollination as well as the carrier of pollen pollinator apples the LF(2)
Abiotic and self-pollination are represented as local pollination(3)
Pollinators are insects, i.e., able to develop flower constancy. It is determined as the possibility of two employed flowers(4)
The transmission of local and international pollination is handled by switch possibilityAlgorithm 1: Pseudocode of FPA
Input: Objective functionfxPopulation initialization: Flower/Pollen gamete with arbitrary solutions;Determine the optimal solutiong∗ in the primary population;Represent a switching possibility p ∈ [0, 1];while (t<Maximum_iterations) dofor i =1:n (everyn flower in the population) doif rand < p thenDraw a (d dimension) step vector L from a Levy delivery;Apply global pollination;ElseDrawϵ from a similar distribution in 0,1;Apply local pollination;EndDetermine new solutions;When better solutions are obtained, updating it in population;EndCompute the present optimum solutiong∗;EndTherefore, the first and second rules are given by(6)xqt+1=xpt+γ×Lλ×g∗−xpt,in which xpt is the pollen vector at iteration t; g∗ indicates a present solution from present generated outcomes; γ=a indicates the level factor to control phase size; and L denotes pollination power, which is related with a step-size of levy allocation. The LF is calculated as a collection of random computations that have the duration of all the leaps and use the levy likelihood distribution function with infinite variation. Following that, L represents a levy distribution:
(7)L∼λ×Γλ×sinπλ/2π×1S1+λS≫S00,in which Γλis the basic gamma function.In the event of local pollination, the second and third rules are formulated as(8)xpt+1=xpt+εxqt−xkt,in which xqt and xkt are 2 pollens from several blooms on the same plant, if xqt and xkt come from the same species and are chosen from a homogenous population; this is represented as local random walks and is included by a standard distribution in zero and one [21].FF acts as an important part of the optimization problem. It estimates a positive integer to represent how better the candidate solution is. In the work, classification error rate is considered as a minimalizing FF. The poorer solutions have high fitness scores and the richer solutions have less fitness scores.(9)fitnessχi=ClassifierErrorRateχi=numberofmisclassifiedinstancesToralnumberofinstances∗100.


## 4. Experimental Validation

In this study, the experimental result analysis of the AIEDAE-ESRS technique takes place using the MIT lecture English speech corpus, called MIT dataset [22]. The MIT corpus includes speech information from invited talks and systematic university classes. The length of a lecture exist between 45 and 90 minutes. First, the error detection result analysis of the AIEDAE-ESRS technique takes place under deletion error detection, confidence estimation, OOV word detection, and CSI classification in Table 1.Table 1
Result analysis of AIEDAE-ESRS technique with existing approaches.
ClassifierAccuracyNECAverageF-scoreConfidence estimationCRF0.84330.36300.7847DNN0.84450.36600.7854DURNN0.84960.38200.7923DULSTM0.85030.38200.7925DBRNN0.85520.40400.8009AIEDAE-ESRS0.88190.44100.8302OOV word detectionCRF0.94290.30500.7046DNN0.94020.27000.6789DURNN0.94320.31400.7034DULSTM0.94420.31200.7014DBRNN0.94600.34800.7212AIEDAE-ESRS0.97080.37200.7497CSI classificationCRF0.82130.34000.6302DNN0.81970.31600.5964DURNN0.82500.34000.6206DULSTM0.82550.33800.6199DBRNN0.83330.38100.6560AIEDAE-ESRS0.85790.41200.6796Deletion error detectionCRF0.96250.18500.6411DNN0.96450.17700.6332DURNN0.96450.19500.6397DULSTM0.96460.19200.6392DBRNN0.96450.24300.6626AIEDAE-ESRS0.99210.26400.6909Figure3 exhibits the comparative result analysis of the AIEDAE-ESRS technique with existing methods under confidence estimation. The figure reported that the AIEDAE-ESRS technique can accomplish effectual outcomes with the increased values of accuracy, average F-score (AFS), and normalized cross entropy (NCE). It is noticed that the CRF and DNN models have shown least performance with the minimal values of accuracy, NEC, and AFS. In line with this, the DURNN, DULSTM, and DBRNN techniques have resulted in moderately closer accuracy, NEC, and AFS values. However, the AIEDAE-ESRS technique has outperformed the other techniques with the higher accuracy, NCE, and AFS of 0.8819, 0.4410, and 0.8302, respectively.Figure 3
Confidence estimation analysis of AIEDAE-ESRS technique.Figure4 displays the comparative result analysis of the AIEDAE-ESRS system with present methodologies under OOV word detection. The figure stated that the AIEDAE-ESRS method has the capacity of achieving efficient outcomes with increased value of accuracy, NCE, and AFS. It is noted that the CRF and DNN models have shown minimum performance with minimal values of accuracy, NEC, and AFS. In line with this, the DURNN, DULSTM, and DBRNN systems have resulted in moderately closer accuracy, NEC, and AFS values. But, the AIEDAE-ESRS method has outperformed the other systems with the high accuracy, NCE, and AFS of 0.9708, 0.3720, and 0.7497, respectively.Figure 4
OOV word detection analysis of AIEDAE-ESRS technique.Figure5 displays the comparative analysis of the AIEDAE-ESRS procedure with current methods under CSI classification. The figure described that the AIEDAE-ESRS method has the capacity of achieving efficient outcomes with the increased values of accuracy, NCE, and AFS. It is noticed that the CRF and DNN models have shown minimum performance with minimal values of accuracy, NEC, and AFS. In line with this, the DURNN, DULSTM, and DBRNN systems have resulted in moderately closer accuracy, NEC, and AFS values. But, the AIEDAE-ESRS system has outperformed the other techniques with the higher accuracy, NCE, and AFS of 0.8579, 0.4120, and 0.6796 correspondingly.Figure 5
CSI classification analysis of AIEDAE-ESRS technique.Figure6 displays the comparative analysis of the AIEDAE-ESRS method with current methods under deletion error detection. The figure described that the AIEDAE-ESRS method has the capacity of achieving efficient outcomes with the increased values of accuracy, NCE, and AFS. It is noticed that the CRF and DNN models have shown smallest performance with the minimal values of accuracy, NEC, and AFS. In line with this, the DURNN, DULSTM, and DBRNN methods have resulted in moderately closer accuracy, NEC, and AFS values.Figure 6
Deletion error detection analysis of AIEDAE-ESRS technique.Figure7 exhibits the accuracy graph analysis of the AIEDAE-ESRS technique on the test MIT speech recognition dataset. The figure portrayed that the AIEDAE-ESRS technique has reached improved training and validation accuracy with increasing amount of epochs. It is also noticed that the training accuracy is considered to be lower compared to the validation accuracy.Figure 7
Accuracy graph analysis of AIEDAE-ESRS technique.Figure8 demonstrates the loss graph analysis of the AIEDAE-ESRS technique on the test MIT speech recognition dataset. The figure depicted that the AIEDAE-ESRS technique has attained decreasing training and validation loss with a rise in the number of epochs. It is noticed the training loss is seemed to be higher than the validation loss.Figure 8
Loss graph analysis of AIEDAE-ESRS technique.Finally, a brief RMSE analysis of the AIEDAE-ESRS technique takes place under distinct sizes of training data is given in Figure9 and Table 2 [23]. The result reported that the AIEDAE-ESRS technique has attained improved performance with the minimal values of RMSE compared to CRF and DBRNN techniques. For instance, with TS of 10%, the AIEDAE-ESRS technique has obtained lower RMSE of 1.12%, whereas the CRF and DBRNN techniques have attained higher RMSE of 2.21% and 1.81%, respectively. Meanwhile, with TS of 40%, the AIEDAE-ESRS system has attained lower RMSE of 0.93%, while the CRF and DBRNN techniques have attained higher RMSE of 1.99% and 1.80% correspondingly. Eventually, with TS of 60%, the AIEDAE-ESRS technique has obtained lower RMSE of 0.89%, whereas the CRF and DBRNN techniques have achieved higher RMSE of 1.98% and 1.75% correspondingly. Moreover, with TS of 80%, the AIEDAE-ESRS technique has obtained lower RMSE of 1.03%, while the CRF and DBRNN methods have accomplished higher RMSE of 2.01% and 1.77% correspondingly. Furthermore, with TS of 100%, the AIEDAE-ESRS technique has obtained lower RMSE of 1.16%, while the CRF and DBRNN systems have reached higher RMSE of 2.01% and 1.73% correspondingly.Figure 9
RMSE analysis of AIEDAE-ESRS technique.Table 2
RMSE analysis of AIEDAE-ESRS technique with distinct training size.
Root mean square error (%)Training size (%)CRFDBRNNAIEDAE-ESRS102.211.811.12202.001.751.06301.971.731.08401.991.800.93502.001.631.04601.981.750.89702.001.650.98802.011.771.03902.021.700.801002.011.731.16By looking into the abovementioned figures and tables, it is ensured that our AIEDAE-ESRS methodology has gained maximal performances over the existing techniques.


## 5. Conclusion

In this study, an effective AIEDAE-ESRS technique has been developed for the accurate estimation and error detection in speech recognition model. The AIEDAE-ESRS technique involves three major processes, namely, preprocessing, DNN-HMM-based speech signal recognition, and FPA-based hyperparameter tuning. The utilization of the FPA helps to properly adjust the hyperparameters of the DNN-HMM model which supports to greatly increase the detection performance. The experimental result analysis of the AIEDAE-ESRS technique take place using benchmark dataset and investigated the results under several aspects. The simulation results reported the outstanding efficiency of the AIEDAE-ESRS methodology over the recent approaches. The improvements in experimental results reported the enhanced outcomes of the AIEDAE-ESRS technique based on various measures. With accuracy, NCE, and AFS values of 0.9921, 0.2640, and 0.6909, respectively, the AIEDAE-ESRS system outperformed the other techniques. With a TS of 100%, the AIEDAE-ESRS technique achieved a reduced root mean square error of 1.16 percent, whereas the CRF and DBRNN systems achieved a higher root mean square error of 2.01% and 1.73 percent, respectively. In the future, the performance of the AIEDAE-ESRS technique is additionally improved by the advanced DL models for speech recognition.



---

*Source: 8279856-2022-04-04.xml*