# A Russian Continuous Speech Recognition System Based on the DTW Algorithm under Artificial Intelligence

Authors: Chunping Yu; Xin Wang
Journal: Journal of Robotics
 (2022)

---

## Abstract

In order to improve the effect of continuous speech recognition, this paper combines the DTW algorithm to construct a continuous Russian speech recognition system and proposes a continuous Russian speech detection method based on VGDTW-MPCA with an unequal interval process. Moreover, considering the influence of the correlation between variables on the synchronization of the DTW algorithm, this paper constructs a DTW algorithm on a local data set to synchronize in different variable groups. Then, this paper integrates the obtained data into complete 3D data for modeling. It can be seen from the simulation research that the Russian continuous speech recognition system based on DTW proposed in this paper has a high continuous Russian speech recognition accuracy.


---

## Body

## 1. Introduction

The training goal of foreign language subjects is to cultivate applied, compound, and innovative talents with international vision and cross-cultural communication skills, familiarity with international conventions, ability to participate in international competition and cooperation, with a sense of social responsibility, and adaptability to the needs of economic construction and social development. This is also the “authentic” that Russian subjects should abide by [1]. Under the background of the new era and historical opportunities, one of the important missions of the Russian subject is to enhance the country’s soft power and shape the country’s hard image in state exchanges. Moreover, the Russian-speaking talents cultivated under this goal and mission should form the consciousness of national culture [2]. In today’s world, Russia is still a pivotal regional power with an important position in the world. At the same time, Russia is still a world leader in basic research, military T, and aerospace technology. Therefore, Russian-speaking talents still need to give full play to their due role in Sino-Russian scientific and technological exchanges and contribute to the development of the country, which is precisely the original mission of foreign language talents given to foreign language talents [3].At present, there is still a big gap between Russian teaching and English. The teaching methods are more traditional and the subject research is relatively lagging behind. Therefore, while establishing a reasonable and complete curriculum system and innovative Russian teaching methods, the teaching staff should be strengthened. Construction is particularly important. As a key factor throughout the entire teaching process, teachers’ professional quality and ideological realm are important conditions for the survival of Russian teaching [4]. Russian teachers also need to change their concepts, strive to improve their own knowledge, and strengthen the integration of theory and practice. Combining and handling the relationship between teaching and scientific research, it is still necessary to make efforts to transform into a “dual-teacher type.” At the same time, Russian teachers should continue to enhance their own knowledge reserves, strengthen connections with other disciplines, and prepare for the challenges of the new era [5].From the perspective of teaching methods, the current teaching activities of public Russian courses in colleges and universities have a single defect in teaching methods. First of all, in content design, more emphasis is placed on guiding students to learn basic Russian vocabulary and grammar, and the application of Russian in actual communication scenarios. Knowledge is not paid enough attention, which leads to insufficient teaching quality of public Russian courses in colleges and universities and it is difficult for students’ practical application ability to effectively fit with the knowledge they have learned [6]. From the perspective of the course evaluation model, many courses and teachers are currently affected by the traditional test-oriented education concept and they do not pay enough attention to the development of students’ application skills in the course teaching process. In the current process of Russian teaching in colleges and universities, many teachers have the phenomenon of “emphasizing theoretical results and ignoring practical application” when setting up a course evaluation system. Most Russian course exams are conducted through written tests, and the written test scores are used as the result of students’ course learning, which makes it difficult for the course evaluation model to accurately reflect students’ learning [7].In terms of education training and job training, colleges and universities should do a good job in education ability training and job training for Russian teachers and fundamentally improve teachers’ information-based teaching by building a good practice platform and collective training for teachers. In addition, it provides opportunities for teachers to communicate with relevant professionals in the course, so as to strengthen the training of teachers of public Russian courses and achieve the effective construction of Russian teaching faculty [8]. Prejob training for young and middle-aged Russian teachers can be increased, as professional training for young teachers who are about to take the stage, learning basic teaching norms, understanding teaching requirements, and inviting old teachers to impart teaching experience. We implement the “mentor-apprentice” system: employ experienced Russian professors or excellent English teachers in our school as teaching tutors for young Russian teachers to help young Russian teachers grow as soon as possible [9]. Through activities such as teacher skills competitions, teaching seminars, and observations, the training of basic teaching skills for young Russian teachers will be strengthened, teachers will be encouraged to carry out teaching research, professional curriculum reform, scientific research, actively participate in social services, and improve teachers’ comprehensive ability. From time to time, experienced experts and professors from outside the school are invited to our school to give special reports and academic lectures to improve the professional qualifications of teachers. Finally, we must work hard on the structure of teachers [10]. In the process of carrying out public Russian courses in colleges and universities, the improvement of teachers not only requires the efforts of teachers themselves but also requires colleges and universities to introduce as many high-quality Russian teachers as possible and promote the structure of the teaching staff from “single” to “Diversity” curriculum transformation. From a specific point of view, colleges and universities can give full play to the advantages of the school-enterprise cooperative teaching mode, and hire Russian-speaking staff with rich business and trade work experience from the cooperative enterprises along the “Belt and Road,” or professional and technical personnel from countries along the “Belt and Road.” As part-time teachers of Russian courses for students, teachers can enrich teacher’s Russian professional knowledge, and improve students’ professional education ability in the process of continuous practice, so as to further improve the teaching staff [11]. In addition, the introduction of postgraduates with overseas experience should be increased, especially bilingual teachers with certain knowledge backgrounds, to meet the needs of Russian teaching in colleges and universities. We improve the educational level of imported foreign teachers and ensure the employment of foreign teachers. Foreign teachers play a crucial role in Russian teaching. The imported foreign teachers should have a master’s degree or above and have certain teaching experience. The subject should be Russian as a foreign language, or bilingual teachers with railway professional background knowledge [12].The core courses in the talent training program for Russian professionals include basic Russian, advanced Russian, Russian grammar, Russian reading, Russian audio-visual, and Russian writing. The curriculum focuses on helping students lay a solid foundation and develop language skills. However, it should be noted that the needs of regional economic development, and the employment direction of students should also be taken into consideration when designing talent training programs and writing syllabuses. In terms of teaching methods, traditional teaching methods are still used, which are mainly explained by teachers and passively accepted by students. This single model is boring to today’s students and it is difficult to obtain good results [13].We give full play to the advantages of comprehensive colleges and universities and actively carry out inter-professional exchanges within the school. Russian teachers and teachers of international trade, international law, and other majors regularly hold academic forums to gain an in-depth understanding of relevant knowledge and cutting-edge theories. Teachers of Russian majors are encouraged to apply for scientific research projects jointly with teachers of other majors to break down disciplinary barriers and promote interdisciplinary research. However, we make full use of research results in teaching, constantly update teaching content, improve teaching quality, encourage students to think and explore, and cultivate the spirit of scientific research and innovation [14]. Advocate exchange visits and enrich teaching experience. We support the exchange of teachers between our school and foreign institutions for further study or visiting exchanges, encourage teachers to actively apply for the China Scholarship Council project, participate in international seminars, and introduce the foreign advanced school-running experience and high-quality educational resources. In terms of inter-school cooperation, it attaches great importance to establishing contacts with domestic colleges and universities, regularly organizes teacher training exchanges, and invites authoritative scholars and experts to give teaching lectures and share valuable teaching experiences. We strengthen communication with enterprises and continuously update the knowledge system. Russian teaching work must be “grounded,” and teachers must first go to practice [15].As an inseparable whole, language and culture play complementary roles in the teaching of Russian. Students integrate their understanding of Russian culture into the process of learning the Russian language, which can better promote their mastery of the language. In this regard, foreign teachers have unique advantages. Teachers and students who have never lived in a Russian-speaking country, they have never experienced the national conditions, culture, customs, etc., of a Russian-speaking country but only learn some superficial knowledge from books [16]. In the process of teaching and working with teachers, foreign teachers can use personal examples to explain to students or set up some special teaching links for students to specifically reflect, so that students can learn relevant language knowledge in these processes, and at the same time enable teachers too. In particular, young teachers have learned knowledge that cannot be learned in books and has a better understanding of Russian culture [17].This paper combines the DTW algorithm to construct a Russian continuous speech recognition system and improves the Russian teaching effect through artificial intelligence methods.


## 2. DTW Continuous Speech Recognition Algorithm

### 2.1. The Principle of the DTW-MPCA Algorithm

We assume that there are two-time seriesR and C, the length of the series R is m, and there is R=r1,r2,L,ri,L,rm. The length of the sequence C is n, there is C=c1,c2,L,cj,L,cn. First, in order to use the DTW method to synchronize the two-time series, a path matrix d of mxn needs to be defined in advance. Among them, the elements of the (i and j) position of the matrix represent the distance between the two points ri and cj. The formula for calculating the distance is as follows:(1)dri,cj=ri−cjp.Among them,⋅p represents the p norm, and p usually takes 2.The goal of dynamic time warping is to find the shortest distanceD (m, n) between two-time series and the point sequence of the optimal path P. The optimal performance indicators of the shortest distance D (m, n) and the optimal path P are as follows:(2)Dm,n=minF∑k=1Kdik,jk,(3)P=argminFDm,n.Among them, there is max (m, n) ≤ K ≤ m + n. The shortest distance D (m and n) is the sum of the local shortest distances of the two sequences along the optimal path. The optimal path P=p1,p2,L,pk,pk=ik,jk is a point sequence in an mxn grid searched on the basis of the shortest distance D (m and n), as shown in Figure 1. The element in the optimal path is the quantization value of the anisotropy of the sequence in the matching process, and the curved path represents the optimal path for the matching of the two sequences.Figure 1
Schematic diagram of the principle of DTW algorithm.According to the feasibility of the actual situation, the curved pathP needs to meet the following two conditions:(1)
Endpoint constraint: the first and last elementsp1 and pk of the optimal path P correspond to the elements at both ends of the diagonal of the distance matrix D, so that the stretched and compressed new data, and the original data keep the same starting point and termination point, as shown in the following formula:(4)p1=D1,1,(5)p2=Dm,n.(2)
Global constraints: the selection of the optimal pathP is generally limited to a certain range. It is beneficial to improve the calculation efficiency of the cumulative distance and avoid the appearance of abnormal paths. As shown in Figure 1, the optimal path is bounded within two dashed diagonals.Local constraints define an optional preceding point for each point and specify the continuity of the midpoints of the path. The continuity of the midpoint of the path avoids excessive distortion and jumping off the original data.As shown in Figure2, the local constraint (1) can only have the following three options before point (i and j) in the grid: point (i−1 and j), point (i and j−1), Point (i−l and j−1). Therefore, formula (2) can be obtained by formula (6):(6)Di,j=Di−1,j+di,jDi,j−1+di,jDi−1,j−1+di,j.Figure 2
Schematic diagram of local constraints.As shown in Figure2, the local constraint (2) can only have the following three options before point (i and j) in the grid: point (i−1 and j), point (i−1 and j−1), point (i−1and j−2). Therefore, formula (2) can be obtained by formula (7):(7)Di,j=Di−1,j+di,jDi,j−1+di,jDi−1,j−2+di,j.


### 2.2. Principle of the MPCA Algorithm

The three-dimensional data set for the unequal-length batch process isXI×J×Kii=1,2,L,I. Among them, I represents the number of batches, J represents the number of variables used for offline modeling, and K represents the reaction time. In order to analyze the batch process using multivariate statistical methods, it is necessary to perform synchronization preprocessing on the 3D data set XI×J×Kii=1,2,L,I. Then, the complete 3D dataset X˜I×J×K after synchronization is expanded into a 2D matrix XI×JK based on batches, and the expansion method is shown in Figure 3.Figure 3
Schematic diagram of a batch expansion of 3D data.In order to obtain the average running trajectories of the measurement variables on multiple batches of data, this paper performs the normalization processing of formulas (1)–(4) on the two-dimensional matrix XI×JK after the three-dimensional data set is expanded. The standardized data set approximately obeys the multidimensional normal distribution and can show the fluctuation of process variables in different batch intermittent operations, which has a certain degree of statistical significance.After the batch process, the 3D dataset is preprocessed, and the expanded 2D matrixXI×JK is analyzed using traditional PCA. In fact, it extracts feature information based on the variance between different batches of data and the covariance of different variables at different sampling times. The variance matrix XTX/I−1 of the two-dimensional matrix XI×JK is obtained by the singular value decomposition algorithm as follows:(8)XTXI−1=1I−1X1TX1X1TX1X1TX1LX1TXKX2TX1X2TX2X2TX3LX1TXKX3TX1X3TX2X3TX3LX1TXKMMMMXKTX1XKTX2XKTX3LXKTXK.Among them,XKTXK represents the variance information between measurement data at the same time, and XK1TXK2 represents the covariance information between different measurement data at different times. The decomposition process is as follows:(9)X=∑r=1Rtrpr+E.Among them,R represents the number of retained principal components, tr represents the relationship between batches, pr represents the relationship between variables and time changes, and E represents the residual matrix containing secondary information and noise.Statistics used to monitor multivariate batch processes are theT2 statistic and the SPE statistic. Among them, the T2 statistic monitors the change information of the data in the pivot space, and the SPE statistic monitors the change information of the data in the residual space. During offline modeling, the T2 statistic and SPE statistic are computed for batch i as follows:(10)Ti2=tiTSt−1ti,(11)SPEi=eieiT,(12)ei=Xi−PPTXi.Among them,ei represents the i-th row of the residual matrix, ti is the score vector, and St is the covariance matrix estimated from the pivot score matrix obtained during modeling.For the monitoring data of the MPCA online process, only the current time and previous values are known. The sampling data in the actual monitoring process are imperfect, resulting in insufficient known data to constitute a complete sampling of the intermittent process. Therefore, it is necessary to use data-filling technology when applying MPCA to online monitoring. This paper adopts the method of filling the current sampling value, that is, it is estimated that the data after the current moment is the same as the current moment.The calculation of theT2 statistic for online monitoring is performed at every sampling time k of batch i as follows:(13)Ti2k=tiTkSt−1tik.Moreover, the SPE statistic is the square of the error at the sampling instantk as follows:(14)SPEik=∑c=k−1j+1JKecTec.Among them,ec is the residual matrix E=Xnew−tnewPT.The control limits for theT2 statistic and the SPE statistic are calculated under the assumption that the modeled data are normally distributed. In this paper, kernel density estimation (KDE) is used to calculate the control limits. For any variable z, the probability density function of its radial basis kernel function is expressed as follows:(15)f^z=1n∑i=1n12πexpz−zi2c2.Among them,c is the nuclear parameter, which is determined by the cross-validation method in this paper. Then, the integral of formula (15) is calculated in the definition domain of the variable z. When the integral reaches the set confidence limit, the corresponding value is the control limit obtained by the KDE method.


### 2.3. Variable Grouping DTW-MPCA Method

Two random variablesX and Y are given, p (x) and p (y) are the marginal probability densities of the random variables X and Y, respectively. p (x and y) is the joint probability density of two random variables X and Y, which can be obtained by kernel density estimation or histogram method. The entropy of X is defined as follows:(16)HX=∑xpxlog1px.The joint entropy of random variablesX and Y is defined as follows:(17)HX,Y=∑x∑ypx,ylog1px,y.Conditional entropy is used to describe the uncertainty of inferring two variables when a random variable is known. The definition of conditional entropy is as follows:(18)HX|Y=∑x∑ypx,ylog1py|x.According to the property of entropy, we obtain the following equation:(19)HX,Y=HX+HY|X=HY+HX|Y.Therefore, the mutual information value is as follows:(20)HX,Y=HX−HX|Y=HY−HY|X=HX+HY−HX,Y=∑xpxlog1px+∑ypylog1py−∑x,ypx,ylog1px,y=∑x,ypx,ylogpx,ypxpy.As shown in Figure4, the mutual information between variable xi and variable xj in a batch of batch process data x (K × J) (J = 10) is Ixi,xji,j=1,2,k,J. The greater the mutual information value between the two variables, the closer the relationship between the two variables. The mutual information value of the two variables x; and x; can be calculated, and the specific calculation method is shown in formula (21).(21)Ixi,xj=∑xipxilog1pxi+∑xjpxjlog1pxj−∑xi,xjpxi,xjlog1pxi,xj=∑xi,xjpxi,xjlogpxi,xjpxipxj.Figure 4
Schematic diagram of the mutual information matrix.Among them,x1,x2 represents the mutual information value between the first variable and the second variable, and the first row of the matrix represents the mutual information value between the first variable and all variables. In order to obtain the variable grouping with the greatest correlation between variables, the first row is used to search for the starting point. First, we compare pairs in the first row of the matrix, and remove the smaller value until only one element remains in the first row as the basis for subsequent variable selection. Then, the variable represented by the remaining elements in the first row is judged, and the variable selection in the next row is performed with the new search starting point indicated by the variable. It iterates in this way until the selected variable coincides with variable 1 in the first row. Finally, a variable grouping that searches for the start and end points with the first row can be obtained.As shown in Figure5, we search for the starting point in the first row to obtain the maximum value x1,x3 of the first row and then search for the maximum value of the third row to obtain x3,x4. Then, the maximum value x4,x9 of the 4th row is searched, and the maximum value x9,x1 of the 9th row is finally terminated. In this way, the mutual information value between the variables obtained by searching for the starting point of the first row is the largest, and the sum of the mutual information values of this group of variables is the largest.Figure 5
Schematic diagram of mutual information matrix search.


### 2.4. Mining and Analysis of Dynamic Characteristics of Variables

We assume thatx∈RJ×1 are data consisting of J process variables. The data array consisting of sampled samples of x is as follows:(22)X=x1,x2,x3,L,xNT∈RN×1.Among them,N is the number of samples, and each row of X represents a sample. In order to consider the dynamic characteristics between different sampling times x (k), X is first extended to as follows:(23)X∗=x1∗,x2∗,x3∗,L,xN−d∗T=X0,X−1,X−2,L,X−d.Among them, there isX∗=X0T,X−1T,X−2T,L,X−dTT. x−i corresponds to the variable obtained by shifting the sampling of x at time i, which is as follows:(24)X−i=x−i1,x−i2,x−i3,L,x−iN−dT=xi+1,xi+2,xi+3,L,xN−d+iT.The matrixX is extended to obtain the augmented matrix X∗∈RN−d×Jd+1 as follows:(25)X∗=xT1xT2LxTd+1xT2xT3LxTd+2MMOMxTN−dxTN−d+1LxTN.The value of the lag parameterd in the general augmented matrix construction is 1 or 2, and for the system with strong dynamics, d can be selected by a specific iterative algorithm. If the value of the lag parameter d is small, the number of measurement values introduced into the data matrix may be insufficient, and the dynamic characteristics of the process data cannot be fully captured. If the value of the lag parameter d is large, it may lead to the introduction of measurement value redundancy in the data matrix, which may cause excessive interference to process modeling and monitoring. For this purpose, the lag parameter in the augmented matrix selected in this section is 4. In the data matrix, four measurements were introduced for every single variable to construct an augmented matrix for the single variable. As shown in Figure 6, an augmented matrix is constructed by taking a batch of data as an example. Among them, each column in the matrix represents the value of a variable at all sampling times.Figure 6
Schematic diagram of constructing augmented matrix with different variables.This approach fully captures the dynamic nature of all variables. However, it inevitably creates redundancy in individual variables. Second, in order to accurately obtain the dynamic characteristics of different variables, it is necessary to filter variables for each augmented matrix. The correlation between each measurement variable in the augmented matrix and all variables in other matrices is calculated, and the calculation method is shown in formula (26):(26)Rm,n=Covm,nVarmVarn,Among them,m and n are any two variables in different augmented matrices, Cov (m, n) is the covariance between the two variables, Var (m) and Var (n) are the variances of variables m and n, respectively. The larger the absolute value of R, the stronger the correlation between the two variables. The closer it is to 0, the weaker the correlation.Finally, the correlation between all measured variables in each augmented matrix isK-means clustered into two categories, and the category with the smaller correlation value in the two categories is used as the basis for eliminating variables. The idea of K-means clustering is to divide the data set through continuous iteration, so that different classes are independent and the same class is compact. In this section, Euclidean distance is selected as the similarity measure between data samples, and the performance of the clustering algorithm is evaluated by using the squared error and objective function. The brief steps of the K-means clustering method are as follows: (a) it arbitrarily selects n samples from the input data samples of the correlation between the measured variables as the initial clustering centers. (b) According to the mean of the clustered samples, the Euclidean distance between each sample, and the sample selected as the cluster center is calculated, and the samples are divided according to the minimum distance. (c) The mean of the new cluster samples is recalculated. (d) b and c are iteratively looped until the cluster centers no longer change. From the results in Figure 7, it can be seen that the dimensions of the augmented matrix of different variables are different after screening, and the difference in the dynamics of different variables leads to the difference in the actual lag order demand.Figure 7
Augmented matrix variable selection.


### 2.5. Distributed Dynamic VGDTW-MPCA Model Modeling Strategy

After the corresponding augmented matrix is constructed according to the dynamics of each variable, under the guidance of the block modeling idea, the augmented matrix of different variables can be divided into different blocks according to a certain relationship. Thus, augmented matrices of different chunks are different and augmented matrices of the same chunk are similar. Therefore, it is considered to use the mutual information between augmented matrices of different variables to measure the intrinsic relationship between such variables. Among them, the principle of mutual information has been introduced in the previous chapter, and the schematic diagram of the grouping of different variable augmentation matrices is shown in Figure8.Figure 8
Schematic diagram of augmented matrix grouping.According to formula (21), the mutual information value between the two variable augmented matrices is calculated and the corresponding mutual information matrix is formed. The formula for calculating the mutual information value between a single measurement variable in the augmented matrix is as follows:(27)IXma,Xnb=∑Xma,XnbpXma,XnblogpXma,XnbpXmapXnb.Among them,Xma represents the a-th measurement variable of the augmented matrix m and Xnb represents the b-th measurement variable of the augmented matrix n.The formula for calculating mutual information between one augmented matrix and another augmented matrix is as follows:(28)Im=∑a=1Jm∑b=1JnIXma,Xnb.Among them,Jm represents the number of all measurement variables in the augmented matrix m and Jn represents the number of all measurement variables in the augmented matrix n.We assume that the PCA model is obtained separately from the two grouped augmented matrix data as follows:(29)Xi=TiPi+Ei.Among them, there isi = {1 and 2}. The control limits Ti,lim2 and Qi,lim are calculated at the same time. In the online monitoring phase, when new data are obtained, new Ti2 and Qi are calculated, respectively. In order to achieve a comprehensive monitoring effect, this paper uses the Bayesian fusion technology to combine multiple statistics into a probability form.The probability of misidentification ofx monitored by the Ti2 statistic is defined as follows:(30)PTi2F|x=PTi2x|FPTi2FPTi2x.Moreover, the probability ofPTi2x is defined as follows:(31)PTi2x=PTi2x|NPTi2N+PTi2x|FPTi2F.Among them,x represents the data at a sampling time of the test data, N and F represent normal and abnormal operating conditions, PTi2N and PTi2F can be simply designated as a and 1−a, and a is the confidence level for calculating the control limit. The conditional probabilities PTi2x|N and PTi2x|F are calculated by the following formulas:(32)PTi2x|N=exp−Ti2Ti,lim2,(33)PTi2x|F=exp−Ti,lim2Ti2.Then, a weighted form is used to combine multiple monitoring results into an overall probability indicator as follows:(34)BICT2=∑i=12PTi2x|FPTi2F|x∑i=12PTi2x|F.Similarly, the final probability indicatorBICQ of the Q statistic can be obtained. Usually, when the value of BICQ or BICT2 exceeds the control limits a, an alarm is misidentified. Otherwise, the monitoring process is considered normal.


### 2.6. Description of Segmentation Strategy of the Ms-VGDTW-MPCA Algorithm

As shown in Figure9, the datasheet XI×J×K is obtained by vertically cutting the three-dimensional matrix X˜kI×J along the third dimension. This slice consists of the k-th sampling time of all batches of batch process data and is referred to by the researchers as the time slice matrix in the batch process. The three-dimensional data array used for cutting is in accordance with the predetermined process and parameters under the normal batch process operating conditions, and the process variables follow the predetermined running track. However, an industrial process cannot be completely repeated and the process variable trajectory must fluctuate under the influence of random disturbances. Therefore, the J process variables of the two-dimensional time slice matrix X˜kI×J, after cutting the three-dimensional data under normal operating conditions approximately obey the Gaussian distribution.Figure 9
Schematic diagram of time slice cutting of the three-dimensional matrix.Next, the PCA method is applied to the time slice matrixX˜kI×J with a Gaussian distribution, which can extract the correlation information between the process variables at K sampling instants. Formula (35) is the PCA model of the time slice matrix:(35)X˜k=TkPkT+Ek,Among them,Tk represents the score matrix, Pk represents the load matrix, and Ek represents the residual matrix. The K load matrices represent the correlation information between process variables and can reflect changes in the internal operating mechanism of the process.In order to capture the change of the load matrix, the principal component correlation degree is used as the similarity measure to measure the changing relationship between the two matrices and as the basis for the stage division. The calculation method of the pivot degree is as follows:(36)PCCU,V=trace∧uTUTV∧v∧uTVT∧u∑i=1cλiuλiv.Among them,U and V, respectively, represent the load matrix obtained by PCA in two different time slices, λiu and λiv are their corresponding eigenvalues, c represents the number of load vectors, and there is ∧u=diagλ1u,λ2u,L,λcu and ∧v=diagλ1v,λ2v,L,λcv.A time seriesX=xtii=1n is given. It is divided into z segments: Xj,j=1,2,L,z, which satisfies Xj≠∅,Xj1∩Xj2=∅,∀j1≠j2,j1,j2∈1,2,L,z,∩j=1kXj=X. The corresponding time interval is tsj,tfj, sj,fj∈1,2,L,n, which represents the position of the start and end points of the j-th line segment in the time series, and there is s1=1 and fz=n. A linear function Fjti=ajti+bj, ∀i∈sj,fj on the interval Xj is found to minimize the following objective function:(37)J=∑j=1k∑i=sjfjxti−Fjti2.The objective functionJ uses the residual sum of squares between the original time series and its linear approximation to measure the degree of fit between the two. The smaller J is, the better the linear approximation fits the original time series. The optimal solution Fj∗,Xj∗j=1k obtained by solving the above optimization problem is called an optimal linear approximation of the time series X.


## 2.1. The Principle of the DTW-MPCA Algorithm

We assume that there are two-time seriesR and C, the length of the series R is m, and there is R=r1,r2,L,ri,L,rm. The length of the sequence C is n, there is C=c1,c2,L,cj,L,cn. First, in order to use the DTW method to synchronize the two-time series, a path matrix d of mxn needs to be defined in advance. Among them, the elements of the (i and j) position of the matrix represent the distance between the two points ri and cj. The formula for calculating the distance is as follows:(1)dri,cj=ri−cjp.Among them,⋅p represents the p norm, and p usually takes 2.The goal of dynamic time warping is to find the shortest distanceD (m, n) between two-time series and the point sequence of the optimal path P. The optimal performance indicators of the shortest distance D (m, n) and the optimal path P are as follows:(2)Dm,n=minF∑k=1Kdik,jk,(3)P=argminFDm,n.Among them, there is max (m, n) ≤ K ≤ m + n. The shortest distance D (m and n) is the sum of the local shortest distances of the two sequences along the optimal path. The optimal path P=p1,p2,L,pk,pk=ik,jk is a point sequence in an mxn grid searched on the basis of the shortest distance D (m and n), as shown in Figure 1. The element in the optimal path is the quantization value of the anisotropy of the sequence in the matching process, and the curved path represents the optimal path for the matching of the two sequences.Figure 1
Schematic diagram of the principle of DTW algorithm.According to the feasibility of the actual situation, the curved pathP needs to meet the following two conditions:(1)
Endpoint constraint: the first and last elementsp1 and pk of the optimal path P correspond to the elements at both ends of the diagonal of the distance matrix D, so that the stretched and compressed new data, and the original data keep the same starting point and termination point, as shown in the following formula:(4)p1=D1,1,(5)p2=Dm,n.(2)
Global constraints: the selection of the optimal pathP is generally limited to a certain range. It is beneficial to improve the calculation efficiency of the cumulative distance and avoid the appearance of abnormal paths. As shown in Figure 1, the optimal path is bounded within two dashed diagonals.Local constraints define an optional preceding point for each point and specify the continuity of the midpoints of the path. The continuity of the midpoint of the path avoids excessive distortion and jumping off the original data.As shown in Figure2, the local constraint (1) can only have the following three options before point (i and j) in the grid: point (i−1 and j), point (i and j−1), Point (i−l and j−1). Therefore, formula (2) can be obtained by formula (6):(6)Di,j=Di−1,j+di,jDi,j−1+di,jDi−1,j−1+di,j.Figure 2
Schematic diagram of local constraints.As shown in Figure2, the local constraint (2) can only have the following three options before point (i and j) in the grid: point (i−1 and j), point (i−1 and j−1), point (i−1and j−2). Therefore, formula (2) can be obtained by formula (7):(7)Di,j=Di−1,j+di,jDi,j−1+di,jDi−1,j−2+di,j.


## 2.2. Principle of the MPCA Algorithm

The three-dimensional data set for the unequal-length batch process isXI×J×Kii=1,2,L,I. Among them, I represents the number of batches, J represents the number of variables used for offline modeling, and K represents the reaction time. In order to analyze the batch process using multivariate statistical methods, it is necessary to perform synchronization preprocessing on the 3D data set XI×J×Kii=1,2,L,I. Then, the complete 3D dataset X˜I×J×K after synchronization is expanded into a 2D matrix XI×JK based on batches, and the expansion method is shown in Figure 3.Figure 3
Schematic diagram of a batch expansion of 3D data.In order to obtain the average running trajectories of the measurement variables on multiple batches of data, this paper performs the normalization processing of formulas (1)–(4) on the two-dimensional matrix XI×JK after the three-dimensional data set is expanded. The standardized data set approximately obeys the multidimensional normal distribution and can show the fluctuation of process variables in different batch intermittent operations, which has a certain degree of statistical significance.After the batch process, the 3D dataset is preprocessed, and the expanded 2D matrixXI×JK is analyzed using traditional PCA. In fact, it extracts feature information based on the variance between different batches of data and the covariance of different variables at different sampling times. The variance matrix XTX/I−1 of the two-dimensional matrix XI×JK is obtained by the singular value decomposition algorithm as follows:(8)XTXI−1=1I−1X1TX1X1TX1X1TX1LX1TXKX2TX1X2TX2X2TX3LX1TXKX3TX1X3TX2X3TX3LX1TXKMMMMXKTX1XKTX2XKTX3LXKTXK.Among them,XKTXK represents the variance information between measurement data at the same time, and XK1TXK2 represents the covariance information between different measurement data at different times. The decomposition process is as follows:(9)X=∑r=1Rtrpr+E.Among them,R represents the number of retained principal components, tr represents the relationship between batches, pr represents the relationship between variables and time changes, and E represents the residual matrix containing secondary information and noise.Statistics used to monitor multivariate batch processes are theT2 statistic and the SPE statistic. Among them, the T2 statistic monitors the change information of the data in the pivot space, and the SPE statistic monitors the change information of the data in the residual space. During offline modeling, the T2 statistic and SPE statistic are computed for batch i as follows:(10)Ti2=tiTSt−1ti,(11)SPEi=eieiT,(12)ei=Xi−PPTXi.Among them,ei represents the i-th row of the residual matrix, ti is the score vector, and St is the covariance matrix estimated from the pivot score matrix obtained during modeling.For the monitoring data of the MPCA online process, only the current time and previous values are known. The sampling data in the actual monitoring process are imperfect, resulting in insufficient known data to constitute a complete sampling of the intermittent process. Therefore, it is necessary to use data-filling technology when applying MPCA to online monitoring. This paper adopts the method of filling the current sampling value, that is, it is estimated that the data after the current moment is the same as the current moment.The calculation of theT2 statistic for online monitoring is performed at every sampling time k of batch i as follows:(13)Ti2k=tiTkSt−1tik.Moreover, the SPE statistic is the square of the error at the sampling instantk as follows:(14)SPEik=∑c=k−1j+1JKecTec.Among them,ec is the residual matrix E=Xnew−tnewPT.The control limits for theT2 statistic and the SPE statistic are calculated under the assumption that the modeled data are normally distributed. In this paper, kernel density estimation (KDE) is used to calculate the control limits. For any variable z, the probability density function of its radial basis kernel function is expressed as follows:(15)f^z=1n∑i=1n12πexpz−zi2c2.Among them,c is the nuclear parameter, which is determined by the cross-validation method in this paper. Then, the integral of formula (15) is calculated in the definition domain of the variable z. When the integral reaches the set confidence limit, the corresponding value is the control limit obtained by the KDE method.


## 2.3. Variable Grouping DTW-MPCA Method

Two random variablesX and Y are given, p (x) and p (y) are the marginal probability densities of the random variables X and Y, respectively. p (x and y) is the joint probability density of two random variables X and Y, which can be obtained by kernel density estimation or histogram method. The entropy of X is defined as follows:(16)HX=∑xpxlog1px.The joint entropy of random variablesX and Y is defined as follows:(17)HX,Y=∑x∑ypx,ylog1px,y.Conditional entropy is used to describe the uncertainty of inferring two variables when a random variable is known. The definition of conditional entropy is as follows:(18)HX|Y=∑x∑ypx,ylog1py|x.According to the property of entropy, we obtain the following equation:(19)HX,Y=HX+HY|X=HY+HX|Y.Therefore, the mutual information value is as follows:(20)HX,Y=HX−HX|Y=HY−HY|X=HX+HY−HX,Y=∑xpxlog1px+∑ypylog1py−∑x,ypx,ylog1px,y=∑x,ypx,ylogpx,ypxpy.As shown in Figure4, the mutual information between variable xi and variable xj in a batch of batch process data x (K × J) (J = 10) is Ixi,xji,j=1,2,k,J. The greater the mutual information value between the two variables, the closer the relationship between the two variables. The mutual information value of the two variables x; and x; can be calculated, and the specific calculation method is shown in formula (21).(21)Ixi,xj=∑xipxilog1pxi+∑xjpxjlog1pxj−∑xi,xjpxi,xjlog1pxi,xj=∑xi,xjpxi,xjlogpxi,xjpxipxj.Figure 4
Schematic diagram of the mutual information matrix.Among them,x1,x2 represents the mutual information value between the first variable and the second variable, and the first row of the matrix represents the mutual information value between the first variable and all variables. In order to obtain the variable grouping with the greatest correlation between variables, the first row is used to search for the starting point. First, we compare pairs in the first row of the matrix, and remove the smaller value until only one element remains in the first row as the basis for subsequent variable selection. Then, the variable represented by the remaining elements in the first row is judged, and the variable selection in the next row is performed with the new search starting point indicated by the variable. It iterates in this way until the selected variable coincides with variable 1 in the first row. Finally, a variable grouping that searches for the start and end points with the first row can be obtained.As shown in Figure5, we search for the starting point in the first row to obtain the maximum value x1,x3 of the first row and then search for the maximum value of the third row to obtain x3,x4. Then, the maximum value x4,x9 of the 4th row is searched, and the maximum value x9,x1 of the 9th row is finally terminated. In this way, the mutual information value between the variables obtained by searching for the starting point of the first row is the largest, and the sum of the mutual information values of this group of variables is the largest.Figure 5
Schematic diagram of mutual information matrix search.


## 2.4. Mining and Analysis of Dynamic Characteristics of Variables

We assume thatx∈RJ×1 are data consisting of J process variables. The data array consisting of sampled samples of x is as follows:(22)X=x1,x2,x3,L,xNT∈RN×1.Among them,N is the number of samples, and each row of X represents a sample. In order to consider the dynamic characteristics between different sampling times x (k), X is first extended to as follows:(23)X∗=x1∗,x2∗,x3∗,L,xN−d∗T=X0,X−1,X−2,L,X−d.Among them, there isX∗=X0T,X−1T,X−2T,L,X−dTT. x−i corresponds to the variable obtained by shifting the sampling of x at time i, which is as follows:(24)X−i=x−i1,x−i2,x−i3,L,x−iN−dT=xi+1,xi+2,xi+3,L,xN−d+iT.The matrixX is extended to obtain the augmented matrix X∗∈RN−d×Jd+1 as follows:(25)X∗=xT1xT2LxTd+1xT2xT3LxTd+2MMOMxTN−dxTN−d+1LxTN.The value of the lag parameterd in the general augmented matrix construction is 1 or 2, and for the system with strong dynamics, d can be selected by a specific iterative algorithm. If the value of the lag parameter d is small, the number of measurement values introduced into the data matrix may be insufficient, and the dynamic characteristics of the process data cannot be fully captured. If the value of the lag parameter d is large, it may lead to the introduction of measurement value redundancy in the data matrix, which may cause excessive interference to process modeling and monitoring. For this purpose, the lag parameter in the augmented matrix selected in this section is 4. In the data matrix, four measurements were introduced for every single variable to construct an augmented matrix for the single variable. As shown in Figure 6, an augmented matrix is constructed by taking a batch of data as an example. Among them, each column in the matrix represents the value of a variable at all sampling times.Figure 6
Schematic diagram of constructing augmented matrix with different variables.This approach fully captures the dynamic nature of all variables. However, it inevitably creates redundancy in individual variables. Second, in order to accurately obtain the dynamic characteristics of different variables, it is necessary to filter variables for each augmented matrix. The correlation between each measurement variable in the augmented matrix and all variables in other matrices is calculated, and the calculation method is shown in formula (26):(26)Rm,n=Covm,nVarmVarn,Among them,m and n are any two variables in different augmented matrices, Cov (m, n) is the covariance between the two variables, Var (m) and Var (n) are the variances of variables m and n, respectively. The larger the absolute value of R, the stronger the correlation between the two variables. The closer it is to 0, the weaker the correlation.Finally, the correlation between all measured variables in each augmented matrix isK-means clustered into two categories, and the category with the smaller correlation value in the two categories is used as the basis for eliminating variables. The idea of K-means clustering is to divide the data set through continuous iteration, so that different classes are independent and the same class is compact. In this section, Euclidean distance is selected as the similarity measure between data samples, and the performance of the clustering algorithm is evaluated by using the squared error and objective function. The brief steps of the K-means clustering method are as follows: (a) it arbitrarily selects n samples from the input data samples of the correlation between the measured variables as the initial clustering centers. (b) According to the mean of the clustered samples, the Euclidean distance between each sample, and the sample selected as the cluster center is calculated, and the samples are divided according to the minimum distance. (c) The mean of the new cluster samples is recalculated. (d) b and c are iteratively looped until the cluster centers no longer change. From the results in Figure 7, it can be seen that the dimensions of the augmented matrix of different variables are different after screening, and the difference in the dynamics of different variables leads to the difference in the actual lag order demand.Figure 7
Augmented matrix variable selection.


## 2.5. Distributed Dynamic VGDTW-MPCA Model Modeling Strategy

After the corresponding augmented matrix is constructed according to the dynamics of each variable, under the guidance of the block modeling idea, the augmented matrix of different variables can be divided into different blocks according to a certain relationship. Thus, augmented matrices of different chunks are different and augmented matrices of the same chunk are similar. Therefore, it is considered to use the mutual information between augmented matrices of different variables to measure the intrinsic relationship between such variables. Among them, the principle of mutual information has been introduced in the previous chapter, and the schematic diagram of the grouping of different variable augmentation matrices is shown in Figure8.Figure 8
Schematic diagram of augmented matrix grouping.According to formula (21), the mutual information value between the two variable augmented matrices is calculated and the corresponding mutual information matrix is formed. The formula for calculating the mutual information value between a single measurement variable in the augmented matrix is as follows:(27)IXma,Xnb=∑Xma,XnbpXma,XnblogpXma,XnbpXmapXnb.Among them,Xma represents the a-th measurement variable of the augmented matrix m and Xnb represents the b-th measurement variable of the augmented matrix n.The formula for calculating mutual information between one augmented matrix and another augmented matrix is as follows:(28)Im=∑a=1Jm∑b=1JnIXma,Xnb.Among them,Jm represents the number of all measurement variables in the augmented matrix m and Jn represents the number of all measurement variables in the augmented matrix n.We assume that the PCA model is obtained separately from the two grouped augmented matrix data as follows:(29)Xi=TiPi+Ei.Among them, there isi = {1 and 2}. The control limits Ti,lim2 and Qi,lim are calculated at the same time. In the online monitoring phase, when new data are obtained, new Ti2 and Qi are calculated, respectively. In order to achieve a comprehensive monitoring effect, this paper uses the Bayesian fusion technology to combine multiple statistics into a probability form.The probability of misidentification ofx monitored by the Ti2 statistic is defined as follows:(30)PTi2F|x=PTi2x|FPTi2FPTi2x.Moreover, the probability ofPTi2x is defined as follows:(31)PTi2x=PTi2x|NPTi2N+PTi2x|FPTi2F.Among them,x represents the data at a sampling time of the test data, N and F represent normal and abnormal operating conditions, PTi2N and PTi2F can be simply designated as a and 1−a, and a is the confidence level for calculating the control limit. The conditional probabilities PTi2x|N and PTi2x|F are calculated by the following formulas:(32)PTi2x|N=exp−Ti2Ti,lim2,(33)PTi2x|F=exp−Ti,lim2Ti2.Then, a weighted form is used to combine multiple monitoring results into an overall probability indicator as follows:(34)BICT2=∑i=12PTi2x|FPTi2F|x∑i=12PTi2x|F.Similarly, the final probability indicatorBICQ of the Q statistic can be obtained. Usually, when the value of BICQ or BICT2 exceeds the control limits a, an alarm is misidentified. Otherwise, the monitoring process is considered normal.


## 2.6. Description of Segmentation Strategy of the Ms-VGDTW-MPCA Algorithm

As shown in Figure9, the datasheet XI×J×K is obtained by vertically cutting the three-dimensional matrix X˜kI×J along the third dimension. This slice consists of the k-th sampling time of all batches of batch process data and is referred to by the researchers as the time slice matrix in the batch process. The three-dimensional data array used for cutting is in accordance with the predetermined process and parameters under the normal batch process operating conditions, and the process variables follow the predetermined running track. However, an industrial process cannot be completely repeated and the process variable trajectory must fluctuate under the influence of random disturbances. Therefore, the J process variables of the two-dimensional time slice matrix X˜kI×J, after cutting the three-dimensional data under normal operating conditions approximately obey the Gaussian distribution.Figure 9
Schematic diagram of time slice cutting of the three-dimensional matrix.Next, the PCA method is applied to the time slice matrixX˜kI×J with a Gaussian distribution, which can extract the correlation information between the process variables at K sampling instants. Formula (35) is the PCA model of the time slice matrix:(35)X˜k=TkPkT+Ek,Among them,Tk represents the score matrix, Pk represents the load matrix, and Ek represents the residual matrix. The K load matrices represent the correlation information between process variables and can reflect changes in the internal operating mechanism of the process.In order to capture the change of the load matrix, the principal component correlation degree is used as the similarity measure to measure the changing relationship between the two matrices and as the basis for the stage division. The calculation method of the pivot degree is as follows:(36)PCCU,V=trace∧uTUTV∧v∧uTVT∧u∑i=1cλiuλiv.Among them,U and V, respectively, represent the load matrix obtained by PCA in two different time slices, λiu and λiv are their corresponding eigenvalues, c represents the number of load vectors, and there is ∧u=diagλ1u,λ2u,L,λcu and ∧v=diagλ1v,λ2v,L,λcv.A time seriesX=xtii=1n is given. It is divided into z segments: Xj,j=1,2,L,z, which satisfies Xj≠∅,Xj1∩Xj2=∅,∀j1≠j2,j1,j2∈1,2,L,z,∩j=1kXj=X. The corresponding time interval is tsj,tfj, sj,fj∈1,2,L,n, which represents the position of the start and end points of the j-th line segment in the time series, and there is s1=1 and fz=n. A linear function Fjti=ajti+bj, ∀i∈sj,fj on the interval Xj is found to minimize the following objective function:(37)J=∑j=1k∑i=sjfjxti−Fjti2.The objective functionJ uses the residual sum of squares between the original time series and its linear approximation to measure the degree of fit between the two. The smaller J is, the better the linear approximation fits the original time series. The optimal solution Fj∗,Xj∗j=1k obtained by solving the above optimization problem is called an optimal linear approximation of the time series X.


## 3. Russian Continuous Speech Recognition System

The most flexible hardware device in the embedded system is the FPGA (Field Programmable Logic Gate Array), which is very suitable for implementing this algorithm. The Russian continuous speech recognition system based on DTW proposed in this paper is shown in Figure10.Figure 10
System framework.The abovementioned framework constructs a Russian continuous speech recognition system based on DTW; then, the effect of the system is verified, the accuracy of speech recognition is counted, and the results shown in Figure11 are obtained.Figure 11
The accuracy of Russian recognition of the DTW-based Russian continuous speech recognition system.It can be seen from the abovementioned research that the Russian continuous speech recognition system based on DTW proposed in this paper has a high continuous Russian speech recognition accuracy.


## 4. Conclusion

To improve the overall level of the teaching staff of public Russian courses in colleges and universities, it is necessary for the teachers of Russian courses in colleges and universities to carry out self-learning and self-improvement. The school should help teachers of Russian courses to develop a sense of self-improvement, and at the same time update the educational concept in the course teaching, constantly enrich the personal knowledge structure, and improve their professional knowledge level and education level. At the same time, it is necessary to continue learning as an important task, make use of modern information-based teaching technology, constantly supplement new knowledge and new content, and learn the teaching and research methods of Russian teachers in other schools. This paper combines the DTW algorithm to construct a Russian continuous speech recognition system and improves the Russian teaching effect through artificial intelligence methods. It can be seen from the simulation results that the Russian continuous speech recognition system based on DTW proposed in this paper has high continuous Russian speech recognition accuracy.



---

*Source: 5777472-2022-09-19.xml*