# Multimodal Intelligent Acoustic Sensor-Assisted English Pronunciation Signal Acquisition and Phonetic Calibration

Authors: Nan Hu; Yongyi Bi
Journal: Journal of Sensors
 (2022)

---

## Abstract

In this paper, a multimodal intelligent acoustic sensor is used for an in-depth study and analysis of English pronunciation signal acquisition and calibration analysis of English phonetic symbols based on the acquired sound signals. This paper proposes a bimodal fusion algorithm around the direction of feature extension and fusion of acoustic recognition features. After each unimodal classification error cost is minimized, the current fusion process is determined by adaptive weights to fix its one decision layer on the fusion. The adaptive weight approach in this algorithm improves the drawback of always identifying one mode as the optimal mode in fixed-weight fusion and further improves applicability and performance compared to unimodal recognition. The random network generation algorithm is used to generate a random network for sound source data acquisition; then, the algorithm is investigated using the decomposition containing fusion center algorithm to each node, and data preprocessing is implemented at each node; finally, the distributed consistency algorithm based on average weights is used for consistent averaging iterations to achieve a consistent speech enhancement effect at each node. The experimental results show that this distributed algorithm can effectively suppress the interference of noncoherent noise, and each node can obtain an enhanced signal close to the source signal-to-noise ratio. In this study, factors that may affect the readability of spoken texts are summarized, analyzed, defined, and extracted, and the difficulty of spoken items obtained from the divisional scoring model is used as the dependent variable, and the extracted influencing factors are used as independent variables for feature screening, model construction, and tuning, and the generated results are interpreted and analyzed. From this, it was found that phonological features have a strong influence on the readability of spoken texts, mainly in features such as phonemes, syllables, and stress. This study is summarized, and the shortcomings of location-based contextual mobile learning of spoken English in terms of student management, device deployment, and empirical evidence are pointed out, to provide references and lessons for the research on IT-supported language learning.


---

## Body

## 1. Introduction

In acoustic emission detection systems, resonant piezoelectric transducers are often used as acoustic receivers, which convert the measured changes into resonant changes. Resonant piezoelectric transducers use resonance technology to analyze parameters such as amplitude, vibration frequency, and phase for resonant oscillators to achieve measurements of parameters such as acoustic pressure and displacement, an algorithm that has the advantage of enabling high accuracy and resolution [1]. Despite the significant advantages of such sensors, they cannot be used in extreme environments, such as high voltage environments and environments subject to electromagnetic interference, and are difficult to embed inside materials. Compared with resonant piezoelectric sensors, fiber optic sensors have many unique properties. Fiber optic sensors have a series of unique advantages: good electrical insulation performance, strong resistance to electromagnetic interference, noninvasive, high sensitivity, easy to achieve long-distance monitoring of the measured signal, corrosion resistance, explosion-proof, the optical path can be flexed, and easy to connect with the computer. They have high sensitivity, strong immunity to electromagnetic interference, extreme corrosion resistance, etc. At the same time, they are simple, compact, and lightweight, can be transmitted over long distances, and can be used as an important part of online telemetry systems. Since the 20th century, fiber optic sensors have received special attention from scholars in various countries; fiber optic sensors have been applied to measure temperature, stress, current, and hundreds of physical quantities, but also in high temperature, corrosion, and other special environments, and with the practical significance of traditional electronic sensors, they cannot be replaced. For solid-state sensors due to the manufacturing process and cost and other factors, the induction area is also smaller [2]. And with the development of embedded and a variety of mobile wearable devices, fingerprint capture devices are also towards the trend of smaller and smaller. For example, Apple’s Touch ID fingerprint capture device is only 6.35mm×6.35mm, and the universal smartphone at the same time, with the development of the trend, is bound to make the size of the fingerprint image capture also becomes smaller, and thus, the information contained in the fingerprint image is also greatly reduced. Multimodal biometrics refers to the integration or fusion of two or more biometric technologies, utilizing the unique advantages of its multiple biometric technologies and combining with data fusion technology to make the authentication and recognition process more accurate and secure. Multimodal biometrics can realize the combination of multiple biometrics such as face, fingerprint, finger vein, iris, and voice print, so as to carry out more accurate identity authentication.It has been shown that when the sensing area of the acquisition device becomes small, the performance of the traditional fingerprint feature point matching recognition method will be greatly affected due to the low number of captured detail points. Therefore, the problems of missing features and recognition accuracy of small-area fingerprints have become the focus of scholars’ research, and feature extension or fusion is the trend to compensate for the missing features [3]. Multimodal biometrics is a new biometric technology that overcomes some drawbacks of unimodal biometrics by fusing no less than two unimodal biometric features (or behavioral features) as data objects, which can get rid of the limitations of usage scenarios with higher stability and security, making the authentication process more accurate and secure. Multimodal fusion also overcomes, for example, the situation where kind of biometric features is lost, such as fingerprints due to cuts, wear, and tear, dryness, or inconspicuous innate features. Multimodal biometrics makes up for the deficiency [4]. One of the more typical methods is the beamforming method, which can obtain different performances depending on different beam constraints, such as minimum variance and minimum mean square error criteria [5]. The speech enhancement performance is proportional to the number of microphones, which means that more microphones will usually result in better speech enhancement performance. Speech recognition technology has a very wide range of application areas and market prospects, such as for voice-activated voice dialing system, voice-activated smart toys, and smart home appliances. It can also be applied to information network inquiries, medical services, banking services, etc. In beamforming algorithms, the microphone array is generally located relatively close to the sound source, which makes both the signal-to-noise ratio and the direct reverberation ratio of the received signal sufficiently large to obtain a better enhancement performance. The pronunciation calibration engine A/D circuit is designed to improve the data sampling efficiency by using analog to digital signal conversion for English pronunciation information acquisition.With the continuous development of speech recognition technology, various derivative technologies based on speech recognition, such as keyword recognition, language recognition, speaker recognition, and ambient sound recognition, have also been devoted increased attention and have made promising progress. At the same time, speech recognition is also penetrating increasingly into our common life. The biggest advantage of speech recognition is that it makes human-computer interaction more natural and convenient. Nowadays, speech recognition technology has been widely used in voice communication systems, data inquiry, ticket booking systems, medical systems, banking services, computer control, industrial control, and many other fields, providing us with a more convenient way of life and playing a significant role in human progress. Social development plays a pivotal role. In summary, speech recognition technology has great research value and application prospects, especially phonetic recognition technology with phonetic symbols as the basic recognition unit, which is outstanding in practical applications under large vocabulary. This has led many researchers to actively participate in the research of phonetic recognition technology.In the second part of the paper, we organize and summarize the existing research, and in the third part, we provide a detailed description of the specific implementation. In the fourth section, we provide a detailed analysis of the results of the implementation and illustrate the advantages of our results.


## 2. Related Work

Speech enhancement research based on distributed microphone arrays is a hot topic in speech signal processing that has emerged in recent years. Scholars initially studied this problem based on the topology of distributed microphone placement and proposed some distributed noise cancellation algorithms, when microphones are distributed throughout the observation area to form an array, still transmitting the received signal from each microphone to the fusion center for unified processing [6]. The spoken language evaluation algorithm models the randomness of natural language pronunciation and the instability of speech processing systems through fuzzy measures and plausibility and then integrates them into the Sugeno integral framework to evaluate the language score rather than specific scores. A delay-free subband adaptive filter is used to implement multichannel speech enhancement in MRI devices. This speech enhancement algorithm is based on a minimum variance estimation criterion to remove noise using a minimum mean square error adaptive filter, where one microphone is used as the reference microphone to receive the noise-containing signal and the other microphones receive only the noise, using the noise as the target signal for the adaptive filter to estimate the source signal [7]. Based on the minimum mean square error criterion to estimate the speech source spectral amplitude and phase, this algorithm assumes that the signal conforms to the Rayleigh distribution and the noise conforms to the Gaussian distribution, which in turn estimates the speech source short-time amplitude spectrum, logarithmic amplitude spectrum, and phase spectrum using the statistical model and the minimum mean square error criterion [8].Two steps are typically followed in the training of speaking assessment algorithms, namely, score generation and score calibration. Score calibration involves adjusting machine scores and combining scores from multiple divisions, to develop scores that match expert judgments as closely as possible [9]. To achieve this goal, it was necessary to collect data including ratings of pronunciation by human expert evaluators to test the validity of this type of assessment algorithm or system. For reliability assessment, it usually means that the scores obtained by the same test-taker after taking the same test under different test conditions are consistent or highly correlated [10]. Device sensing technology can be divided into five basic categories: vector pressure sensing technology touch screen, resistive technology touch screen, capacitive technology touch screen, infrared technology touch screen, and surface acoustic wave technology touch screen. Traditional speaking tests have experts as scorers, where more factors can lead to unfair scoring, such as scorer evaluation concerns, experience, and ability, age, gender, and psychology. Machine-based speaking assessments, on the other hand, exclude a range of subjective factors that can cause problems, thus ensuring that the speaking assessment algorithm or system has high reliability [11]. Most of these systems are based on ubiquitous learning theory and generally push location-related content information to learners based on their different location information [12].The focus of research has shifted to the construction of ubiquitous learning environments with a rich variety of implementation technologies [13]. The gradual integration of mobile phones, radios, and sensor technologies into learning activities has facilitated the formation of new learning environments that are highly location-portable and context-aware. From foreign studies, it is found that technologies such as mobile phones, wireless networks, and multimedia are popular in language teaching and learning research. Many researchers had placed RFID tags on top of many objects, and sensors can sense these objects when learners are near, allowing the learners’ mobile devices to receive information about these objects [14]. Most of the platforms targeting language learning or context awareness lack and should have empirical studies and rarely do long-term tracking of the systems, but there is no shortage of new technologies introduced and applied. The corresponding system platforms pay attention to theoretical guidance, as well as the analytical design and implementation techniques of the whole system [15]. In addition, although domestic language learning systems make use of the characteristics of mobile devices, they seldom point to specific should-Chan scenarios or contextual information as well as lack specific feedback mechanisms and are rarely integrated with specific speaking instruction.


## 3. Analysis of English Pronunciation Signal Acquisition and Phonetic Calibration with Multimodal Intelligent Acoustic Sensors

### 3.1. Multimodal Intelligent Acoustic Sensor Sound Acquisition Design

Acoustic emission is a type of elastic wave, and in the real environment, it can be divided into transverse waves, longitudinal waves, and surface waves, depending on how the sound waves propagate in the medium and the direction of vibration. When propagating in solids, it causes local deformation of the medium, and two types of waves appear simultaneously: longitudinal (compressional) and shear (transverse) waves, which have different speeds and are automatically separated after leaving the source [16]. When the acoustic signal propagates to the junction of air and solid media, the acoustic wave is reflected and refracted, which causes it to undergo a waveform transition, followed by the simultaneous appearance of two waveforms. On free surfaces, acoustic emission waves also form surface waves, and plate waves are generated when the thickness of the solid medium is like the wavelength. After reflection and other transformations of the acoustic signal, the various forms of sound waves will be transmitted to the sensor according to their wave speed, time sequence, and wave range, so that the pulsed signals generated by the acoustic emission sources can be superimposed on each other to produce long and complex waveforms. The online learning environment provides course resources for learning management. Students carry out independent learning, teachers assist through inspiration and demonstration, and students learn cooperatively, communicate with each other, and share wisdom.When the device receives an acoustic signal, the alternating forces during the resonance of the sound on the piezoelectric film will cause deformation of the PVDF-TrFE piezoelectric film, which causes a piezoelectric effect and generates an electrical charge. To accurately measure the voltage signal generated on the PVDF-TrFE piezoelectric film due to the piezoelectric effect of the acoustic device in operation, we use function generators, ultrasonic transducers, loudspeakers, high sampling rate voltage data acquisition cards, electronic computers, and software data analysis to build a test platform for acoustic device performance characterization, and as shown in Figure1, the working principle of the experimental platform is as follows: use function generator to generate alternating voltage and frequency-controlled electrical signal, through the ultrasonic transducer electroacoustic conversion, the electrical signal into the required acoustic signal, or use the electronic computer connected to the speaker to play the required sound samples, the acoustic signal reached the PVDF-TrFE thin-film acoustic device, through the high sampling rate voltage data acquisition card for the device response voltage signal acquisition. The voltage signal is recorded, analyzed, and processed by an electronic computer and software.Figure 1
Simplified waveform diagram of the acoustic emission signal.An acoustic emission signal is a process in which the emission signal reaches its highest amplitude and then gradually decays. The ringing count reflects the number and frequency of acoustic emission events, and the amplitude is the maximum amplitude of the signal waveform, which is used to distinguish the type of wave source as well as to measure the strength of the wave. The duration is the time from when the acoustic signal first rises above the threshold limit to when it falls to the threshold. The rise time is the time when the acoustic emission signal first exceeds the threshold to reach its maximum amplitude. The mathematical expression can be expressed as a decaying sine function as shown in Equation (1).
(1)xt=A0expαtcosπf0t.A is the amplitude of the acoustic signal Pa; α is the attenuation factor of the signal; f0 is the resonant frequency of the acoustic emission sensor (Hz). When there is acoustic emission generation, the acoustic emission sensor located on the surface of the component or inside the component converts the acoustic signal into an electrical signal, amplifies it through a signal amplifier, and then after acquisition and data processing, analyzes the relevant characteristic parameters and calculates the specific location of the measured acoustic emission source by combining mathematical geometric formulae with data analysis.The MZI-based wavelength demodulation technique is based on the sensing principle of fiber optic interferometer, by converting the DFB fiber laser sensor wavelength drift into phase change after entering the nonequilibrium fiber interferometer, and using the high-resolution phase demodulation technique can finally get the sensor weak wavelength change [17]. It transmits the narrow-band laser to the MZI by exciting the 980 nm pump light source and 980/1550 nm fiber wavelength division multiplexer (WDM) guide and DFB fiber laser sensor output, and then convert its optical signal into an electrical signal output by photodetector; by the demodulation system for the analysis of data processing, the output electrical signal is proportional to the magnitude of the acoustic signal.In the actual environment, the speech signal observed by each microphone in the array will be disturbed by some environmental signals such as noise and reverberation, and there are two common microphone array signal models, the ideal model and the actual reverberation model. The ideal model considers an array ofN microphones, and the array structure is chosen to be either one of the structures, and let the source signal be st; then, the signal observed by the ith microphone yit can be expressed as follows:
(2)yit=αist+τi−nit,where αi is the attenuation coefficient of the source signal due to distance, τi is the time delay of the source signal to the ith microphone, nit represents the background noise at the ith microphone, and t is the time factor. Since the actual environment contains other disturbances such as reverberation and multipath effects, for an array of N microphones, let the source signal be st and the acoustic transfer function from the source to the ith microphone be hi; then, the signal observed by the ith microphone yit can be expressed as follows:
(3)yit=hist∗st−nit.For the ideal model, the acoustic transfer function can be equated as follows:(4)hist=αist+τi,where zt is the output beamforming result and αi is the weighting factor, generally taken as 1/N. This can be described by the following expression.
(5)zt=∑i=1Nαiyit+τi.Wireless sensor networks (WSNs) consist of multiple inexpensive miniature sensor nodes, and each node contains one or more sensors, which constitute a distributed network with less power consumption and lower cost through wireless collaborative information sharing, which are used in a wide range of applications such as environmental science, medical health, space exploration, remote environmental monitoring, and target tracking. Especially in recent years, the development of smaller, cheaper, and smart sensors has greatly enhanced the scope of wireless sensor network applications [18]. Sound wave is a mechanical wave, generated by the vibration of an object (sound source); the space in which the sound wave propagates is called the sound field. It is a longitudinal wave when propagating in gas and liquid media but may be mixed with transverse waves when propagating in solid media. These wireless sensors are equipped with wireless interfaces for communicating with each other to form a network. These sensors are smaller and are capable of only limited computation and processing of information or data at a low cost compared to conventional sensors. These sensor nodes can collect information by sensing and measuring in the environment and can transmit the sensed data to the user based on some local discriminant criterion. In this paper, we focus on speech processing; at this point, the sensors become microphone sensors, so this network is also called wireless acoustic sensor networks (WASNs).Acoustic sensors are mainly used for sensing and measuring environmental information; processors and memories perform limited processing and storage of data; power supplies are generally composed of batteries, which are the main energy for this sensor network and are limited in energy; wireless transmitter-receiver devices are used for information transmission between nodes, and actuators are mainly for local discrimination (see Figure2 for the specific structure).Figure 2
Wireless acoustic sensor network with 20 nodes.For unstructured WASNs, network maintenance management and detection of faults are more difficult due to many nodes. In contrast, for structured WASNs, some or all the sensor nodes are arranged in a way that is required for certain purposes. The advantage of structured networks is the ability to deploy fewer nodes to obtain smaller network maintenance and management costs. But fewer nodes mean that providing network coverage is significantly reduced.Unlike traditional networks, wireless acoustic sensor networks are subject to design and resource constraints. Resource constraints include limited energy per node, short communication range, low bandwidth, and limited computational and storage capacity. Design constraints are application-dependent and based on the environment being monitored. The environment plays a key role in the size of the network, the deployment scheme, and the network topology. The size of the network varies with the environment being monitored. For an indoor environment, a network can be formed with fewer nodes, while an outdoor environment may require more nodes to cover a larger area. Obstacles in the environment can also limit communication between nodes, which in turn can interfere with the connectivity of the network. Acoustic measurement software is a measurement software that runs on a smartphone or tablet PC. It is paired with an instrument to achieve accurate measurements and supports acquisition of measurement data from WIFI and analysis of calculations.


### 3.2. Experimental Design for English Pronunciation Phonetic Calibration

Due to the coarticulation phenomenon, phonetic pronunciations do not exist completely independently in time, and neighboring phonetic pronunciations partially overlap and influence each other [19]. These indicate the need to apply features that can capture long-time information in speech recognition. To address the above problem, we try to extend the spectral features in the time domain by combining the Mel subband energy of the current frame and the subband energies of multiple frames before and after it to obtain the time-domain extended features, which are called temporal pattern (TRAP) features. The extraction process of TRAP features is shown in Figure 3.Figure 3
TRAP feature extraction process.First, the speech signal is windowed in frames, and the energy value of the output of each frame through the Mayer filter bank is calculated. And with the current frame as the center point, the same number of frames is taken before and after, and these several frames’ features are concatenated to get a long-time feature. Then, the DCT method is used for dimensionality reduction, and the features are normalized on the mean and variance to finally obtain the TRAP features. TRAP reflects the long-time feature variation and effectively exploits the correlation between speech signals. Compared to short-time features such as MFCC, TRAP features effectively exploit the correlation between speech signal contexts, which not only improves the noise robustness of acoustic features, but it helps to solve the copronunciation phenomenon among phonetic features, which can improve the performance of speech recognition.Phoneme posterior probability features are commonly used feature representations in speech recognition with high discriminative and robust features. Define the phoneme model space to consist ofM classes (for an English corpus with 39 phonemes); then, M=39. Then, the phoneme posterior probability feature vector for a frame of speech at moment t is as follows:
(6)Pt=ps1ot,ps2ot,⋯,siot,⋯,sMot,where ot denotes the corresponding speech frame at that moment and si denotes the corresponding ith phoneme; then, ps1ot denotes the posterior probability that the corresponding phoneme is si for a known speech frame ot.
(7)∑i=1Mps1ot=2.Rhythmic features are a typical feature of natural human language, with many common features across languages. When people communicate with each other using language, it is not only the phonetic sounds of the words but also the suprasegmental information of the language, i.e., the rhythmic features, which play a very important role in the naturalness and intelligibility of the language. Rhythmic features respond to variations in pitch, length, and intensity in addition to phonetic features: variations in pitch form the tone of speech, variations in length from the length of speech, and variations in intensity from the stress. Speech rhythm features are widely used in the fields of speech recognition, speaker recognition, and language recognition because of the rich dynamic information they carry. No state can be transferred to any of the previous states:(8)aij=0,j≥i.A GMM model is a multidimensional probability density function model that utilizes a weighted sum of multiple Gaussian probability density functions to represent arbitrary probability distributions. For example, Equation (9) is a GMM model with M Gaussian components:
(9)px⟶λ=∑i=1Mwipxi⟶μi,(10)∑i=1Mwi=2.The ball in Equations (9) and (10) is a D-dimensional eigenvector, pxi⟶μi is the probability density function of one of the Gaussian components, μi denotes the mean vector of the Gaussian components, and x⟶ is the covariance matrix and is calculated in Equation (11).
(11)pxi⟶μi=12π∑ii2exp12x⟶−μiT∑ii2x⟶−μi.The observed probability distribution is what is characterized using the GMM model, and the superiority of the GMM as an observed probability model is that the continuous distribution model provides a more accurate description of the probability distribution compared to the discrete or semicontinuous model. This is because the characteristics of the inputs in the continuous distribution model do not need to be quantified. In addition, GMM models can fit arbitrary distributions infinitely accurately by increasing or decreasing the number of mixing components. Thus, the CMU-SPHINX phoneme recognition mechanism sees distribution of a single Gaussian distribution when the state is not bound at the beginning of parameter training. The state binding complicates the model parameters by Gaussian splitting, which in turn leads to more accurate parameter estimates, as shown in Table1.Table 1
Steps of Viterbi algorithm.
(1) Initializationδ1=πibio1,i∈Nψii=1,i∈N(2) RecursionFort=2,3,⋯,Tψiti=argminπibio1,i∈Nδti=minπibio1,i∈N(3) TerminationP∗=maxψitiS∗=argmaxψiti(4) Optimal path backtrackingFort=T−1,T−2,⋯,0st=ψt+1si(5) Find the optimal pathS∗S∗=s1,s2,⋯,sNAccording to the dynamic programming principle, the optimal path has the property that if the optimal path passes through noderi at time ti, then the partial path of this path from node ri to endpoint pi must be optimal for all possible partial paths from ti to ri. For if this is not the case, then another better partial path exists from ti to ri. If it is connected to the partial path from ti to pi, it will form a path that is better than the original path, which is contradictory. According to this principle, we only need to compute recursively the maximum probability of each partial path with state i at moment t=1 until we get the maximum probability of each path with state i at moment t=T. The maximum probability at moment t=T is the probability of the optimal path, and the endpoint pi of the optimal path is obtained at the same time. After that, to find the individual nodes of the optimal path, starting from the endpoint ri, the nodes are gradually solved from back to front iT−1,iT−2,⋯,iT, and the optimal path is obtained I=iT−1,iT−2,⋯,iT.The preparation of data has always been a prerequisite and key to system experiments. Especially in systems using fully connected neural networks as the underlying model, when the number of input and hidden layer neurons is relatively large and the number of model parameters is huge, the need for a large amount of training data is more urgent. Only by collecting enough training data can the acoustic model be trained more adequately and accurately. In addition, the quality of the data preparation is also essential for the training of good acoustic models. When the quality of the prepared data is poor, it will lead to the existence of outliers, which will affect the accuracy of the model training. Resource limitation is the speed of program execution limited by computer hardware resources or software resources during concurrent programming. In concurrent programming, the principle of making code execution faster is to turn the serial part into concurrent execution. A tree network can contain branches, and each branch can contain multiple nodes. The tree topology is an expanded form of the bus topology, and the transmission medium is an unclosed branch cable. The tree topology is the same as the bus topology, where one station sends data and all other stations can receive it.The meaning of tree topology network is the network left by removing the edges that form a circle (ring) in the network, like the spanning tree algorithm [20]. This network has a minimum connection structure and most efficient information transfer rate and is suitable for information transfer between nodes. This topological network uses the knowledge of tree to divide the adjacent nodes into parent and child nodes so that information transfer can be divided into two processes: convergence and dispersion. Convergence starts from the leaf nodes, and the child nodes converge their information to the parent nodes to the root node; dispersion is exactly the opposite, starting from the root node and the parent node disperse their information to the child nodes, all the way to the leaf nodes, to achieve all nodes can get consistent fusion results, and due to the use of spanning-tree structure, in each node, just convergence or dispersion process can greatly reduce the duplication of information transmission, improve the efficiency of information transmission, and reduce the energy overhead of the sensor network, as shown in Algorithm 1.Algorithm 1: Algorithm pseudocode.
1: Input: The matching scores of the two classifiers are respectively denoted as X=(X, X), X=Px, |R ), X,=Py, R), i=1,2.... N, j =1,..,k; N represents a total of N categories, and k represents a total of k samples.α is the weight of the fingerprint identification system; α is the weight of the voiceprint system: o is the quality threshold. In addition, set o to be the average score value obtained by the current user paired with N templates.2: output: which category the feature belongs to3: loop4: ifδti=minπibio1,i∈N5: ifhist=αist+τi6: return∑i=1Mps1ot=27: elsepx⟶λ=∑i=1Mwipxi⟶μi8: returnzt=∑i=1Nαiyit+τiStudents can easily use the online communication function provided by mobile terminals for discussion and interaction when learning English speaking around the location context. According to the participating objects, this communication includes teacher-student and student-student communication; according to the timeliness of communication, the communication can be divided into synchronous and asynchronous communication; according to the form of communication, it can be divided into text communication, voice communication, and video communication. Learners can ask questions online about the problems they encounter in learning spoken English, and they can also answer other people’s questions, making communication more convenient. In addition, learners can also expand the field of interaction through the social sharing function provided by the system. Through discussion and interaction, learners can enhance their internalization of speaking knowledge.The first layer is the response layer, which considers the system’s response and satisfaction to the learners; while the second layer is the learning layer assessment, which measures the users’ understanding and proficiency in English-speaking learning such as English knowledge, speaking skills, and attitudes. Given the minimum period of two years required for the assessment of the behavioral and outcome layers of the Koch model, the period is too long, and the students who participated in this experiment have already graduated. Therefore, this study assessed the effects of the positional context in terms of the first two layers of learner response and cognitive transfer. It is important to note that this implementation is not a true, rigorous experiment in the true sense of the word, but rather a design-based research approach to explore the effects of positional contexts to be closer to natural contexts and closer to reality.


## 3.1. Multimodal Intelligent Acoustic Sensor Sound Acquisition Design

Acoustic emission is a type of elastic wave, and in the real environment, it can be divided into transverse waves, longitudinal waves, and surface waves, depending on how the sound waves propagate in the medium and the direction of vibration. When propagating in solids, it causes local deformation of the medium, and two types of waves appear simultaneously: longitudinal (compressional) and shear (transverse) waves, which have different speeds and are automatically separated after leaving the source [16]. When the acoustic signal propagates to the junction of air and solid media, the acoustic wave is reflected and refracted, which causes it to undergo a waveform transition, followed by the simultaneous appearance of two waveforms. On free surfaces, acoustic emission waves also form surface waves, and plate waves are generated when the thickness of the solid medium is like the wavelength. After reflection and other transformations of the acoustic signal, the various forms of sound waves will be transmitted to the sensor according to their wave speed, time sequence, and wave range, so that the pulsed signals generated by the acoustic emission sources can be superimposed on each other to produce long and complex waveforms. The online learning environment provides course resources for learning management. Students carry out independent learning, teachers assist through inspiration and demonstration, and students learn cooperatively, communicate with each other, and share wisdom.When the device receives an acoustic signal, the alternating forces during the resonance of the sound on the piezoelectric film will cause deformation of the PVDF-TrFE piezoelectric film, which causes a piezoelectric effect and generates an electrical charge. To accurately measure the voltage signal generated on the PVDF-TrFE piezoelectric film due to the piezoelectric effect of the acoustic device in operation, we use function generators, ultrasonic transducers, loudspeakers, high sampling rate voltage data acquisition cards, electronic computers, and software data analysis to build a test platform for acoustic device performance characterization, and as shown in Figure1, the working principle of the experimental platform is as follows: use function generator to generate alternating voltage and frequency-controlled electrical signal, through the ultrasonic transducer electroacoustic conversion, the electrical signal into the required acoustic signal, or use the electronic computer connected to the speaker to play the required sound samples, the acoustic signal reached the PVDF-TrFE thin-film acoustic device, through the high sampling rate voltage data acquisition card for the device response voltage signal acquisition. The voltage signal is recorded, analyzed, and processed by an electronic computer and software.Figure 1
Simplified waveform diagram of the acoustic emission signal.An acoustic emission signal is a process in which the emission signal reaches its highest amplitude and then gradually decays. The ringing count reflects the number and frequency of acoustic emission events, and the amplitude is the maximum amplitude of the signal waveform, which is used to distinguish the type of wave source as well as to measure the strength of the wave. The duration is the time from when the acoustic signal first rises above the threshold limit to when it falls to the threshold. The rise time is the time when the acoustic emission signal first exceeds the threshold to reach its maximum amplitude. The mathematical expression can be expressed as a decaying sine function as shown in Equation (1).
(1)xt=A0expαtcosπf0t.A is the amplitude of the acoustic signal Pa; α is the attenuation factor of the signal; f0 is the resonant frequency of the acoustic emission sensor (Hz). When there is acoustic emission generation, the acoustic emission sensor located on the surface of the component or inside the component converts the acoustic signal into an electrical signal, amplifies it through a signal amplifier, and then after acquisition and data processing, analyzes the relevant characteristic parameters and calculates the specific location of the measured acoustic emission source by combining mathematical geometric formulae with data analysis.The MZI-based wavelength demodulation technique is based on the sensing principle of fiber optic interferometer, by converting the DFB fiber laser sensor wavelength drift into phase change after entering the nonequilibrium fiber interferometer, and using the high-resolution phase demodulation technique can finally get the sensor weak wavelength change [17]. It transmits the narrow-band laser to the MZI by exciting the 980 nm pump light source and 980/1550 nm fiber wavelength division multiplexer (WDM) guide and DFB fiber laser sensor output, and then convert its optical signal into an electrical signal output by photodetector; by the demodulation system for the analysis of data processing, the output electrical signal is proportional to the magnitude of the acoustic signal.In the actual environment, the speech signal observed by each microphone in the array will be disturbed by some environmental signals such as noise and reverberation, and there are two common microphone array signal models, the ideal model and the actual reverberation model. The ideal model considers an array ofN microphones, and the array structure is chosen to be either one of the structures, and let the source signal be st; then, the signal observed by the ith microphone yit can be expressed as follows:
(2)yit=αist+τi−nit,where αi is the attenuation coefficient of the source signal due to distance, τi is the time delay of the source signal to the ith microphone, nit represents the background noise at the ith microphone, and t is the time factor. Since the actual environment contains other disturbances such as reverberation and multipath effects, for an array of N microphones, let the source signal be st and the acoustic transfer function from the source to the ith microphone be hi; then, the signal observed by the ith microphone yit can be expressed as follows:
(3)yit=hist∗st−nit.For the ideal model, the acoustic transfer function can be equated as follows:(4)hist=αist+τi,where zt is the output beamforming result and αi is the weighting factor, generally taken as 1/N. This can be described by the following expression.
(5)zt=∑i=1Nαiyit+τi.Wireless sensor networks (WSNs) consist of multiple inexpensive miniature sensor nodes, and each node contains one or more sensors, which constitute a distributed network with less power consumption and lower cost through wireless collaborative information sharing, which are used in a wide range of applications such as environmental science, medical health, space exploration, remote environmental monitoring, and target tracking. Especially in recent years, the development of smaller, cheaper, and smart sensors has greatly enhanced the scope of wireless sensor network applications [18]. Sound wave is a mechanical wave, generated by the vibration of an object (sound source); the space in which the sound wave propagates is called the sound field. It is a longitudinal wave when propagating in gas and liquid media but may be mixed with transverse waves when propagating in solid media. These wireless sensors are equipped with wireless interfaces for communicating with each other to form a network. These sensors are smaller and are capable of only limited computation and processing of information or data at a low cost compared to conventional sensors. These sensor nodes can collect information by sensing and measuring in the environment and can transmit the sensed data to the user based on some local discriminant criterion. In this paper, we focus on speech processing; at this point, the sensors become microphone sensors, so this network is also called wireless acoustic sensor networks (WASNs).Acoustic sensors are mainly used for sensing and measuring environmental information; processors and memories perform limited processing and storage of data; power supplies are generally composed of batteries, which are the main energy for this sensor network and are limited in energy; wireless transmitter-receiver devices are used for information transmission between nodes, and actuators are mainly for local discrimination (see Figure2 for the specific structure).Figure 2
Wireless acoustic sensor network with 20 nodes.For unstructured WASNs, network maintenance management and detection of faults are more difficult due to many nodes. In contrast, for structured WASNs, some or all the sensor nodes are arranged in a way that is required for certain purposes. The advantage of structured networks is the ability to deploy fewer nodes to obtain smaller network maintenance and management costs. But fewer nodes mean that providing network coverage is significantly reduced.Unlike traditional networks, wireless acoustic sensor networks are subject to design and resource constraints. Resource constraints include limited energy per node, short communication range, low bandwidth, and limited computational and storage capacity. Design constraints are application-dependent and based on the environment being monitored. The environment plays a key role in the size of the network, the deployment scheme, and the network topology. The size of the network varies with the environment being monitored. For an indoor environment, a network can be formed with fewer nodes, while an outdoor environment may require more nodes to cover a larger area. Obstacles in the environment can also limit communication between nodes, which in turn can interfere with the connectivity of the network. Acoustic measurement software is a measurement software that runs on a smartphone or tablet PC. It is paired with an instrument to achieve accurate measurements and supports acquisition of measurement data from WIFI and analysis of calculations.


## 3.2. Experimental Design for English Pronunciation Phonetic Calibration

Due to the coarticulation phenomenon, phonetic pronunciations do not exist completely independently in time, and neighboring phonetic pronunciations partially overlap and influence each other [19]. These indicate the need to apply features that can capture long-time information in speech recognition. To address the above problem, we try to extend the spectral features in the time domain by combining the Mel subband energy of the current frame and the subband energies of multiple frames before and after it to obtain the time-domain extended features, which are called temporal pattern (TRAP) features. The extraction process of TRAP features is shown in Figure 3.Figure 3
TRAP feature extraction process.First, the speech signal is windowed in frames, and the energy value of the output of each frame through the Mayer filter bank is calculated. And with the current frame as the center point, the same number of frames is taken before and after, and these several frames’ features are concatenated to get a long-time feature. Then, the DCT method is used for dimensionality reduction, and the features are normalized on the mean and variance to finally obtain the TRAP features. TRAP reflects the long-time feature variation and effectively exploits the correlation between speech signals. Compared to short-time features such as MFCC, TRAP features effectively exploit the correlation between speech signal contexts, which not only improves the noise robustness of acoustic features, but it helps to solve the copronunciation phenomenon among phonetic features, which can improve the performance of speech recognition.Phoneme posterior probability features are commonly used feature representations in speech recognition with high discriminative and robust features. Define the phoneme model space to consist ofM classes (for an English corpus with 39 phonemes); then, M=39. Then, the phoneme posterior probability feature vector for a frame of speech at moment t is as follows:
(6)Pt=ps1ot,ps2ot,⋯,siot,⋯,sMot,where ot denotes the corresponding speech frame at that moment and si denotes the corresponding ith phoneme; then, ps1ot denotes the posterior probability that the corresponding phoneme is si for a known speech frame ot.
(7)∑i=1Mps1ot=2.Rhythmic features are a typical feature of natural human language, with many common features across languages. When people communicate with each other using language, it is not only the phonetic sounds of the words but also the suprasegmental information of the language, i.e., the rhythmic features, which play a very important role in the naturalness and intelligibility of the language. Rhythmic features respond to variations in pitch, length, and intensity in addition to phonetic features: variations in pitch form the tone of speech, variations in length from the length of speech, and variations in intensity from the stress. Speech rhythm features are widely used in the fields of speech recognition, speaker recognition, and language recognition because of the rich dynamic information they carry. No state can be transferred to any of the previous states:(8)aij=0,j≥i.A GMM model is a multidimensional probability density function model that utilizes a weighted sum of multiple Gaussian probability density functions to represent arbitrary probability distributions. For example, Equation (9) is a GMM model with M Gaussian components:
(9)px⟶λ=∑i=1Mwipxi⟶μi,(10)∑i=1Mwi=2.The ball in Equations (9) and (10) is a D-dimensional eigenvector, pxi⟶μi is the probability density function of one of the Gaussian components, μi denotes the mean vector of the Gaussian components, and x⟶ is the covariance matrix and is calculated in Equation (11).
(11)pxi⟶μi=12π∑ii2exp12x⟶−μiT∑ii2x⟶−μi.The observed probability distribution is what is characterized using the GMM model, and the superiority of the GMM as an observed probability model is that the continuous distribution model provides a more accurate description of the probability distribution compared to the discrete or semicontinuous model. This is because the characteristics of the inputs in the continuous distribution model do not need to be quantified. In addition, GMM models can fit arbitrary distributions infinitely accurately by increasing or decreasing the number of mixing components. Thus, the CMU-SPHINX phoneme recognition mechanism sees distribution of a single Gaussian distribution when the state is not bound at the beginning of parameter training. The state binding complicates the model parameters by Gaussian splitting, which in turn leads to more accurate parameter estimates, as shown in Table1.Table 1
Steps of Viterbi algorithm.
(1) Initializationδ1=πibio1,i∈Nψii=1,i∈N(2) RecursionFort=2,3,⋯,Tψiti=argminπibio1,i∈Nδti=minπibio1,i∈N(3) TerminationP∗=maxψitiS∗=argmaxψiti(4) Optimal path backtrackingFort=T−1,T−2,⋯,0st=ψt+1si(5) Find the optimal pathS∗S∗=s1,s2,⋯,sNAccording to the dynamic programming principle, the optimal path has the property that if the optimal path passes through noderi at time ti, then the partial path of this path from node ri to endpoint pi must be optimal for all possible partial paths from ti to ri. For if this is not the case, then another better partial path exists from ti to ri. If it is connected to the partial path from ti to pi, it will form a path that is better than the original path, which is contradictory. According to this principle, we only need to compute recursively the maximum probability of each partial path with state i at moment t=1 until we get the maximum probability of each path with state i at moment t=T. The maximum probability at moment t=T is the probability of the optimal path, and the endpoint pi of the optimal path is obtained at the same time. After that, to find the individual nodes of the optimal path, starting from the endpoint ri, the nodes are gradually solved from back to front iT−1,iT−2,⋯,iT, and the optimal path is obtained I=iT−1,iT−2,⋯,iT.The preparation of data has always been a prerequisite and key to system experiments. Especially in systems using fully connected neural networks as the underlying model, when the number of input and hidden layer neurons is relatively large and the number of model parameters is huge, the need for a large amount of training data is more urgent. Only by collecting enough training data can the acoustic model be trained more adequately and accurately. In addition, the quality of the data preparation is also essential for the training of good acoustic models. When the quality of the prepared data is poor, it will lead to the existence of outliers, which will affect the accuracy of the model training. Resource limitation is the speed of program execution limited by computer hardware resources or software resources during concurrent programming. In concurrent programming, the principle of making code execution faster is to turn the serial part into concurrent execution. A tree network can contain branches, and each branch can contain multiple nodes. The tree topology is an expanded form of the bus topology, and the transmission medium is an unclosed branch cable. The tree topology is the same as the bus topology, where one station sends data and all other stations can receive it.The meaning of tree topology network is the network left by removing the edges that form a circle (ring) in the network, like the spanning tree algorithm [20]. This network has a minimum connection structure and most efficient information transfer rate and is suitable for information transfer between nodes. This topological network uses the knowledge of tree to divide the adjacent nodes into parent and child nodes so that information transfer can be divided into two processes: convergence and dispersion. Convergence starts from the leaf nodes, and the child nodes converge their information to the parent nodes to the root node; dispersion is exactly the opposite, starting from the root node and the parent node disperse their information to the child nodes, all the way to the leaf nodes, to achieve all nodes can get consistent fusion results, and due to the use of spanning-tree structure, in each node, just convergence or dispersion process can greatly reduce the duplication of information transmission, improve the efficiency of information transmission, and reduce the energy overhead of the sensor network, as shown in Algorithm 1.Algorithm 1: Algorithm pseudocode.
1: Input: The matching scores of the two classifiers are respectively denoted as X=(X, X), X=Px, |R ), X,=Py, R), i=1,2.... N, j =1,..,k; N represents a total of N categories, and k represents a total of k samples.α is the weight of the fingerprint identification system; α is the weight of the voiceprint system: o is the quality threshold. In addition, set o to be the average score value obtained by the current user paired with N templates.2: output: which category the feature belongs to3: loop4: ifδti=minπibio1,i∈N5: ifhist=αist+τi6: return∑i=1Mps1ot=27: elsepx⟶λ=∑i=1Mwipxi⟶μi8: returnzt=∑i=1Nαiyit+τiStudents can easily use the online communication function provided by mobile terminals for discussion and interaction when learning English speaking around the location context. According to the participating objects, this communication includes teacher-student and student-student communication; according to the timeliness of communication, the communication can be divided into synchronous and asynchronous communication; according to the form of communication, it can be divided into text communication, voice communication, and video communication. Learners can ask questions online about the problems they encounter in learning spoken English, and they can also answer other people’s questions, making communication more convenient. In addition, learners can also expand the field of interaction through the social sharing function provided by the system. Through discussion and interaction, learners can enhance their internalization of speaking knowledge.The first layer is the response layer, which considers the system’s response and satisfaction to the learners; while the second layer is the learning layer assessment, which measures the users’ understanding and proficiency in English-speaking learning such as English knowledge, speaking skills, and attitudes. Given the minimum period of two years required for the assessment of the behavioral and outcome layers of the Koch model, the period is too long, and the students who participated in this experiment have already graduated. Therefore, this study assessed the effects of the positional context in terms of the first two layers of learner response and cognitive transfer. It is important to note that this implementation is not a true, rigorous experiment in the true sense of the word, but rather a design-based research approach to explore the effects of positional contexts to be closer to natural contexts and closer to reality.


## 4. Analysis of Results

### 4.1. Performance Results of Multimodal Smart Acoustic Sensors

From Figure4, the 20 nodes of the simulated wireless acoustic sensor network are relatively evenly distributed, and there is no node overlap, and cover the whole 10m∗10m plane area. The whole network edge connection is more reasonable, satisfying the experimental hypothesis, and closer to the real environment. It can also be found that no matter for noise-containing sources or clean noise-free sources, it is clearer to distinguish the speech-free segments from speech-containing segments, which is more conducive to estimating the noise power spectrum by VAD method and paving the way for the subsequent speech enhancement by using beamforming algorithm.Figure 4
Nodal signal-to-noise ratio.After transmission noise addition, the signal received by each node contains significant noise and has a large difference in signal-to-noise ratio, some are still higher, such as 12.5 dB for node 5, and some are lower, such as -1.18 dB for node 18. At this point, speech enhancement must be performed to obtain a better output for the whole node. So, after that, these two special cases are selected for experimental observation to confirm the effectiveness of the algorithm in this paper. Neural networks have a wide and attractive prospect in the fields of system identification, pattern recognition, and intelligent control. Especially in intelligent control, people are particularly interested in the self-learning function of neural networks and regard this important feature of neural networks as one of the keys to solve the difficult problem of controller adaptability in automatic control.Due to the unique directivity characteristics of the DFB fiber laser sensor, the acoustic emission sensor was pressed onto the pigtail of the DFB fiber laser sensor, allowing for a complete acoustic emission signal with maximum responsiveness in the axial direction, and a 100 mW pump laser output was loaded on the DFB fiber laser sensor, allowing the DFB fiber laser sensor to reflect a sufficiently large light intensity. The bandwidth of the demodulation system was tested using a 2 MHz wide band acoustic emission sensor transmitting acoustic wave signals from 20 kHz to 2 MHz as shown in Figure5. The results of the bandwidth test of the fiber demodulation system are shown in Figure 5. The signals of the acoustic emission transducer operating at 20 kHz, 40 kHz, 100 kHz, 1000 kHz, 1500 kHz, and 2000 kHz were tested separately. It was able to obtain that the DFB demodulation system can stably demodulate acoustic waves from 20 kHz to 2 MHz frequency, and the small resonance does not have a large impact on the main frequency. Also, when the sampling rate is F and the number of sampling points is set to N, the frequency resolution can be derived from the fast Fourier change equation.Figure 5
Bandwidth test results of fiber laser sensor demodulation system.The delay caused by the system is unavoidable and will have a large impact on the accuracy during the positioning process, so the system delay needs to be calibrated. A self-transmitting signal generator was used to transmit a signal with a center frequency of 100 kHz from the acoustic emission sensor for the experiment. The fiber laser sensor is placed on a 2 cm thick Plexiglas plate, impedance matching is performed using an ultrasonic coupling agent, and the acoustic emission sensor is placed on the back of the Plexiglas plate to emit a 1 kHz pulse signal.Sixteen acoustic emission signal points at fixed locations were acquired and demodulated and localized by the hyperbolic localization method, with 10 measurements per point and averaged as the result. The minimum error of the 16 acoustic emission sources of the hyperbolic localization algorithm is 0.35 cm, and the maximum error is 0.98 cm, in which the average error of the horizontal axis is 0.39 cm and the average error of the vertical axis is 0.45 cm, and the summation of the errors of these 16 sources is averaged to obtain the overall localization error of 0.67 cm. The model constructs the initial load of a node from the global and local perspectives by combining the node mesonumber, node degree, node weight, and neighbor node weight and establishes the proportional relationship between the node capacity and the initial load. When a node fails, the load redistribution rules are formulated by combining the capacity of the neighbors of the failed node, and then, the evolutionary process of the load parameters is deduced through the analysis of the network cascade failure, resulting in the parameters in the model.


### 4.2. English Pronunciation Phonetic Calibration Results

It can be seen from Figure6 above that the loss function of the training process of the neural net will occur a steep drop in the initial stage of training, after which it will gradually smooth out. However, the change of the loss function in this experiment is still drastic near the end of the training, indicating that the learning rate is set too large in the end stage, causing the loss function to oscillate too much and make it difficult to converge. Since the learning rate of the neural network is updated in epochs, Figure 6 shows the change of the loss function at the end of each epoch. It is seen through Figure 6 that the number of iterations in this experiment is low, and there is still a large reduction in the loss function at the end of training. Therefore, an attempt was made to change the training end condition to make the model more fully trained and to increase the initial learning rate to prevent a local optimum.Figure 6
Variation of loss function.It is seen that the number of iterations becomes more in this experiment, and the value of the final loss function is smaller than that of the last experiment, indicating that the training is more adequate this time. It can also be seen that the recognition rate of the system has been improved after the network parameters have been adjusted.To test whether the model has high variance or high bias and whether collecting more data would help solve this problem, we use learning curve in sci-kit-learn, which is used to return test scores for different size datasets, and use matplotlib’s plot with the “fill_between” function to add the standard deviation of the MSE to the plot to obtain the learning curve of the model as shown in Figure7. The scores obtained by using 10-fold cross-validation are calculated. For the mean and standard deviation of MSE, the fold line represents the corresponding mean MSE at the size of this dataset, and the band around the fold line indicates the fluctuation range of MSE, where blue represents the training set and the green represents the test set. It can be found that the MSE of the model is stable around 0.0026, compared to the slightly larger standard deviation of MSE in the test set, but the fluctuation range is between 0.0022 and 0.0031 with less variation in the values.Figure 7
Phonetic calibration change curve.It can be found that the learning effect in the elastic network has been greatly improved. The linear regression results are more unstable, and the reasons for this result are that there is still covariance in the parameters of the model or the model is overfitted, which makes the model ineffective, but these problems are well circumvented in the elastic network. In the learning layer, students’ attitude towards learning spoken English through M-Oral was more positive and they imitated the pronunciation of authentic spoken English more during the interaction. The observation shows that compared to the control group, the experimental group is more motivated to learn spoken English due to the simulation of contextual perception.


## 4.1. Performance Results of Multimodal Smart Acoustic Sensors

From Figure4, the 20 nodes of the simulated wireless acoustic sensor network are relatively evenly distributed, and there is no node overlap, and cover the whole 10m∗10m plane area. The whole network edge connection is more reasonable, satisfying the experimental hypothesis, and closer to the real environment. It can also be found that no matter for noise-containing sources or clean noise-free sources, it is clearer to distinguish the speech-free segments from speech-containing segments, which is more conducive to estimating the noise power spectrum by VAD method and paving the way for the subsequent speech enhancement by using beamforming algorithm.Figure 4
Nodal signal-to-noise ratio.After transmission noise addition, the signal received by each node contains significant noise and has a large difference in signal-to-noise ratio, some are still higher, such as 12.5 dB for node 5, and some are lower, such as -1.18 dB for node 18. At this point, speech enhancement must be performed to obtain a better output for the whole node. So, after that, these two special cases are selected for experimental observation to confirm the effectiveness of the algorithm in this paper. Neural networks have a wide and attractive prospect in the fields of system identification, pattern recognition, and intelligent control. Especially in intelligent control, people are particularly interested in the self-learning function of neural networks and regard this important feature of neural networks as one of the keys to solve the difficult problem of controller adaptability in automatic control.Due to the unique directivity characteristics of the DFB fiber laser sensor, the acoustic emission sensor was pressed onto the pigtail of the DFB fiber laser sensor, allowing for a complete acoustic emission signal with maximum responsiveness in the axial direction, and a 100 mW pump laser output was loaded on the DFB fiber laser sensor, allowing the DFB fiber laser sensor to reflect a sufficiently large light intensity. The bandwidth of the demodulation system was tested using a 2 MHz wide band acoustic emission sensor transmitting acoustic wave signals from 20 kHz to 2 MHz as shown in Figure5. The results of the bandwidth test of the fiber demodulation system are shown in Figure 5. The signals of the acoustic emission transducer operating at 20 kHz, 40 kHz, 100 kHz, 1000 kHz, 1500 kHz, and 2000 kHz were tested separately. It was able to obtain that the DFB demodulation system can stably demodulate acoustic waves from 20 kHz to 2 MHz frequency, and the small resonance does not have a large impact on the main frequency. Also, when the sampling rate is F and the number of sampling points is set to N, the frequency resolution can be derived from the fast Fourier change equation.Figure 5
Bandwidth test results of fiber laser sensor demodulation system.The delay caused by the system is unavoidable and will have a large impact on the accuracy during the positioning process, so the system delay needs to be calibrated. A self-transmitting signal generator was used to transmit a signal with a center frequency of 100 kHz from the acoustic emission sensor for the experiment. The fiber laser sensor is placed on a 2 cm thick Plexiglas plate, impedance matching is performed using an ultrasonic coupling agent, and the acoustic emission sensor is placed on the back of the Plexiglas plate to emit a 1 kHz pulse signal.Sixteen acoustic emission signal points at fixed locations were acquired and demodulated and localized by the hyperbolic localization method, with 10 measurements per point and averaged as the result. The minimum error of the 16 acoustic emission sources of the hyperbolic localization algorithm is 0.35 cm, and the maximum error is 0.98 cm, in which the average error of the horizontal axis is 0.39 cm and the average error of the vertical axis is 0.45 cm, and the summation of the errors of these 16 sources is averaged to obtain the overall localization error of 0.67 cm. The model constructs the initial load of a node from the global and local perspectives by combining the node mesonumber, node degree, node weight, and neighbor node weight and establishes the proportional relationship between the node capacity and the initial load. When a node fails, the load redistribution rules are formulated by combining the capacity of the neighbors of the failed node, and then, the evolutionary process of the load parameters is deduced through the analysis of the network cascade failure, resulting in the parameters in the model.


## 4.2. English Pronunciation Phonetic Calibration Results

It can be seen from Figure6 above that the loss function of the training process of the neural net will occur a steep drop in the initial stage of training, after which it will gradually smooth out. However, the change of the loss function in this experiment is still drastic near the end of the training, indicating that the learning rate is set too large in the end stage, causing the loss function to oscillate too much and make it difficult to converge. Since the learning rate of the neural network is updated in epochs, Figure 6 shows the change of the loss function at the end of each epoch. It is seen through Figure 6 that the number of iterations in this experiment is low, and there is still a large reduction in the loss function at the end of training. Therefore, an attempt was made to change the training end condition to make the model more fully trained and to increase the initial learning rate to prevent a local optimum.Figure 6
Variation of loss function.It is seen that the number of iterations becomes more in this experiment, and the value of the final loss function is smaller than that of the last experiment, indicating that the training is more adequate this time. It can also be seen that the recognition rate of the system has been improved after the network parameters have been adjusted.To test whether the model has high variance or high bias and whether collecting more data would help solve this problem, we use learning curve in sci-kit-learn, which is used to return test scores for different size datasets, and use matplotlib’s plot with the “fill_between” function to add the standard deviation of the MSE to the plot to obtain the learning curve of the model as shown in Figure7. The scores obtained by using 10-fold cross-validation are calculated. For the mean and standard deviation of MSE, the fold line represents the corresponding mean MSE at the size of this dataset, and the band around the fold line indicates the fluctuation range of MSE, where blue represents the training set and the green represents the test set. It can be found that the MSE of the model is stable around 0.0026, compared to the slightly larger standard deviation of MSE in the test set, but the fluctuation range is between 0.0022 and 0.0031 with less variation in the values.Figure 7
Phonetic calibration change curve.It can be found that the learning effect in the elastic network has been greatly improved. The linear regression results are more unstable, and the reasons for this result are that there is still covariance in the parameters of the model or the model is overfitted, which makes the model ineffective, but these problems are well circumvented in the elastic network. In the learning layer, students’ attitude towards learning spoken English through M-Oral was more positive and they imitated the pronunciation of authentic spoken English more during the interaction. The observation shows that compared to the control group, the experimental group is more motivated to learn spoken English due to the simulation of contextual perception.


## 5. Conclusion

Around the feature extension perspective, vocal recognition technology is introduced and a Bayesian decision-based dynamic weight bimodal fusion algorithm is proposed to alleviate the unimodal biometric feature acquisition; recognition generic drawbacks in addition to making the recognition rate of the small area improved and improving the performance of the algorithm. The algorithm determines the classification weights of the two recognition modalities in that environment by adaptive weights and finally fuses them at the decision level based on the weights. The theory of the algorithm is based on Bayesian decision-making which minimizes the cost of classification errors and improves the inflexibility problem caused by traditional fixed-weight fusion. Distributed consistency-based and distributed speech enhancement algorithm: this algorithm decomposes the traditional array algorithm to each wireless acoustic sensor network node, transforms distributed speech enhancement into a distributed consistency problem, and then achieves consistency averaging based on a distributed consistency iterative algorithm. In the process of consistency averaging, an average Metropolis right is proposed in this paper, and the experimental results show that the efficiency of iteration based on this right can be second only to the consistency iteration algorithm based on the optimal constant right and outperforms the consistency iteration algorithm based on the maximum degree right and Metropolis right. The speech enhancement effect of this algorithm can achieve the effect of the algorithm containing the fusion center, and the signal-to-noise ratio of the enhanced speech of each node can approximate the source, which can offset the signal energy attenuation caused by the distance. For English speech sound data, the changes of the loss function during the training of the neural network of the LC-RC system are tracked, and the learning rate and iterations of the benchmark system are optimized as necessary according to their changes, and certain results are achieved.



---

*Source: 3383685-2022-03-02.xml*