# Optimization of Vocal Singing Training Methods Based on Multimedia Data Analysis

Authors: Dajun Zhang; Xiaoning Wang
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

This paper constructs a vocal singing training optimization model based on multimedia data analysis and conducts in-depth research and analysis on the optimization of vocal singing training methods using multimedia data analysis system. For the problems of poor real-time and large analysis error of the traditional system, a massive real-time multimedia data analysis system based on DSP is designed, with high-speed DSP as the core architecture, multipoint modeling of the experimenter, and data extraction and fusion processing are completed with model association characteristics as the analysis basis. The experimental results show that the real-time performance and accuracy of the system are improved by 15.14% and 13.42%, respectively. To address the time-space domain consistency problem in traditional time series data forecasting, two high-order time series forecasting algorithms based on tensor decomposition are proposed in this paper: the multilinear orthogonal autoregressive model and the multilinear constrained autoregressive model. Using the local structure information of the data to reduce the dimensionality of the data is more beneficial to extract the true low-dimensional representation of the data. Most current supervised cross-modal retrieval methods use a strict binary label matrix (i.e., 0 and 1); however, the interval between labels 0 and 1 is small, which may increase the risk of classification errors. To solve some of the above existing problems, a two-stage hashing method based on smooth matrix decomposition and label relaxation is proposed, which proposes a novel label relaxation strategy that can adaptively control the interval between label intervals and can reduce the quantization loss by about 5.21%. The multimedia data analysis vocal singing training optimization model designed in this paper can improve enhance students’ motivation and accuracy and provide technical support and a rational basis for vocal singing practice and overall musical expression.


---

## Body

## 1. Introduction

In recent years, with the rapid popularity and development of smart devices and online media platforms, multimedia data has not only exploded in terms of data volume but has also become diversified and complex in terms of data sources and types. Multimedia data prediction has become an important research topic in the field of multimedia content analysis due to its broad research value and application prospect [1]. With the advent of the AI 2.0 era, the research on multimedia data prediction has gradually shifted from single-media data analysis in the early stage to cross-media intelligent analysis driven by demand. In addition to the rapid growth of multimedia data, the data structure has also become more complex and the dimensionality has been increasing. In the face of such huge and complex multimedia data, how to manage and analyze it effectively has become an issue of close attention in both industrial technology and academic circles [2]. Since these multimodal data are characterized by avian heterogeneity, rich sources, and large quantities, the traditional structured data management methods are no longer suitable for the processing of these multimedia data. The rapid development of Internet technology makes multimedia data become one of the “biggest data,” which has the characteristics of large quantity, high complexity, diversity, high dimensionality, and multimodality, and puts forward higher requirements for efficient retrieval. Hashing technology encodes data into a set of binary hash codes, and the compact binary representation allows efficient pairwise comparison in Hamming space, which has great advantages in efficient similarity retrieval of large-scale data. In the era of multimedia big data, learning a more effective semantic feature representation of multimedia data and achieving efficient retrieval using hashing technology has become an urgent problem to solve [3]. Teachers can use the “Sound Spectrum Waveform Analysis” function of multimedia to analyze students’ pronunciation and generate atlases so that both teachers and students can have a better grasp of students’ vocalization and make more targeted and purposeful vocalizations train. Multimedia data analysis is a hot research topic in recent years, and the high-dimensional characteristics of multimedia data bring great challenges to the research, for which researchers have proposed many methods for data dimensionality reduction, among which the method based on stream learning is one of the effective ways to reduce data dimensionality.The so-called “song singing” is an aural art with aesthetic nature, and song singing is accompanied by unique attraction and infectious power, and this artistic infectious power of singing is derived from the singing skills of the singer and the expression of the emotion of the song. With the existence of this artistic expression of song singing, the problem of singing pitch also arises. There are various reasons for the deviation of song singing pitch; some of them are the effects of congenital defects of the auditory organs or neural tissues, and some are the reasons for singing skills or psychological factors [4]. In our daily life, we can always hear people ask: “Can a person who is not in tune learn to sing? “Teacher, can I sing out of tune and still be saved?” We always hear people ask: “Can I learn to sing if I am out of tune?” There are only a few people who are born with incompatibility or congenital defects. Most vocalists or learners are not so much incompetent but lack a “sense of tone.” In plain language, this means that the subconscious brain does not have the concept of “pitch position” and is unable to control the position of the tones concerning each other [5]. There are many factors involved in evaluating a singer’s singing: the physiological condition of the voice, the musical sense, the singing technique, and the overall stage performance. Regardless of the singing style, the singer must use the voice as the basis and carrier to complete the complete artistic expression. The quality of the singing voice plays a decisive role in the evaluation of singing, and the complete form of the singing voice is composed of static physiological conditions of voice and dynamic singing skills. The training set size is set to vary from 10k to 180k.In traditional vocal music teaching, the main teaching tool is the piano, and the teacher plays and models singing to teach, but the vocal classroom is different from other subjects, the vocal organs of vocal music are the vocal folds, and the vocal folds are the source of articulation. The student cannot see the state of his or her vocal organs when singing and cannot control his or her voice freely. Therefore, vocal music teaching is somewhat abstract, but it is very difficult to accomplish with the teacher’s modeling and guidance, which makes vocal music the most difficult art subject to teach and learn [6]. In addition to vocal exercises, students in the actual vocal music class should also understand the music of the sung work, the background of the work, analyze the lyrics, melody, tune, tonality, piano accompaniment, and so on, and explore how to sing vocal music so that they can better and more accurately grasp the style of the work, give a better interpretation of the work, and truly reflect the true meaning of the work. In traditional teaching, it is difficult to make a breakthrough based on the teacher’s knowledge and experience and the students’ sense and understanding. Therefore, both the teaching and learning sides must spend great effort to complete the learning task, how to optimize the effectiveness of the teaching classroom is particularly important, today’s advanced multimedia technology provides us with a convenient channel, that is, with the help of multimedia technology to assist in vocal music teaching [7]. With the development of computer technology and the enhancement of digital audio equipment, computers have been widely used in music composition, recording, and education. Artificial intelligence has developed rapidly in recent years, and major universities have opened artificial intelligence majors one after another. The promotion of music is closely related to the development of science and technology. Without the invention of tools and the progress of the production process, a wide variety of musical instruments cannot be produced and improved, and without the development of computer technology, electronic music will not appear. In this paper, we use multimedia data analysis methods to study the optimization of vocal singing training methods, hoping to provide reliable methods and a basis for music teaching.


## 2. Related Works

To overcome the problems caused by the high-dimensional characteristics of multimedia data, many feature extraction algorithms have been proposed, among which, Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two very classical linear feature extraction algorithms [8]. The principal component analysis is to calculate a set of projection vectors by using the covariance matrix of the data without labeling the data and project the original data onto these projection vectors to obtain the features of the data in the low-dimensional space, to complete the dimensionality reduction of the original data, and make the features in the low-dimensional space keep the useful information of the original data as much as possible, which can remove the redundant information in the original data. Gałka et al. used PCA for scene change detection and motion classification, etc. Linear discriminant analysis is different from principal component analysis in that it is a supervised linear dimensionality reduction algorithm [9]. The linear discriminant analysis uses the sample data and its label information to find such a set of projection vectors so that the projected low-dimensional features are as close as possible to the data of the same category and as far as possible from the data of different categories so that the learned low-dimensional features can be recognized with a better correct rate. When the data satisfy Gaussian distribution, using simple linear dimensionality reduction algorithms (e.g., PCA, LDA, etc.) can achieve more satisfactory results, but in practice, multimedia data generally do not satisfy Gaussian distribution, when using linear dimensionality reduction algorithms will make the results unsatisfactory, so many kernel-based and streamlet-based feature extraction algorithms have been proposed [10]. Usually, the minimum frequency is set to 80 and the maximum frequency is set to 1200. Some scholars introduce kernel functions based on linear feature extraction algorithms and use kernel functions to nonlinearly map the original data to a high-dimensional space and then use PCA, LDA, and so on in the high-dimensional space to reduce the dimensionality of the data, which is kernel PCA and kernel LDA based on the kernel, but this method is greatly influenced by the kernel function, and if the kernel function is not selected properly, it may even cause performance degradation, so it has relatively large limitations.Artificial Intelligence (AI) is a technical science used to study, simulate, and extend human intelligence. It is within the scope of computer science. Since the 21st century, AI, nanoscience, and genetic engineering have been called the world’s three most advanced technologies, and the rapid development of AI technology has had an impact on various areas of our lives. In recent years, several companies such as Ali, Baidu, Huawei, Tencent, and Xiaomi have started to get involved in the AI field; for example, Xiaomi AI audio “Xiaoyi” can already communicate with customers by voice and complete a variety of preset skills. Sun and Ifeachor believe that the combination of AI technology and musical instruments can combine software and hardware, making the complex playing process easier and more interesting, while the emergence of online education forms makes the current music education more convenient [11]. Usman et al. discussed that some “weak artificial intelligence” can more efficiently do the work that needs to be repeated by the human brain “mechanically” [12]. At the same time, AI can learn by itself, identify relevant features in the rich and diverse Chinese music, and store huge amounts of data. This can give us more references when we encounter music problems that are not easy to conclude. Jiang et al. used techniques, such as spectral analysis, to find corresponding acoustic evaluation parameters for different forms of the singing voice and explored the value of application in the singing evaluation and vocal teaching [13]. Zhao and other scholars also studied the relevant parameters affecting the evaluation of singing vocal quality in terms of the fundamental frequency, standardized noise energy, and sound intensity [14].In the traditional teaching of sight-singing and ear training, students of vocal singing tend to have the habit of ignoring dictation at the expense of sight-singing and using a single material. As a result, vocal singing students have a blurred understanding of the basic elements of music, such as pitch, rhythm, modulation, and harmony. In the process of sight-singing and ear training, in addition to the basic exercises, score-reading exercises, listening and recognition training, and the practice of various sight-singing materials, the use of actual vocal works is also a very important element [7]. Compared to traditional sight-singing training methods, the incorporation of poetry gives them a richer meaning, thus making them unique individuals and adding more interest and variety to a relatively single technical exercise. In addition, art songs also present a wide variety of styles and compositional techniques depending on the era in which they are written, including not only skillful pieces but also musical ones, making it easier for us to explore and apply them. With the reform and development of music education in recent years, the content and teaching methods of sight-singing and ear training as a basic music subject have been gradually improved. In addition to strengthening students’ basic musical knowledge and basic reading and writing skills, the emphasis of sight-singing and ear training should also be on strengthening students’ musical literacy, their ability to feel the music, their ability to express themselves, and their ability to appreciate music.


## 3. Optimization Model Construction of Vocal Singing Training Based on Multimedia Data Analysis

### 3.1. Multimedia Data Analysis Design

Multimedia data analysis has always been a popular research area, and multimedia data is usually high dimensional, which brings great challenges to the research. How to reduce the dimensionality of multimedia data and extract effective low-dimensional features from it has become a difficult research problem. Fortunately, through observation and research, it is found that many high-dimensional data can be represented by a small number of low-dimensional features, and the data are essentially distributed on top of some low-dimensional manifolds. To effectively reveal the underlying essential structure of the data, it is necessary to find the low-dimensional features hidden in the high-dimensional space [15]. The discrete Karhunen–Loève (K-L) transform provides the theoretical support for the principal component analysis algorithm, which uses the covariance matrix to orthogonally transform the data to remove the correlation of the data. The mathematical idea of principal component analysis has two equivalent interpretations: minimizing reconstruction error and maximizing variance. Minimizing reconstruction error requires that the error between the original data and the reconstructed data be minimized. Maximizing the variance requires that the variance of the projected low-dimensional data is maximized. First, a lot of time frames are obtained by adding windows to frames, and then a fast Fourier transform (FFT) is performed on each frame to obtain a time domain signal. Finally, each frame frequency domain signal after the FFT is stacked in time. A spectrogram (linear spectrum) containing frequency domain information can be obtained, and its intuitive understanding can visualize the surface.Let the high-dimensional data beX=x1,x2,...xM∈Rn×M, that is, where ∑i=1xi=0n denotes the dimensionality of the original data and M denotes the total number of training samples. The expected projection matrix is W=w1,w2,...,wd∈Rn×d, where d denotes the number of projection vectors. Assume that the low-dimensional representation of the data Xii=1,2,...,n isfii=1,2,...,nfi=wTxi such that F=fi,f2,...,fn∈Rd×n is the low-dimensional representation of the data set X=x1,x2,xM∈Rn×M; then F=wTxi.(1)Wopt=argmaxwTx2,∑i,j=1fi+fj2=Si,j.The principal component analysis is an unsupervised feature extraction algorithm that can replace the original data with a small number of features, and each principal component is orthogonal to each other to remove the correlation between the original data, and the calculation is relatively simple and easy to implement in computer coding. However, there are some disadvantages of principal component analysis. In some cases, the principal components with a small contribution may contain more discrepant information, and there is no good guideline to limit the number of principal components [16]. The local hold projection algorithm is a linear approximation of the Laplace feature mapping algorithm. The local hold projection constructs the neighborhood structure between data pairs from the original high-dimensional data while maintaining this neighborhood structure in the reduced dimensional low-dimensional space.Tensor, also known as a multidimensional array, is a very effective tool for mathematical analysis and is widely used in mechanics and physics. In a broad sense, a tensor is a generalization of the traditional vector concept. In the sense of isomorphism, a 0th-order tensor is represented as a scalar, a 1st-order tensor as a vector, and a 2nd-order tensor as a matrix. In today’s era, the tensor theory has been popularized and applied in many fields. And in the field of multimedia analysis, tensor decomposition is widely used in image retrieval, personalized recommendation, entity mining, and so on. A slice is a representation of a two-dimensional section of a tensor. Taking a third-order tensor as an example, a slice of a tensor is a tensor representation in which all dimensions are fixed except for two dimensions whose indices are not fixed. A third-order tensorx has three slices i, j, and k along the horizontal, horizontal, and forward directions, denoted by the symbols x1, xj, and xk, respectively. Like tensor slices, a tensor fiber is a tensor representation with all dimensions fixed except for one dimension that is not fixed. The tensor x is expanded along the n− module to obtain a matrix Xn∈RI1×I2×...×In−1×In×In+1. In this matrix, all column vectors are obtained by arranging the fibers of the n− module of the original tensor x in a specific order, as shown in Figure 1.Figure 1
Schematic diagram of the third-order tensor expansion operation.Hashing methods can transform the original data into a series of fixed-length binary codes (hash codes), while the similarity information of the original data can be well maintained. Thanks to the binary representation, so retrieval can be performed quickly by efficient operations between hash codes, and the storage cost can be greatly reduced, so hashing methods have gained a lot of attention. Early hashing methods mainly focus on similarity retrieval tasks in single-modal data scenarios (i.e., unimodal hashing methods) [17]. Unimodal hashing methods can be classified according to the way the hash function is generated: data-independent methods and data-dependent methods. In data-independent methods, the hash function is usually pregiven by hand or directly obtained by random generation. Typical methods include Locality-Sensitive Hashing (LSH), which provides a series of random hash functions so that similar samples have a high probability of collision in Hamming space, while dissimilar samples have as little chance of a collision as possible. The black points are the ideal target positions for mapping, the points in the red area are like the ideal points, and the points in the blue area are the dissimilar sample points.(2)Pt∈Ret×rc,Zi,j=1Zi,j<1Zi,jZi,j=10Zi,j>1.The approach proposed in this chapter is a two-stage model. Specifically, firstly, the hash representation of the binary is learned in the first stage. The second requires learning the hash function in the next stage. The work proposed in this chapter learns the hash codes shared by different modalities in the first stage; however, for cross-modal retrieval, data is obtained from one modality and it is desired to retrieve the content of another modality related to its semantics, so for this task, the hash function for each modality needs to be learned to generate its corresponding hash code. The role of the hash function is to convert the data in the original space into a binary hash code in Hamming space. Many effective classification models can be used to learn the hash functions in this stage, such as support vector machines, deep neural networks, and linear regression. Complex moduli usually lead to better pick-up results. However, synthetically considering the computational complexity and retrieval accuracy, this chapter chooses the linear regression model of the cylinder alone to learn the hash function, as shown in Figure2. The black points are the ideal target positions for mapping, the points in the red area are like the ideal points, and the points in the blue area are the dissimilar sample points.Figure 2
Schematic diagram of the mapping operation.Some cross-modal retrieval methods learn hash functions and obtain good results. However, the above schemes may limit the capability of hash functions, for example, when mapping the original data from its original space to Hamming space, which may lead to bias and thus affect the retrieval performance [18]. It is assumed that when mapping a sample r to another space, its ideal position is indicated by black dots, and the other three red dots are the points where bias may occur. Points of the same shape indicate similarity, otherwise not similar. From Figure 2, it can be seen that x may map to one of the three possible candidate points. However, it can also be observed from the above figure that x deviated data point 1 is like the ideal and labeled point; in other words, point 1 is more suitable for retrieval than points 2 and 3 when the projection is shifted from a retrieval perspective. Therefore, the search performance can also be improved if the offset point is found to be favorable for retrieval when an offset occurs. In other words, if supervised semantic knowledge is considered when learning linear mappings, it can make its retrieval performance improved.


### 3.2. Vocal Singing Training Optimization Model Construction

Digital Signal Processor (DSP) is a dedicated digital processing chip, generally using the classical Harvard structure to form a dedicated processing chip. As DSP is dedicated to processing digital signals, it has the advantages of accurate data processing, good reliability, and easy large-scale integration. In recent years, with the improvement of the DSP manufacturing process, its application scope spreads across the field of data statistics and information technology; DSP minimum system design includes two aspects of dedicated hardware design and supporting software design [19]. The hardware design part of the main work is to select the appropriate DSP chip, based on the chip as the core, auxiliary to other peripheral equipment chip, data acquisition chip, system power supply circuit, system chip reset circuit, and set up in the DSP access memory external expansion memory circuit, chip work crystal circuit, simulation connection circuit, and so on, to build the DSP minimum system. The classical TMS320C5502 is selected as the core processor to build the DSP minimum system schematic. The system, due to the need to design a low-power microintegrated system, so the system’s power supply design and volume of higher requirements. This paper uses TI’s special power supply chip to achieve the system power supply.Retrieval can be performed quickly by efficient XOR operation between hash codes, and the storage cost can also be greatly reduced, so the hash method has gained a lot of attention. Different channel shapes generally produce different resonance peaks, directly reflected in the spectrum where the intensity of the harmonics is different. We can approximate the resonance peaks by looking at the spectral envelope. If the window size of the spectrum is adjusted to a small size, then although the frequency axis is blurred to the extent that it is impossible to distinguish each harmonic, we can vaguely see a few obvious areas of high intensity, which are the resonance peaks. Resonance peaks are usually expressed in terms of central frequency and bandwidth [20]. The original audio file is usually stored in the computer as a one-dimensional time-length sequence, the length of which is determined by the length of the audio and the sampling rate; for example, 1-second audio with a sampling rate of 24 kHz is stored in the computer as 24,000+ points. To extract the acoustic features of Meier’s spectrum and fundamental frequency, it is necessary to first transform the original audio pair domain representation into the class domain representation. First, divide the frames and add windows to get many married time frames, then do Fast Fourier Transform (FFT) on each frame to get the time domain signal, and finally do a stacking of each frame of frequency domain signal in time after FFT; you can get the acoustic spectrum map (linear spectrum) containing frequency domain information, and its intuitive understanding can be visualized in table. A schematic diagram of the waveform acoustic spectrum conversion diagram is shown in Figure 3.Figure 3
Schematic diagram of waveform sound spectrum conversion diagram.The corresponding linear spectrogram is obtained, through which the corresponding acoustic characteristic Meier spectrum and fundamental frequency can be obtained. The fundamental frequency is usually the frequency value corresponding to the first wave peak of the linear spectrum, so we only need to find the frequency corresponding to the first wave peak of each frame of the spectrum map; that is, we can get the corresponding fundamental frequency sequence. Sometimes the clear tone also contains higher frequency peaks, so we need to set the frequency range of the fundamental frequency; usually, the minimum frequency is set to 80 and the maximum frequency is set to 1200. The extracted fundamental frequency sequence also needs to do a smoothing operation to eliminate some abnormal extracted values. In some cases, principal components with a small contribution rate may contain more differentiated information, and the number of principal components is not limited by a good criterion. At the same time, principal component analysis analyzes the data from the overall structure of the data, ignoring the difference between the data.Human hearing is sensitive only in certain specific frequency domain ranges and therefore does not require the input of a complete linear spectrum. In contrast, Meier’s frequency analysis is an analysis based on human auditory perception [21]. Experimental observations have revealed that the human ear is like a filter bank, which focuses only on certain specific frequencies. Therefore, according to human hearing perception (more sensitive to low frequencies and less sensitive to high frequencies), it is possible to design a corresponding Meier filter set to convert the linear spectrum into a Meier spectrum that is consistent with human hearing.(3)Fmel=ln1+log700f.As a basic music discipline aimed at developing pitch, rhythm, and aural discrimination, the learning content included is inextricably linked to subjects, such as music theory, harmony, pattern analysis, and improvisation accompaniment, in addition to singing and dictation in the traditional sense. “If one learns sight-reading and ear training well, the ability to read music is strong, and good score-reading skills can reduce the time spent reading music, allowing students more time to invest in their feelings and communicate their feelings about the piece to others.” The process of musical expression, as well as communication, occupies an important position in vocal singing. Vocal singing is a kind of art of voice expression; if the vocal link is lost, the artistic effect will not exist. The process of vocal singing combines many kinds of pitch, rhythm, speed, strength, and other musical elements to show diverse emotions and thematic ideas [22]. Thus, it seems that, in the training process of vocal singing, on the one hand, it is necessary to understand the basic vocal singing techniques; on the other hand, the emotional expression in singing and the imagination training in music should become indispensable and important. The training of musical senses focuses on firstly opening the students’ desire for emotional expression so that they can feel the emotions shown by the music during the musical activity.


## 3.1. Multimedia Data Analysis Design

Multimedia data analysis has always been a popular research area, and multimedia data is usually high dimensional, which brings great challenges to the research. How to reduce the dimensionality of multimedia data and extract effective low-dimensional features from it has become a difficult research problem. Fortunately, through observation and research, it is found that many high-dimensional data can be represented by a small number of low-dimensional features, and the data are essentially distributed on top of some low-dimensional manifolds. To effectively reveal the underlying essential structure of the data, it is necessary to find the low-dimensional features hidden in the high-dimensional space [15]. The discrete Karhunen–Loève (K-L) transform provides the theoretical support for the principal component analysis algorithm, which uses the covariance matrix to orthogonally transform the data to remove the correlation of the data. The mathematical idea of principal component analysis has two equivalent interpretations: minimizing reconstruction error and maximizing variance. Minimizing reconstruction error requires that the error between the original data and the reconstructed data be minimized. Maximizing the variance requires that the variance of the projected low-dimensional data is maximized. First, a lot of time frames are obtained by adding windows to frames, and then a fast Fourier transform (FFT) is performed on each frame to obtain a time domain signal. Finally, each frame frequency domain signal after the FFT is stacked in time. A spectrogram (linear spectrum) containing frequency domain information can be obtained, and its intuitive understanding can visualize the surface.Let the high-dimensional data beX=x1,x2,...xM∈Rn×M, that is, where ∑i=1xi=0n denotes the dimensionality of the original data and M denotes the total number of training samples. The expected projection matrix is W=w1,w2,...,wd∈Rn×d, where d denotes the number of projection vectors. Assume that the low-dimensional representation of the data Xii=1,2,...,n isfii=1,2,...,nfi=wTxi such that F=fi,f2,...,fn∈Rd×n is the low-dimensional representation of the data set X=x1,x2,xM∈Rn×M; then F=wTxi.(1)Wopt=argmaxwTx2,∑i,j=1fi+fj2=Si,j.The principal component analysis is an unsupervised feature extraction algorithm that can replace the original data with a small number of features, and each principal component is orthogonal to each other to remove the correlation between the original data, and the calculation is relatively simple and easy to implement in computer coding. However, there are some disadvantages of principal component analysis. In some cases, the principal components with a small contribution may contain more discrepant information, and there is no good guideline to limit the number of principal components [16]. The local hold projection algorithm is a linear approximation of the Laplace feature mapping algorithm. The local hold projection constructs the neighborhood structure between data pairs from the original high-dimensional data while maintaining this neighborhood structure in the reduced dimensional low-dimensional space.Tensor, also known as a multidimensional array, is a very effective tool for mathematical analysis and is widely used in mechanics and physics. In a broad sense, a tensor is a generalization of the traditional vector concept. In the sense of isomorphism, a 0th-order tensor is represented as a scalar, a 1st-order tensor as a vector, and a 2nd-order tensor as a matrix. In today’s era, the tensor theory has been popularized and applied in many fields. And in the field of multimedia analysis, tensor decomposition is widely used in image retrieval, personalized recommendation, entity mining, and so on. A slice is a representation of a two-dimensional section of a tensor. Taking a third-order tensor as an example, a slice of a tensor is a tensor representation in which all dimensions are fixed except for two dimensions whose indices are not fixed. A third-order tensorx has three slices i, j, and k along the horizontal, horizontal, and forward directions, denoted by the symbols x1, xj, and xk, respectively. Like tensor slices, a tensor fiber is a tensor representation with all dimensions fixed except for one dimension that is not fixed. The tensor x is expanded along the n− module to obtain a matrix Xn∈RI1×I2×...×In−1×In×In+1. In this matrix, all column vectors are obtained by arranging the fibers of the n− module of the original tensor x in a specific order, as shown in Figure 1.Figure 1
Schematic diagram of the third-order tensor expansion operation.Hashing methods can transform the original data into a series of fixed-length binary codes (hash codes), while the similarity information of the original data can be well maintained. Thanks to the binary representation, so retrieval can be performed quickly by efficient operations between hash codes, and the storage cost can be greatly reduced, so hashing methods have gained a lot of attention. Early hashing methods mainly focus on similarity retrieval tasks in single-modal data scenarios (i.e., unimodal hashing methods) [17]. Unimodal hashing methods can be classified according to the way the hash function is generated: data-independent methods and data-dependent methods. In data-independent methods, the hash function is usually pregiven by hand or directly obtained by random generation. Typical methods include Locality-Sensitive Hashing (LSH), which provides a series of random hash functions so that similar samples have a high probability of collision in Hamming space, while dissimilar samples have as little chance of a collision as possible. The black points are the ideal target positions for mapping, the points in the red area are like the ideal points, and the points in the blue area are the dissimilar sample points.(2)Pt∈Ret×rc,Zi,j=1Zi,j<1Zi,jZi,j=10Zi,j>1.The approach proposed in this chapter is a two-stage model. Specifically, firstly, the hash representation of the binary is learned in the first stage. The second requires learning the hash function in the next stage. The work proposed in this chapter learns the hash codes shared by different modalities in the first stage; however, for cross-modal retrieval, data is obtained from one modality and it is desired to retrieve the content of another modality related to its semantics, so for this task, the hash function for each modality needs to be learned to generate its corresponding hash code. The role of the hash function is to convert the data in the original space into a binary hash code in Hamming space. Many effective classification models can be used to learn the hash functions in this stage, such as support vector machines, deep neural networks, and linear regression. Complex moduli usually lead to better pick-up results. However, synthetically considering the computational complexity and retrieval accuracy, this chapter chooses the linear regression model of the cylinder alone to learn the hash function, as shown in Figure2. The black points are the ideal target positions for mapping, the points in the red area are like the ideal points, and the points in the blue area are the dissimilar sample points.Figure 2
Schematic diagram of the mapping operation.Some cross-modal retrieval methods learn hash functions and obtain good results. However, the above schemes may limit the capability of hash functions, for example, when mapping the original data from its original space to Hamming space, which may lead to bias and thus affect the retrieval performance [18]. It is assumed that when mapping a sample r to another space, its ideal position is indicated by black dots, and the other three red dots are the points where bias may occur. Points of the same shape indicate similarity, otherwise not similar. From Figure 2, it can be seen that x may map to one of the three possible candidate points. However, it can also be observed from the above figure that x deviated data point 1 is like the ideal and labeled point; in other words, point 1 is more suitable for retrieval than points 2 and 3 when the projection is shifted from a retrieval perspective. Therefore, the search performance can also be improved if the offset point is found to be favorable for retrieval when an offset occurs. In other words, if supervised semantic knowledge is considered when learning linear mappings, it can make its retrieval performance improved.


## 3.2. Vocal Singing Training Optimization Model Construction

Digital Signal Processor (DSP) is a dedicated digital processing chip, generally using the classical Harvard structure to form a dedicated processing chip. As DSP is dedicated to processing digital signals, it has the advantages of accurate data processing, good reliability, and easy large-scale integration. In recent years, with the improvement of the DSP manufacturing process, its application scope spreads across the field of data statistics and information technology; DSP minimum system design includes two aspects of dedicated hardware design and supporting software design [19]. The hardware design part of the main work is to select the appropriate DSP chip, based on the chip as the core, auxiliary to other peripheral equipment chip, data acquisition chip, system power supply circuit, system chip reset circuit, and set up in the DSP access memory external expansion memory circuit, chip work crystal circuit, simulation connection circuit, and so on, to build the DSP minimum system. The classical TMS320C5502 is selected as the core processor to build the DSP minimum system schematic. The system, due to the need to design a low-power microintegrated system, so the system’s power supply design and volume of higher requirements. This paper uses TI’s special power supply chip to achieve the system power supply.Retrieval can be performed quickly by efficient XOR operation between hash codes, and the storage cost can also be greatly reduced, so the hash method has gained a lot of attention. Different channel shapes generally produce different resonance peaks, directly reflected in the spectrum where the intensity of the harmonics is different. We can approximate the resonance peaks by looking at the spectral envelope. If the window size of the spectrum is adjusted to a small size, then although the frequency axis is blurred to the extent that it is impossible to distinguish each harmonic, we can vaguely see a few obvious areas of high intensity, which are the resonance peaks. Resonance peaks are usually expressed in terms of central frequency and bandwidth [20]. The original audio file is usually stored in the computer as a one-dimensional time-length sequence, the length of which is determined by the length of the audio and the sampling rate; for example, 1-second audio with a sampling rate of 24 kHz is stored in the computer as 24,000+ points. To extract the acoustic features of Meier’s spectrum and fundamental frequency, it is necessary to first transform the original audio pair domain representation into the class domain representation. First, divide the frames and add windows to get many married time frames, then do Fast Fourier Transform (FFT) on each frame to get the time domain signal, and finally do a stacking of each frame of frequency domain signal in time after FFT; you can get the acoustic spectrum map (linear spectrum) containing frequency domain information, and its intuitive understanding can be visualized in table. A schematic diagram of the waveform acoustic spectrum conversion diagram is shown in Figure 3.Figure 3
Schematic diagram of waveform sound spectrum conversion diagram.The corresponding linear spectrogram is obtained, through which the corresponding acoustic characteristic Meier spectrum and fundamental frequency can be obtained. The fundamental frequency is usually the frequency value corresponding to the first wave peak of the linear spectrum, so we only need to find the frequency corresponding to the first wave peak of each frame of the spectrum map; that is, we can get the corresponding fundamental frequency sequence. Sometimes the clear tone also contains higher frequency peaks, so we need to set the frequency range of the fundamental frequency; usually, the minimum frequency is set to 80 and the maximum frequency is set to 1200. The extracted fundamental frequency sequence also needs to do a smoothing operation to eliminate some abnormal extracted values. In some cases, principal components with a small contribution rate may contain more differentiated information, and the number of principal components is not limited by a good criterion. At the same time, principal component analysis analyzes the data from the overall structure of the data, ignoring the difference between the data.Human hearing is sensitive only in certain specific frequency domain ranges and therefore does not require the input of a complete linear spectrum. In contrast, Meier’s frequency analysis is an analysis based on human auditory perception [21]. Experimental observations have revealed that the human ear is like a filter bank, which focuses only on certain specific frequencies. Therefore, according to human hearing perception (more sensitive to low frequencies and less sensitive to high frequencies), it is possible to design a corresponding Meier filter set to convert the linear spectrum into a Meier spectrum that is consistent with human hearing.(3)Fmel=ln1+log700f.As a basic music discipline aimed at developing pitch, rhythm, and aural discrimination, the learning content included is inextricably linked to subjects, such as music theory, harmony, pattern analysis, and improvisation accompaniment, in addition to singing and dictation in the traditional sense. “If one learns sight-reading and ear training well, the ability to read music is strong, and good score-reading skills can reduce the time spent reading music, allowing students more time to invest in their feelings and communicate their feelings about the piece to others.” The process of musical expression, as well as communication, occupies an important position in vocal singing. Vocal singing is a kind of art of voice expression; if the vocal link is lost, the artistic effect will not exist. The process of vocal singing combines many kinds of pitch, rhythm, speed, strength, and other musical elements to show diverse emotions and thematic ideas [22]. Thus, it seems that, in the training process of vocal singing, on the one hand, it is necessary to understand the basic vocal singing techniques; on the other hand, the emotional expression in singing and the imagination training in music should become indispensable and important. The training of musical senses focuses on firstly opening the students’ desire for emotional expression so that they can feel the emotions shown by the music during the musical activity.


## 4. Analysis of Results

### 4.1. Multimedia Data Analysis Performance Testing

We randomly select one experiment to report the convergence of the proposed optimization algorithm. Since theUii=1N matrix is used to obtain the kernel tensor slices of the mapping, we measure the perturb ability Uii=1N during the iteration by employing the expression of (3).(4)Dt=∑i=1UitF+Uitf.The use of the above function measure criterion ensures that the feature representations of the samples do not change drastically during the optimization process. In Figure4, the variation pattern of the MOAR and MCAR algorithms on the four databases is shown with the number of iterations Dt values. From Figure 4, we can see that the algorithms MOAR and MCAR converge quickly when the convergence analysis is verified on different databases, indicating the effectiveness of the proposed solution algorithm. Both algorithms converge on four different databases when the number of iterations is close to 30. In addition, MOAR has a faster convergence rate compared to MCAR, which indicates that the MOAR algorithm is more stable. Based on the above observations, we uniformly set the convergence condition for the algorithm that the difference between two consecutive iterations Dt does not exceed 10–2 and the maximum number of iterations does not exceed 30.Figure 4
Convergence curve of the analyzed media data algorithm on the database.This chapter proposes the Discrete Asymmetric Hashing (DAH) method for unequal-length discrete asymmetric cross-modal hashing. The method designs a flexible hash model that can handle not only traditional but also novel unequal-length coding problems. The complexity analysis shows that the complexity of DAH is related ton (the size of the training data) of the Great Wall Current. To further investigate its efficiency, this section conducts some experiments on MIR-Flickr. The results of the experiments are shown in Figure 5. This section at NMLI-F IITA yields some faster than all other methods; it is even faster than those relaxation algorithms. The DAH proposed in this chapter is faster than all the comparison methods. The research on multimedia data forecasting has gradually shifted from the early single-media data analysis to the demand-driven cross-media intelligent analysis. It is even faster than those relaxation algorithms (e.g., SCM, CMFH, and LCMFII). Moreover, the graph-based methods (i.e., SMFH, GSePH, and SePH) spend a lot of training time, which indicates that these methods cannot scale to large data sets. For the discrete method DCH, which learns hash codes through a per-bit scheme, the training time of DCHI increases significantly with the length of the hash code. For DAH, on the other hand, the increase in its training time is not significant. The main reason is that DAH learns all bits of the hash code synchronously instead of a bit-by-bit learning scheme.Figure 5
Training time of DAH on NUS-WIDE.To further validate the scalability of DAH, this section conducts some experiments on a large-scale dataset (SP, NUS-WIDE) and sets the hash code length to 32 bits and the training set size to vary from 10 k to 180 k. The training time is linearly related to the amount of training data in Qi. In addition, as mentioned above, the counting complexity of DAH is linearly related, which is consistent with the experimental results. In this regard, based on the above analysis, it can be concluded that DAH can be scaled to larger datasets and is very efficient.To further test the effectiveness of the model designed in this chapter, this section chooses to conduct a series of experiments on the Wiki dataset and designs several variants of DAH, namely, DAH-1, DAH-2, DAH-3, and DAH-4. Specifically, DAH-1 denotes using the original features instead of kernelized features. DAH-2 denotes discarding the label reconstruction terms from the original model. DAH-3 means learning both hash functions and hash codes in one stage instead of learning them separately in two stages. DAH outperforms all variants, which further proves the validity of the proposed model. The performance of DAII outperforms DAH-3, indicating that the two-stage scheme does improve the retrieval performance. The possible reason is that the two-stage strategy can reduce the complexity and generate more discriminative hash codes in the first stage. The use of kernelized features can improve its performance. This is because kernelization of features can better capture the nonlinear structure of the data. The semantic intersection scheme helps to improve retrieval performance. The main reason is that this scheme can reduce the heterogeneous divide and can learn more powerful hash functions. Data structures are also becoming more complex and their dimensions are increasing. Faced with such huge and complex multimedia data, how to carry out effective management and analysis has become a problem that the industrial technology and academic circles pay close attention to.


### 4.2. Vocal Singing Training Optimization Model Simulation Experiment

At present, the examination of melodic pitches is generally done by identifying the contour diagram formed by the change of the next tone relative to the preceding one in this way, and most do not use the pitch values themselves. This approach, if represented in a coordinate diagram, would be like the melodic line, and would represent the line of pitch change very well. Like the first key in music, and the fact that the melody is raised or lowered to a different key when performing melody recognition is not noticed by the computer does not affect the recognition result. The fundamental pitch extraction, which corresponds to the fundamental frequency, is the lowest frequency at which the vocal body vibrates. The harmonics in the fundamental frequency correspond to the overtones in the music. By capturing the harmonics, both the fundamental and its overtones corresponding to a pitch can be detected. However, the current main purpose for accurate harmonics is so that the overtones do not interfere with the detection of the fundamental ones. After checking with the template, the chords are finally checked against the chords marked by the music expert to verify the accuracy of the machine recognition. It can be observed that the major triads, minor triads, and pendent chords can be checked (generally, chords formed by replacing the original third with a second or a pure fourth, mostly used in pop music), but the recognition rate is not high due to the instability and uncertainty of pendent chords.There are many reasons for the deviation of singing intonation. Some people are affected by congenital defects of auditory organs or nervous tissue, and some people are caused by singing skills or psychological factors and many other reasons. The average percentage values of the contribution of the silent knowledge on how to master scientific vocal methods in singing are higher for A, B, and F, 30.6%, 29.3%, and 25.5%, respectively, with a percentage sum of 85.4%. The values of A, B, and F were 94.7%, 90.8%, and 81.6%, respectively, and more students thought they were useful (81% or more), while the values of the other items were C38.2%, D31.6%, E22.4%, and G18.4%, and fewer students thought they were useful (39% or less). This indicates that most of the students believed that the teacher’s classroom language, singing demonstration, and repeated practice were the most important processes for acquiring silent knowledge about how to master the scientific method of vocal expression in singing. Most of the students believed that classroom instruction, singing demonstrations, and repetitive practice were the most important processes for acquiring silent knowledge of how to master the scientific method of breathing in singing. A PCP diagram of a vocal singing passage is shown in Figure6.Figure 6
Vocal singing PCP chart.In the process of vocal teaching, improving students’ awareness of their voice is a very important key point, because we know that one hears one’s voice through bone conduction, so one hears one’s voice differently from what others hear, and students will not be able to have a good awareness of their misconceptions in pronunciation. To change students’ problems with pronunciation, teachers need to spend a lot of time and effort guiding them. However, with the recording function of multimedia technology, teachers can let students listen to their vocal effects and look for their vocal deficiencies; moreover, teachers can use the function of “sound spectrum waveform analysis” to analyze students’ pronunciation and generate a chart. In this way, both teachers and students can have a better grasp of students’ vocal situation and conduct more targeted and purposeful vocal training. Many students are confused when they hear themselves singing beautifully, but others say they do not sing well, and when they listen to their recordings, they feel that the voice is unfamiliar. At this point, teachers should be patient and explain this “strange” situation to students carefully and lead them out from misunderstanding their voices. At the same time, teachers can use multimedia technology to familiarize students with their voices and make them clear about their pronunciation to promote students’ enthusiasm for vocal training. Many people do not have the concept of “pitch position” in the subconscious mind of the brain and have no way to control the positional relationship between the sound and the sound, so they mistakenly think that they are “partisan” and cannot sing.In terms of MOS scores, MelGAN still has a certain gap with several other vocoders, and the GAN-based multidiscriminative song synthesis vocoder proposed in this study has a certain improvement in all dimensional scores compared with the original PWG vocoder, which has not completely reached the effect of WaveRNN but is very close to it. The main reason is that the generalization of the vocoder proposed in this study is not very good. Some speakers can synthesize better and some speakers synthesize less well, and there is some variability. In terms of RTF, MelGAN has an absolute advantage in RTF, while the vocoder proposed in this study is close to PWG in RTF, which is mainly because the structure of the vocoder generator proposed in this study is like PWG in that both contain many inflated convolutional stacks. In comparison with the overall synthesis effect, we continue to compare and analyze the improvements of the proposed multidiscriminative vocal synthesis vocoder in terms of spectrograms.Figure7shows that the proposed vocoder in this study is smooth without the raised part in the low-frequency part due to the introduction of the excitation source constructed from the fundamental frequency. And with the multiwindow multiband discriminator, the mid-frequency energy diffusion phenomenon still exists, but it is relatively reduced compared with the PWG vocoder. And in the high-frequency part, the harmonization problem of high frequency is solved. Although there is still some difference between the spectrum of the synthesized singing voice and that of the real audio, the overall effect of the proposed vocoder is still significantly better than that of the PWG vocoder, especially in the low- and middle-frequency parts.Figure 7
Results of comparative analysis of the singing frequency spectrum.


## 4.1. Multimedia Data Analysis Performance Testing

We randomly select one experiment to report the convergence of the proposed optimization algorithm. Since theUii=1N matrix is used to obtain the kernel tensor slices of the mapping, we measure the perturb ability Uii=1N during the iteration by employing the expression of (3).(4)Dt=∑i=1UitF+Uitf.The use of the above function measure criterion ensures that the feature representations of the samples do not change drastically during the optimization process. In Figure4, the variation pattern of the MOAR and MCAR algorithms on the four databases is shown with the number of iterations Dt values. From Figure 4, we can see that the algorithms MOAR and MCAR converge quickly when the convergence analysis is verified on different databases, indicating the effectiveness of the proposed solution algorithm. Both algorithms converge on four different databases when the number of iterations is close to 30. In addition, MOAR has a faster convergence rate compared to MCAR, which indicates that the MOAR algorithm is more stable. Based on the above observations, we uniformly set the convergence condition for the algorithm that the difference between two consecutive iterations Dt does not exceed 10–2 and the maximum number of iterations does not exceed 30.Figure 4
Convergence curve of the analyzed media data algorithm on the database.This chapter proposes the Discrete Asymmetric Hashing (DAH) method for unequal-length discrete asymmetric cross-modal hashing. The method designs a flexible hash model that can handle not only traditional but also novel unequal-length coding problems. The complexity analysis shows that the complexity of DAH is related ton (the size of the training data) of the Great Wall Current. To further investigate its efficiency, this section conducts some experiments on MIR-Flickr. The results of the experiments are shown in Figure 5. This section at NMLI-F IITA yields some faster than all other methods; it is even faster than those relaxation algorithms. The DAH proposed in this chapter is faster than all the comparison methods. The research on multimedia data forecasting has gradually shifted from the early single-media data analysis to the demand-driven cross-media intelligent analysis. It is even faster than those relaxation algorithms (e.g., SCM, CMFH, and LCMFII). Moreover, the graph-based methods (i.e., SMFH, GSePH, and SePH) spend a lot of training time, which indicates that these methods cannot scale to large data sets. For the discrete method DCH, which learns hash codes through a per-bit scheme, the training time of DCHI increases significantly with the length of the hash code. For DAH, on the other hand, the increase in its training time is not significant. The main reason is that DAH learns all bits of the hash code synchronously instead of a bit-by-bit learning scheme.Figure 5
Training time of DAH on NUS-WIDE.To further validate the scalability of DAH, this section conducts some experiments on a large-scale dataset (SP, NUS-WIDE) and sets the hash code length to 32 bits and the training set size to vary from 10 k to 180 k. The training time is linearly related to the amount of training data in Qi. In addition, as mentioned above, the counting complexity of DAH is linearly related, which is consistent with the experimental results. In this regard, based on the above analysis, it can be concluded that DAH can be scaled to larger datasets and is very efficient.To further test the effectiveness of the model designed in this chapter, this section chooses to conduct a series of experiments on the Wiki dataset and designs several variants of DAH, namely, DAH-1, DAH-2, DAH-3, and DAH-4. Specifically, DAH-1 denotes using the original features instead of kernelized features. DAH-2 denotes discarding the label reconstruction terms from the original model. DAH-3 means learning both hash functions and hash codes in one stage instead of learning them separately in two stages. DAH outperforms all variants, which further proves the validity of the proposed model. The performance of DAII outperforms DAH-3, indicating that the two-stage scheme does improve the retrieval performance. The possible reason is that the two-stage strategy can reduce the complexity and generate more discriminative hash codes in the first stage. The use of kernelized features can improve its performance. This is because kernelization of features can better capture the nonlinear structure of the data. The semantic intersection scheme helps to improve retrieval performance. The main reason is that this scheme can reduce the heterogeneous divide and can learn more powerful hash functions. Data structures are also becoming more complex and their dimensions are increasing. Faced with such huge and complex multimedia data, how to carry out effective management and analysis has become a problem that the industrial technology and academic circles pay close attention to.


## 4.2. Vocal Singing Training Optimization Model Simulation Experiment

At present, the examination of melodic pitches is generally done by identifying the contour diagram formed by the change of the next tone relative to the preceding one in this way, and most do not use the pitch values themselves. This approach, if represented in a coordinate diagram, would be like the melodic line, and would represent the line of pitch change very well. Like the first key in music, and the fact that the melody is raised or lowered to a different key when performing melody recognition is not noticed by the computer does not affect the recognition result. The fundamental pitch extraction, which corresponds to the fundamental frequency, is the lowest frequency at which the vocal body vibrates. The harmonics in the fundamental frequency correspond to the overtones in the music. By capturing the harmonics, both the fundamental and its overtones corresponding to a pitch can be detected. However, the current main purpose for accurate harmonics is so that the overtones do not interfere with the detection of the fundamental ones. After checking with the template, the chords are finally checked against the chords marked by the music expert to verify the accuracy of the machine recognition. It can be observed that the major triads, minor triads, and pendent chords can be checked (generally, chords formed by replacing the original third with a second or a pure fourth, mostly used in pop music), but the recognition rate is not high due to the instability and uncertainty of pendent chords.There are many reasons for the deviation of singing intonation. Some people are affected by congenital defects of auditory organs or nervous tissue, and some people are caused by singing skills or psychological factors and many other reasons. The average percentage values of the contribution of the silent knowledge on how to master scientific vocal methods in singing are higher for A, B, and F, 30.6%, 29.3%, and 25.5%, respectively, with a percentage sum of 85.4%. The values of A, B, and F were 94.7%, 90.8%, and 81.6%, respectively, and more students thought they were useful (81% or more), while the values of the other items were C38.2%, D31.6%, E22.4%, and G18.4%, and fewer students thought they were useful (39% or less). This indicates that most of the students believed that the teacher’s classroom language, singing demonstration, and repeated practice were the most important processes for acquiring silent knowledge about how to master the scientific method of vocal expression in singing. Most of the students believed that classroom instruction, singing demonstrations, and repetitive practice were the most important processes for acquiring silent knowledge of how to master the scientific method of breathing in singing. A PCP diagram of a vocal singing passage is shown in Figure6.Figure 6
Vocal singing PCP chart.In the process of vocal teaching, improving students’ awareness of their voice is a very important key point, because we know that one hears one’s voice through bone conduction, so one hears one’s voice differently from what others hear, and students will not be able to have a good awareness of their misconceptions in pronunciation. To change students’ problems with pronunciation, teachers need to spend a lot of time and effort guiding them. However, with the recording function of multimedia technology, teachers can let students listen to their vocal effects and look for their vocal deficiencies; moreover, teachers can use the function of “sound spectrum waveform analysis” to analyze students’ pronunciation and generate a chart. In this way, both teachers and students can have a better grasp of students’ vocal situation and conduct more targeted and purposeful vocal training. Many students are confused when they hear themselves singing beautifully, but others say they do not sing well, and when they listen to their recordings, they feel that the voice is unfamiliar. At this point, teachers should be patient and explain this “strange” situation to students carefully and lead them out from misunderstanding their voices. At the same time, teachers can use multimedia technology to familiarize students with their voices and make them clear about their pronunciation to promote students’ enthusiasm for vocal training. Many people do not have the concept of “pitch position” in the subconscious mind of the brain and have no way to control the positional relationship between the sound and the sound, so they mistakenly think that they are “partisan” and cannot sing.In terms of MOS scores, MelGAN still has a certain gap with several other vocoders, and the GAN-based multidiscriminative song synthesis vocoder proposed in this study has a certain improvement in all dimensional scores compared with the original PWG vocoder, which has not completely reached the effect of WaveRNN but is very close to it. The main reason is that the generalization of the vocoder proposed in this study is not very good. Some speakers can synthesize better and some speakers synthesize less well, and there is some variability. In terms of RTF, MelGAN has an absolute advantage in RTF, while the vocoder proposed in this study is close to PWG in RTF, which is mainly because the structure of the vocoder generator proposed in this study is like PWG in that both contain many inflated convolutional stacks. In comparison with the overall synthesis effect, we continue to compare and analyze the improvements of the proposed multidiscriminative vocal synthesis vocoder in terms of spectrograms.Figure7shows that the proposed vocoder in this study is smooth without the raised part in the low-frequency part due to the introduction of the excitation source constructed from the fundamental frequency. And with the multiwindow multiband discriminator, the mid-frequency energy diffusion phenomenon still exists, but it is relatively reduced compared with the PWG vocoder. And in the high-frequency part, the harmonization problem of high frequency is solved. Although there is still some difference between the spectrum of the synthesized singing voice and that of the real audio, the overall effect of the proposed vocoder is still significantly better than that of the PWG vocoder, especially in the low- and middle-frequency parts.Figure 7
Results of comparative analysis of the singing frequency spectrum.


## 5. Conclusion

In this paper, we construct an optimization model for vocal singing training based on multimedia data analysis. The model is tested, and the real-time and accuracy rates of the model reach 87.58% and 83.15, respectively, improving 15.14% and 13.42% based on the current technology. A discrete iterative optimization scheme is proposed, which allows hash codes to be learned efficiently and discrete, avoiding large quantization losses. The semantic embedding hash function learning scheme makes the learned hash functions more efficient. Experimental results on several datasets show that this method can obtain better retrieval performance compared with existing methods. The high-order time series prediction algorithm, based on tensor decomposition, multilinear orthogonal autoregressive model, and multilinear restricted autoregressive model, uses the idea of heterogeneous learning to solve the time-space domain consistency problem in time series data feature characterization. The existing generative adversarial network-based vocoder MelGAN and Parallel WaveGAN are used as a starting point to verify the effectiveness of both vocoders on a multispeaker song dataset. Although the research work in this thesis addresses the problem of multimedia data prediction from the perspective of feature representation and achieves relatively good results on specific application domains, direct feature representation learning and prediction studies on incompletely observed data are highly susceptible to loss of important information, leading to wrong or inaccurate conclusions.



---

*Source: 7609516-2022-09-17.xml*