# Machine Learning-Based Improvement of Musical Digital Processing Technology on Musical Performance

Authors: Lingyang Sun; Mishal Sohail
Journal: Security and Communication Networks
 (2022)

---

## Abstract

The modern stage focuses more on structural changes, and the numerical control technology realizes the complex changes of the stage scenes and the precise movement of stage props. The study object in this article is musical drama, and it uses digital technologies to digitally manipulate the soundtrack. This study offers a low-complexity feature space minimal variance algorithm that combines the power approach to address the concerns of inadequate resolution improvement, excessive complexity, poor real-time performance, and low resilience of standard MV methods. The method has a high resolution, low complexity, and great resilience, and it may be utilized in a variety of stages. In addition, this paper combines digital technology to process music, enhances the promotion of music in musical performances, and allows performers to integrate in more effectively. Finally, through experimental research, it can be known that the music digital processing technology proposed in this paper can play a good role in promoting musical performances.


---

## Body

## 1. Introduction

Although musicals have similarities with opera, dance drama, musicals, and other stage performance forms, it is unique in that it places equal emphasis on songs, dialogues, body movements, performances, and other factors. Musicals are staged all over the world, but the most frequent places are Broadway in New York City, USA, and West London in the UK. Musical is a stage art that balances popularity, commerciality, entertainment, and artistry. Commerciality is a major feature of musicals, and its value is more reflected in the balance between commerciality and artistry [1].Musical drama is a dramatic art integrating song, dance, and drama. The requirements for actors are quite strict, but only a handful of talents are born with almighty qualities [2]. Most actors only possess certain specialties and talents, and then they are cultivated through intensive training. When cultivating actors’ singing talents and skills, we must first consider whether the actors have the conditions and ability to sing. The specific content includes testing the actor’s voice conditions, musical sense, scientific vocalization, and singing skills. Moreover, it is necessary to test the actor’s listening, sight-singing, and music theory. At the same time, it is necessary to comprehensively train the actor’s singing skills, voice conditions, sensitive ears, and musical sense. These inherent advantages and singing skills that can only be acquired through acquired training must be strengthened through relevant teaching. Musical actors should be all-rounders, one specializing in multiple skills, and there is no weakness that ordinary audiences can feel. This “one specialty” in most cases should be dedicated to singing; this is called weakness, and in most cases they cannot be weak in singing [3].Whether it is the evolution of the stage itself or the evolution of the stage performance form, the evolution of stage art has gone through the development process from the traditional framed stage to the variable stage and then to the centre protruding stage, open stage, mobile stage, and natural stage. Among them, the rapid development of multimedia technology has greatly promoted the development of stage art, and it is not an exaggeration to call it the revolution of stage art. At present, with the rapid development of multimedia technology, it is difficult to meet the aesthetic needs and viewing experience of the current audience compared to more traditional stage art forms. Instead, those more novel and more modern technology stage art designs and media images have become modern stage performances. The audience prefers the mainstream kind of design. The cornerstone of stage design video image design is video image, which is complemented by stage installation equipment, and it is via this design that the content qualities of the performance on the stage and the mood of the stage are created. The evolution of numerous technologies such as lighting, audio and video technology, playback technology, and computer technology has shifted the whole stage art towards digital development, providing the audience with a new style of stage art performance. In comparison with the conventional theatre, the arrival of computer lighting has given the stage a new lease of life. Scanners can show both high-resolution images and smooth video. The gap between stage art and lighting design has increasingly shrunk since the invention of this technology. The scenery, lighting, and sound effects are interwoven to provide a unique visual and audio experience. The growth of media art improves the entire feeling of space onstage, boosts the stage’s creative awareness, improves the overall picture of the art, and completely mobilises the stage components. This unique stage idea elevates us to a new level by allowing the audience to participate in the performance. At the same time, the advancement of science and technology has allowed Chinese stage art to expand its growth area.Musicals’ soundtracks will also help to boost their performance to some level. This article uses digital processing technologies to digitally process music and, as a result, examines ways for enhancing musical performance.


## 2. Related Work

The coding principle of PCM is to turn a continuous analog signal into a discrete amplitude signal and then turn the discrete amplitude signal into a digital discrete signal. The signal has two important parameters: one is frequency, and the other is amplitude [4]. The frequency of the signal determines the sampling frequency, and the amplitude variation range of the signal determines the number of bits of the binary number that each sample needs to be allocated. When a uniform quantization interval is employed to quantize a sampled signal, it is referred to as uniform quantization. A certain amount of error or distortion is introduced into the sample value during the quantization process. Quantization noise [5] is the term for this kind of mistake. If the quantization interval is narrow and there are more digits to represent the amplitude with a decimal number, then the quantization noise is lower for a signal with the same amplitude range. It is required to increase the number of bits per sample in order to guarantee the input signal’s dynamic range while simultaneously reducing quantization noise. This is why when high-quality super-high-fidelity audio uses PCM encoding, the number of bits per sample uses 16 bits [6]. When uniform quantization is used, the same quantization interval is used for both large and small signals [7]. In order to adapt to the large changes in the dynamic range of the input signal while requiring low quantization noise, one of the solutions is to increase the number of samples. However, the probability of a large signal is very small for voice signals, so the PCM coding system is not fully utilized [8].Music is a way to communicate emotions like language. Although people all over the world have different language and cultural backgrounds, they can communicate emotionally in a common-way music. Along with the progress and development of mankind, music has also undergone thousands of years of development and changes, resulting in a variety of music forms, but rhythm has always maintained its important position in expressing the beauty of music [9]. Rhythm is like the pulse of music. If music has no rhythm, it loses the ability to express musical thinking. The idea of music rhythm is a human experience of music, although it is more difficult to explain in words [10]. The rhythm is defined as the organizational shape generated by the duration of the pitch and the location of the strong and weak beats in “Basic Music Theory.” In comparison to rhythm, “beat” is a feature of rhythm. The term “beat” is defined in literature [11] as “the impression of impetus split into equal time intervals.” At the same time, “beat” is a unit for measuring the tempo of music, commonly expressed in the number of beats per minute (BPM). In fact, music has a hierarchical structure organized in a binary tree or a trinomial tree, and the accents in each layer constitute the steps of the upper layer. The beat is the middle layer of the music hierarchy. Humans have a natural ability to “listen” to music; that is, even people who do not know anything about music will clap and stomp their feet with the beat of the music [12].Humans can best communicate their musical sentiments via dance. With the advancement of computer technology in recent years, utilizing computers to imitate this inherent talent of human beings has become a hot topic among academics [13]. Its goal is to create a computer program that can automatically abstract specific signals in music in order to convey human sense of musical rhythm. This technique requires a theoretical understanding of music psychoacoustics and signal processing. Eric Scheirer believes that the research in this area can be summarized into two levels. The lower level is to construct an analysis system of the perception of “beat”; the higher level is based on the pattern recognition of accents and musical instruments. Above, infer the model of the hierarchical structure of the rhythm. The content involved in this topic is to “perceive” the moment and strength of each beat in the music and the speed of the music [14]. Because the human auditory system has amazing detection, discrimination and recognition capabilities for speech, music, and other surrounding sounds, people have done a lot of theoretical and experimental research on the principles of human auditory function in recent decades [15].


## 3. Digital Processing of Music Waveform

This section proposes a low-complexity feature space minimum variance algorithm (hereinafter referred to as the LCMV algorithm) fused with the power method to address the problems of insufficient resolution improvement, high complexity, poor real-time performance, and low robustness of traditional MV algorithms. The method has a high resolution, low complexity, and great resilience, and it may be utilized in a variety of stages. Figure1 depicts the algorithm schematic representation. First, the method converts the music waveform echo data to the low-dimensional beam domain using the discrete cosine transform (DCT) and then calculates the dimensionality reduction coefficient based on the notion of minimising distortion and beam domain conversion efficiency. This minimises the size of the covariance matrix and the amount of time it takes to execute the adaptive algorithm’s dynamic weighting factors. Second, the approach uses the power method to extract the sample covariance matrix’s maximum eigenvalue and associated eigenvector, decreasing the complexity of the eigenspace method by simplifying the covariance matrix eigenvalue decomposition and sorting processes. Finally, by discarding the low-energy signals corresponding to nonmaximum eigenvalues, the approach reduces the inversion of the covariance matrix into vector multiplication and eigenspace projection operations. It lowers the computation and complexity of the LCMV method while keeping great image quality. The following are the specific principles of the beamforming algorithm.Figure 1
Schematic diagram of low-complexity feature space minimum variance algorithm fused with power method.First, the algorithm uses DCT to construct a conversion matrixT of (p + 1) × L dimensions. Among them, L is the number of elements contained in the spatial smoothing subarray, and p is the dimensionality reduction parameter of the beam domain. The form of the constructed beam domain conversion matrix is as follows:(1)Tk,n=1L,k=0,0≤n≤L−1,2Lcosπkn+1/2L,1≤k≤p,0≤n≤L−1,The matrixT satisfies TTH=I, where I is the identity matrix. The selection of the beam-domain dimensionality reduction parameter p should be based on the principle of minimising distortion to ensure the imaging quality of the adaptive algorithm. In order to reduce the complexity of the algorithm, the value range of the dimensionality reduction parameter is 2<p+1<L. The larger the dimensionality reduction parameter p, the closer the imaging result is to the original image, but the dimension of the beam domain covariance matrix will increase accordingly. On the contrary, the calculation amount of the algorithm dynamic weighting vector is obviously reduced, but the imaging quality is reduced. Therefore, in order to select the best dimensionality reduction parameters, on the one hand, the mean squared error (MSE) parameter is introduced to characterize the beamforming quality of the algorithm. The smaller the MSE, the higher the imaging quality of the LCMV algorithm [16].On the other hand, the beam domain conversion efficiency is introduced to represent the time it takes to solve all of the dynamic weight vectors of the dimensionality reduction signal in the beam domain. Figure2 depicts the MSE value and beam-domain conversion efficiency curve graphs when various dimensionality reduction values pare used. The ideal beam domain dimension reduction parameter p may be derived as illustrated in Figure 2 by getting the value of the intersection of the curves. That is, when the dimensionality reduction parameter p=8 is used to build the beam domain conversion matrix T, not only is the LCMV algorithm’s complexity reduced, but the difference between the dimensionality reduction data and the original data is also reduced, ensuring the LCMV algorithm’s imaging quality.Figure 2
The efficiency and MSE with differentp.The dimension of the sample covariance matrix of the echo signal of the traditional adaptive beamforming algorithm is determined by the number of spatial smoothing subarraysL. When applying DCT to multiply the spatially smoothed subarray echo signal by the conversion matrix T, the beam-domain echo signal can be obtained. By extracting part of the beam-domain echo signal according to the dimensionality reduction parameter, a dimensionality-reduced beam-domain covariance matrix can be obtained, and the matrix dimension is reduced from L × L to p+1×p+1. Therefore, the complexity of the inversion of the covariance matrix after beam domain conversion is reduced from OL3 to OP+13. Taking the first subarray as an example, the beam domain signal conversion is as follows [17]:(2)xblkp+1×1=Tp+1×1xnlkL×1.Among them,xnlk is the time-domain subarray signal, and xblk is the beam-domain subarray signal. As a result, the beam domain sample covariance matrix is correspondingly changed to(3)R^bk=1N−L+1∑I−1N−L+1xblkp+1×1xblk1×p+1H=Tp+1×1R^kTL×p+1×1H.In summary, the optimal dimensionality reduction beam-domain sample covariance matrix is obtained by constructing the DCT conversion matrix according to the optimal dimensionality reduction parameterp, which further reduces the algorithm complexity by ensuring the LCMV algorithm’s beamforming quality.Second, the eigenvalue of the covariance matrix may be used to quantify the energy of the received music waveform echo signal, with a big eigenvalue corresponding to a strong energy effective echo signal. The signal subspace is made up of the relevant eigenvectors, whereas the noise subspace is made up of the eigenvectors corresponding to the low-energy component. As a result, the maximum eigenvalue and eigenvector of the eigenspace projection can be extracted using the power method, and the low-energy signal corresponding to the nonmaximum eigenvalue can be ignored, simplifying the inversion of the beam domain covariance matrix as well as the eigenvalue decomposition and sorting operations. At the same time, the inversion of the covariance matrix is simplified into vector multiplication and projection operations, thereby reducing the amount and complexity of the LCMV algorithm. The specific steps of applying the power method to obtain the maximum eigenvalue of the sample covariance matrix of the music waveform echo signal and its corresponding eigenvector are shown in Figure3 [18].Figure 3
The principle diagram of the power method to solve the maximum eigenvalue and eigenvector.We set the initial vectore0=1,1,…,1 and perform the following iterative operations:(4)ei+1=R^bkeiλi+1=maxei+1.Among them,ei+1 is the eigenvector after the i + 1th iteration, and max⋅ is the maximum element solution function. After each iteration, the acquired feature vector is normalized and substituted into the next iteration operation. We use the ith iteration as an example, that is, ei=ei/maxei. After the normalization step is over, the iterative operation of formula (4) is performed again. When the iterative operation satisfies the judgment condition λi+1−λi/λi+1<ε, the operation is terminated, and the maximum eigenvalue λmax and its corresponding eigenvector emax are output. Among them, ε is the iteration precision value. In engineering practice, s = 0.01 is selected according to the empirical value, and a higher ε value can also be selected according to the imaging quality requirements. In this paper, ε is set to 0.001.Third, low-energy signals can be ignored to simplify the eigenvalue decomposition and matrix inversion budget. In order to ensure the stability of the energy of the music waveform echo signal, while keeping the music waveform echo signal covariance matrix trace unchanged, the eigenvalues corresponding to the noise subspace can be set to the same value, as shown in the following formula [19]:(5)λq+1=λq+2,=…=λp+1=1p−q+1traceRb−∑i=1qλi.If we setλq+1=λq+2=…=λp+1=1/p−q+1traceRb−∑i=1qλi=α, the inversion result of the covariance matrix after diagonal loading can be simplified as(6)R^b−1=α−1I−∑i=1qλi−αλieieiH.In order to further simplify formula (6), the beam domain signal corresponding to the nonmaximum eigenvalue can be ignored and q = 1, and the inversion operation of the matrix can be converted into a vector multiplication operation to reduce the number of cycles. Then, the computational complexity of the inversion of the covariance matrix is reduced from OL3 to Op+12; namely,(7)R^b−1=α−1I−λi−αλieieiH.Among them, the maximum feature value of feature space projection is the maximum feature value obtained by the power method and its corresponding feature vector, namely,λ1=λmax, e1=emax. When the inverse matrix of the beam-domain covariance matrix is substituted into the MV beamforming expression, the beam-domain dynamic weight vector can be obtained as follows:(8)wib=R^b−1ababHR^b−1ab.Among them,ab=Ta is the desired direction vector of the beam domain.Finally, the algorithm projects the weighted vector to the signal subspace formed by the largest eigenvectoremax and uses the orthogonality between the eigenvectors to eliminate the noise signal. The optimal weighting vector wkmv of the low-complexity feature space minimum variance algorithm of the fusion power method can be obtained as follows [20]:(9)wkmv=emaxemaxHwib,=emaxemaxHR^b−1ababHR^b−1ab.We set up a linear music waveform transducer array withN elements. The traditional subband minimum variance beamforming algorithm uses discrete STFT to convert the time-domain echo signal xk=x1k−Δ1,…,xNk−ΔNT of the music waveform after delay and presteering processing into several equally spaced frequency-domain subband signals for processing. Among them, k represents the kth time sampling point, Δi represents the delay time imposed on the echo signal of the ith element, and ⋅T is the transposition operation. Then, for each frequency domain subband, its adaptive beamforming output can be expressed as follows:(10)yω=wHωXω,=∑i=1NwiωXiω.Among them,Xω=X1ω,…,XNωT represents a vector of Fourier variables of the segmented subband signal, wω=w1ω,…,wNω represents an adaptive weighting vector in the frequency domain, and ⋅H represents a conjugate transpose operation.The traditional MVS algorithm is similar in principle to the MV algorithm. By constantly updating the weighting vector, the output power of each frequency domain subband beamforming is minimized, and the response of the focus is passed without distortion. The expression is(11)P=Eyω2=wHEXωXHωw,minwwHRw,subject towHa=1.Among them,P represents the subband beamforming power, E{·} represents the expected operation, Rω=EXωXHω is the frequency domain subband sample covariance matrix, and a is the expected direction vector of all ones. It can be seen that formula (11) can be solved by the Lagrangian multiplier method, and the optimal weighting vector in the frequency domain can be obtained as follows:(12)wopt=R−1aaHR−1a.The above calculation is performed on each subband, and the final beamforming output of the MVS is the sum of the beamforming output of each frequency domain subband. For the MVS beamforming of thekth subband, its frequency domain output can be expressed as follows:(13)yω=∑k=1Kyωk=∑i=1N∑k=1KwiωkXiωk.Finally, using IFFT, the output expression of MVS beamforming can be obtained as follows [21]:(14)yk=IFFTyω.


## 4. Enhancement of Musical Performance Based on Digital Processing Technology

The parallel configuration of the DSP array is chosen in this design. The system’s scalability must be addressed while constructing the connecting bus between DSPs. The synchronisation bus between DSPs in this system can accommodate up to 16 DSPs at the same time. Simultaneously, in order to limit the effect of DSP on the AHB bus even further, a memory unit is included in the DSP array. This memory unit may significantly increase the DSP’s operational efficiency while also reducing the DSP’s access to the AHB. Figure4 depicts the structure of the DSPs that were ultimately chosen.Figure 4
Effect of DSP and AHB connection.In the single AHB bus structure, it will be different depending on the type of external memory. It is divided into two cases: SDRAM’s external memory and the external memory is DDR2. The external memory is shown in Figure5 of the SDRAM structure.Figure 5
AHB and SDRAM.According to the application of this SOC, the priority of each core in the chip for the bus occupation is the highest in the synthesis DSP, followed by the ARM, followed by the effect DSP array, DMA, and so on. The process of synthesis DSP can synthesis 256 sounds at once, but while doing so, it takes up a lot of space on the bus, and the bus overhead is rather significant. 256 sounds need 512 bus accesses, which essentially address that reach memory. This will prevent ARM from receiving the bus for a long period, preventing it from accessing the external memory. This would significantly diminish ARM’s efficiency while also having a significant influence on the impacts of DSP arrays, DMA, and other similar technologies. I conducted a particular test throughout the design process. When the audio source in the system has a sample rate of 44.1 KHz and there are more than 200 pronunciations, ARM often performs 500 or 600 system clock cycles in my tests. The access right to the bus cannot be obtained once; the limit is that the access right to the bus cannot be obtained in more than 1000 system clock cycles. In addition, when using SDRAM as an external memory, it is also faced with: SDRAM is slow and has a small capacity. The tiny capacity of a single chip, in particular, will significantly influence the system’s future applicability. Because the Wave Table of the timbre is engaged in this SOC’s system application, a big memory capacity (i.e., a large-capacity sound library) is necessary. SDRAM currently has a maximum capacity of 32 megabytes. If we need to expand the sound library to 512 megabytes, we will require 16 SDRAM, which will strain the system board’s architecture and significantly raise the cost. A strategy to utilize DDR2 instead of SDRAM is offered in this plan to alleviate the speed and capacity difficulties of SDRAM. Use the DDR2 interface memory scheme box as shown in Figure6.Figure 6
AHB and DDR2.The dual AHB bus structure is proposed when the efficiency of ARM cannot be fully utilized in the case of the previous single AHB bus, and the synthetic DSP’s preemption of the bus is relatively serious. In the case of dual AHB buses, the external use of dual memory buses for coordination can perfectly solve the occupancy of the synthetic DSP on the bus and improve the efficiency of ARM. Figure7 uses dual AHB bus and dual memory bus structure.Figure 7
Dual bus structure.Using twin AHB buses and dual external memory buses may address not only the bus occupancy issue, but also the memory capacity problem. If you require a lot of memory on the synthetic DSP, you may utilize Flash instead of SDRAM; the access speed issue is handled since the synthetic DSP will monopolise a bus. After testing the dual AHB bus solution, the Linux operating system can be successfully booted on the ARM9 hard-core development platform (0.13 um, 300 MHz, FA616TE). This clearly demonstrates that this structure can fully use ARM9’s performance benefits. It also has no influence on the performance of the synthesis and effect DSPs. To summarize, a suitable chip structure was eventually chosen in the structural design of the SOC, beginning with the chip’s performance, taking into account the chip’s later application environment, the chip’s cost, the cost of the system design, and many other variables. The effect DSP is composed of an effect DSP array that works simultaneously under the control of ARM; at the same time, two AHB buses and two external memory buses are selected for the SOC; it has laid a solid foundation to ensure the success of the SOC.This paper studies and discusses the architecture of SOC, focusing on the analysis of the relationship between MCU, synthesis DSP, and effect DSP, and the system block diagram of SOC is shown in Figure8.Figure 8
SOC chip structure block diagram.Among them, ARM9 (FA616TE), DMA, effect DSP array (four effect DSP), SDRAM interface, USB, and so on are connected to an AHB bus. The synthesized DSP is hung on a single AHB bus. On this AHB bus, SDRAM and Flash compatible memory interfaces are used.After constructing the above music digital model, the performance of the model is verified, and its digitalization is first evaluated, and the results are shown in Table1 and Figure 9.Table 1
Music digitization effect.
NumberDigitizingNumberDigitizing191.981196.55296.031290.83392.451396.58496.001493.40591.291591.26696.031692.01794.551794.44893.971892.53994.381991.611092.872095.87Figure 9
Evaluation of the effect of digital music processing.From the above research, we can see that the music digital processing method constructed in this paper can effectively process the music of musicals. On this basis, the improvement of musical digital processing on the performance of musicals is studied, and the results are shown in Table2 and Figure 10.Table 2
The effect of musical digital processing technology on musical performance.
NumberEvaluationNumberEvaluation187.061181.21286.821278.79389.591387.09487.371484.72583.301584.82675.891671.82782.641780.84885.941886.90975.791972.331090.152079.82Figure 10
The promotion of musical digital processing technology on musical performance.From the above analysis, it can be seen that the music digital processing technology proposed in this paper can play a good role in promoting musical performances.


## 5. Conclusion

Musical literacy is very important for the cultivation of musical literacy in the stage performance of musicals. Music literacy is generally composed of several aspects, such as listening, sight, and singing. In the musical stage, the actors themselves are required to have good musical qualities. Actors must not only match the speed of the music and each beat point, but also have a strong reaction ability to adapt to the change of the beat without interruption to adapt to the rhythm of the performance, thereby achieving a unified combination of internal and external rhythms. In terms of emotional expression, the organic fusion of musical literacy and stage effects can increase the rhythm of the overall stage performance and make the performance more infectious. This article combines digital technology to process music, enhances the promotion of music in musical performances, and allows performers to integrate in more effectively. Through experimental research, it can be known that the music digital processing technology proposed in this paper can play a good role in promoting musical performances.



---

*Source: 8318928-2022-05-17.xml*