# Recognition Algorithm of Piano Playing Music in Intelligent Background

Authors: Weina Gu
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Currently, music recognition research is primarily focused on single note recognition, with some limitations in recognition accuracy and antinoise performance. This paper proposes a new algorithm for piano playing music recognition against the backdrop of intelligent interaction. The method of spectrum peak sorting is extended to the field of multifundamental frequency detection, and high and low channel processing is achieved. The statistical properties of spectral entropy of coefficients in compressed domain are used, resulting in more stable fingerprints. This statistical feature will not be destroyed after the original segment is processed, ensuring that the calculated feature maintains its high stability. This method can effectively improve the accuracy of fundamental frequency extraction by highlighting the peak characteristics of the periodic position of frame samples, avoiding the influence of half-frequency and frequency doubling, and thus avoiding the influence of half-frequency and frequency doubling. In comparison to traditional methods, achieve higher accuracy and fault tolerance. The feasibility and efficiency of the algorithm proposed in this paper are confirmed by a simulation experiment. This method’s overall performance meets certain practical requirements and achieves the expected results, laying the groundwork for future research in this field.


---

## Body

## 1. Introduction

With the rapid development of the Internet, digital music has become the mainstream media of music storage, and the number is huge [1]. More and more users began to use online music applications, which brought diversified music information needs. Therefore, how to organize and manage the massive music data automatically and effectively and how to extract all kinds of different information from music have become urgent problems. A series of problems, such as music recognition, classification, and feature extraction [2] based on computer and artificial intelligence technology [3, 4], have been paid attention by more and more researchers. Music information retrieval is a research field born and developed under this background. According to the traditional manual retrieval and recognition of music, it has become very difficult, and it cannot meet people’s requirements for retrieval methods. Automatic music recognition is a new interdisciplinary subject, and its research involves many disciplines such as physics, signal processing [5, 6], music theory, and music psychology. Automatic music recognition is of great application value in the field of music retrieval and automatic music composition technology. At present, the traditional identification of playing music has the following problems. (1) The accuracy of piano music note recognition for note segmentation cannot be met. (2) Cannot meet the requirement of fundamental frequency extraction accuracy. (3) The adaptive problem of piano music recognition cannot be solved. Piano has become an indispensable tool in modern music class. It is convenient, flexible, and emotional to use piano teaching [6]. However, there are also shortcomings, such as in class, teachers have to spend part of their energy on playing the piano and cannot devote themselves to the organization of teaching activities. Piano playing is not only restricted by the technical level of performance but also by people’s mental and physical state [7]. Moreover, most piano beginners are prone to play notes incorrectly without the guidance of music teachers. If the computer can detect and evaluate the piano music played in real-time, users can use the technology of music recognition to find out the wrong points in time and correct them [8].The number of songs on the Internet has exploded as a result of the popularity of the Internet and the rapid development of multimedia technology [9], and music search, audition, and download have become very popular applications among network end users. The process and technology of organizing and storing information in a specific way, as well as finding relevant information based on the needs of information users, is known as information retrieval. We have enjoyed the text retrieval provider with the rapid development of the Internet. The massive amount of music data and growing number of users present both opportunities and challenges for music service providers. In the face of massive amounts of music data, the traditional model of music organization and management based on manpower has clearly failed to adapt. Pattern recognition [10–12] is defined as a system of human ability to describe phenomena and identify objects, or the ability to process and analyze complex data. Speaker recognition and speech recognition are the two types of speech recognition systems. The music recognition associated with it is widely used in the era of spiritual civilization development. Regular audio is divided into sound effects, speech, and music, and irregular audio is divided into sound effects, speech, and music, based on the different characteristics of sound waves. The horizontal axis in music coordinates is rhythm alternation, and the vertical axis is sound height [13]. It is critical to pay attention to the level of discrimination in the flow of music. It is necessary to carry out the necessary intonation training in order to cultivate students’ ability to distinguish the sound level [14]. The use of a computer system can greatly aid students’ intonation training. This paper investigates the algorithm of piano playing music recognition against a background of intelligent interaction.People’s demand for multimedia information retrieval is growing in tandem with the rapid development of Internet technology [15]. As the proportion of multimedia information, such as audio and video, contained in network information grows, so does their demand for multimedia information retrieval. The large number of users has resulted in a wide range of music information needs, as well as making the original single music service appear outdated. As a result, how to automatically and effectively organize and manage music data, as well as how to provide a variety of services to meet various user needs, has become a critical issue [16]. Audio information retrieval is a branch of multimedia information retrieval that uses auditory feature value as its foundation. Audio recognition is the foundation of audio retrieval, and one of the most important steps in audio recognition is structuring audio information. Music, as an artistic tool for expressing feelings and resonance, contains complex contents and levels that cannot be analyzed directly. As a result of studying music recognition technology, we can extract the common features of music, thereby enhancing music recognition’s auxiliary function in human production and life. This study, which is based on an in-depth understanding of the music system and is based on the background of intelligent interaction, focuses on a number of key technologies in piano music recognition, such as multinote start-stop point detection, accurate fundamental frequency detection, and accurate fundamental frequency calculation of notes. This paper starts with the basics of music recognition and then moves on to the analysis and extraction of piano music features. The feature recognition of piano music performance can be achieved through computer-assisted analysis and processing of audio signals. The musical characteristics of the piano are used to identify energy jump points, which can improve the detection accuracy of note beginning and end points and prevent missing and incorrect judgments. This study is useful in the field of music recognition when playing the piano.


## 2. Related Work

Literature [17] first resets the harmonic and corrects the octave of a MIDI-generated electronic music signal and then uses the harmonic product spectrum to achieve multi-fundamental frequency detection with high detection accuracy. According to literature [18], the future field of music recognition will provide humans with unexpected user experiences, and it will undoubtedly open a new chapter in music recognition by inferring from other users’ functions and technological innovations. Literature [19] developed a music recognition algorithm based on linear matching. To improve music recognition accuracy, the two characteristics of music rhythm and pitch are extracted and compared. A multiclass and hierarchical music retrieval method was proposed in the literature [20]. To divide the audio signal into different sound types, first set the environmental noise segment, then extract the statistical music-related feature data to divide the audio signal into music segment, environment segment, silent segment, and so on. Finally, using a hidden Markov model, detect time-domain features and perform hierarchical retrieval of music segments. The automatic classification of music styles using the minimum information length criterion has been studied in the literature [21], but there has been little research on identifying all playing instruments from a given set of music signals. The methods for recognizing plucked instruments, keyboard instruments, and percussion instruments are presented in literature [22]. The identification of a single piano note using an RBF neural network model was proposed in the literature [23]. It has a high recognition rate, but it cannot be recognized in real time. According to literature [24], the computer recognition system for piano performance will be a novel application of music recognition theory. Literature [25] completed the spectrum analysis of piano music signals and the reconstruction of piano timbre by using the short-time Fourier transform theory to analyze the fundamental frequency characteristics of piano music signals. An improved tone model is proposed in literature [26], which employs a constrained Gaussian mixture model to model each group of harmonics in a unified manner. And, using expectation maximization, calculate the fundamental frequency of music.This paper finds that traditional playing music recognition has a number of issues, based on an in-depth review of related literature. This paper proposes a new algorithm for piano playing music recognition in the context of intelligent interaction based on this. The music signal is first converted into a time-frequency representation, and then the synchronization of music in the time and frequency domains is examined using this representation. Using the example of piano continuous multinote detection, fundamental frequency extraction accuracy is improved on the basis of solving the adaptive problem of music recognition. And music recognition is important on a global scale. Basic algorithms suitable for the characteristics of musical instruments can be used to realize music recognition of various playing musical instruments, depending on the situation.


## 3. Methodology

### 3.1. Music Feature Extraction and Music Signal Preprocessing

Sound is a natural occurrence. When an object vibrates, sound waves are produced, which are transmitted through the air to people’s eardrums and perceived as sound by the brain. The range of a musical instrument is defined as the distance between the lowest bass and the highest pitch. Its frequency is fixed when pronounced as simple harmonic motion, and the resulting sound is known as “pure tone.” The sound produced when the pronunciation body moves randomly is known as “polyphony.” Many sounds combine to form a fixed artistic system that is used to describe music ideas and create musical images [27]. People in long-term life practice select the sounds mentioned in music, which can express people’s lives, thoughts, and feelings, and form a fixed system to express music thoughts and shape music images. Notes, as the fundamental component of music, play a critical role in analyzing and recognizing music. The foundation for detecting and recognizing piano music signals is accurate note segmentation.The duration, frequency, spectrum distribution, and amplitude of vibration are the physical quantities that correspond to the four basic characteristics of sound: length, pitch, timbre, and intensity. The duration of music that people listen to is determined by the vibration duration of the instrument’s sounding position, take, for example, the piano. There is a sound for every Bai Jian and black key on the piano keyboard. Regardless of whether the white or black keys are used, the sound level is the same. Every two adjacent keys are separated by a semitone. The sound gets longer as the key is pressed longer, and vice versa. The sound level is determined by the vibration times per unit time of the instrument’s sounding position. The louder the sound, the higher the frequency. The sound is lower the lower the frequency.A single isolated sound cannot shape a musical image; in music, sounds are always linked together in a specific relationship to express specific musical ideas. Melody, rhythm, and harmony are the three basic elements of music. Overall features, basic features, and complex features are the three types of music features. Complex features are reflected through the basic features of music, and then the overall features are expressed. Finally, demonstrate the musical form structure, artistic style, and emotional connotation contrast. Figure1 depicts the music feature system.Figure 1
Music characteristic system diagram.Musical tone system refers to a set of notes with a fixed pitch that is used in music. A sound train is a musical system in which sounds are arranged in ascending or descending order. A tone level is the name given to each tone in the musical system. Basic and variable levels are the two types of levels. Seven levels with distinct names are referred to as basic levels in the musical system. On the piano, fifty-two white key cycles use the same seven basic level names over and over again. Music’s sensory characteristic is timbre. Overtones of various frequencies are present in music played by various musical instruments, and it is precisely because of these complex and changeable overtones that people can distinguish the timbres of various musical instruments. The piano has a powerful and full-bodied bass, a natural and smooth alto, and a bright and lovely treble. In its string array, more than 200 tight strings and soundboards are coupled into a beautiful timbre. The amplitude of music determines the intensity of sound, as the name implies. The louder the sound, the greater the amplitude of the music.The four basic characteristics of sound play an important role in the artistic expression of music. The term “mode” refers to a grouping of notes in music that form a system. One of the tones is arranged in high and low order, making it modal. This tonic arrangement is known as scale, and it demonstrates the regularity of modal. Understanding the aforementioned music characteristics is the first step in delving into the music recognition algorithm. The four properties of sound mentioned above are all important in music performance, but the height and length of sound are the most important. You can tell the name of the music if you play the same piece of music with different instruments and change the sound intensity. The music image will be severely harmed if the length and pitch of the music are changed, and music recognition will not be completed. As a result, the primary goal of music recognition is to obtain information about these four parameters through recognition, with a particular focus on pitch and duration. The extraction of basic features of music is the foundation for extracting complex features of music. Recognition of rhythm, melody, and harmony is part of the former. Melody and rhythm are the most important of these three elements, so knowing the melody and rhythm of a piece of music is necessary and accurate.Taking rhythm recognition as an example, it reflects the regularity of music in time length. Rhythm recognition directly trains rhythm samples with fixed rhythm as the model and then compares the music to be recognized with the training sample music to recognize the rhythm. The difficulty lies in that the rhythm changes of music are random, and there are almost no music with the same rhythm in the world, which requires many sample music with different rhythms, which is not feasible in practice. Let us decompose the music, extract the time value characteristics of each note, and then get the rhythm characteristics of the music. If the time value of several consecutive notes is small, it indicates that the rhythm is faster; otherwise, the rhythm is gentle. At the same time, the mode and tonality of the music also represent the characteristics of the music very well. If the mode characteristics of the music can be well extracted, it can help the performer to perceive and express the characteristics of the music to a certain extent. The audio recognition framework is shown in Figure2.Figure 2
Audio recognition framework.The step of music identification is to first extract the complex features of music by using the basic features of music, then study and analyze its complex features, and finally get the overall features of the music. Audio information structuring refers to dividing the audio into independent segments of different lengths and shorts based on a certain standard and having the same attributes under the standard. Each audio signal contains a lot of information, which can be divided into three levels in terms of content: physical sample level, acoustic feature level, and semantic level. Information structuring is generally based on acoustic features.The pitch of the note, that is, the fundamental frequency of the note, is strictly regulated; the method of specifying the pitch of each note is called temperament. The temperament system generally follows the following principles to determine the pitch of each note: first specify the pitch of a standard note; then use a certain mathematical algorithm to calculate the pitch of other notes within an octave from the pitch of the standard note. The modern piano has the widest range of musical instruments, capable of playing 88 different pitched tones. It contains 52 white keys corresponding to the ancient basic independent scale names. Any two adjacent keys on a piano are separated by a half-tone. If you look at the white keys on the keyboard, it can be seen as composed of several groups of seven keys divided into groups, each group represents a distance from a certain note to its higher octave. The relationship between the groups is a relationship that gradually increases or decreases from low to high or from high to low. If there is a black key inserted between the two white keys, the two white keys contain two semitones, that is, a whole tone; if there is no black key inserted between the two white keys, the two white keys contain a semitone.The preprocessing of the music signal is the measure taken before extracting the characteristic parameters of the music signal, in order to achieve the purpose of eliminating the DC component and trend items and reducing the noise of the music signal. This process includes audio sample quantization, preemphasis processing, windowing, filtering, and other operational procedures. For the performance music collected by ordinary microphones and stored in wave format for a piece of primary piano work, the system reads its performance information and automatically extracts each main music feature in its performance. Then, the basic features of music are extracted from the performance, and the feature recognition result of the performance music is output. In order to make the results more intuitive and facilitate the system users to analyze the music they play, the system provides the feature analysis results in the form of text or images.


### 3.2. Recognition of Piano Playing Music under the Background of Intelligent Interaction

The power spectrum of music will decrease with the increase of frequency, and most of its energy is concentrated in the low-frequency range, which may lead to the unacceptable signal-to-noise ratio of the high-frequency part of the message signal. However, because the energy of the high-frequency part of the music signal is small, it is not enough to generate the maximum frequency offset, so it is mostly the maximum frequency offset caused by the low-frequency component. Similar to the zero-crossing rate, the short-term energy of silence is relatively low, which is mainly used to distinguish silence with high signal-to-noise ratio from other types of audio signals and needs hard threshold judgment. In addition, for unvoiced speech, its short-term energy is obviously lower than voiced speech. In order to improve the accuracy of hard threshold judgment, both the zero-crossing rate and short-time energy are generally selected as double threshold judgment.The machine is capable of extracting individual music features. The research work that goes into the development of this module is crucial to the overall system’s success. This module realizes the extraction of the essential features of music, which includes the extraction of four features of the most fundamental elements of music-monophonic, as well as the overall characteristics of music as a piece of music. Audio fingerprints are used in music recognition technology, and audio fingerprints should meet the following requirements: (1) differentiating, each piece of music should have its own unique fingerprint; (2) it has a compact appearance that makes storage and comparison easier; (3) easy math problems; (4) resilience to common signal distortion and transmission interference.The music waveform time-domain signal isxn, and the ith frame music signal obtained after frame-by-frame processing by the windowing function wn is yin. Then, yin satisfies the following formula:
(1)yin=wn∗xi−1∗inc+n,1≤n≤L,1≤i≤fn,where wn is the window function, and yin is the amplitude of one frame. n=1,2⋯,L,i=1,2,.⋯fn, L is the frame length. inc is the frame shift length; fn is the total number of frames after frame division. Calculate the short-term average energy of any frame of music signal yin according to the following formula:
(2)Ei=∑n=0L−1yi2n.The music signal is an example of a non-stationary signal with limited duration and time-varying second-order statistics. Time-frequency analysis is based on the construction of a time-frequency joint function, and it can describe changes in music signals in the time-frequency domain as well as estimate signal characteristic parameters. Before adding noise, increase the high-frequency component of the input modulation signal. At the output end, de-emphasis processing is used to restore the original signal’s power distribution and improve the output signal-to-noise ratio. The amplitude of the zero-crossing rate does not change much if the audio signal is relatively stable, such as mute or music. The degree of signal nonstationarity is high from silence to voice or music, and the zero-crossing rate changes dramatically. Different types of audio signals have different zero-crossing rates in theory. Different types of audio signals can be easily and effectively distinguished if more accurate thresholds can be found.LetYjω be the discrete Fourier transform of the discrete-time signal yn. Then, Yjω can be obtained by the following formula:
(3)Yijω=Sijω∗Fijω,Sijω=Aiαiαi2+ω−ωi2,ωi−αi≤ω≤ωi+αi,Ai2ω−ωi,other,Yjω=∑i=1nYijω.In the formula,ωi is the fundamental frequency or octave of the specified tone, Ai is the amplitude of the fundamental frequency or the octave point, αi is used to adjust the width of the waveform near the fundamental frequency or the octave, and Fi is the function of sine and cosine.Because the short-time Fourier transform has a fixed window length, the time width and bandwidth in the time-frequency plane are also fixed, implying that the frequency resolution and time resolution are both fixed. The fundamental frequency of piano notes ranges from 27.5 Hz to 4186 Hz, and the piano music signal has a wide range. The narrow time window, which is suitable for high-pitched signals, can distinguish the signal’s high-frequency details, whereas the wide time window has higher frequency resolution. The fixed window length cannot meet the resolution requirements of piano treble and bass at the same time because it is suitable for bass signals with close frequencies. The wavelet transform’s time-frequency window is variable, allowing for high-frequency resolution for low-frequency components and high time resolution for high-frequency components. Because the noise signal in music affects the detection accuracy of the music recognition algorithm, when filtering the music signal with noise in the music recognition process, the designed filter must reduce the noise as much as possible without changing the signal waveform. It is possible to use the error caused by the fixed filter and adaptive filter.It can be assumed thatHz is an all-pole system excited by the input sequence un. Then, we can regard the audio signal xn as the output result of this all-pole system, and the transfer function of this system is shown in the following formula.
(4)Hz=G1−∑i=1paiz−i.Among them,G is a constant, ai is a real number, and p is the order of the model. Obviously, this model is an all-pole model with coefficient ai and gain G as model parameters. A p-order linear predictor Fz can be defined with the coefficients ai as shown in the following equation.
(5)Fz=∑i=1paiz−i.From the perspective of the time domain, it can be understood that the predicted valuex~n obtained by using the first p samples of the signal to predict the current sample is shown in the following formula.
(6)x~n=∑i=1paixn−i.The mathematical description of the prediction erroren is shown in the following formula.
(7)en=xn−x~n.In terms of minimum mean square error, a set of optimal prediction coefficients can be calculated. When the calculation method is changed, the result of the linear prediction coefficient changes. The resolution of the wavelet transform decreases as the scale of the wavelet transform increases, which can be used for signal profile analysis. When the scale is small, the resolution is higher, allowing for more detailed observation, but the quality factor of analysis remains constant across scales. In practice, a dynamic sampling grid is used, which allows the wavelet transform’s time and frequency resolution to be varied to adapt to signal nonstationarity. The conversion of a music score into a music score and image in the form of a piano roll window is simple and straightforward to compare. The change of pitch on the time axis can be well represented by the expression of music score in the form of rolling curtain. The linear prediction method is a linear approximation process that involves estimating a set of unique prediction coefficients. If the linear combination of several audio signal samples is known, for example, the actual audio signal sample can be approximated using the minimum mean square error.Subwindow processing transforms nonstationary piano music signals into short-term stationary signals by weighting them with moving windows of fixed length. The method of overlapping segmentation allows for a smooth transition between dip and dip while maintaining signal continuity. Dun shift refers to the overlapping area between the front and back dips. Pitch, duration, timbre, and intensity are difficult to summarize. The array number indicates the pronunciation order, and the time value pitch is stored according to the pronunciation order of each note. The summary of time value and pitch in this recognition system is only used to determine whether the sequentially pronounced notes are of the specified pitch and whether the duration is consistent with the score requirements.


## 3.1. Music Feature Extraction and Music Signal Preprocessing

Sound is a natural occurrence. When an object vibrates, sound waves are produced, which are transmitted through the air to people’s eardrums and perceived as sound by the brain. The range of a musical instrument is defined as the distance between the lowest bass and the highest pitch. Its frequency is fixed when pronounced as simple harmonic motion, and the resulting sound is known as “pure tone.” The sound produced when the pronunciation body moves randomly is known as “polyphony.” Many sounds combine to form a fixed artistic system that is used to describe music ideas and create musical images [27]. People in long-term life practice select the sounds mentioned in music, which can express people’s lives, thoughts, and feelings, and form a fixed system to express music thoughts and shape music images. Notes, as the fundamental component of music, play a critical role in analyzing and recognizing music. The foundation for detecting and recognizing piano music signals is accurate note segmentation.The duration, frequency, spectrum distribution, and amplitude of vibration are the physical quantities that correspond to the four basic characteristics of sound: length, pitch, timbre, and intensity. The duration of music that people listen to is determined by the vibration duration of the instrument’s sounding position, take, for example, the piano. There is a sound for every Bai Jian and black key on the piano keyboard. Regardless of whether the white or black keys are used, the sound level is the same. Every two adjacent keys are separated by a semitone. The sound gets longer as the key is pressed longer, and vice versa. The sound level is determined by the vibration times per unit time of the instrument’s sounding position. The louder the sound, the higher the frequency. The sound is lower the lower the frequency.A single isolated sound cannot shape a musical image; in music, sounds are always linked together in a specific relationship to express specific musical ideas. Melody, rhythm, and harmony are the three basic elements of music. Overall features, basic features, and complex features are the three types of music features. Complex features are reflected through the basic features of music, and then the overall features are expressed. Finally, demonstrate the musical form structure, artistic style, and emotional connotation contrast. Figure1 depicts the music feature system.Figure 1
Music characteristic system diagram.Musical tone system refers to a set of notes with a fixed pitch that is used in music. A sound train is a musical system in which sounds are arranged in ascending or descending order. A tone level is the name given to each tone in the musical system. Basic and variable levels are the two types of levels. Seven levels with distinct names are referred to as basic levels in the musical system. On the piano, fifty-two white key cycles use the same seven basic level names over and over again. Music’s sensory characteristic is timbre. Overtones of various frequencies are present in music played by various musical instruments, and it is precisely because of these complex and changeable overtones that people can distinguish the timbres of various musical instruments. The piano has a powerful and full-bodied bass, a natural and smooth alto, and a bright and lovely treble. In its string array, more than 200 tight strings and soundboards are coupled into a beautiful timbre. The amplitude of music determines the intensity of sound, as the name implies. The louder the sound, the greater the amplitude of the music.The four basic characteristics of sound play an important role in the artistic expression of music. The term “mode” refers to a grouping of notes in music that form a system. One of the tones is arranged in high and low order, making it modal. This tonic arrangement is known as scale, and it demonstrates the regularity of modal. Understanding the aforementioned music characteristics is the first step in delving into the music recognition algorithm. The four properties of sound mentioned above are all important in music performance, but the height and length of sound are the most important. You can tell the name of the music if you play the same piece of music with different instruments and change the sound intensity. The music image will be severely harmed if the length and pitch of the music are changed, and music recognition will not be completed. As a result, the primary goal of music recognition is to obtain information about these four parameters through recognition, with a particular focus on pitch and duration. The extraction of basic features of music is the foundation for extracting complex features of music. Recognition of rhythm, melody, and harmony is part of the former. Melody and rhythm are the most important of these three elements, so knowing the melody and rhythm of a piece of music is necessary and accurate.Taking rhythm recognition as an example, it reflects the regularity of music in time length. Rhythm recognition directly trains rhythm samples with fixed rhythm as the model and then compares the music to be recognized with the training sample music to recognize the rhythm. The difficulty lies in that the rhythm changes of music are random, and there are almost no music with the same rhythm in the world, which requires many sample music with different rhythms, which is not feasible in practice. Let us decompose the music, extract the time value characteristics of each note, and then get the rhythm characteristics of the music. If the time value of several consecutive notes is small, it indicates that the rhythm is faster; otherwise, the rhythm is gentle. At the same time, the mode and tonality of the music also represent the characteristics of the music very well. If the mode characteristics of the music can be well extracted, it can help the performer to perceive and express the characteristics of the music to a certain extent. The audio recognition framework is shown in Figure2.Figure 2
Audio recognition framework.The step of music identification is to first extract the complex features of music by using the basic features of music, then study and analyze its complex features, and finally get the overall features of the music. Audio information structuring refers to dividing the audio into independent segments of different lengths and shorts based on a certain standard and having the same attributes under the standard. Each audio signal contains a lot of information, which can be divided into three levels in terms of content: physical sample level, acoustic feature level, and semantic level. Information structuring is generally based on acoustic features.The pitch of the note, that is, the fundamental frequency of the note, is strictly regulated; the method of specifying the pitch of each note is called temperament. The temperament system generally follows the following principles to determine the pitch of each note: first specify the pitch of a standard note; then use a certain mathematical algorithm to calculate the pitch of other notes within an octave from the pitch of the standard note. The modern piano has the widest range of musical instruments, capable of playing 88 different pitched tones. It contains 52 white keys corresponding to the ancient basic independent scale names. Any two adjacent keys on a piano are separated by a half-tone. If you look at the white keys on the keyboard, it can be seen as composed of several groups of seven keys divided into groups, each group represents a distance from a certain note to its higher octave. The relationship between the groups is a relationship that gradually increases or decreases from low to high or from high to low. If there is a black key inserted between the two white keys, the two white keys contain two semitones, that is, a whole tone; if there is no black key inserted between the two white keys, the two white keys contain a semitone.The preprocessing of the music signal is the measure taken before extracting the characteristic parameters of the music signal, in order to achieve the purpose of eliminating the DC component and trend items and reducing the noise of the music signal. This process includes audio sample quantization, preemphasis processing, windowing, filtering, and other operational procedures. For the performance music collected by ordinary microphones and stored in wave format for a piece of primary piano work, the system reads its performance information and automatically extracts each main music feature in its performance. Then, the basic features of music are extracted from the performance, and the feature recognition result of the performance music is output. In order to make the results more intuitive and facilitate the system users to analyze the music they play, the system provides the feature analysis results in the form of text or images.


## 3.2. Recognition of Piano Playing Music under the Background of Intelligent Interaction

The power spectrum of music will decrease with the increase of frequency, and most of its energy is concentrated in the low-frequency range, which may lead to the unacceptable signal-to-noise ratio of the high-frequency part of the message signal. However, because the energy of the high-frequency part of the music signal is small, it is not enough to generate the maximum frequency offset, so it is mostly the maximum frequency offset caused by the low-frequency component. Similar to the zero-crossing rate, the short-term energy of silence is relatively low, which is mainly used to distinguish silence with high signal-to-noise ratio from other types of audio signals and needs hard threshold judgment. In addition, for unvoiced speech, its short-term energy is obviously lower than voiced speech. In order to improve the accuracy of hard threshold judgment, both the zero-crossing rate and short-time energy are generally selected as double threshold judgment.The machine is capable of extracting individual music features. The research work that goes into the development of this module is crucial to the overall system’s success. This module realizes the extraction of the essential features of music, which includes the extraction of four features of the most fundamental elements of music-monophonic, as well as the overall characteristics of music as a piece of music. Audio fingerprints are used in music recognition technology, and audio fingerprints should meet the following requirements: (1) differentiating, each piece of music should have its own unique fingerprint; (2) it has a compact appearance that makes storage and comparison easier; (3) easy math problems; (4) resilience to common signal distortion and transmission interference.The music waveform time-domain signal isxn, and the ith frame music signal obtained after frame-by-frame processing by the windowing function wn is yin. Then, yin satisfies the following formula:
(1)yin=wn∗xi−1∗inc+n,1≤n≤L,1≤i≤fn,where wn is the window function, and yin is the amplitude of one frame. n=1,2⋯,L,i=1,2,.⋯fn, L is the frame length. inc is the frame shift length; fn is the total number of frames after frame division. Calculate the short-term average energy of any frame of music signal yin according to the following formula:
(2)Ei=∑n=0L−1yi2n.The music signal is an example of a non-stationary signal with limited duration and time-varying second-order statistics. Time-frequency analysis is based on the construction of a time-frequency joint function, and it can describe changes in music signals in the time-frequency domain as well as estimate signal characteristic parameters. Before adding noise, increase the high-frequency component of the input modulation signal. At the output end, de-emphasis processing is used to restore the original signal’s power distribution and improve the output signal-to-noise ratio. The amplitude of the zero-crossing rate does not change much if the audio signal is relatively stable, such as mute or music. The degree of signal nonstationarity is high from silence to voice or music, and the zero-crossing rate changes dramatically. Different types of audio signals have different zero-crossing rates in theory. Different types of audio signals can be easily and effectively distinguished if more accurate thresholds can be found.LetYjω be the discrete Fourier transform of the discrete-time signal yn. Then, Yjω can be obtained by the following formula:
(3)Yijω=Sijω∗Fijω,Sijω=Aiαiαi2+ω−ωi2,ωi−αi≤ω≤ωi+αi,Ai2ω−ωi,other,Yjω=∑i=1nYijω.In the formula,ωi is the fundamental frequency or octave of the specified tone, Ai is the amplitude of the fundamental frequency or the octave point, αi is used to adjust the width of the waveform near the fundamental frequency or the octave, and Fi is the function of sine and cosine.Because the short-time Fourier transform has a fixed window length, the time width and bandwidth in the time-frequency plane are also fixed, implying that the frequency resolution and time resolution are both fixed. The fundamental frequency of piano notes ranges from 27.5 Hz to 4186 Hz, and the piano music signal has a wide range. The narrow time window, which is suitable for high-pitched signals, can distinguish the signal’s high-frequency details, whereas the wide time window has higher frequency resolution. The fixed window length cannot meet the resolution requirements of piano treble and bass at the same time because it is suitable for bass signals with close frequencies. The wavelet transform’s time-frequency window is variable, allowing for high-frequency resolution for low-frequency components and high time resolution for high-frequency components. Because the noise signal in music affects the detection accuracy of the music recognition algorithm, when filtering the music signal with noise in the music recognition process, the designed filter must reduce the noise as much as possible without changing the signal waveform. It is possible to use the error caused by the fixed filter and adaptive filter.It can be assumed thatHz is an all-pole system excited by the input sequence un. Then, we can regard the audio signal xn as the output result of this all-pole system, and the transfer function of this system is shown in the following formula.
(4)Hz=G1−∑i=1paiz−i.Among them,G is a constant, ai is a real number, and p is the order of the model. Obviously, this model is an all-pole model with coefficient ai and gain G as model parameters. A p-order linear predictor Fz can be defined with the coefficients ai as shown in the following equation.
(5)Fz=∑i=1paiz−i.From the perspective of the time domain, it can be understood that the predicted valuex~n obtained by using the first p samples of the signal to predict the current sample is shown in the following formula.
(6)x~n=∑i=1paixn−i.The mathematical description of the prediction erroren is shown in the following formula.
(7)en=xn−x~n.In terms of minimum mean square error, a set of optimal prediction coefficients can be calculated. When the calculation method is changed, the result of the linear prediction coefficient changes. The resolution of the wavelet transform decreases as the scale of the wavelet transform increases, which can be used for signal profile analysis. When the scale is small, the resolution is higher, allowing for more detailed observation, but the quality factor of analysis remains constant across scales. In practice, a dynamic sampling grid is used, which allows the wavelet transform’s time and frequency resolution to be varied to adapt to signal nonstationarity. The conversion of a music score into a music score and image in the form of a piano roll window is simple and straightforward to compare. The change of pitch on the time axis can be well represented by the expression of music score in the form of rolling curtain. The linear prediction method is a linear approximation process that involves estimating a set of unique prediction coefficients. If the linear combination of several audio signal samples is known, for example, the actual audio signal sample can be approximated using the minimum mean square error.Subwindow processing transforms nonstationary piano music signals into short-term stationary signals by weighting them with moving windows of fixed length. The method of overlapping segmentation allows for a smooth transition between dip and dip while maintaining signal continuity. Dun shift refers to the overlapping area between the front and back dips. Pitch, duration, timbre, and intensity are difficult to summarize. The array number indicates the pronunciation order, and the time value pitch is stored according to the pronunciation order of each note. The summary of time value and pitch in this recognition system is only used to determine whether the sequentially pronounced notes are of the specified pitch and whether the duration is consistent with the score requirements.


## 4. Result Analysis and Discussion

In the field of piano performance, the means of intelligent interaction mainly depends on the recording and analysis of electronic devices. In the research of music recognition, what are the basic features of music and how to use these features for classification and recognition have been puzzling researchers. In fact, the definition of music features can be started from the perspective of music aesthetics. If we analyze the process of collecting music signals by human ears and processing them by human brain, we can know that the four basic characteristics of single tones make people feel the ups and downs of melody. The overall characteristics of music make people distinguish music styles, emotions, themes, and so on. The main task of music recognition is to identify the sound name and time value information of each note from a multinote audio signal.Short-term energy is the characteristic of detecting energy intensity of music signal, which has the advantages of low computational complexity and high speed. Because piano music signals usually have a high signal-to-noise ratio, the energy of musical segments is much larger than that of silent segments and noise segments for single music with obvious rhythm, so the starting point of musical segments can be detected by short-time energy characteristics. After the start and end points of notes are detected, each note contains several gaps, and each tilt signal corresponds to a fundamental frequency sample. The traditional autocorrelation method is used to extract the fundamental frequency of an audio signal. The experimental results are shown in Figure3. Figure 3(a) is a short-time energy waveform diagram of an audio signal. The horizontal axis is the time axis, and the vertical axis is the normalized short-term energy amplitude. Figure 3(b) is the corresponding fundamental frequency extraction result, in Hz, and the corresponding fundamental frequency curve of each note containing frame has obvious fluctuation characteristics.Figure 3
Time domain waveform diagram and fundamental frequency extraction result of an audio signal.
(a)(b)It can be seen from Figure3 that in the duration of a note, the fundamental frequency of each frame is different, and taking scientific methods to extract the fundamental frequency of the frame is the basis for accurately calculating the fundamental frequency of the note. Each piece has a certain performance speed requirement, which is not necessarily accurately expressed in beats per minute, but there is a certain range. Performance speed is also an integral feature of music performance, and the speed used by the performer will have a direct impact on the mood, style, and even feelings of music. Short-term energy can be used to distinguish music segments from environmental noise segments. Because the short-term energy of the environmental noise segment is relatively small, and the short-term energy of the music segment will be significantly increased due to the vibration of the strings of playing musical instruments, this method can be used to detect the music segment. In order to verify the effectiveness and feasibility of the algorithm in this paper, different algorithms are used for sample set experiments, and the results shown in Figure 4 are obtained.Figure 4
Comparison of calculation errors of different algorithms.The error rate of this algorithm is the lowest, as evidenced by comparative experiments of different algorithm errors. In comparison to the other two algorithms, this one has a few advantages. The physical amplitude reflects the vibration’s intensity. A piano hammer is used to strike the strings, and the sound intensity is determined by the maximum vibration amplitude after the strings have fully vibrated. Because extracting the total energy of each string under each stroke is difficult, the vibration of the strings pronounced before and after each stroke has a direct impact on the extraction of this parameter, and in terms of musical hearing, the maximum vibration amplitude is essentially consistent with the psychological effect of the percussion energy on the audience, so the maximum vibration amplitude can be used as the sound intensity information of each tone. A piano piece of music contains several notes, the connecting segment between them may be extremely short, and the ending point of one note may be the beginning and ending point of the next note. As a result, the energy in the connecting segment between notes must be greater than that of the ambient noise.Because the onset point causes a bright change in the spectrum energy of music signals, if the degree of energy change is expressed in distance, the greater the distance between adjacent inclinations, the more likely the onset point, that is, the starting point of the note, will be detected. Music is a convolution signal with a short-time spectrum that is logarithmically transformed, and the inverse Fourier transform spectrum is called cepstrum. The music signal’s cepstrum and impulse signal have the same periodicity. The short-time autocorrelation algorithm, short-time amplitude difference algorithm, and time-frequency domain algorithm are used for comparative experiments to analyze and verify the feasibility of the above algorithms, and the results are shown in Figure5.Figure 5
Comparison of frame fundamental frequency extraction results of different algorithms.The experimental results show that the low pitch period at the beginning of a note can be improved by using the time-frequency domain algorithm, but the fundamental frequency curve of each note is not smooth enough, and there are a large number of outliers, which will cause the final fundamental frequency calculation result to be inaccurate. In addition, both the short-time autocorrelation method and the short-time amplitude difference method belong to the time domain method. Although this method has good antinoise performance, it will be interfered by half-frequency and double-frequency interference. The pitch period can be determined by comparing the similarity between the delayed signal and the original signal. Usually, the maximum value of the short-time autocorrelation function is taken as the pitch period, and then the fundamental frequency of the signal is determined. The experimental results based on the time-frequency domain algorithm and this algorithm are shown in Figure6.Figure 6
Comparison of frame fundamental frequency extraction results of two algorithms.The experimental results show that the fundamental frequency curve of the same note obtained by the improved algorithm has less fluctuation and is relatively smooth. Moreover, this method is based on the combination of time-frequency domain, so it not only has good noise resistance but also can eliminate the frequency doubling and half-frequency interference. As a method in the field of applied mathematics, wavelet analysis transforms the time and frequency of the signal locally, thus effectively extracting the fundamental frequency information in the music signal. When wavelet transform theory is applied to music recognition, good frequency resolution can be obtained in the low-frequency part of the signal, and good time resolution can be obtained in the high-frequency part, which solves the difficult problem of Fourier transform. Modal tonality is a very important concept in music performance and learning. When the performance learning or appreciation level enters a certain level, an accurate grasp of modal tonality and an understanding of the characteristics and functions of each modal tonality can enable the performer or the appreciator to grasp music from a higher level. Similarly, because of the influence of modal tonality on grasping the style, emotion, and emotion of works, it is very meaningful to pay attention to this element in feature extraction and recognition. In order to verify the practicability of piano playing music recognition using this method, we use three different music recognition methods to experiment. Get the trend chart of recognition accuracy of piano playing music, as shown in Figure7.Figure 7
Comparison of recognition accuracy of different algorithms.It can be seen from Figure7 that the recognition accuracy of short-time energy difference method and harmonic peak method is low, and the recognition accuracy of this method is the highest. It shows that it is of practical value to use this method to identify piano playing music. The low-pass decomposition of the music signal into about 3 scales by wavelet transform can clearly reflect the pitch period on the waveform of its approximate function, and then the pitch frequency can be obtained by calculation. This method mainly considers that wavelet low-pass decomposition can filter out a large part of high-frequency components, thus fully retaining low-frequency components. Similar to the short-time autocorrelation function, the short-time average amplitude function has the same periodicity as the original signal. The difference is that the short-time autocorrelation function reaches the maximum value at the integer multiple of the period, while the short-time average amplitude difference function reaches the minimum value. Because the latter reflects the amplitude difference between the delayed signal and the original signal, the smaller the amplitude difference, the closer the music signal is to the original signal. The function has the valley characteristic of integer times of period.In this section, the playing speed is obtained by the playing time value of the first tone of a piece of music and the known time value, and the playing speed can be calculated by calculating the proportional relationship of the corresponding beats through the sampling rate. The time scaling and pitch shifting of audio signals are modeled as time scaling and frequency scaling of spectrogram images at corresponding logarithmic frequencies, respectively, and scale invariant feature transformation is extracted from spectrogram images for music matching. By analyzing the experimental results in this chapter, it is proved that the relative error of fundamental frequency calculation by this method is small, and the accuracy of note fundamental frequency extraction is greatly improved. This is because the method in this paper has high weight concentration and good system fault tolerance.


## 5. Conclusions

The fundamental goal of electronic music, as a new interdisciplinary subject, is to use computers to realize music recognition and even music creation. Pattern recognition technology, for example, encompasses a variety of research topics such as face recognition, license plate recognition, and fingerprint recognition technology. As a result, high-precision music recognition is challenging. This paper investigates the algorithm of piano playing music recognition against the backdrop of intelligent interaction. The feature extraction of music pitch, duration, and mode tonality is realized using a computer recognition system for piano playing music that can sample, preprocess, feature analysis, and score recognition of piano music signals. The time-varying music signal is in a time-invariant state in each investigation thanks to windowing and framing. The music signal is then processed using median smoothing and linear smoothing. The single-mouth limit decision of the short-time energy method is used to extract the music segment. Finally, the music segment is judged twice, and the detection of note time values is completed. The simulation results show that the system recognizes ideal piano music signals, providing an experimental platform for further research. This method can be used to aid piano teaching, allowing students to analyze whether the music they are playing is correct or not, and to continually improve their performance level. On the other hand, the algorithm presented in this paper can be applied to a variety of musical instruments.



---

*Source: 1245078-2022-04-27.xml*