# Analysis of Different Keying Modes and Brain Sound Image Establishment Mode in Piano Playing

Authors: Jingle Huang
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

The piano is known as the king of musical instruments for its rich expressiveness. Pianists get rich emotion with tone control. Touch key mode is the primary method of tone control. The purpose of this paper is to conduct research and analysis on different keying modes and brain-sound image establishment modes in piano performance. In this paper, we first propose a fuzzy mathematical proximity comparison method and an audio feature extraction method. Most of the audio features are derived from speech recognition tasks. They can reduce the original waveform sampling signal, thereby accelerating the machine’s understanding of the semantic meaning of audio. Using comparative analysis method, fuzzy mathematical proximity comparison method, and audio feature extraction as research methods, a model analysis research model based on different key touch methods in piano playing was established, and the relationship between piano key touch and brain sound image was studied. The experimental results of this paper show that after using the research method in this paper, the error rate of the data is controlled within 5%. Compared with previous research methods, the error rate is lower and has certain practical value.


---

## Body

## 1. Introduction

If you want to study piano sound, you should first understand the working mechanism of piano music and grasp the influence and function of each part of piano on music sound from the perspective of acoustic composition. The sound of the piano has gone through very complicated steps from the beginning. The excitation of the sound mainly depends on the percussion system, which directly determines the volume and tone of the piano music. Without further research, it is difficult to know how the system works. Unlike other stringed instruments, the percussion part of the piano is completely wrapped in the piano shell. Some people think they can play the piano, but they do not know where the piano sound comes from. If we want to understand the relationship between keys and sounds, we must have a full understanding of the mechanism of piano sound. In order to pursue a “pleasant” sound, one must also master the acoustic components of the piano.After the development of science and technology in the nineteenth century, visual arts and recording technology were applied in the field of music research for the first time, and these researches even became a truly independent field. In the historical development of music visualization, piano music has played an important role. The purpose of this paper is to analyze and demonstrate the different keying modes and brain-sound image establishment modes in piano performance, in order to make certain contributions in the field of piano music.The current vowel neural coding models are usually based on the linear description of the auditory edge. These models rely on the auditory nerve firing rate or the phase locking of the temporal fine structure. However, the discharge rate and phase lock will be saturated at medium and high sound levels, and at medium and high frequencies, the phase lock of the central nervous system will be reduced. The fact that speech intelligibility is robust over a wide range of sound levels is a problem for codes that deteriorate as the sound level increases. In addition, a successful neural coding must function in the background noise at a level that the listener can tolerate. Carney et al.’s model solves these problems and combines several key response characteristics of the nonlinear auditory periphery, including saturation, synchronization capture, and phase locking of fine structure and envelope time characteristics. The model also includes the characteristics of the auditory midbrain, where the discharge rate is adjusted according to the amplitude fluctuation rate. Nonlinear peripheral response characteristics have formed a contrast in the amplitude of low-frequency nerve rate fluctuations in the entire population [1]. In the arbitration between the representation theory of perception and the relational theory, the perceptual illusion that a subject’s perceptual experience deviates from the real existence of the world is an important battlefield. However, this debate is mainly about visual perception. When attempting to extend the argument to hearing, it is appropriate to first consider a key case considered to be an auditory illusion. Judge considers the phenomenon of “lack of fundamentals” and examines a concept often used by representatives to explain it, namely, perceptual inference. Although perceptual reasoning is often used by reasoners as an explanatory concept, its concept is somewhat opaque. Here, Judge developed a “job description” for perceptual reasoning, including following the rules [2]. The characteristics of the melody have nothing to do with surface characteristics, such as keys (pitch), speed (speed), and timbre (instrument). Schellenberg and Habashi studied the memory duration of a melody (tune) and whether this memory is affected by changes in the main key, rhythm, or timbre. After listening to a previously unfamiliar melody twice, the participants rated the same (old) melody and the same number of new melody. In the experiment, half of the old melody was replaced by six semitones or moved at a speed of sixty-four times per minute. Half of the old melody has changed in tone (the piano changes to a saxophone, and vice versa) [3]. Nonmusicians remember that vocal melody is better than instrumental melody. If more exposure to sound helps to produce these effects, then long-term experience with musical instrument sounds should produce instrument-specific advantages. Here, Weiss et al. evaluate this hypothesis by comparing pianists with other musicians and nonmusicians. Weiss et al. also evaluated the possibility of absolute pitch (AP) affecting melodic memory, which involves special memory of isolated pitch. Participants heard 24 melodies played by four tones (vocal, piano, banjo, and xylophone) and then asked them to distinguish the melodies previously heard from the 24 novel melodies presented in the same tone. Musicians performed better than nonmusicians, but the two groups showed considerable memory advantages in vocal melody. In addition, the performance of pianists on the melody played on the piano is not better than other instruments, and AP musicians perform the same as non-AP musicians [4].Zhang et al.’s research shows that different dimensions of sound, such as pitch and timbre, cause different patterns of neural activity. Here, Zhang et al. reports event-related potential (ERP) and functional magnetic resonance imaging (fMRI) data on a selective music and sound listening task. The subjects of the study are normal adults. Stimulus is a musical tone with contrasting 4 tones (piano and violin) and tones (high and low). The duration and intensity of the stimulation are standardized. Under the condition of timbre, the listener needs to specify the type of instrument. Under pitch conditions, the listener needs to indicate the pitch level. Although these two conditions have reached the highest accuracy, the pitch condition requires a longer response time [5]. In Anderson and Strongs’s experiment, the synthetic piano tones were produced based on the recording of an upright piano. One set of tones has inharmonic parts that match the partial frequencies of piano tones, while the other set of tones consists of harmonic parts. The audience compared the synthesized, nonharmonic piano tones with a set of synthesized, and harmonic piano tones with different fundamental frequencies. They were asked to choose the harmonics whose tone best matched the inharmonic tones. [6].Based on the research results at home and abroad, this paper uses comparative analysis method, fuzzy mathematics closeness comparison method, and audio feature extraction as research methods to establish a research model. During the research process, this article mainly focuses on analyzing the strength, speed, and angle and explained it through typical musical examples and studied the relationship between piano key touch and the establishment of acoustic images in the brain [7].


## 2. Research Method of Establishing Model of Sound Image in Brain

### 2.1. The Closeness Comparison Method of Fuzzy Mathematics

Fuzzy sets are relative to ordinary sets. In the general set theory, the relationship between the subordination of elementX and set a is absolute. X belongs to a or X does not belong to a. This is a kind of binary logic. In fuzzy sets, the membership relationship between element X and set a is not only whether there is a binary relationship [8, 9]. The membership relationship between X and a can be measured and expressed by a function called membership.Assuming that all the objects to be discussed are called universe, denoted byX, it is the fuzzy set a membership function μax of universe x=x, which can reflect the membership degree of any element X as a in X. The value range of μax is 0,1. The larger the value, the higher the degree of subordination of X. On the contrary, the smaller the value of μax, the lower the degree of subordination of X [10, 11].Assuming that there are n fuzzy setsA1, A2, and a in the universe x, which represent n modes or categories, respectively, the criterion for any element x to belong to a fuzzy set can be expressed as
(1)AX=maxυA1X,υA2X.In other words,X belongs to a category with a large membership function. The above formula shows that the key to classification with fuzzy theory is the determination of the membership function. There is no uniform rule on how to reasonably determine the function of membership. In practical applications, the membership function is mainly determined by people’s empirical knowledge of classification. With the above knowledge, we can formally express the image as a fuzzy set [12, 13].A regular imagex with a size of M×n is represented as a two-dimensional matrix, and the pixel is located at I,J,I=0, m−1, j=0, n−1I,J,I=0, using Xij means [14]. Therefore, for an n-bit grayscale image, the range of values that Xij can use is from 0 to L−1, where l=2n, which is mathematically expressed as
(2)X=U∗Uxijxij.The definition of blurred image includes the gray valueIXJij and its related membership value μXij [15, 16]. The member value μXij indicates that the pixel at the position I,J belongs to the range of this category and has specific attributes. Therefore, in the fuzzy set theory, the image x with M×N size and l-level gray can be regarded as the display of a fuzzy single-element set, and each single-element set is related to the membership value or function [17, 18].In order to better analyze the image, people need to perform some processing on the original image so that the processed image can meet the specific application requirements. This image enhancement method has the characteristics of image processing, which can meet some people’s needs for image processing without being affected by space [19, 20]. Generally, image enhancement technologies can be divided into two categories based on image and algorithm characteristics: spatial domain image enhancement methods and frequency domain image enhancement methods.The spatial domain image enhancement method is based on the direct processing of image pixels. The main content includes basic gray scale transformation (image inversion, logarithmic transformation, power transformation, and piecewise linear function transformation), histogram processing (histogram equalization, histogram matching), arithmetic/logic enhancement (image subtraction, image averaging)), spatial smoothing filtering, in the frequency domain image enhancement method, Fourier transform, or wavelet transform is used to introduce the image into the frequency domain, and frequency domain enhancement operations (such as low-pass filtering, high-pass filtering, and band-pass) are performed [21, 22]. Then band-stop filtering and homomorphic filtering are performed on the transformed spectral components, and then, the desired image is obtained through inverse Fourier transform.


### 2.2. Audio Feature Extraction

Sound is a sound wave produced by the vibration of an object. The auditory organs of humans or other animals perceive it through the transmission medium. Sounds with a vibration frequency between 20 Hz and 20000 Hz can be recognized by the human ear. Among them, the most sensitive sound is 1000 Hz~3000 Hz, the sound below 20 Hz is called infrasonic wave, and the sound above 20 Hz is called infrasonic wave [23–25]. 20000 Hz is called ultrasound.In real life, sound exists in the form of analog signals, and three steps of “sampling,” “quantization,” and “coding” are required to express the sound as a digital signal form that can be processed by a computer.Sound sampling uses an “analog/digital” converter to obtain the vibration intensity of the sound analog signal according to a certain sampling frequency and then convert the vibration intensity to a voltage level, thereby realizing the conversion from analog signal to digital signal.. The Nyquist sampling law states that in order to ensure that the sampled digital signal can recover the analog signal without distortion, and the sampling frequency should not be less than 2 times the highest frequency in the analog signal spectrum. It can be seen that the higher the sampling frequency, the smaller the sound distortion and the larger the amount of data. Therefore, the specific sampling rate needs to be determined according to the actual application [26, 27].To extract the frequency domain features, first perform the fast Fourier transform to obtain the STFT energy spectrum and STFT power spectrum and then calculate the following features on the spectrum.aktm represents the amplitude of the spectrum obtained by STFT conversion, and pktm represents the normalized value of the spectrum obtained by STFT conversion. The energy of the spectrum (FrameEnergy) is the sum of the square of the amplitude ak in the time tm after Fourier transform, as shown in the following formula. In the experiment, extract features from the STFT energy spectrum and STFT power spectrum of all frames of the signal and find their mean and variance.
(3)Ettm=∑kak2tm.The strings of many stringed instruments, including the piano, have a certain stiffness. When the string vibrates, part of the restoring force comes from the stiffness of the string itself. Therefore, with the increase of the number of harmonics in the spectrum, the frequency interval of harmonics gradually increases, that is, disharmony appears. In this case, the harmonic frequency can be approximately expressed as(4)fm=m×f01+m2−1B.Image preprocessing usually starts with image preprocessing. Image restoration through filtering is an important content in the field of image preprocessing. Image filtering is the most basic and important research content in computer vision. It is a key technology for the success of boundary extraction, image analysis, image understanding, and image description, because the quality of image filtering will directly affect subsequent processing. Image filtering technology is just to design an appropriate filtering algorithm for noise-polluted image signals, so that the filtered image signal can best approximate the original image signal. After decades of research and exploration, domestic and foreign scholars have proposed many successful image filtering algorithms. Image filtering is to filter out image noise and protect image details as much as possible [28].


### 2.3. Comparative Analysis Method

The content of image saliency detection method based on multiscale local feature and global feature difference is as follows:(1)
Preliminary detection of image saliency, because the proposed image saliency detection method based on the difference between multiscale image local features and global features is based on both the image and the block to calculate the image saliency, so the image is. It is divided into uniform blocks at multiple scales, and then, the local color features of each block are extracted. Finally, the global color difference of each block is calculated according to the spatial information between the blocks, and a preliminary saliency map is generated(2)
This paper analyzes the characteristics of the saliency maps initially detected at each scale. Since the image saliency detection method that gives the difference between multiscale local features and global features is performed at multiple scales, the saliency map at each scale can reflect the importance of the image from all aspects, so each saliency map scale is compared and analyzed(3)
According to the characteristics of the saliency map under each scale, divide the saliency map under each scale to enhance each other, thereby obtaining the final saliency mapIn the field of image salient area detection, there are some classic methods, among which the IT method that imitates the human visual mechanism is the first to appear, which the beginning of image salient area detection is. Since then, a variety of salient area detection models have appeared, including the area-based RC method for image salient area detection. These two models not only represent the two directions of image saliency detection but also reflect the development of image saliency detection. At the same time, in the following experimental chapters, these two methods are used as a comparison method, and the detection effect of the method is compared. This section describes and analyzes two classic and representative saliency models in order to describe image saliency area detection more specifically and vividly and deepen the understanding of the role of visual features in saliency area detection.


## 2.1. The Closeness Comparison Method of Fuzzy Mathematics

Fuzzy sets are relative to ordinary sets. In the general set theory, the relationship between the subordination of elementX and set a is absolute. X belongs to a or X does not belong to a. This is a kind of binary logic. In fuzzy sets, the membership relationship between element X and set a is not only whether there is a binary relationship [8, 9]. The membership relationship between X and a can be measured and expressed by a function called membership.Assuming that all the objects to be discussed are called universe, denoted byX, it is the fuzzy set a membership function μax of universe x=x, which can reflect the membership degree of any element X as a in X. The value range of μax is 0,1. The larger the value, the higher the degree of subordination of X. On the contrary, the smaller the value of μax, the lower the degree of subordination of X [10, 11].Assuming that there are n fuzzy setsA1, A2, and a in the universe x, which represent n modes or categories, respectively, the criterion for any element x to belong to a fuzzy set can be expressed as
(1)AX=maxυA1X,υA2X.In other words,X belongs to a category with a large membership function. The above formula shows that the key to classification with fuzzy theory is the determination of the membership function. There is no uniform rule on how to reasonably determine the function of membership. In practical applications, the membership function is mainly determined by people’s empirical knowledge of classification. With the above knowledge, we can formally express the image as a fuzzy set [12, 13].A regular imagex with a size of M×n is represented as a two-dimensional matrix, and the pixel is located at I,J,I=0, m−1, j=0, n−1I,J,I=0, using Xij means [14]. Therefore, for an n-bit grayscale image, the range of values that Xij can use is from 0 to L−1, where l=2n, which is mathematically expressed as
(2)X=U∗Uxijxij.The definition of blurred image includes the gray valueIXJij and its related membership value μXij [15, 16]. The member value μXij indicates that the pixel at the position I,J belongs to the range of this category and has specific attributes. Therefore, in the fuzzy set theory, the image x with M×N size and l-level gray can be regarded as the display of a fuzzy single-element set, and each single-element set is related to the membership value or function [17, 18].In order to better analyze the image, people need to perform some processing on the original image so that the processed image can meet the specific application requirements. This image enhancement method has the characteristics of image processing, which can meet some people’s needs for image processing without being affected by space [19, 20]. Generally, image enhancement technologies can be divided into two categories based on image and algorithm characteristics: spatial domain image enhancement methods and frequency domain image enhancement methods.The spatial domain image enhancement method is based on the direct processing of image pixels. The main content includes basic gray scale transformation (image inversion, logarithmic transformation, power transformation, and piecewise linear function transformation), histogram processing (histogram equalization, histogram matching), arithmetic/logic enhancement (image subtraction, image averaging)), spatial smoothing filtering, in the frequency domain image enhancement method, Fourier transform, or wavelet transform is used to introduce the image into the frequency domain, and frequency domain enhancement operations (such as low-pass filtering, high-pass filtering, and band-pass) are performed [21, 22]. Then band-stop filtering and homomorphic filtering are performed on the transformed spectral components, and then, the desired image is obtained through inverse Fourier transform.


## 2.2. Audio Feature Extraction

Sound is a sound wave produced by the vibration of an object. The auditory organs of humans or other animals perceive it through the transmission medium. Sounds with a vibration frequency between 20 Hz and 20000 Hz can be recognized by the human ear. Among them, the most sensitive sound is 1000 Hz~3000 Hz, the sound below 20 Hz is called infrasonic wave, and the sound above 20 Hz is called infrasonic wave [23–25]. 20000 Hz is called ultrasound.In real life, sound exists in the form of analog signals, and three steps of “sampling,” “quantization,” and “coding” are required to express the sound as a digital signal form that can be processed by a computer.Sound sampling uses an “analog/digital” converter to obtain the vibration intensity of the sound analog signal according to a certain sampling frequency and then convert the vibration intensity to a voltage level, thereby realizing the conversion from analog signal to digital signal.. The Nyquist sampling law states that in order to ensure that the sampled digital signal can recover the analog signal without distortion, and the sampling frequency should not be less than 2 times the highest frequency in the analog signal spectrum. It can be seen that the higher the sampling frequency, the smaller the sound distortion and the larger the amount of data. Therefore, the specific sampling rate needs to be determined according to the actual application [26, 27].To extract the frequency domain features, first perform the fast Fourier transform to obtain the STFT energy spectrum and STFT power spectrum and then calculate the following features on the spectrum.aktm represents the amplitude of the spectrum obtained by STFT conversion, and pktm represents the normalized value of the spectrum obtained by STFT conversion. The energy of the spectrum (FrameEnergy) is the sum of the square of the amplitude ak in the time tm after Fourier transform, as shown in the following formula. In the experiment, extract features from the STFT energy spectrum and STFT power spectrum of all frames of the signal and find their mean and variance.
(3)Ettm=∑kak2tm.The strings of many stringed instruments, including the piano, have a certain stiffness. When the string vibrates, part of the restoring force comes from the stiffness of the string itself. Therefore, with the increase of the number of harmonics in the spectrum, the frequency interval of harmonics gradually increases, that is, disharmony appears. In this case, the harmonic frequency can be approximately expressed as(4)fm=m×f01+m2−1B.Image preprocessing usually starts with image preprocessing. Image restoration through filtering is an important content in the field of image preprocessing. Image filtering is the most basic and important research content in computer vision. It is a key technology for the success of boundary extraction, image analysis, image understanding, and image description, because the quality of image filtering will directly affect subsequent processing. Image filtering technology is just to design an appropriate filtering algorithm for noise-polluted image signals, so that the filtered image signal can best approximate the original image signal. After decades of research and exploration, domestic and foreign scholars have proposed many successful image filtering algorithms. Image filtering is to filter out image noise and protect image details as much as possible [28].


## 2.3. Comparative Analysis Method

The content of image saliency detection method based on multiscale local feature and global feature difference is as follows:(1)
Preliminary detection of image saliency, because the proposed image saliency detection method based on the difference between multiscale image local features and global features is based on both the image and the block to calculate the image saliency, so the image is. It is divided into uniform blocks at multiple scales, and then, the local color features of each block are extracted. Finally, the global color difference of each block is calculated according to the spatial information between the blocks, and a preliminary saliency map is generated(2)
This paper analyzes the characteristics of the saliency maps initially detected at each scale. Since the image saliency detection method that gives the difference between multiscale local features and global features is performed at multiple scales, the saliency map at each scale can reflect the importance of the image from all aspects, so each saliency map scale is compared and analyzed(3)
According to the characteristics of the saliency map under each scale, divide the saliency map under each scale to enhance each other, thereby obtaining the final saliency mapIn the field of image salient area detection, there are some classic methods, among which the IT method that imitates the human visual mechanism is the first to appear, which the beginning of image salient area detection is. Since then, a variety of salient area detection models have appeared, including the area-based RC method for image salient area detection. These two models not only represent the two directions of image saliency detection but also reflect the development of image saliency detection. At the same time, in the following experimental chapters, these two methods are used as a comparison method, and the detection effect of the method is compared. This section describes and analyzes two classic and representative saliency models in order to describe image saliency area detection more specifically and vividly and deepen the understanding of the role of visual features in saliency area detection.


## 3. Research Design

### 3.1. The Piano Frequency Design Produced by Different Touch Keys

After determining the selection, representation, and classification of eigenvalues, a dual neural network model composed of the Hopfield neural network and GRNN neural network is proposed. This section will improve the design idea of dual neural network based on this to evaluate piano keys of different touch keys.(1)
Sound signal extraction: first, in a low-noise environment, the hardware platform collects the signal of piano key sound samples(2)
Extraction of feature value data: extract the physical expression of acoustic features from sound samples to prevent singular samples from affecting the evaluation results. The eigenvalues are normalized before being input to the neural network(3)
Set the evaluation results of training samples: short, long, and piano key length when playing punctually, pressing the sustain pedal, playing vigorously, playing softly and pitch deviation. They have objective evaluation standards, but because the training samples are just a few pianos in the laboratory, it is impossible to determine their sound quality based on the relationship between thematic differences between them. For this characteristic value, the difference between them is determined according to their relationship, and then subjectively scored by professional musicians according to scientific standards, so as to establish a benchmark for these pianos in the entire piano category. Figure1 shows the design of piano keysFigure 1
Piano key design.


### 3.2. Auditory Image Feature Extraction

According to the scale space theory, the maximum or minimum image in the corresponding scale space is taken as the key point of image feature extraction. The scale space theory was first used in the field of computer vision to simulate the multiscale features of image data [29]. Then, the diffusion equation describes the filtering process in the scale space and proves that the Gaussian convolution kernel is the only transformation kernel that realizes the scale transformation. Therefore, the main idea of the scale space theory is to use Gaussian kernel function to transform the original image to obtain the scale space representation sequence under the multiscale image and then extract the scale space characteristics of these sequences. The two-dimensional Gaussian kernel function is defined as follows:
(5)Gx,y,σ=12πσe−x2+y2/2σ2.Among them,σLX,Y represents the scale space of the image IX,y under the σ scale, x,Y represents the coordinates of each pixel in the image, and σ is called the scale space factor. The smaller the σ value, the smaller the scale of the image, the less the details of the image are smoothed, and the clearer the details of the image. The large scale can represent the general features of the image, and the small scale can represent the detailed features of the image. Therefore, the determination of the size of the scale space factor is very important for the establishment of the scale space, and it is also helpful for the detection of key points.In traditional sparse representation, coefficient vectors and dictionaries are used to approximate the original signal. The coefficient vector records the weight of each atom in the dictionary when the signal is reconstructed, and the dictionary is a matrix in which each atom vector is arranged, so the reconstructed signal can be expressed as the product of the coefficient vector and the dictionary matrix. The so-called “sparseness” means that the number of nonzero elements in the coefficient vector is small, that is, only a single atom is used to reconstruct the original signal to represent the most important feature of the signal. The atoms in the dictionary are composed of different atomic parameters. In traditional methods, coefficient vectors are used to represent weights, and dictionaries are used to represent atoms. However, the atomic coefficient represents the strength of the corresponding atom and also reflects the ratio of the atomic parameters in the signal, so it is difficult to combine the two together through a single representation method. Traditional methods usually select parameters corresponding to nonzero coefficients and directly calculate the average of these parameters as features. However, this method ignores the difference between selected atoms, only calculates the average value, and ignores information about individual atoms.


## 3.1. The Piano Frequency Design Produced by Different Touch Keys

After determining the selection, representation, and classification of eigenvalues, a dual neural network model composed of the Hopfield neural network and GRNN neural network is proposed. This section will improve the design idea of dual neural network based on this to evaluate piano keys of different touch keys.(1)
Sound signal extraction: first, in a low-noise environment, the hardware platform collects the signal of piano key sound samples(2)
Extraction of feature value data: extract the physical expression of acoustic features from sound samples to prevent singular samples from affecting the evaluation results. The eigenvalues are normalized before being input to the neural network(3)
Set the evaluation results of training samples: short, long, and piano key length when playing punctually, pressing the sustain pedal, playing vigorously, playing softly and pitch deviation. They have objective evaluation standards, but because the training samples are just a few pianos in the laboratory, it is impossible to determine their sound quality based on the relationship between thematic differences between them. For this characteristic value, the difference between them is determined according to their relationship, and then subjectively scored by professional musicians according to scientific standards, so as to establish a benchmark for these pianos in the entire piano category. Figure1 shows the design of piano keysFigure 1
Piano key design.


## 3.2. Auditory Image Feature Extraction

According to the scale space theory, the maximum or minimum image in the corresponding scale space is taken as the key point of image feature extraction. The scale space theory was first used in the field of computer vision to simulate the multiscale features of image data [29]. Then, the diffusion equation describes the filtering process in the scale space and proves that the Gaussian convolution kernel is the only transformation kernel that realizes the scale transformation. Therefore, the main idea of the scale space theory is to use Gaussian kernel function to transform the original image to obtain the scale space representation sequence under the multiscale image and then extract the scale space characteristics of these sequences. The two-dimensional Gaussian kernel function is defined as follows:
(5)Gx,y,σ=12πσe−x2+y2/2σ2.Among them,σLX,Y represents the scale space of the image IX,y under the σ scale, x,Y represents the coordinates of each pixel in the image, and σ is called the scale space factor. The smaller the σ value, the smaller the scale of the image, the less the details of the image are smoothed, and the clearer the details of the image. The large scale can represent the general features of the image, and the small scale can represent the detailed features of the image. Therefore, the determination of the size of the scale space factor is very important for the establishment of the scale space, and it is also helpful for the detection of key points.In traditional sparse representation, coefficient vectors and dictionaries are used to approximate the original signal. The coefficient vector records the weight of each atom in the dictionary when the signal is reconstructed, and the dictionary is a matrix in which each atom vector is arranged, so the reconstructed signal can be expressed as the product of the coefficient vector and the dictionary matrix. The so-called “sparseness” means that the number of nonzero elements in the coefficient vector is small, that is, only a single atom is used to reconstruct the original signal to represent the most important feature of the signal. The atoms in the dictionary are composed of different atomic parameters. In traditional methods, coefficient vectors are used to represent weights, and dictionaries are used to represent atoms. However, the atomic coefficient represents the strength of the corresponding atom and also reflects the ratio of the atomic parameters in the signal, so it is difficult to combine the two together through a single representation method. Traditional methods usually select parameters corresponding to nonzero coefficients and directly calculate the average of these parameters as features. However, this method ignores the difference between selected atoms, only calculates the average value, and ignores information about individual atoms.


## 4. Simulation Result

As shown in Table1, according to the calculation method of fuzzy mathematical proximity, the sound intensities of 4 pianos are compared through experiments [30]. The following are the different changes of the sound intensity produced by different touch keys. Comparing the entire playback process of 4 sound sources is to compare the rate of change of the entire playback process of the two sound sources. The experimental data includes the pressure on the keys and the pressure on the strings. The pressure of the keys comes from the player’s fingers, and the intensity depends on the strength of the player’s fingers. The string pressure mainly comes from the force generated by hammering the string, and the force depends on the hammer force. Since 2000 data will be generated in one second during the measurement process, there will not be too many data listed here. Instead, it will be described by combining words, data, and statistical graphs. For piano sounds, depending on the piano, the performance from performance to attenuation is usually 5 seconds to 10 seconds, and the sampling frequency is 44100 Hz. After digitization, select the number of samples processed. If the one-to-one comparison is a large amount of data, the comparison workload will be huge. One of the most important problems is that the rate of change of sound intensity is fuzzy, so it is most appropriate to compare the mathematical proximity of sound intensity to the fuzzy.Table 1
Variation chart of piano sound intensity.
First sound intensity (db)706865697273Second sound intensity(db)706865647275Third sound intensity(db)667065727571Fourth sound intensity(db)627367727571In the research, this article only compares 4 kinds of recorded piano tones. Because the general sampling frequency is 44100 Hz, according to the Nyquist theory, the highest frequency recorded can reach 22050 Hz, which is not far from the hearing threshold. The highest note of the piano is C8, and the frequency is 4186.01 Hz. The fifth harmonic is 20930.05 Hz, which just exceeds the hearing threshold.Compared with the energy of the first harmonic (C8), this value is very small. Based on the principle of simplicity, this study does not filter or consider physiological factors, because this issue is complex and needs to be studied separately. As shown in Figure2, we will compare the length of the two piano sounds by comparing the intensity change patterns of the two piano sounds to observe the length of the sound. When the sound intensity of the first sound intensity and the second sound intensity changes, their rate of change is the same. Third sound intensity and fourth sound intensity have different sound intensities in the first half but the same rate of change, and the sound intensity in the second half is the same.Figure 2
The relationship between key touch mode and sound intensity.Because the fingers stick to the surface of the piano keys when the keys are pressed, that is, the fingers are in contact with the pressure sensor, and the player’s slight movement will cause the sensor’s recorded value to change. After repeated experiments, it is found that when the finger is weakly stuck to the surface of the sensor, the data received by the sensor is 0 or 0.5. Therefore, when analyzing the key data, start counting from the first occurrence of 1 value.As shown in Figure3, after a relatively slow upward trend, the force on the string suddenly increases from the third data, and then gradually decreases. After the 6th data, the reduction range becomes slower, and the maximum value in the 6th data is 3.4. Although the key touch time is very short, there is still a phenomenon of “multiple strokes.” In other words, when the hammer strikes the string once, it will not return immediately but will stay on the string for a while. In fact, the hammer and string touched many times at a time, and there is only one wave crest in Figure 2.Figure 3
The curve of the string force change.Touch the keys with different forces, and the strength of the keys is proportional to the strength of the strings. The smaller the keystroke, the smaller the string’s hitting power. Conversely, the greater the keystroke force, the greater the string strike force. Within the same playing intensity range, the relationship between keystroke intensity and string intensity is not proportional. For example, if you use MF twice, the first keystroke is slightly stronger than the second keystroke, but the first keystroke does not necessarily have to be stronger than the second keystroke. This is different from the player’s hand, wrist, and arm strength is related to speed. But when the keystroke reaches a certain intensity, the string intensity will not increase.As shown in Figure4, the harmonic energy at the initial stage of keystroke is usually higher than the harmonic energy at the initial stage of keystroke. In addition, the same initial harmonic energy analysis was performed on the four sets of data, and the conclusion basically conformed to the law. Among the 4 sets of data on the tone of piano keys with different forces, one set of “keystroke” energy range is greater than the “keystroke” energy range, that is, the minimum value of “keystroke” is smaller than the minimum value of “keystroke.”Figure 4
Analysis of the long-time frequency spectrum of a keystroke.To some extent, the change in loudness is consistent with the change in the overtone above the sound, indicating that the tone has indeed changed. However, since the number of harmonics and the energy of low harmonics are almost unchanged, the change in tone color is very weak. In the initial stage of the sound, the appearance time of the homophonic energy of the “key” and “keystroke” modes is different, which is mainly reflected in the different time series of the homophonic sequence of the two modes, and each homophony is different. The harmonic energy of the two touch modes is different.The analysis of piano sound signal should adopt frequency domain analysis method. A typical piano sound spectrum is shown in Figure5. The frequency spectrum of piano sound can also be divided into several representative parts to analyze its main characteristics. Due to the different characteristics of the piano sound signal in different frequency domains, the main characteristics of the piano sound excited by each piano key whose fundamental frequency is in a different frequency range are also different. After comparing the difference between the recorded spectra of each key, it is found that the first 12 data have a single law. In this critical area, as long as there is a difference between the two, it will be found that the high-order harmonic components excited by the first sound intensity and the second sound intensity are greater than the harmonic components of the other two groups, and there is no opposite situation. The difference between them is not only that the harmonic component of the former is greater than or greater than the harmonic component of the latter but also the opposite.Figure 5
Time domain signal envelope of piano sound.To some extent, the change in loudness is consistent with the change in the overtone above the sound, indicating that the tone has indeed changed. However, since the number of harmonics and the energy of low harmonics are almost unchanged, the change in tone color is very weak. In the initial stage of sound, the appearance time of the homophonic energy of the “keystroke” and “keystroke” modes is different, which is mainly reflected in the different time series of the homophonic sequence of the two modes, and each homophony is different. The harmonic energy of the two touch modes is different. Obviously, when the hammer completes a complete stroke, there will be “multiple hits.” The contact strength between the hammer and the strings did not gradually increase and then gradually disappeared, but it experienced several increases and decreases. The number of increases or decreases varies with the intensity of the performance. The phenomenon of “multiple strokes” is formed, which is very valuable for studying the relationship between vibration characteristics and timbre.


## 5. Conclusions

Based on the research results at home and abroad, this paper uses comparative analysis method, fuzzy mathematics closeness comparison method, and audio feature extraction as research methods to establish the relationship between the auditory sense and the acoustic image in the brain generated by different touch keys in piano performance. A model analysis research model was established. In the research process, this article mainly focuses on analyzing the strength, speed, and angle of the keys and explains them through typical music examples and discusses the relationship between the touch of piano keys and the establishment of acoustic images in the brain. The research results show that one of the decisive factors affecting the sound tone is whether the way of touching the keys is correct. On how to correctly play sounds of different intensities, we should work hard on the strength of the piano keys, using different intensities, different ways of applying force, and different ways of transmission, so as to play a beautiful sound.



---

*Source: 6713468-2022-04-11.xml*