# Automatic Evaluation System for Piano Performance Based on the Internet of Things Technology under the Background of Artificial Intelligence

Authors: Jianan Yu
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

Ubiquitous sensors cover many areas of modern society. As the sensor network matures, various applications based on the Internet of Things are setting off a new revolution in all aspects of social life. In order to in-depth study whether the Internet of Things technology can be used in the automatic evaluation of piano performance, this article uses artificial system comparison method, database establishment method, and model construction method to collect samples, analyze the automatic evaluation model, and streamline the algorithm, and based on these foundations, a practical automatic evaluation system for piano performance was created. However, the role of this article does not stop there. There are also a variety of algorithm-like models and the construction of technical models. First, the practicality of the created model is studied, and it is found that the traditional manual recognition rate is about 52%, while the recognition rate of the system in this paper is more than 90%, and the average recognition time of the system is 1.1 s. The start-up process and recognition process time of other systems are all no more than 6 s, indicating the superior performance of the system. On this basis, select the classic piano textbook: Thompson’s Simple Piano Tutorial. From it, select representative pieces as test samples. We can find that the overallF-measure value is above 90%, and the average F-measure value is 96.8%; the system performance test is good and can provide accurate evaluation results for piano learners. The results show that the number of identifications and missing numbers of the system are not much different from those of manual identification, which further proves its superiority. It is basically realized that starting from the Internet of Things technology, a system model that can automatically evaluate most piano repertoires has been designed.


---

## Body

## 1. Introduction

With such a fulfilling material life, the pursuit of spiritual food has become the yearning of most people. Modern education pays more and more attention to people’s ideological and moral quality, ability training, and personality development. Music education, as an important part of quality education, plays a very important role in improving personal cultivation and cultural quality and enhancing aesthetic ability. Of course, we cannot ignore the many shortcomings of traditional teaching methods, especially the inability to meet the needs of current science teaching. Therefore, the teaching platform built by the Internet is our current best choice, and it can also help us effectively get out of the existing predicament. As long as students can access the Internet, you can receive music education anytime and anywhere. With the rapid development of mobile Internet today, if we can introduce some of the advantages of mobile Internet to make up for some of the shortcomings of traditional music education, such an attempt will be very meaningful.Corresponding to this, in recent years, as an important part of the latest intelligent information technology, the Internet of Things has received extensive attention from everyone. The Internet of Things is the current advanced technology concept and application closest to this model, and it is the product of the world’s information industry tide. It replaces the revolutionary role of computers, the Internet, and mobile communication networks. The proposal of the concept of the Internet of Things technology crosses the fetters of the traditional network concept and introduces a new way of thinking. The intelligent analysis system established with reference to this framework can classify all kinds of effective piano playing information for evaluation and management. The use of this framework will improve the quality of the repertoire played by pianists and better adapt to the market. For listeners, it will get a better listening experience and increase the penetration rate. If this can be achieved, it is bound to cause a great wave in the music field.Piano music is a relatively elegant current art, and more and more listeners choose to join in. Based on this, it is very necessary to develop an evaluation and screening system. In fact, domestic and foreign scholars are already studying this topic. In 2021, the modules included in Sisavath’s design include Getaway Board module, node module, and App module. The system builds an Ethernet service controller through the LPC1769 embedded Ethernet module and DP83848 chip. The software is based on the UC/OS-II real-time operating system, equipped with UIP protocol tasks and other task modules, and is built into the Web server function. Although the configuration is quite complete, the practicability of the system is approximately equal to 0 [1]. In 2016, Hwang et al. designed and implemented a reliable message transmission system, using the MQTT protocol, to maintain the ordering of messages in the working environment. For simulation, they implemented a reliable message transmission system with Mosquitto MQTT message broker, and the simulation showed that the proposed system can enhance message transmission in the IOT environment. Despite the meticulousness, it is a pity that the stability of the system cannot be guaranteed [2]. In 2016, Gergelitsová introduced the design of a system for distributing and evaluating tasks in geometric areas. The goal of this design is to help teachers specify and test/verify the initial knowledge and ability levels required by students entering the relevant courses. In the past 4 years, it has been used by more than 100 teachers and 5,000 students in universities, middle schools, and primary schools. The feedback received from teachers confirmed the availability of the application and both teachers and students expressed positive acceptance. The system is very complete, but it deviates from the main point [3]. In 2016, Ji adopted the best dynamic resource allocation method of the Internet of Things (IoT) parking guidance system using the Q learning resource allocation model. Through simulation results, it is proved that the method can increase the total throughput, reduce the fines issued by the service-level agreement (SLA), and reduce the response time through dynamic users. Unfortunately, there are deviations in the simulation process [4]. In 2019, Bashir introduced a parking system that uses radio frequency identification (RFID) technology for entry and exit terminals. The research results show that the proposed RFID-based entrance/exit system is significantly better than the current system. Similarly, the verification results confirm the robustness of the simulation model. Therefore, it can be concluded that the RFID parking system will provide an effective solution for parking facilities. However, the system is not complete [5]. In 2019, the focus of Sharma’s research is to design a hybrid convolutional neural network (CNN) architecture by using the firefly optimization algorithm (FOA/FFA) to detect all human blood cells, all blood cells from human blood cells, or normal blood cells. When the proposed model was tested on different microscopic images, the evaluation parameters were calculated and compared with some other state-of-the-art methods; they obtained the best performance of the proposed method in terms of classification accuracy. It is not known that the result of changing the research object is still not valid [6]. In 2018, Watanabe proposed a dispatch and control system for automatic train operation (ATO) that can save energy. For the experiment on the track, the efficiency of the linear motor train was measured in the preliminary experiment and used to perform accurate numerical calculations. In summary, the numerical study showed that the energy efficiency was increased by 7.3%, and a plan for further experiments was determined. However, the author did not introduce the further results of the system [7].The innovations of this article are as follows: (1) Based on the three-tier architecture of the Internet of Things, a four-tier IoT intelligent system framework is proposed. Through this subdivided framework, it is possible to guide the establishment of an intelligent system for the Internet of Things. (2) The application of intelligent evaluation of the Internet of Things cannot be separated from the support of the cloud platform and the support of management software. And this article is a good combination of these two parts. (3) The system has a high degree of modularity, the structure of each module is clear, and the maintainability is strong. The back-end module adopts the mature and stable LNMP framework system, and the whole system adopts the design method of high cohesion and low coupling, and the abnormal conditions of the system are fully considered. Through the above work, it is basically realized and ensured that the system constructed from the theory can independently complete the identification, analysis, and evaluation of the piano repertoire, making the system more perfect.


## 2. Piano Performance Evaluation System Based on the Internet of Things Technology in the Context of Artificial Intelligence

### 2.1. Introduction to Piano Performance and Evaluation Module

Traditional piano music education needs professional piano training teachers to guide students face to face, listen carefully in the process of students playing the piano, and point out the errors and deficiencies in the process of students playing the piano from the perspective of teachers’ specialty. This process often needs teachers’ face-to-face, one-to-one guidance, and is usually completed in a very short piano class [8]. With the development of quality education, students’ learning needs in art are increasing, which makes the resources to play the role of piano performance evaluation very scarce. If an automatic evaluation system is developed through computer technology to evaluate and display the user’s performance, this will be a good aid to students and piano lovers, so that they can continue to practice and correct their performance without the company of a piano teacher [9].


#### 2.1.1. Audio Preprocessing

The analysis and processing of music signals are often carried out in the frequency domain. The input in this system is the recorded audio data played by the user, which is a continuous time domain signal. In order to analyze and evaluate the audio, it is necessary to preprocess the audio first, divide the audio data into frames, and then carry out fast Fourier transform processing.When processing audio signals, the continuous audio data is usually divided into frames, and the length of each frame is about tens of milliseconds. The steady-state signal characteristics obtained in this way are convenient for subsequent analysis [10].Then, the Fourier transform is usually used to convert the time domain signal to the frequency domain signal. Through the introduction of framing, it can be seen that the frequency domain analysis of the framing audio signal needs to be based on the discrete Fourier transform. For a sequence of lengtho, the DFT calculation formula and its inverse operation are shown as follows:(1)ah=wlsao=∑o=0o−1aof−k2ρ/oho0<h≤o−1,0others,(2)ao=uwlsah=1o∑h=0o−1ahfk2ρ/oho0<o≤o−1,oothers.Among them,ah and ao are, respectively, the sum of values. Because the time complexity of the WLS algorithm is too high, the optimized fast Fourier transform is generally used in implementation [11]. LLS is a fast calculation method of WLS, which reduces the time complexity of PO2 to POLOGO, and has been widely used in the field of signal processing.


#### 2.1.2. Multitone Detection Algorithm

In this system, a nonnegative matrix factorization algorithm based on the degree of dispersion is selected. The general principle of the algorithm is to update the degree of dispersion by loop iteration, and the iteration can be ended when the degree of dispersion is small enough [12]. The update calculation process of the dispersion degree in the iterative process is shown in formula (3). The d and g values will be updated in iteration. When the dispersion degree is sufficiently small, the d or g value obtained is the result of the calculation we need.(3)wmdg=∑l−1l∑o−1owmlodglo.Among them,wm/dg is the final calculation result, and the spectrum template of the piano sound is also obtained by nonnegative matrix decomposition, but the control input variables and output results are different [13]. The input audio frequency spectrum is the standard frequency spectrum data of a certain single tone, and then the coefficient matrix g is set to a fixed value corresponding to the single tone, and finally the spectrum template vector d can be calculated through nonnegative matrix decomposition. The process of obtaining the piano spectrum template through nonnegative matrix factorization is shown in Figure 1.Figure 1
Flowchart of piano spectrum template generation.As shown in Figure1, a specific framework diagram can give us an overall grasp of the system content. From the figure, it can be seen that if the spectrum template D is available, then the input audio spectrum data can be multitone detection. We can also draw a further subdivided framework diagram, as shown in Figure 2 [14].Figure 2
Flowchart of multitone detection algorithm.It can be seen from Figure2 that for a piece of piano audio frequency spectrum data of a specified length, the nonnegative matrix decomposition is performed on the preacquired spectrum template d to obtain the coefficient matrix g of the piano single tone, and the multitone detection algorithm is obtained by filtering g. The result: which notes appear in the input audio [15]. There are many ways to process the g value, which can be obtained through deep learning and other methods. In this system, artificial thresholds are used to filter, and the part of the coefficient matrix that exceeds the artificial threshold is selected as the final result. This method is simple and efficient. It is practical and has a good performance on the existing piano performance evaluation system.


#### 2.1.3. Score Tracking Algorithm

The core technology of the music score following system is to compare audio and music scores. The essence of the comparison is the similarity comparison between audio feature values and music score feature values. This system uses a relatively simple and efficient dynamic time warping algorithm. The principle ofD algorithm is summarized below [16].Given two discrete sequences, the sample sequencea and the sequence to be tested b define the Euclidean distance between the corresponding points of the two sequences as the similarity matrix, which can also be seen as the distance matrix w from another angle. The calculation formula of the matrix w is as follows:(4)wu,k=∑au,bk2.The core of theD algorithm is to pass a given distance matrix w, where wu,k is the distance between u and k.The process of calculating the cumulative sum of distances is the process of selecting paths, and finding the optimal path through comparison and analysis. The update process of the cumulative sum of distances is shown in the following formula:(5)wsdu,k=wu,k+maxwu−1,k,wu−1,k−1.In the formula,wsdu,k is equal to the mean value. In this system, the standard music score is first digitized, and the note sequence is regarded as the sample sequence a in the D algorithm.


### 2.2. Internet of Things Technology

In recent years, as an important part of the new generation of information technology, the Internet of Things has received extensive attention from the scientific and technological circles. The Internet of Things technology originated in the media field and is the third revolution in the information technology industry. The Internet of Things refers to the connection of any object to the network through information sensing equipment and the network. Objects exchange and exchange information through information media in accordance with the agreed agreement, so as to realize functions such as intelligent identification, positioning, tracking, and supervision. The Internet of Things is the current advanced technology concept and application closest to this model. It is the third wave of the world’s information industry, replacing computers. It is the third wave of the world’s information industry and has a revolutionary role in replacing computers, the Internet, and mobile communication networks. The proposal of the concept of the Internet of Things technology crosses the fetters of the traditional network concept and introduces a new way of thinking [17]. When the industry first proposed the concept of the Internet of Things in 1999, it mainly refers to the combination of various types of information perception transmission equipment, such as radio frequency identification devices, IR sensors, GPS positioning systems, ray instruments, and the Internet. But unlike the traditional Internet, H2T represents the connection between people using special devices and goods, thereby simplifying the connection of goods, while H2H is the interconnection between each other without relying on PC [18]. To some extent, the Internet of Things is to solve the problem of goods connection that the traditional Internet has not considered.The Internet of Things integrates various sensors with the existing Internet [19], and its purpose is to connect all goods to the Internet. The essence of the Internet of Things is M2M, that is, wireless data transmission from machine to machine. At present, the transmission protocols supporting M2M mainly include CDMA, GPRS, and WLAN [20]. The characteristics of the Internet of Things can be summarized in 4A, as shown in Table 1.Table 1
Features of the Internet of Things.
Transfer protocolFeaturesPracticality (%)Reliability (%)CDMAAB88.690.6GPRSAT96.196.6IEEEAW85.595.3WLANATH91.691.5The basic framework of the Internet of Things of the software system is shown in Figure3.Figure 3
Basic frame of the Internet of Things.It can be seen from Figure3 that it includes IoT middleware, radio frequency readers, RFID electronic tags, physical servers, object name resolution servers, and Internet network servers [21]. The most important part is middleware. Middleware is a bridge tool that connects the external resource software of the Internet of Things and the internal resources of the Internet of Things software system and network communication, and is a set of universal services. Middleware needs to consider the compatibility of access devices.In short, the Internet of Things is the transmission and control of information among things, people, and things. The application of the Internet of Things mainly has the following key technologies, such as RFID technology. Let me focus on the radio frequency identification technology. With the development of the Internet of Things, the method of information collection has also changed from manual collection to the current automated collection. The current Internet of Things perception technology is mainly RFID technology, that is, radio frequency identification technology. RFID tags are also a sensor technology. RFID technology is a comprehensive technology integrating radio frequency technology and embedded technology. RFID has broad application prospects in automatic identification and cargo logistics management. Under normal circumstances, a complete RFID system refers to a combination of a card reader and an electronic tag, which is generally referred to as a transponder, and a data management system. Therefore, the composition structure of the RFID system is shown in Figure4.Figure 4
The structure of the radio frequency identification system.As can be seen from Figure4, the working principle of RFID is that when the electronic tag enters the corresponding magnetic field, it receives a specific interrogation radio frequency signal from the reader. When the electronic tag senses this signal, it can be generated by its own coil. Data transmission technology, data source and data receiver through one or more data channels, or data transmission technology links, methods and equipment, jointly follow a communication protocol. The data transmission process is that DTE converts the text, image, or language information that people want to transmit into electrical signals in the device through electromechanical conversion, photoelectric conversion or sound-electric conversion man-machine interface, and then becomes a suitable channel through DCE. The transmitted signal is sent to the data transmission channel. The induced current obtains the corresponding energy, and finally transmits the identification object data previously stored in the tag chip to the reader; after the reader reads the information of the identification object, it performs related processing and decoding, and finally sends it to the reader. The data management system performs corresponding data processing [22]. Sensor technology is also a key technology in computer applications. Most computers process digital signals. Since the advent of computers, sensors must convert analog signals into digital signals before computers can process them.Because the sensor has dynamic characteristics such as frequency response and step response, as well as static characteristics such as drift, repeatability, accuracy, sensitivity, resolution, and linearity, changes and turbulence in external factors will inevitably cause the small stability of the sensor’s own characteristics. Therefore, it has a greater impact on its practical application. This requires us to specify the basic requirements of the sensor in the same situation according to the working principle and structure of the sensor, so as to optimize its performance parameters and indicators to the greatest extent, such as high sensitivity and anti-interference. Stability, linearity, easy adjustment, high precision, no hysteresis, long working life, repeatability, antiaging, high response rate, antienvironmental influence, interchangeability, low cost/wide measurement range/small size/light weight and high strength, etc.


### 2.3. Artificial Intelligence and Evaluation Algorithm

Artificial intelligence is a branch of computer science. Since the 1970s, it has been known as one of the world’s three cutting-edge technologies (space technology, energy technology, and artificial intelligence). Artificial intelligence technology is the study of computers used to simulate certain human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), mainly including the principles of computer realization of intelligence, manufacturing computers similar to human brain intelligence, so that computers can realize higher-level applications. The intelligence of the intelligent algorithm is reflected in the algorithm’s certain adaptability, which can continuously adjust the search strategy according to the algorithm in the search process, so as to better search. According to the different simulated natural phenomena, different search strategies are formed, and different intelligent optimization algorithms are formed. “Intelligent algorithm” refers to some relatively “novel” algorithms or theories often encountered in engineering practice, such as simulated annealing, genetic algorithm, tabu search, neural network, as well as cattle whisker search algorithm, sparrow search algorithm, etc. This type of intelligent algorithm is different from the general Turing machine-based precise calculation program, especially the artificial neural network. They are new interpretations of computer models, out of the circle of von Neumann machines. Computers designed according to computers have broad prospects for the development of this kind of thinking.


#### 2.3.1. Genetic Algorithm

Genetic algorithm is a bionic algorithm, and its basic principle is to simulate the evolutionary principle of survival of the fittest in natural biological populations. A solution of the optimization problem corresponds to an individual of a biological population, and a set of solutions corresponds to a population of organisms. The evolution of natural biological populations is simulated by selection, crossover, and mutation operators in genetic algorithms [23]. The recombination operator is the reason why the algorithm can sample key areas with exponentially increasing probability. The mutation operator enables the algorithm to cover the entire solution space and enables the algorithm to jump out of the local optimization solution.(6)cg+1,u=acrossagx,agy,u=1,…,o,bg+1,u=mutatecg+1,u,u=1,…,o.Among them,cg+1,u is the intermediate result of the crossover operator and bg+1,u is the result of the g + 1th sampling [24].The information collection model of genetic algorithm is a selection operator, which selects a certain number of individuals from the sampling results as the information structure through certain strategies.(7)ag+1,u=choosebg+1,u,u=1,…,o.Choose is the selection operator.The genetic algorithm generates new solutions in two steps, corresponding to the two operators: the crossover operator and the mutation operator, respectively. The crossover operator randomly selects the parent body and randomly selects the crossover position [25]. Therefore, each individual produced by the crossover operator has the same probability. Although in this crossover process, the same individuals will be produced. If we distinguish the creation gates, then the distribution of new solutions produced by the crossover operator can be regarded as a uniform distribution,(8)qa=1oacrossπ2∗qdac.In the formula,oacross is the total number of individuals that may be produced in the current population.The probability that the mutation operator produces a new solution is affected by the mutation probabilityqn, and the probability of mutation from a to b is(9)qnb,a=1−qno−gqng.In the formula,a and b have g different bits.Then, the probability that the genetic algorithm produces a new solution is(10)qb=∑a∈aacross1oacross1−qno−gqng.In the formula,aacross is the total number of individuals generated by the crossover operator, and g is the number of genes that are mutated.


#### 2.3.2. Particle Swarm Algorithm

Particle swarm algorithm is a swarm intelligence algorithm designed by imitating the principle of bird swarm foraging [26]. In the process of foraging for food, the flock of birds will automatically gather around the food source. In this process, each bird controls the speed and direction of flight through very simple principles.The position of the particles is transformed as follows:(11)qug+1=qug+z∗mug+∑gog.Among them,qug+1 represents the uth individual in the g + 1 generation, mug represents the speed of the uth individual in the gth generation, and z is the control parameter.When a new sampling point is obtained, the global information of the particle must be updated:(12)qgood=qug+1,to otheru∈1,owhenlqgood>lqug+1,qgood,another.Among them,qgood is a global information parameter. When a new sampling point is obtained, the individual information of the particle must be updated:(13)qugood=qug+1,whenlqugood>lqug+1qug,another,(14)mug+1=mug+zi∗qugood−qug+1+z2∗qgood−qug+1.In the formula,qugood represents the optimal position corresponding to the uth individual.The generation operator of the particle swarm algorithm is a typical incremental generation operator. According to formulas (13) and (14), the new solution can be simplified.(15)aug+1=aug+Δmug+θ∗Δaigood+π∗Δahgood.Among them,θ and π are uniformly distributed random variables; Δmug is the increment brought by aug+1. In the gth iteration, Δmug is a constant; Δaigood is the distance between the current particle and the optimal value of the current particle. Then, this distribution is the uniform distribution in a parallelogram area in the feasible solution domain determined by θ and π.The pure MC method is used to estimate the optimal solution probability, and the estimation is made by uniform sampling from the feasible solution area [27].Set up(16)qn=qla≤n+ABC¯.qn is a hypothetical value; then, the CMC estimator of l is(17)q^cmc=1o∑u=1oulau≤n.Among them,au is uniformly sampled from the feasible solution domain, and u is the indicator function. The relative estimation error in the formula increases with the speed of 1/oi as i decreases. When i is very small, the number of samples o needs to be greatly increased to reduce its estimation error.In order to solve the problem that the pure MC method requires a large amount of sampling, a very common method is to sample the key sampling method. The key sampling method can get better results while reducing the number of samples, and it has quite a wide range of applications in the field of statistics [28].For (18), the standard key sampling estimator is(18)i^it=1o∑u=1oqauhauulau≤n.Among them,qau is the uniform distribution as 1/IW, and HAu is the key sampling probability density. According to the optimal key sampling probability density,(19)h∗a||n=ula≤niw∗in.h∗ itself depends on in, so this key sampling probability density cannot be used directly. It can only be estimated through a series of techniques. The crossover method and the minimum variance method are parameter methods used to approximate the optimal key sampling probability density.


## 2.1. Introduction to Piano Performance and Evaluation Module

Traditional piano music education needs professional piano training teachers to guide students face to face, listen carefully in the process of students playing the piano, and point out the errors and deficiencies in the process of students playing the piano from the perspective of teachers’ specialty. This process often needs teachers’ face-to-face, one-to-one guidance, and is usually completed in a very short piano class [8]. With the development of quality education, students’ learning needs in art are increasing, which makes the resources to play the role of piano performance evaluation very scarce. If an automatic evaluation system is developed through computer technology to evaluate and display the user’s performance, this will be a good aid to students and piano lovers, so that they can continue to practice and correct their performance without the company of a piano teacher [9].


### 2.1.1. Audio Preprocessing

The analysis and processing of music signals are often carried out in the frequency domain. The input in this system is the recorded audio data played by the user, which is a continuous time domain signal. In order to analyze and evaluate the audio, it is necessary to preprocess the audio first, divide the audio data into frames, and then carry out fast Fourier transform processing.When processing audio signals, the continuous audio data is usually divided into frames, and the length of each frame is about tens of milliseconds. The steady-state signal characteristics obtained in this way are convenient for subsequent analysis [10].Then, the Fourier transform is usually used to convert the time domain signal to the frequency domain signal. Through the introduction of framing, it can be seen that the frequency domain analysis of the framing audio signal needs to be based on the discrete Fourier transform. For a sequence of lengtho, the DFT calculation formula and its inverse operation are shown as follows:(1)ah=wlsao=∑o=0o−1aof−k2ρ/oho0<h≤o−1,0others,(2)ao=uwlsah=1o∑h=0o−1ahfk2ρ/oho0<o≤o−1,oothers.Among them,ah and ao are, respectively, the sum of values. Because the time complexity of the WLS algorithm is too high, the optimized fast Fourier transform is generally used in implementation [11]. LLS is a fast calculation method of WLS, which reduces the time complexity of PO2 to POLOGO, and has been widely used in the field of signal processing.


### 2.1.2. Multitone Detection Algorithm

In this system, a nonnegative matrix factorization algorithm based on the degree of dispersion is selected. The general principle of the algorithm is to update the degree of dispersion by loop iteration, and the iteration can be ended when the degree of dispersion is small enough [12]. The update calculation process of the dispersion degree in the iterative process is shown in formula (3). The d and g values will be updated in iteration. When the dispersion degree is sufficiently small, the d or g value obtained is the result of the calculation we need.(3)wmdg=∑l−1l∑o−1owmlodglo.Among them,wm/dg is the final calculation result, and the spectrum template of the piano sound is also obtained by nonnegative matrix decomposition, but the control input variables and output results are different [13]. The input audio frequency spectrum is the standard frequency spectrum data of a certain single tone, and then the coefficient matrix g is set to a fixed value corresponding to the single tone, and finally the spectrum template vector d can be calculated through nonnegative matrix decomposition. The process of obtaining the piano spectrum template through nonnegative matrix factorization is shown in Figure 1.Figure 1
Flowchart of piano spectrum template generation.As shown in Figure1, a specific framework diagram can give us an overall grasp of the system content. From the figure, it can be seen that if the spectrum template D is available, then the input audio spectrum data can be multitone detection. We can also draw a further subdivided framework diagram, as shown in Figure 2 [14].Figure 2
Flowchart of multitone detection algorithm.It can be seen from Figure2 that for a piece of piano audio frequency spectrum data of a specified length, the nonnegative matrix decomposition is performed on the preacquired spectrum template d to obtain the coefficient matrix g of the piano single tone, and the multitone detection algorithm is obtained by filtering g. The result: which notes appear in the input audio [15]. There are many ways to process the g value, which can be obtained through deep learning and other methods. In this system, artificial thresholds are used to filter, and the part of the coefficient matrix that exceeds the artificial threshold is selected as the final result. This method is simple and efficient. It is practical and has a good performance on the existing piano performance evaluation system.


### 2.1.3. Score Tracking Algorithm

The core technology of the music score following system is to compare audio and music scores. The essence of the comparison is the similarity comparison between audio feature values and music score feature values. This system uses a relatively simple and efficient dynamic time warping algorithm. The principle ofD algorithm is summarized below [16].Given two discrete sequences, the sample sequencea and the sequence to be tested b define the Euclidean distance between the corresponding points of the two sequences as the similarity matrix, which can also be seen as the distance matrix w from another angle. The calculation formula of the matrix w is as follows:(4)wu,k=∑au,bk2.The core of theD algorithm is to pass a given distance matrix w, where wu,k is the distance between u and k.The process of calculating the cumulative sum of distances is the process of selecting paths, and finding the optimal path through comparison and analysis. The update process of the cumulative sum of distances is shown in the following formula:(5)wsdu,k=wu,k+maxwu−1,k,wu−1,k−1.In the formula,wsdu,k is equal to the mean value. In this system, the standard music score is first digitized, and the note sequence is regarded as the sample sequence a in the D algorithm.


## 2.1.1. Audio Preprocessing

The analysis and processing of music signals are often carried out in the frequency domain. The input in this system is the recorded audio data played by the user, which is a continuous time domain signal. In order to analyze and evaluate the audio, it is necessary to preprocess the audio first, divide the audio data into frames, and then carry out fast Fourier transform processing.When processing audio signals, the continuous audio data is usually divided into frames, and the length of each frame is about tens of milliseconds. The steady-state signal characteristics obtained in this way are convenient for subsequent analysis [10].Then, the Fourier transform is usually used to convert the time domain signal to the frequency domain signal. Through the introduction of framing, it can be seen that the frequency domain analysis of the framing audio signal needs to be based on the discrete Fourier transform. For a sequence of lengtho, the DFT calculation formula and its inverse operation are shown as follows:(1)ah=wlsao=∑o=0o−1aof−k2ρ/oho0<h≤o−1,0others,(2)ao=uwlsah=1o∑h=0o−1ahfk2ρ/oho0<o≤o−1,oothers.Among them,ah and ao are, respectively, the sum of values. Because the time complexity of the WLS algorithm is too high, the optimized fast Fourier transform is generally used in implementation [11]. LLS is a fast calculation method of WLS, which reduces the time complexity of PO2 to POLOGO, and has been widely used in the field of signal processing.


## 2.1.2. Multitone Detection Algorithm

In this system, a nonnegative matrix factorization algorithm based on the degree of dispersion is selected. The general principle of the algorithm is to update the degree of dispersion by loop iteration, and the iteration can be ended when the degree of dispersion is small enough [12]. The update calculation process of the dispersion degree in the iterative process is shown in formula (3). The d and g values will be updated in iteration. When the dispersion degree is sufficiently small, the d or g value obtained is the result of the calculation we need.(3)wmdg=∑l−1l∑o−1owmlodglo.Among them,wm/dg is the final calculation result, and the spectrum template of the piano sound is also obtained by nonnegative matrix decomposition, but the control input variables and output results are different [13]. The input audio frequency spectrum is the standard frequency spectrum data of a certain single tone, and then the coefficient matrix g is set to a fixed value corresponding to the single tone, and finally the spectrum template vector d can be calculated through nonnegative matrix decomposition. The process of obtaining the piano spectrum template through nonnegative matrix factorization is shown in Figure 1.Figure 1
Flowchart of piano spectrum template generation.As shown in Figure1, a specific framework diagram can give us an overall grasp of the system content. From the figure, it can be seen that if the spectrum template D is available, then the input audio spectrum data can be multitone detection. We can also draw a further subdivided framework diagram, as shown in Figure 2 [14].Figure 2
Flowchart of multitone detection algorithm.It can be seen from Figure2 that for a piece of piano audio frequency spectrum data of a specified length, the nonnegative matrix decomposition is performed on the preacquired spectrum template d to obtain the coefficient matrix g of the piano single tone, and the multitone detection algorithm is obtained by filtering g. The result: which notes appear in the input audio [15]. There are many ways to process the g value, which can be obtained through deep learning and other methods. In this system, artificial thresholds are used to filter, and the part of the coefficient matrix that exceeds the artificial threshold is selected as the final result. This method is simple and efficient. It is practical and has a good performance on the existing piano performance evaluation system.


## 2.1.3. Score Tracking Algorithm

The core technology of the music score following system is to compare audio and music scores. The essence of the comparison is the similarity comparison between audio feature values and music score feature values. This system uses a relatively simple and efficient dynamic time warping algorithm. The principle ofD algorithm is summarized below [16].Given two discrete sequences, the sample sequencea and the sequence to be tested b define the Euclidean distance between the corresponding points of the two sequences as the similarity matrix, which can also be seen as the distance matrix w from another angle. The calculation formula of the matrix w is as follows:(4)wu,k=∑au,bk2.The core of theD algorithm is to pass a given distance matrix w, where wu,k is the distance between u and k.The process of calculating the cumulative sum of distances is the process of selecting paths, and finding the optimal path through comparison and analysis. The update process of the cumulative sum of distances is shown in the following formula:(5)wsdu,k=wu,k+maxwu−1,k,wu−1,k−1.In the formula,wsdu,k is equal to the mean value. In this system, the standard music score is first digitized, and the note sequence is regarded as the sample sequence a in the D algorithm.


## 2.2. Internet of Things Technology

In recent years, as an important part of the new generation of information technology, the Internet of Things has received extensive attention from the scientific and technological circles. The Internet of Things technology originated in the media field and is the third revolution in the information technology industry. The Internet of Things refers to the connection of any object to the network through information sensing equipment and the network. Objects exchange and exchange information through information media in accordance with the agreed agreement, so as to realize functions such as intelligent identification, positioning, tracking, and supervision. The Internet of Things is the current advanced technology concept and application closest to this model. It is the third wave of the world’s information industry, replacing computers. It is the third wave of the world’s information industry and has a revolutionary role in replacing computers, the Internet, and mobile communication networks. The proposal of the concept of the Internet of Things technology crosses the fetters of the traditional network concept and introduces a new way of thinking [17]. When the industry first proposed the concept of the Internet of Things in 1999, it mainly refers to the combination of various types of information perception transmission equipment, such as radio frequency identification devices, IR sensors, GPS positioning systems, ray instruments, and the Internet. But unlike the traditional Internet, H2T represents the connection between people using special devices and goods, thereby simplifying the connection of goods, while H2H is the interconnection between each other without relying on PC [18]. To some extent, the Internet of Things is to solve the problem of goods connection that the traditional Internet has not considered.The Internet of Things integrates various sensors with the existing Internet [19], and its purpose is to connect all goods to the Internet. The essence of the Internet of Things is M2M, that is, wireless data transmission from machine to machine. At present, the transmission protocols supporting M2M mainly include CDMA, GPRS, and WLAN [20]. The characteristics of the Internet of Things can be summarized in 4A, as shown in Table 1.Table 1
Features of the Internet of Things.
Transfer protocolFeaturesPracticality (%)Reliability (%)CDMAAB88.690.6GPRSAT96.196.6IEEEAW85.595.3WLANATH91.691.5The basic framework of the Internet of Things of the software system is shown in Figure3.Figure 3
Basic frame of the Internet of Things.It can be seen from Figure3 that it includes IoT middleware, radio frequency readers, RFID electronic tags, physical servers, object name resolution servers, and Internet network servers [21]. The most important part is middleware. Middleware is a bridge tool that connects the external resource software of the Internet of Things and the internal resources of the Internet of Things software system and network communication, and is a set of universal services. Middleware needs to consider the compatibility of access devices.In short, the Internet of Things is the transmission and control of information among things, people, and things. The application of the Internet of Things mainly has the following key technologies, such as RFID technology. Let me focus on the radio frequency identification technology. With the development of the Internet of Things, the method of information collection has also changed from manual collection to the current automated collection. The current Internet of Things perception technology is mainly RFID technology, that is, radio frequency identification technology. RFID tags are also a sensor technology. RFID technology is a comprehensive technology integrating radio frequency technology and embedded technology. RFID has broad application prospects in automatic identification and cargo logistics management. Under normal circumstances, a complete RFID system refers to a combination of a card reader and an electronic tag, which is generally referred to as a transponder, and a data management system. Therefore, the composition structure of the RFID system is shown in Figure4.Figure 4
The structure of the radio frequency identification system.As can be seen from Figure4, the working principle of RFID is that when the electronic tag enters the corresponding magnetic field, it receives a specific interrogation radio frequency signal from the reader. When the electronic tag senses this signal, it can be generated by its own coil. Data transmission technology, data source and data receiver through one or more data channels, or data transmission technology links, methods and equipment, jointly follow a communication protocol. The data transmission process is that DTE converts the text, image, or language information that people want to transmit into electrical signals in the device through electromechanical conversion, photoelectric conversion or sound-electric conversion man-machine interface, and then becomes a suitable channel through DCE. The transmitted signal is sent to the data transmission channel. The induced current obtains the corresponding energy, and finally transmits the identification object data previously stored in the tag chip to the reader; after the reader reads the information of the identification object, it performs related processing and decoding, and finally sends it to the reader. The data management system performs corresponding data processing [22]. Sensor technology is also a key technology in computer applications. Most computers process digital signals. Since the advent of computers, sensors must convert analog signals into digital signals before computers can process them.Because the sensor has dynamic characteristics such as frequency response and step response, as well as static characteristics such as drift, repeatability, accuracy, sensitivity, resolution, and linearity, changes and turbulence in external factors will inevitably cause the small stability of the sensor’s own characteristics. Therefore, it has a greater impact on its practical application. This requires us to specify the basic requirements of the sensor in the same situation according to the working principle and structure of the sensor, so as to optimize its performance parameters and indicators to the greatest extent, such as high sensitivity and anti-interference. Stability, linearity, easy adjustment, high precision, no hysteresis, long working life, repeatability, antiaging, high response rate, antienvironmental influence, interchangeability, low cost/wide measurement range/small size/light weight and high strength, etc.


## 2.3. Artificial Intelligence and Evaluation Algorithm

Artificial intelligence is a branch of computer science. Since the 1970s, it has been known as one of the world’s three cutting-edge technologies (space technology, energy technology, and artificial intelligence). Artificial intelligence technology is the study of computers used to simulate certain human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), mainly including the principles of computer realization of intelligence, manufacturing computers similar to human brain intelligence, so that computers can realize higher-level applications. The intelligence of the intelligent algorithm is reflected in the algorithm’s certain adaptability, which can continuously adjust the search strategy according to the algorithm in the search process, so as to better search. According to the different simulated natural phenomena, different search strategies are formed, and different intelligent optimization algorithms are formed. “Intelligent algorithm” refers to some relatively “novel” algorithms or theories often encountered in engineering practice, such as simulated annealing, genetic algorithm, tabu search, neural network, as well as cattle whisker search algorithm, sparrow search algorithm, etc. This type of intelligent algorithm is different from the general Turing machine-based precise calculation program, especially the artificial neural network. They are new interpretations of computer models, out of the circle of von Neumann machines. Computers designed according to computers have broad prospects for the development of this kind of thinking.


### 2.3.1. Genetic Algorithm

Genetic algorithm is a bionic algorithm, and its basic principle is to simulate the evolutionary principle of survival of the fittest in natural biological populations. A solution of the optimization problem corresponds to an individual of a biological population, and a set of solutions corresponds to a population of organisms. The evolution of natural biological populations is simulated by selection, crossover, and mutation operators in genetic algorithms [23]. The recombination operator is the reason why the algorithm can sample key areas with exponentially increasing probability. The mutation operator enables the algorithm to cover the entire solution space and enables the algorithm to jump out of the local optimization solution.(6)cg+1,u=acrossagx,agy,u=1,…,o,bg+1,u=mutatecg+1,u,u=1,…,o.Among them,cg+1,u is the intermediate result of the crossover operator and bg+1,u is the result of the g + 1th sampling [24].The information collection model of genetic algorithm is a selection operator, which selects a certain number of individuals from the sampling results as the information structure through certain strategies.(7)ag+1,u=choosebg+1,u,u=1,…,o.Choose is the selection operator.The genetic algorithm generates new solutions in two steps, corresponding to the two operators: the crossover operator and the mutation operator, respectively. The crossover operator randomly selects the parent body and randomly selects the crossover position [25]. Therefore, each individual produced by the crossover operator has the same probability. Although in this crossover process, the same individuals will be produced. If we distinguish the creation gates, then the distribution of new solutions produced by the crossover operator can be regarded as a uniform distribution,(8)qa=1oacrossπ2∗qdac.In the formula,oacross is the total number of individuals that may be produced in the current population.The probability that the mutation operator produces a new solution is affected by the mutation probabilityqn, and the probability of mutation from a to b is(9)qnb,a=1−qno−gqng.In the formula,a and b have g different bits.Then, the probability that the genetic algorithm produces a new solution is(10)qb=∑a∈aacross1oacross1−qno−gqng.In the formula,aacross is the total number of individuals generated by the crossover operator, and g is the number of genes that are mutated.


### 2.3.2. Particle Swarm Algorithm

Particle swarm algorithm is a swarm intelligence algorithm designed by imitating the principle of bird swarm foraging [26]. In the process of foraging for food, the flock of birds will automatically gather around the food source. In this process, each bird controls the speed and direction of flight through very simple principles.The position of the particles is transformed as follows:(11)qug+1=qug+z∗mug+∑gog.Among them,qug+1 represents the uth individual in the g + 1 generation, mug represents the speed of the uth individual in the gth generation, and z is the control parameter.When a new sampling point is obtained, the global information of the particle must be updated:(12)qgood=qug+1,to otheru∈1,owhenlqgood>lqug+1,qgood,another.Among them,qgood is a global information parameter. When a new sampling point is obtained, the individual information of the particle must be updated:(13)qugood=qug+1,whenlqugood>lqug+1qug,another,(14)mug+1=mug+zi∗qugood−qug+1+z2∗qgood−qug+1.In the formula,qugood represents the optimal position corresponding to the uth individual.The generation operator of the particle swarm algorithm is a typical incremental generation operator. According to formulas (13) and (14), the new solution can be simplified.(15)aug+1=aug+Δmug+θ∗Δaigood+π∗Δahgood.Among them,θ and π are uniformly distributed random variables; Δmug is the increment brought by aug+1. In the gth iteration, Δmug is a constant; Δaigood is the distance between the current particle and the optimal value of the current particle. Then, this distribution is the uniform distribution in a parallelogram area in the feasible solution domain determined by θ and π.The pure MC method is used to estimate the optimal solution probability, and the estimation is made by uniform sampling from the feasible solution area [27].Set up(16)qn=qla≤n+ABC¯.qn is a hypothetical value; then, the CMC estimator of l is(17)q^cmc=1o∑u=1oulau≤n.Among them,au is uniformly sampled from the feasible solution domain, and u is the indicator function. The relative estimation error in the formula increases with the speed of 1/oi as i decreases. When i is very small, the number of samples o needs to be greatly increased to reduce its estimation error.In order to solve the problem that the pure MC method requires a large amount of sampling, a very common method is to sample the key sampling method. The key sampling method can get better results while reducing the number of samples, and it has quite a wide range of applications in the field of statistics [28].For (18), the standard key sampling estimator is(18)i^it=1o∑u=1oqauhauulau≤n.Among them,qau is the uniform distribution as 1/IW, and HAu is the key sampling probability density. According to the optimal key sampling probability density,(19)h∗a||n=ula≤niw∗in.h∗ itself depends on in, so this key sampling probability density cannot be used directly. It can only be estimated through a series of techniques. The crossover method and the minimum variance method are parameter methods used to approximate the optimal key sampling probability density.


## 2.3.1. Genetic Algorithm

Genetic algorithm is a bionic algorithm, and its basic principle is to simulate the evolutionary principle of survival of the fittest in natural biological populations. A solution of the optimization problem corresponds to an individual of a biological population, and a set of solutions corresponds to a population of organisms. The evolution of natural biological populations is simulated by selection, crossover, and mutation operators in genetic algorithms [23]. The recombination operator is the reason why the algorithm can sample key areas with exponentially increasing probability. The mutation operator enables the algorithm to cover the entire solution space and enables the algorithm to jump out of the local optimization solution.(6)cg+1,u=acrossagx,agy,u=1,…,o,bg+1,u=mutatecg+1,u,u=1,…,o.Among them,cg+1,u is the intermediate result of the crossover operator and bg+1,u is the result of the g + 1th sampling [24].The information collection model of genetic algorithm is a selection operator, which selects a certain number of individuals from the sampling results as the information structure through certain strategies.(7)ag+1,u=choosebg+1,u,u=1,…,o.Choose is the selection operator.The genetic algorithm generates new solutions in two steps, corresponding to the two operators: the crossover operator and the mutation operator, respectively. The crossover operator randomly selects the parent body and randomly selects the crossover position [25]. Therefore, each individual produced by the crossover operator has the same probability. Although in this crossover process, the same individuals will be produced. If we distinguish the creation gates, then the distribution of new solutions produced by the crossover operator can be regarded as a uniform distribution,(8)qa=1oacrossπ2∗qdac.In the formula,oacross is the total number of individuals that may be produced in the current population.The probability that the mutation operator produces a new solution is affected by the mutation probabilityqn, and the probability of mutation from a to b is(9)qnb,a=1−qno−gqng.In the formula,a and b have g different bits.Then, the probability that the genetic algorithm produces a new solution is(10)qb=∑a∈aacross1oacross1−qno−gqng.In the formula,aacross is the total number of individuals generated by the crossover operator, and g is the number of genes that are mutated.


## 2.3.2. Particle Swarm Algorithm

Particle swarm algorithm is a swarm intelligence algorithm designed by imitating the principle of bird swarm foraging [26]. In the process of foraging for food, the flock of birds will automatically gather around the food source. In this process, each bird controls the speed and direction of flight through very simple principles.The position of the particles is transformed as follows:(11)qug+1=qug+z∗mug+∑gog.Among them,qug+1 represents the uth individual in the g + 1 generation, mug represents the speed of the uth individual in the gth generation, and z is the control parameter.When a new sampling point is obtained, the global information of the particle must be updated:(12)qgood=qug+1,to otheru∈1,owhenlqgood>lqug+1,qgood,another.Among them,qgood is a global information parameter. When a new sampling point is obtained, the individual information of the particle must be updated:(13)qugood=qug+1,whenlqugood>lqug+1qug,another,(14)mug+1=mug+zi∗qugood−qug+1+z2∗qgood−qug+1.In the formula,qugood represents the optimal position corresponding to the uth individual.The generation operator of the particle swarm algorithm is a typical incremental generation operator. According to formulas (13) and (14), the new solution can be simplified.(15)aug+1=aug+Δmug+θ∗Δaigood+π∗Δahgood.Among them,θ and π are uniformly distributed random variables; Δmug is the increment brought by aug+1. In the gth iteration, Δmug is a constant; Δaigood is the distance between the current particle and the optimal value of the current particle. Then, this distribution is the uniform distribution in a parallelogram area in the feasible solution domain determined by θ and π.The pure MC method is used to estimate the optimal solution probability, and the estimation is made by uniform sampling from the feasible solution area [27].Set up(16)qn=qla≤n+ABC¯.qn is a hypothetical value; then, the CMC estimator of l is(17)q^cmc=1o∑u=1oulau≤n.Among them,au is uniformly sampled from the feasible solution domain, and u is the indicator function. The relative estimation error in the formula increases with the speed of 1/oi as i decreases. When i is very small, the number of samples o needs to be greatly increased to reduce its estimation error.In order to solve the problem that the pure MC method requires a large amount of sampling, a very common method is to sample the key sampling method. The key sampling method can get better results while reducing the number of samples, and it has quite a wide range of applications in the field of statistics [28].For (18), the standard key sampling estimator is(18)i^it=1o∑u=1oqauhauulau≤n.Among them,qau is the uniform distribution as 1/IW, and HAu is the key sampling probability density. According to the optimal key sampling probability density,(19)h∗a||n=ula≤niw∗in.h∗ itself depends on in, so this key sampling probability density cannot be used directly. It can only be estimated through a series of techniques. The crossover method and the minimum variance method are parameter methods used to approximate the optimal key sampling probability density.


## 3. Experiments and Conclusions of the Design and Implementation Method of the Piano Performance Evaluation System Based on the Internet of Things Technology in the Context of Artificial Intelligence

### 3.1. Demand Analysis

The system implemented in this topic is mainly aimed at piano enthusiasts. When using piano auxiliary tools, users often need to check the score, understand the performance evaluation results, and save performance records. Therefore, the main function of this system is to display electronic music scores; record and upload when users play music scores, and finally give an intuitive visual performance evaluation result, save user performance history. According to the abovementioned user demand analysis, the specific overall system demand is determined as shown in Figure5.Figure 5
Analysis of overall system requirements.From Figure5, combining the abovementioned user demand analysis and the overall system demand analysis diagram in the figure, it can be seen that the system has six aspects of demand. The overall system demand is embodied through the user demand analysis, which is mainly divided into the following 6 modules: user registration and login, add/delete scores, view the content of scores, upload and upload performance recordings, obtain performance evaluation results, and view historical performance records.


### 3.2. Database Design

It can be seen from the above system module design that each module in this system has corresponding files and data, which need to be stored in the database for easy query and use. At the same time, there is a strong correlation between the data between the modules in this system, so the relational database MySQL is used as the database of the system. The main data includes the user’s login information, the score file and its corresponding series of pictures and data files, the recording file uploaded by the user and the corresponding evaluation data, etc. The following focuses on the design of the core data table in this system, and the functions of the entire system can be completed through the association between the data tables. There are four main data tables in this system, namely, user table, music score table, user login table, and performance table, which will be introduced in detail below.The user’s nickname and the user’s identifier are stored in the user table. The specific field information in the table is shown in Table2.Table 2
User field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>12 KEY95.6Open IDUser IDVAR (250)>10 KEY99.5UserUser’s nicknameVAR (250)Chinese (1)92.5The specific field information in the user login table is shown in Table3.Table 3
User login table field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>10 KEY92.53rd sessionSession informationVAR (250)>15 KEY96.5Open IDUser IDVAR (250)<5 KEY90.6The specific field information in the music score table is shown in Table4. The picture path and evaluation data path in the score table provide data for the performance evaluation module, which closely connects the entity of the score and the entity of performance.Table 4
Music xml field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>15 KEY93.4NameSheet music nameVAR (250)Chinese (1)97.1X-roadFile pathVAR (250)>16 KEY93.1P-roadPicture pathVAR (250)>20 KEY93.5D-roadFile pathVAR (250)>21 KEY92.8The specific field information in the performance table is shown in Table5.Table 5
Performance field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>20 KEY94.3User IDPerformance userINT (10)<15 KEY95.8MidRepertoireINT (10)<1 KEY96.1D-pathFile pathVAR (250)Chinese (1)90.3The data information in this system is stored in the database cloud server, and some files in the system are stored in the back-end server. In the back-end server, the file directory is divided according to the functional modules, and then the files are stored in the corresponding location.


### 3.3. System Implementation

The ultimate requirement of the piano evaluation system is not only to achieve automatic collection and evaluation of the music, but also to realize real-time online query evaluation results, so as to achieve the purpose of automatic evaluation of piano performance.In the B/S mode of the Web system, system users can access two parts: the front-end web page displayed on the client browser and the back-end business management program on the Web server side. The other key part of the system is the database. The system is not directly accessible.For users, the client browser is the presentation layer, the server-side program is the business logic layer, and the database system includes the data access layer and the database itself. These three layers constitute the three-tier architecture of the B/S model. The technical structure of this three-tier system structure not only has a clear division of labor and a clear process; it can not only reflect the internal requirements of the system very efficiently, but also achieve high-efficiency work between the data and files within the system and the ease of user operations.The overall technical architecture of the piano performance automatic evaluation system is a three-tier system structure consisting of a presentation layer, a business layer, and a data layer, as shown in Figure6.Figure 6
Architecture design of piano automatic evaluation system.It can be seen from Figure6 that the user can send a message requesting an operation to the Web server by accessing the client program of the presentation layer (IE, a web browser). The message requesting an operation includes information such as the operation requirement and the purpose of the operation. For example, in this system, trainees request to query the evaluation results of piano repertoire at a certain time through the client, and the operation message will be sent to the business layer. When the message sent by the client program is received, logic analysis, business process processing, etc., are performed. Finally, realize the data access and data management in the system. For example, after the data layer receives a request to access a database object, the system will query the evaluation information table of the database through the database server, and finally achieve the purpose of the user inquiring the evaluation results of the piano music at a certain time.


### 3.4. System Testing and Practical Analysis

In order to promote the efficiency and accuracy of system evaluation, this is the purpose of system testing. The function experiment is carried out by checking one by one and using appropriate methods. After testing, the basic components of each part are complete. Using an Android-based client to perform long-term control through a mobile data network, the system can respond to commands correctly and smoothly, and manage tasks in accordance with the sequence of operating instructions. The experimental results are shown in Figure7.Figure 7
Recognition rate of piano tracks.As shown in Figure7, the traditional manual recognition rate is about 52%, while the recognition rate of the system in this paper is more than 90%. We can also conclude that although the recognition accuracy of the algorithm is less than 100%, because of the superiority of the system, even if the recognition fails, it can be resolved. The existence of the database provides an excellent system for this system. Fault-tolerant space, because of so many advantages, we can say that the system design conforms to international standards.Test the performance indicators of the Internet of Things evaluation system, and get the data shown in Figure8.Figure 8
Performance test data.It can be seen from the performance test in Figure8 that the average recognition time of the system is 1.1 s, and the start-up process and recognition process time of other systems does not exceed 6s, indicating that the system has superior performance.


### 3.5. Data Analysis

At present, the mainstream multitone detection evaluation standard in the music retrieval field uses the note-levelF-measure. F-measure is the weighted harmonic average of precision and recall. It is a common evaluation standard in the field of information retrieval. The calculation process of F-measure uses intelligent calculation formulas (13) and (14). This system is oriented to piano learners, so in the performance test of this system, the classic piano textbook: Thompson’s Simple Piano Tutorial is selected in the test set, and representative repertoires are selected as the test samples, as shown in Figure 9.Figure 9
F-measure rate of systematic reviews.It can be seen from Figure9 that for the sample repertoire of Thompson’s Simple Piano Tutorial for piano learners, the overall F-measure value is above 90%, and the average F-measure value is 96.8%. The system performance test is good, and it can be provided for piano learners. Then study the identification numbers and missed numbers of these tracks to further reflect their performance, as shown in Figure 10.Figure 10
Number of identifications and number of omissions.As can be seen from Figure10, it can also be seen from the above four combination diagrams that compared with manual, the system does not have too many missing people, and combined with the other three combination diagrams, we can conclude that the system in this article is effective, while ensuring its recognition ability.


## 3.1. Demand Analysis

The system implemented in this topic is mainly aimed at piano enthusiasts. When using piano auxiliary tools, users often need to check the score, understand the performance evaluation results, and save performance records. Therefore, the main function of this system is to display electronic music scores; record and upload when users play music scores, and finally give an intuitive visual performance evaluation result, save user performance history. According to the abovementioned user demand analysis, the specific overall system demand is determined as shown in Figure5.Figure 5
Analysis of overall system requirements.From Figure5, combining the abovementioned user demand analysis and the overall system demand analysis diagram in the figure, it can be seen that the system has six aspects of demand. The overall system demand is embodied through the user demand analysis, which is mainly divided into the following 6 modules: user registration and login, add/delete scores, view the content of scores, upload and upload performance recordings, obtain performance evaluation results, and view historical performance records.


## 3.2. Database Design

It can be seen from the above system module design that each module in this system has corresponding files and data, which need to be stored in the database for easy query and use. At the same time, there is a strong correlation between the data between the modules in this system, so the relational database MySQL is used as the database of the system. The main data includes the user’s login information, the score file and its corresponding series of pictures and data files, the recording file uploaded by the user and the corresponding evaluation data, etc. The following focuses on the design of the core data table in this system, and the functions of the entire system can be completed through the association between the data tables. There are four main data tables in this system, namely, user table, music score table, user login table, and performance table, which will be introduced in detail below.The user’s nickname and the user’s identifier are stored in the user table. The specific field information in the table is shown in Table2.Table 2
User field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>12 KEY95.6Open IDUser IDVAR (250)>10 KEY99.5UserUser’s nicknameVAR (250)Chinese (1)92.5The specific field information in the user login table is shown in Table3.Table 3
User login table field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>10 KEY92.53rd sessionSession informationVAR (250)>15 KEY96.5Open IDUser IDVAR (250)<5 KEY90.6The specific field information in the music score table is shown in Table4. The picture path and evaluation data path in the score table provide data for the performance evaluation module, which closely connects the entity of the score and the entity of performance.Table 4
Music xml field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>15 KEY93.4NameSheet music nameVAR (250)Chinese (1)97.1X-roadFile pathVAR (250)>16 KEY93.1P-roadPicture pathVAR (250)>20 KEY93.5D-roadFile pathVAR (250)>21 KEY92.8The specific field information in the performance table is shown in Table5.Table 5
Performance field design table.
Field nameIntroduceTypesRemarkAccuracy (%)IDSelf-increment IDINT (10)>20 KEY94.3User IDPerformance userINT (10)<15 KEY95.8MidRepertoireINT (10)<1 KEY96.1D-pathFile pathVAR (250)Chinese (1)90.3The data information in this system is stored in the database cloud server, and some files in the system are stored in the back-end server. In the back-end server, the file directory is divided according to the functional modules, and then the files are stored in the corresponding location.


## 3.3. System Implementation

The ultimate requirement of the piano evaluation system is not only to achieve automatic collection and evaluation of the music, but also to realize real-time online query evaluation results, so as to achieve the purpose of automatic evaluation of piano performance.In the B/S mode of the Web system, system users can access two parts: the front-end web page displayed on the client browser and the back-end business management program on the Web server side. The other key part of the system is the database. The system is not directly accessible.For users, the client browser is the presentation layer, the server-side program is the business logic layer, and the database system includes the data access layer and the database itself. These three layers constitute the three-tier architecture of the B/S model. The technical structure of this three-tier system structure not only has a clear division of labor and a clear process; it can not only reflect the internal requirements of the system very efficiently, but also achieve high-efficiency work between the data and files within the system and the ease of user operations.The overall technical architecture of the piano performance automatic evaluation system is a three-tier system structure consisting of a presentation layer, a business layer, and a data layer, as shown in Figure6.Figure 6
Architecture design of piano automatic evaluation system.It can be seen from Figure6 that the user can send a message requesting an operation to the Web server by accessing the client program of the presentation layer (IE, a web browser). The message requesting an operation includes information such as the operation requirement and the purpose of the operation. For example, in this system, trainees request to query the evaluation results of piano repertoire at a certain time through the client, and the operation message will be sent to the business layer. When the message sent by the client program is received, logic analysis, business process processing, etc., are performed. Finally, realize the data access and data management in the system. For example, after the data layer receives a request to access a database object, the system will query the evaluation information table of the database through the database server, and finally achieve the purpose of the user inquiring the evaluation results of the piano music at a certain time.


## 3.4. System Testing and Practical Analysis

In order to promote the efficiency and accuracy of system evaluation, this is the purpose of system testing. The function experiment is carried out by checking one by one and using appropriate methods. After testing, the basic components of each part are complete. Using an Android-based client to perform long-term control through a mobile data network, the system can respond to commands correctly and smoothly, and manage tasks in accordance with the sequence of operating instructions. The experimental results are shown in Figure7.Figure 7
Recognition rate of piano tracks.As shown in Figure7, the traditional manual recognition rate is about 52%, while the recognition rate of the system in this paper is more than 90%. We can also conclude that although the recognition accuracy of the algorithm is less than 100%, because of the superiority of the system, even if the recognition fails, it can be resolved. The existence of the database provides an excellent system for this system. Fault-tolerant space, because of so many advantages, we can say that the system design conforms to international standards.Test the performance indicators of the Internet of Things evaluation system, and get the data shown in Figure8.Figure 8
Performance test data.It can be seen from the performance test in Figure8 that the average recognition time of the system is 1.1 s, and the start-up process and recognition process time of other systems does not exceed 6s, indicating that the system has superior performance.


## 3.5. Data Analysis

At present, the mainstream multitone detection evaluation standard in the music retrieval field uses the note-levelF-measure. F-measure is the weighted harmonic average of precision and recall. It is a common evaluation standard in the field of information retrieval. The calculation process of F-measure uses intelligent calculation formulas (13) and (14). This system is oriented to piano learners, so in the performance test of this system, the classic piano textbook: Thompson’s Simple Piano Tutorial is selected in the test set, and representative repertoires are selected as the test samples, as shown in Figure 9.Figure 9
F-measure rate of systematic reviews.It can be seen from Figure9 that for the sample repertoire of Thompson’s Simple Piano Tutorial for piano learners, the overall F-measure value is above 90%, and the average F-measure value is 96.8%. The system performance test is good, and it can be provided for piano learners. Then study the identification numbers and missed numbers of these tracks to further reflect their performance, as shown in Figure 10.Figure 10
Number of identifications and number of omissions.As can be seen from Figure10, it can also be seen from the above four combination diagrams that compared with manual, the system does not have too many missing people, and combined with the other three combination diagrams, we can conclude that the system in this article is effective, while ensuring its recognition ability.


## 4. Discussion

Here we have an overall grasp of the first half, which is a bit different from the main body of the article. Here, the second paragraph mainly discusses the algorithm, and the first paragraph mainly discusses the system. For details, refer to the following two paragraphs.This paper puts forward the requirements of the evaluation system from the aspects of system practicability, information sharing, terminal structure design, and performance requirements, and expounds the overall design idea and network architecture of the piano automatic evaluation system. The selection and design of hardware system and software system are introduced in detail, which paves the way for the realization of the following system. This paper describes the implementation process of hardware system and software system, respectively. The hardware part of the system mainly includes database, data transmission terminal, and controller. The system software mainly includes web software platform and mobile client. The construction of this project can inject new vitality into the evaluation and research of modern piano repertoire, promote technological progress, popularize the application of new technologies, increase communication benefits, improve the efficiency of music appreciation, promote the development speed of industrial development towards automation, informatization and intelligence, comprehensively improve the industry level, and provide a solid foundation for the expansion and strengthening of China’s piano industry. On the basis of building the demonstration base, it can provide an information technology application model for the development of surrounding music, which is conducive to the further development and technological upgrading of the national music industry.The intelligent optimization algorithm proposed in this paper is also unique. The intelligent optimization algorithm obtains the characteristic information of the optimization problem through sampling, and then uses the characteristic information to guide the search of the algorithm. Therefore, the process of optimizing the intelligent optimization algorithm can be regarded as a process of cognition of the characteristic information of the optimization problem. Similar to target recognition, the optimization process is equivalent to identifying the optimal solution in the feasible solution domain. The information fusion of target recognition has been widely used in various fields. The fusion of intelligent optimization algorithms is inspired by the theory of information fusion. Corresponding to the fusion of the three levels of information fusion, the decision-making layer, the feature layer, and the data layer, the algorithm fusion is also divided into three levels of fusion methods based on the algorithm fusion of results, information structure, and sample information.


## 5. Conclusions

According to the actual needs of the research, combined with the study and research of the relevant literature and products of the performance evaluation system, this paper realizes a portable and practical piano performance evaluation system. In order to achieve this goal, this paper uses the algorithm model combination method, parameter setting method, and system platform construction method to integrate the system and algorithm, so as to ensure the integrity of the system. Thus, an effective automatic evaluation system can be formed. The first step is to study whether the system can be applied in practice. We can find that in comparison with the traditional manual recognition rate, the recognition rate of the system in this paper is more than 90%, nearly half higher than that of the manual, and the system recognition time is also very fast. The average recognition time is 1.1 s, and the slowest is no more than 6S. Further, taking the traditional piano tracks as the recognition object, it is found that the averageF-measure value is 96.8%, and the lowest is more than 90%. Moreover, the follow-up research tells us that there is little difference between the number of recognition and omission of the system and the number of manual recognition, and the advantages of the system are at a glance. The shortcomings of this paper are as follows: firstly, as the most popular and widely used data communication technology, mobile communication technology has the characteristics of low cost, low power consumption, and easy to use. At the same time, it has also been criticized for the security of data transmission; secondly, as the carrier connecting intelligent devices, human-computer interaction, and various application services, the amount of data and access concurrency of intelligent system platform also need to be deeply analyzed. Therefore, in further research, we should not only pay attention to the efficiency of the system, but also pay attention to the evaluation of the data transmission path, so that our evaluation results can be transported more smoothly without jamming, and the one-time data transmission capacity can carry the situation of multiple people online at the same time. Finally, our system has a wider application range, which is both efficient and accurate.



---

*Source: 8501960-2021-10-14.xml*