# Music Singing Based on Computer Analog Piano Accompaniment and Digital Processing for 5G Industrial Internet of Things

Authors: Linlin Qi; Na Liu
Journal: Mobile Information Systems
 (2022)

---

## Abstract

At present, most professional music colleges and universities have made large-scale investment in computer software and hardware equipment and have also continuously opened various relevant courses. However, how to give full play to the advantages of computer music at the software level and make software and hardware systems suitable for music education application is still a subject being explored. The industrial Internet of Things turns every link and equipment in the production process into a data terminal, comprehensively collects the underlying basic data, carries out deeper data analysis and mining, and improves efficiency. To adapt to the trend of the times, digitization is the inevitable trend of the development of music accompaniment, and the deep integration of computer is an important way to realize digitization. Combined with the basic music theory and digital processing method, this paper designs the virtual piano auxiliary accompaniment and realizes the digital music accompaniment of artificial intelligence through digital signal processing. This paper will start with the auditory effect of auxiliary singing, take the rhythm and style of piano auxiliary accompaniment as the research object, build the accompaniment generation tool through the piano music simulation system model, and generate the accompaniment chord of the main melody by extracting the style, treble, and other characteristics of the main melody. The simulation results show that the frequency response is evenly distributed, the model effectively uses the harmonic structure information, and the extracted feature dimension is only 88. The reduction of feature dimension can save the time of subsequent processing and has the characteristics of real time, automation, embedded (software), security, and information interoperability.


---

## Body

## 1. Introduction

With the increasing maturity of 5G cutting-edge technology, its application in the music field is also accelerating. The music industry now has favorable policies and cutting-edge technology applications and other development conditions, and the scale of China’s digital music market grows steadily. 5G technology is boosting online music platforms to extend the music scene. The development and application of computer science has brought a revolutionary breakthrough for the development of music. Algorithmic composition technology has developed rapidly, and the thinking mode of music creation is also changing gradually. It is no longer a dream for one person to play the effect of a band, and there are more and more automatic composing and automatic accompaniment software on the market. Digital music will accelerate the application of technology, further enrich the music scene, gradually infiltrate offline, continuously expand the music scene, and provide users with a more extreme audiovisual feast [1]. With the continuous development of science and technology, there is a great demand for fast, wide capacity, low delay, and other characteristics of the network. And the birth of 5G network just meets the emerging technology for these network features. The 5G Internet provides an excellent platform for emerging technologies and promotes the deep integration of music singing. In the music singing industry, the collaborative application of 5G and digital technology will bring great changes to the music industry. 5G has a great impact in the field of communication. Due to the high speed of 5G, video expression will become the mainstream. From the original static text pictures to dynamic images, real-time and low-cost features meet the needs of the audience.With the arrival of the era of artificial intelligence, digital processing technology to further promote the development of music system to the direction of intelligence, people have a deeper pursuit of music, hope that the computer can “understand” the content of music, and replace professionals to complete music classification, music personality recommendation, music creation and other work [2]. Therefore, the application of artificial intelligence in music singing to assist people to complete music singing is a research direction combining computer technology, signal digital processing, and musicology [3, 4]. As the musical signal changes with time, that is, the musical state or feature will change with time, and the state at different moments will have a certain connection. In popular songs, an accompaniment provides rhythm, bass, harmony, and counterpoint support for the melody of the song. The main accompaniment instruments used in pop music are generally divided into percussion instruments, monophonic instruments, and multivoice instruments. Musical instruments can play harmonic texture and bass part at the same time. The emergence of virtual piano accompaniment is the result of the combination of accompaniment and digital technology. At present, many researchers have conducted research on automatic piano accompaniment. However, the automatic piano accompaniment also needs to consider the piano skills and accompaniment tone in the automation, which is the weak part of the current research. The premise of digital processing is to find out the rhythm differences that cause different styles. How to digitally process the accompaniment has become a popular research direction. In this context, on the basis of relevant research work, this paper will expand the melody of the song, extract pitch, and rhythm characteristics and construct virtual piano accompaniment through digital processing of accompaniment chords. This study provides a theoretical basis for enriching and perfecting automatic accompaniment and at the same time lowers the threshold of musical accompaniment.


## 2. Related Work

In the context of 5G Internet, piano accompaniment is actively integrating with computer digitalization, and integrating big data, artificial intelligence (AI), virtual reality (VR), augmented reality (AR), and other new technologies to promote the integration of music singing and realize digitalization. Since entering the digital music era, China’s online music industry to a new development opportunity speeds up the 5G technology and the integration process of the music industry, the online music platform development new form, in the digital economy status under the background of rising, digital music for the music industry technology development, and the application to provide a good policy environment. The research and application of music artificial intelligence technology in performance mainly includes six directions: audio processing, music analysis, music creation, performance, music retrieval, and music education. Different from the earlier research on artistic works, the current research on music creation focuses on the generation of pop music. In terms of melody and accompaniment generation of pop music, Li mainly uses piano timbre, supplemented by other timbre, as the carrier of tone-textured texture, which has the widest and most representative audience in response to different styles of accompaniment [5]. In complex melodies, the musical structure cannot be expressed only by the relationship between the notes before and after or the tone length. In order to better express the complex musical structure, Tabuena extended the 2-dimensional conversion table to the N-dimensional conversion table, which can make the style of the set of accompaniment samples generated [6]. Based on the theory of virtual acoustic modelling, Campbell et al. introduced the application of an analysis method based on finite element analysis software ANSYS for acoustic quality analysis of piano soundboards [7]. In the music world, the characteristics of sound correspond to the physical properties of sound and are called pitch, length, force, and timbre, respectively. The frequency of the sounds used in music is about 60 Hz to 1000 Hz. The sounds in this band are divided into different absolute pitches in the unit of semitone. The collection of these sounds is called the musical system. Because of the different descriptions of the relationship between the sounds, the musical sound system can be arranged according to the order of pitch or reverse order of a number of sounds called tone series. Each tone in the tone column is a scale. For example, every white key and black key in the piano is a tone level. With the continuous advancement of “new infrastructure,” the application of 5G will enable it to present more diversified carriers and give more instant feedback to music content, which will boost the innovation and development of the music industry.In recent years, the research of computer automatic accompaniment has made some achievements [8]. Computer digital accompaniment can complete the accompaniment task instantly on the premise of satisfying the main melody of the song. Virtual piano accompaniment technology is a new direction in the field of artificial intelligence. It takes 5G network as the media and follows certain rules to intelligently simulate the process of human accompaniment according to the melody theme of songs. Piano in the accompaniment closely with the characteristics of the melody has become the main form of accompaniment. Computer automatic accompaniment is the result of the development of computer automatic composition to a certain extent. According to the direction of song melody, computer is used to simulate the process of human accompaniment to achieve automatic accompaniment. But music is an art form, and computers only have established procedures and rules, without human emotions and thinking. Therefore, computer automatic composition not only needs more diversified programs but also needs the further support of digital technology. Mitra et al. proposed a hybrid system that automatically generates specific style accompaniment and automatically selects chords for melody to perform automatic accompaniment [9]. Dongmei and Binqi proposed an automatic generation accompaniment system based on harmony progression that can evoke specific emotions of a given melody to realize human-machine concert and proposed real-time scoring that can truly reflect errors, repetition, and jumps [10]. Tabuena proposed a real-time accompaniment system based on the dynamic programming algorithm to realize the real-time positioning of notes, and an automatic piano accompaniment method based on the structure of tone elements to realize the automatic piano accompaniment in the form of human-computer interaction [11]. Jin proposed an automatic accompaniment method based on MIDI technology to convert songs into musical instruments, which to a certain extent met people’s demand for song accompaniment [12]. The continuous development of virtual piano accompaniment provides conditions for the emergence of automatic accompaniment, so the academic circle has set off a research upsurge of automatic accompaniment. To a certain extent, virtual piano accompaniment is a continuation and development on the basis of automatic composition method and an exploration of song melody accompaniment.


## 3. Virtual Piano-Assisted Accompaniment Generation Design

### 3.1. Virtual Piano-Assisted Pitch Extraction for Accompaniment

The extraction of pitch in this design is in the digital audio workstation (DAW) operating environment, through the input of source musical instrument digital interface (MIDI) keyboard or in the piano window when the input of chord notes MIDI events, and MIDI events contain various types of MIDI information. It is to extract the MIDI note information corresponding to chord notes. When you press a key on a MIDI keyboard, the keyboard sends a note on message on the MIDI OUT port. The pitch information of MIDI Note can be obtained in the Note on information, and then, the pitch corresponding to each value can be obtained according to the MIDI pitch comparison table. MIDI pitch information of complete chords was obtained through the MIDI keyboard input or MIDI track of DAW, and the pitch information was screened and reset according to Figure1. Musical instrument digital interface is an industrial standard electronic communication protocol. Various musical notes record musical performance musical note events in the form of instruction sequence binary instructions. The attributes and related information of each musical note, such as the start time, end time and pitch of the note, as well as the instrument used for performance, are clearly recorded. So, through the analysis of the format can get more accurate music melody.Figure 1
Extraction flow chart of virtual piano-assisted accompaniment pitch.MIDI Note information input is to obtain MIDI pitch information values in descending order, each chord as a series of numbers sent to the next step. To judge the number of chord tones is to judge the number of elements in the sequence according to the pitch sequence obtained in the previous step, that is, the number of chord tones. The elements in the series of pitch values have been arranged smoothly from small to large, so the value of the first element can be directly extracted from the series, which is the pitch information value of chord and bass. By judging the number of chord tones, if the number of chord tones is greater than or equal to 5, the omission tone processing step is entered. If the number of chord tones is less than 5, skip this step and enter the valid chord tones directly. By omitting the sound processing chord of the pitch after the processing sequence, the chord pitches the number of elements in the sequence in 4, the sequence contains information for effective chord pitches. The valid chord pitch obtained for the first time is the original pitch input by MIDI, which may not conform to the conventional playing range. Therefore, it is necessary to identify the valid chord pitch and rearrange the pitch in the unreasonable range. The rearrangement of register is divided into two parts: one is the rearrangement of bass and the other is the rearrangement of other chord tones [13]. The pitch used in the bass region for rearrangement comes from the extracted chord bass. When the chord bass pitch value is greater than 56, the pitch will be moved down an octave and recirculated to check whether it is still greater than 56. If it is still greater than 56, the operation of moving down an octave will be repeated. When the chord bass pitch value is lower than 48, the pitch will be moved up an octave and also recirculated to check whether it is still less than 48. If it is still less than 48, the operation of moving up an octave will be repeated. The pitches of other chords used for rearrangement are derived from the extracted valid chord pitches, and if there is a chord pitch value greater than 69, the pitch of that part above 69 will be moved down an octave to below 69. After the rearrangement, the bass pitch information value is output as an integer and the other pitch information value is output as a sequence.


### 3.2. Digital Processing of Musical Notes Assisted by Virtual Piano

Is accompanied by a different sound frequency with certain rules are combined to form, continuity and periodicity, therefore and section and section and between notes, there is a certain internal connection between the in this article, in pitch, as the most basic characteristics of the notes, to note the starting point and end point as feature extraction based on notes, The law of beat strength is used as the basis for section division, and section information is extracted [14]. According to the existing MIDI music (including only the music of the main melody), the basic notes are C, D, E, F, G, A, and B, and the value of the notes is set as 0 ~128. The note and pitch are one-to-one corresponding relationship, but when the value of the note is 0, it indicates that the note does not have pitch value, which is A rest, indicating the pause. The length of a note is the interval between the start of that note and the start of the next note. The pitch strength of a note refers to the intensity of the sound. In order to facilitate feature extraction, the notes are coded with 0 as the medium, 1 as the sharp and -1 as the flat. In some cases, the sounds of the sharp and the flat are the same, but written in different ways. The codes are shown in Table 1.Table 1
Note encoding.
Serial number123456789101112Encode0C1C0D-1E0E0F1F0G-1A0A-1B0BAccording to note the time interval, by the length of the statistical time to obtain the position of the notes, in general, as a starting point to note before speak often accompanied by a certain time interval, there will be a certain compactness, and note the ending point is a pause, a sound came to an end, then the starting point of the next note. For the whole song, there is only one beginning note and one ending note. In order to accurately extract features, the starting point of each note is extracted by counting the interval time of each note, and then, the occurrence frequency of each note is counted, so as to obtain the characteristics of notes.Select a segment of the melody of the song, andNT is the time interval of each note, N is the starting point of the note, i is the number of notes, NF is the frequency of the note, Z is the total number of notes, and T is the pitch of the note, the pitch can be represented by the number 1-7, and the value is C [15].
(1)NT=Ni−Ni−1,(2)NF=∑i=1znumT−1Z.Through the above results, we can set time interval of each note, the note in the frequency of collection for NF notes the interval between the main is between 0.5 s and 1 s, the starting point will continue to a certain amount of time, with the former one note time interval, extracted to note after the starting point, and can get the note’s pitch. The length of a note is the sum of the time value and the delay value of the note, which is related to the beat number. The delay value is the duration of time caused by objective factors (such as tone key and release time). The features of pitch intensity can also be extracted from the features of musical notes.Rules of bar strength are helpful to extract bar position. Generally, bar lines are used to divide music into several bars. According to the strength of the beat, it is generally a combination of strong and weak alternating, which is usually periodic and fixed within a certain range. Therefore, based on the appearance of the strong beat, when the sum of the time values of several notes is equal to the molecule of the beat, it is a bar. Generally, the time value includes the time of the note, the time value of the dotted note and the time value of the rest. To facilitate the calculation of the time value, a quarter note is indicated by the number 1. The dotted note is indicated by half the time value of the modified note, which is the note with a small black dot to the right of the note. Rest is the time to pause or pause, and the number 0 is often used to indicate the rest note [16]. The time values of musical notes are shown in Table 2.Table 2
Timing of notes.
Note namesValueNote namesValueNote namesValueSemibreve4Dotted semibreve6Semibreve rest1Half note2Dotted half note3Half rest1/2Quarter note1Dotted quarter-note3/2Quarter rest1/4Eighth note1/2Dotted eighth note3/4Eight rest1/8Sixteenth note1/4Dotted with sixteenth notes3/8Sixteenth rest1/16Thirty-second note1/8Dotted with 32 quarter notes3/16Thirty-second rest1/32A bar is composed of several notes in a certain sequence, and the feature of a bar is essentially a collection of note features. On the basis of the strong beat information, the measure is judged, and then, the feature of notes in the measure is extracted, so as to get the feature of the measure.


### 3.3. Piano-Assisted Accompaniment Rhythm and Style Generation

The rhythm generation of the piano part is accomplished by a rhythm sequencer, which sets the minimum rhythm unit to 16 partials, with a beat of 2/4 or 4/4, and a cycle of 4 beats. Since the extraction of pitch has been divided into bass pitch information and other chord pitch information, the rhythm generation process is also divided into bass rhythm generation and middle and treble rhythm generation, which are used to output rhythm filling for pitch data of these two parts [16, 17]. The virtual piano part generation process is shown in Figure 2. There are three ways in which chord material appears in musical texture: the “chorus” mode based on four-part harmony, the rhythmic chord mode, and the decomposed chord mode.Figure 2
Virtual piano part generation flowchart.This design takes CyanogenMod (CM7) chord as an example. Chord sounds are ranked from low too high in order of C, E, G, and B, marked with numbers as 1, 2, 3, and 4, so 1 represents the lowest chord sound, and 4 represents the highest chord sound. The number pointed to by the arrow at the back of each pattern represents the order in which each note appears after the chord is broken up. The pitch combinations are successively carried out. CM7 chord can obtain 13 combination results, and up mode can obtain pitch combinations C, E, G, and B. In this mode, chords are ordered from low to high. Up and down mode gets pitch combinations C, E, G, B, B, G, E, C. Swing to top mode: get the pitch combination C, B, E, B, G, B. In this mode, chords start in the bass and swing upward toward the top. Swing to bottom down mode: get the pitch combination C, E, C, G, C, B, C, G, C, E. This pattern is an extension of the swing to bottom pattern, where chord notes start at the bottom, swing up from low to high order to the lowest note, then swing down order.After the establishment of the system, it is necessary to train the pattern structure first, input sample music score into the system, update the reference table of pattern structure in the database, and store the sound group structure and accompaniment pattern structure extracted from the sample songs into the corresponding data table according to the style and tempo classification. The harmonic rules are extracted from the sample songs used for training the musical structure and stored in the rule library according to the classification. After the main melody to be accompanied is input into the automatic accompaniment system, the processing process of the system is as follows: after the melody to be accompanied is input, the overall information of the melody is firstly extracted, and the series generation module of musical pattern structure and the generation module of accompaniment melody are generated. The accompaniment pattern structure is transmitted to the accompaniment melody generation module, and the accompaniment chord sequence of the phrase is aggregated into the accompaniment chord sequence of the main melody and transmitted to the accompaniment melody generation module. The accompaniment chord generated by the accompaniment chord sequence generation algorithm is output together with MIDI music.


### 3.4. Piano Music Simulation System Model

In this study, the musical sound simulation system model is divided into two parts: resonance system and excitation system from the perspective of piano physical sound mechanism. String vibration is simulated by means of sinusoidal wave superposition, and timbre is modified from frequency domain and time domain, respectively, to make the music simulation more realistic [18]. In the frequency domain, the spectrum envelope extracted by cestrum method is used to model the filter bank to simulate the resonance system. In time domain, the natural attenuation of strings is simulated by fitting the envelope function of the time-domain envelope extracted from the short-time root mean square energy. The simulation system model adopted in this research is shown in Figure 3. The string is the basic sound source, and the vibration of the string is essentially the superposition of sinusoidal mode, which contains the components of high harmonics and fundamental waves. The vibration of the string is simulated by means of addition synthesis, and the specific parameters required for simulation are the duration of each tone in the music, harmonic frequency component, and fundamental frequency. Harmonic components need to be added to the simulation model. Since there is a certain degree of multiple relationship between each harmonic frequency and fundamental frequency, harmonic components can be added to the model through curve function [19, 20]. In this study, the piano C major monophonic as the material, do, re, mi, fa, so, la, si implemented Fourier transform successively, and then, the harmonic normalized amplitude and frequency, the spectral data of monophonic signal obtained., in turn, the implementation of curve fitting based on the notes at various levels of data processing, to obtain the relationship between amplitude and frequency of notes function, and on this basis, adopt the way of additive synthesis will be various tones of the duration, amplitude relationship, and every harmonic parameters such as frequency, input to the amplitude-frequency relation function can complete according to the string vibration simulation.Figure 3
Piano music simulation system model.The temperament filter bank is designed based on the twelve-equal temperament, and its function is to obtain the distribution of energy of each frame in the musical signal on the scale frequency, which is an important basis for note starting point detection [21]. The tone filter bank contains 88 band pass filters, and the number of filters is the same as the number of piano keys. In 88 bandpass filters, each filter has only one passband, and the center frequency of the passband is the fundamental frequency of the key. According to the international standard, the pitch of central C is 440 Hz, that is, the fundamental frequency of the key number 49 is 440 Hz, and the fundamental frequency of other piano keys can be calculated according to the law of duodenal equal temperament. The boundary frequency of the pass band is the frequency of the first half of the center frequency and the frequency of the second half of the center frequency, that is, the bandwidth is the difference between the frequencies of the next half of the center frequency. Due to spectral resolution, there will be errors between the frequency points on the spectrum and the scale frequencies, so a passband needs to be set up in order to give some weight to the frequency points near the central frequency. In this paper, the cosine shape is chosen as the passband shape. Compared with the traditional triangle bandpass filter, the cosine shape filter can give higher weight to the frequency components near the center frequency and lower weight to the frequency components farther from the center frequency.


## 3.1. Virtual Piano-Assisted Pitch Extraction for Accompaniment

The extraction of pitch in this design is in the digital audio workstation (DAW) operating environment, through the input of source musical instrument digital interface (MIDI) keyboard or in the piano window when the input of chord notes MIDI events, and MIDI events contain various types of MIDI information. It is to extract the MIDI note information corresponding to chord notes. When you press a key on a MIDI keyboard, the keyboard sends a note on message on the MIDI OUT port. The pitch information of MIDI Note can be obtained in the Note on information, and then, the pitch corresponding to each value can be obtained according to the MIDI pitch comparison table. MIDI pitch information of complete chords was obtained through the MIDI keyboard input or MIDI track of DAW, and the pitch information was screened and reset according to Figure1. Musical instrument digital interface is an industrial standard electronic communication protocol. Various musical notes record musical performance musical note events in the form of instruction sequence binary instructions. The attributes and related information of each musical note, such as the start time, end time and pitch of the note, as well as the instrument used for performance, are clearly recorded. So, through the analysis of the format can get more accurate music melody.Figure 1
Extraction flow chart of virtual piano-assisted accompaniment pitch.MIDI Note information input is to obtain MIDI pitch information values in descending order, each chord as a series of numbers sent to the next step. To judge the number of chord tones is to judge the number of elements in the sequence according to the pitch sequence obtained in the previous step, that is, the number of chord tones. The elements in the series of pitch values have been arranged smoothly from small to large, so the value of the first element can be directly extracted from the series, which is the pitch information value of chord and bass. By judging the number of chord tones, if the number of chord tones is greater than or equal to 5, the omission tone processing step is entered. If the number of chord tones is less than 5, skip this step and enter the valid chord tones directly. By omitting the sound processing chord of the pitch after the processing sequence, the chord pitches the number of elements in the sequence in 4, the sequence contains information for effective chord pitches. The valid chord pitch obtained for the first time is the original pitch input by MIDI, which may not conform to the conventional playing range. Therefore, it is necessary to identify the valid chord pitch and rearrange the pitch in the unreasonable range. The rearrangement of register is divided into two parts: one is the rearrangement of bass and the other is the rearrangement of other chord tones [13]. The pitch used in the bass region for rearrangement comes from the extracted chord bass. When the chord bass pitch value is greater than 56, the pitch will be moved down an octave and recirculated to check whether it is still greater than 56. If it is still greater than 56, the operation of moving down an octave will be repeated. When the chord bass pitch value is lower than 48, the pitch will be moved up an octave and also recirculated to check whether it is still less than 48. If it is still less than 48, the operation of moving up an octave will be repeated. The pitches of other chords used for rearrangement are derived from the extracted valid chord pitches, and if there is a chord pitch value greater than 69, the pitch of that part above 69 will be moved down an octave to below 69. After the rearrangement, the bass pitch information value is output as an integer and the other pitch information value is output as a sequence.


## 3.2. Digital Processing of Musical Notes Assisted by Virtual Piano

Is accompanied by a different sound frequency with certain rules are combined to form, continuity and periodicity, therefore and section and section and between notes, there is a certain internal connection between the in this article, in pitch, as the most basic characteristics of the notes, to note the starting point and end point as feature extraction based on notes, The law of beat strength is used as the basis for section division, and section information is extracted [14]. According to the existing MIDI music (including only the music of the main melody), the basic notes are C, D, E, F, G, A, and B, and the value of the notes is set as 0 ~128. The note and pitch are one-to-one corresponding relationship, but when the value of the note is 0, it indicates that the note does not have pitch value, which is A rest, indicating the pause. The length of a note is the interval between the start of that note and the start of the next note. The pitch strength of a note refers to the intensity of the sound. In order to facilitate feature extraction, the notes are coded with 0 as the medium, 1 as the sharp and -1 as the flat. In some cases, the sounds of the sharp and the flat are the same, but written in different ways. The codes are shown in Table 1.Table 1
Note encoding.
Serial number123456789101112Encode0C1C0D-1E0E0F1F0G-1A0A-1B0BAccording to note the time interval, by the length of the statistical time to obtain the position of the notes, in general, as a starting point to note before speak often accompanied by a certain time interval, there will be a certain compactness, and note the ending point is a pause, a sound came to an end, then the starting point of the next note. For the whole song, there is only one beginning note and one ending note. In order to accurately extract features, the starting point of each note is extracted by counting the interval time of each note, and then, the occurrence frequency of each note is counted, so as to obtain the characteristics of notes.Select a segment of the melody of the song, andNT is the time interval of each note, N is the starting point of the note, i is the number of notes, NF is the frequency of the note, Z is the total number of notes, and T is the pitch of the note, the pitch can be represented by the number 1-7, and the value is C [15].
(1)NT=Ni−Ni−1,(2)NF=∑i=1znumT−1Z.Through the above results, we can set time interval of each note, the note in the frequency of collection for NF notes the interval between the main is between 0.5 s and 1 s, the starting point will continue to a certain amount of time, with the former one note time interval, extracted to note after the starting point, and can get the note’s pitch. The length of a note is the sum of the time value and the delay value of the note, which is related to the beat number. The delay value is the duration of time caused by objective factors (such as tone key and release time). The features of pitch intensity can also be extracted from the features of musical notes.Rules of bar strength are helpful to extract bar position. Generally, bar lines are used to divide music into several bars. According to the strength of the beat, it is generally a combination of strong and weak alternating, which is usually periodic and fixed within a certain range. Therefore, based on the appearance of the strong beat, when the sum of the time values of several notes is equal to the molecule of the beat, it is a bar. Generally, the time value includes the time of the note, the time value of the dotted note and the time value of the rest. To facilitate the calculation of the time value, a quarter note is indicated by the number 1. The dotted note is indicated by half the time value of the modified note, which is the note with a small black dot to the right of the note. Rest is the time to pause or pause, and the number 0 is often used to indicate the rest note [16]. The time values of musical notes are shown in Table 2.Table 2
Timing of notes.
Note namesValueNote namesValueNote namesValueSemibreve4Dotted semibreve6Semibreve rest1Half note2Dotted half note3Half rest1/2Quarter note1Dotted quarter-note3/2Quarter rest1/4Eighth note1/2Dotted eighth note3/4Eight rest1/8Sixteenth note1/4Dotted with sixteenth notes3/8Sixteenth rest1/16Thirty-second note1/8Dotted with 32 quarter notes3/16Thirty-second rest1/32A bar is composed of several notes in a certain sequence, and the feature of a bar is essentially a collection of note features. On the basis of the strong beat information, the measure is judged, and then, the feature of notes in the measure is extracted, so as to get the feature of the measure.


## 3.3. Piano-Assisted Accompaniment Rhythm and Style Generation

The rhythm generation of the piano part is accomplished by a rhythm sequencer, which sets the minimum rhythm unit to 16 partials, with a beat of 2/4 or 4/4, and a cycle of 4 beats. Since the extraction of pitch has been divided into bass pitch information and other chord pitch information, the rhythm generation process is also divided into bass rhythm generation and middle and treble rhythm generation, which are used to output rhythm filling for pitch data of these two parts [16, 17]. The virtual piano part generation process is shown in Figure 2. There are three ways in which chord material appears in musical texture: the “chorus” mode based on four-part harmony, the rhythmic chord mode, and the decomposed chord mode.Figure 2
Virtual piano part generation flowchart.This design takes CyanogenMod (CM7) chord as an example. Chord sounds are ranked from low too high in order of C, E, G, and B, marked with numbers as 1, 2, 3, and 4, so 1 represents the lowest chord sound, and 4 represents the highest chord sound. The number pointed to by the arrow at the back of each pattern represents the order in which each note appears after the chord is broken up. The pitch combinations are successively carried out. CM7 chord can obtain 13 combination results, and up mode can obtain pitch combinations C, E, G, and B. In this mode, chords are ordered from low to high. Up and down mode gets pitch combinations C, E, G, B, B, G, E, C. Swing to top mode: get the pitch combination C, B, E, B, G, B. In this mode, chords start in the bass and swing upward toward the top. Swing to bottom down mode: get the pitch combination C, E, C, G, C, B, C, G, C, E. This pattern is an extension of the swing to bottom pattern, where chord notes start at the bottom, swing up from low to high order to the lowest note, then swing down order.After the establishment of the system, it is necessary to train the pattern structure first, input sample music score into the system, update the reference table of pattern structure in the database, and store the sound group structure and accompaniment pattern structure extracted from the sample songs into the corresponding data table according to the style and tempo classification. The harmonic rules are extracted from the sample songs used for training the musical structure and stored in the rule library according to the classification. After the main melody to be accompanied is input into the automatic accompaniment system, the processing process of the system is as follows: after the melody to be accompanied is input, the overall information of the melody is firstly extracted, and the series generation module of musical pattern structure and the generation module of accompaniment melody are generated. The accompaniment pattern structure is transmitted to the accompaniment melody generation module, and the accompaniment chord sequence of the phrase is aggregated into the accompaniment chord sequence of the main melody and transmitted to the accompaniment melody generation module. The accompaniment chord generated by the accompaniment chord sequence generation algorithm is output together with MIDI music.


## 3.4. Piano Music Simulation System Model

In this study, the musical sound simulation system model is divided into two parts: resonance system and excitation system from the perspective of piano physical sound mechanism. String vibration is simulated by means of sinusoidal wave superposition, and timbre is modified from frequency domain and time domain, respectively, to make the music simulation more realistic [18]. In the frequency domain, the spectrum envelope extracted by cestrum method is used to model the filter bank to simulate the resonance system. In time domain, the natural attenuation of strings is simulated by fitting the envelope function of the time-domain envelope extracted from the short-time root mean square energy. The simulation system model adopted in this research is shown in Figure 3. The string is the basic sound source, and the vibration of the string is essentially the superposition of sinusoidal mode, which contains the components of high harmonics and fundamental waves. The vibration of the string is simulated by means of addition synthesis, and the specific parameters required for simulation are the duration of each tone in the music, harmonic frequency component, and fundamental frequency. Harmonic components need to be added to the simulation model. Since there is a certain degree of multiple relationship between each harmonic frequency and fundamental frequency, harmonic components can be added to the model through curve function [19, 20]. In this study, the piano C major monophonic as the material, do, re, mi, fa, so, la, si implemented Fourier transform successively, and then, the harmonic normalized amplitude and frequency, the spectral data of monophonic signal obtained., in turn, the implementation of curve fitting based on the notes at various levels of data processing, to obtain the relationship between amplitude and frequency of notes function, and on this basis, adopt the way of additive synthesis will be various tones of the duration, amplitude relationship, and every harmonic parameters such as frequency, input to the amplitude-frequency relation function can complete according to the string vibration simulation.Figure 3
Piano music simulation system model.The temperament filter bank is designed based on the twelve-equal temperament, and its function is to obtain the distribution of energy of each frame in the musical signal on the scale frequency, which is an important basis for note starting point detection [21]. The tone filter bank contains 88 band pass filters, and the number of filters is the same as the number of piano keys. In 88 bandpass filters, each filter has only one passband, and the center frequency of the passband is the fundamental frequency of the key. According to the international standard, the pitch of central C is 440 Hz, that is, the fundamental frequency of the key number 49 is 440 Hz, and the fundamental frequency of other piano keys can be calculated according to the law of duodenal equal temperament. The boundary frequency of the pass band is the frequency of the first half of the center frequency and the frequency of the second half of the center frequency, that is, the bandwidth is the difference between the frequencies of the next half of the center frequency. Due to spectral resolution, there will be errors between the frequency points on the spectrum and the scale frequencies, so a passband needs to be set up in order to give some weight to the frequency points near the central frequency. In this paper, the cosine shape is chosen as the passband shape. Compared with the traditional triangle bandpass filter, the cosine shape filter can give higher weight to the frequency components near the center frequency and lower weight to the frequency components farther from the center frequency.


## 4. Simulation Analysis of Virtual Piano Auxiliary Accompaniment

### 4.1. Analysis of Frequency Response Characteristics of Digital Accompaniment

Based on the resonance principle of the resonance system, this study simulated the effect of the piano sound box with the help of filters, that is, to strengthen or weaken a specific frequency band through filtering, and then generate digital signal spectrum [22]. According to the principle of sound equalizer EQ, the following sound sense characteristics of piano music signals in various frequency bands can be summarized: 8 ~12 KHz, 6 ~ 8 KHz, 1 ~ 6 KHz, 250~1 KHz, 100~250 Hz, and 50~100 Hz. Specific filtering settings in each frequency band can achieve the adjustment of music, for example, a small increase in the energy of 100-250 Hz frequency band can increase the fullness of music, in the piano music signal frequency segment based on the design of band-pass filter. For music signal segmentation implementation of filtering, since the input signal of each bandpass filter is not a series filter, but a sine wave signal output by the string vibration model [23], the weight of the i circuit is ki, the output signal of the i circuit is yin, and the willing signal of the sine wave model is xn. Then the filtered signal yn can be expressed by the following formula:
(3)yn=∑i=1Nkiyinxn.By setting filter parameters according to the characteristics of music signals and writing corresponding programs, the frequency response characteristic curve of filter banks can be obtained as shown in Figure4. Combined in this paper, the piano music is polyphony music harmonic peak method to detect pitch features of music signal, the method of harmonic peaks for harmonic are particularly rich in error, the frequency of the main peak is the largest place often is not necessarily a pitch frequency, and in many cases, it will also play the piano music itself has a harmonic relationship between notes. The original harmonic peak method is improved to transform the extracted pitch into the feature sequence containing pitch.Figure 4
Frequency response characteristic curve of harmonic filter banks.The function of the temperament filter bank is to extract the energy components of the fundamental frequency of 88 notes from the music signal, that is to say, the spectrum of the music signal can get 88 energy outputs after passing through the temperament filter bank, and each energy output represents the energy components of the corresponding fundamental frequency contained in the music signal. According to the figure, in this case, the weights of all frequency bands are 1. Only proper adjustment of the weights of each frequency band can realize the enhancement or attenuation of music energy in a certain frequency band. The distribution of energy in the musical signal on the scale frequency can be obtained through the tonal filter bank.


### 4.2. Digital Vibration Frequency Analysis

Virtual modeling of phase difference physical parameters is carried out in the piano, and the data are constantly modified according to the ideal state until better vibration modes are obtained. Virtual acoustic modeling is used to grasp the vibration modes of mMusical instruments, and the rational cognition of the vibration modes of musical instruments is obtained, so as to provide virtual samples for musical instrument acoustic restoration [24]. Virtual acoustic modeling simulates the acoustic parameters of musical instruments to a certain extent. There are many factors that affect the vibration sound of musical instruments, which cannot be completely covered by virtual modeling technology. Through the repeated analysis of piano vibration modes and frequencies, the vibration modes of piano soundboard are shown in Figure 5.Figure 5
Analysis and comparison diagram of vibration frequency and mode after virtual modeling.The analysis results of vibration frequency and mode after virtual modeling show that the resonance region of this digital design increases. Although the basic mode changes little, the frequency below 100 Hz decreases again, and the frequency response distribution is relatively uniform. From the point of view of vibration mode, the vibration energy is obviously increased and the radiation range is greatly expanded, which can effectively simulate the acoustic quality of piano. For low frequency, high-order frequency has little effect on it. And the high frequencies are reflected in the higher frequencies. Anyhow, this kind of design enlarging radiate range, the vibration displacement degree that raises soundboard reached the effect that enlarging volume, and timbre is even inside the range of the whole frequency band on sound expression.


### 4.3. Simulation Data Set Analysis

The multifundamental frequency estimation method based on the timbre filter bank was analyzed through experiments. The experimental data was from the RAND subset of the database. The experimental evaluation indexes were the recall rate, accuracy rate, andF measure in MIREX standard. Accuracy is measured by the proportion of correct points detected, and recall is measured by the proportion of standard points detected. F measure 1 is a comprehensive measure produced by combining accuracy rate and recall rate. RAND subset contains 300 WAV format audio files; each audio file only has a set of random notes composed of multiple fundamental frequency, so there are a total of 300 groups of notes in the combination; the number of notes in the combination range from 2 to 7, according to the number of notes can be divided into 6 categories; and each category of music is 50. The multifundamental frequency estimation results obtained through experiments are shown in Figure 6. The data set indicators of the timbre filter banks are shown in Figure 7.Figure 6
Cluster analysis of hidden node activation values.Figure 7
Data set index diagram of the timbre filter bank.As can be seen from the figure, with the increase of the number of notes, the recall rate and precision rate will gradually decline, leading to the decline ofF measure, especially when the number of notes is more than 5, the recall rate and precision rate will decrease rapidly. The method performs multifundamental frequency estimation experiments on RAND data, and the final recall rate, accuracy rate, and F measure are 84.07%, 78.71%, and 81.30%, respectively. Compared with other methods, the accuracy rate is slightly lower, but the recall rate and F measure are significantly improved, among which the recall rate is increased by about 6% and F measure by about 3%. From the perspective of comprehensive index F measure, the detection performance of the method in this paper is higher. On the other hand, the feature dimension extracted based on the energy spectrum envelope decomposition method is 890, while the information of harmonic structure is effectively used in this paper, and the extracted feature dimension is only 88. The reduction of feature dimension can save the time of subsequent processing and is conducive to the improvement of calculation efficiency.


### 4.4. Network Rating Analysis of Music Singing Application in 5G Era

In this section, a network scoring platform is constructed to upload the original music and the music after harmony arrangement to the cloud server, so that professional musicians or music lovers can listen to it through the browser and evaluate the effect of harmony arrangement. The network scoring platform includes front-end page display and back-end business logic, and I am responsible for the back-end business logic realization, mainly including request processing, file uploading, and scoring data management. A total of 10 groups of sample music were uploaded, each of which contained one original monophonic music and one music arranged with virtual piano-assisted accompaniment. Considering the large size of WAV files, I cut 30-second music clips from the synthesized WAV music and compressed them into MP3 format and then uploaded them to the cloud server. There are 5 grades in total, with the lowest score being 1 and the highest score being 5. The grades from low to high correspond to the evaluation of low quality, barely acceptable, acceptable, relatively satisfactory, and very satisfactory, respectively. After listening to the music, the audience can evaluate the effect of the arrangement, and they need to fill in the correct invitation code when submitting the score. I invited 100 music lovers to evaluate the effect of harmony arrangement, and the scoring results obtained after averaging the scores of each sample are shown in Figure8.Figure 8
Each sample effect score result diagram.As can be seen from the scoring results, sample 1 got the highest score of 4.6, and sample 9 got the lowest score of only 3.1. The average score of most of the samples was more than 3.5, and the overall average score was about 4, indicating that the listeners were generally satisfied with the effect of virtual piano accompaniment arrangement. When the scale of works was expanded to 50 pieces and the scale of evaluation was expanded to 1000 people, including freshmen and juniors from conservatory of music and nonconservatory of music, four evaluation experiments were conducted on the works considering the psychological changes of people’s evaluation. Sample 1 got the highest score of 4.5, while sample 9 got the lowest score of 3.4. Most of the samples averaged more than 3.8, and the overall average was about 4.2. The effect of virtual piano accompaniment orchestration was evaluated, and most of the songs scored high, indicating that the audients were quite satisfied with the orchestration effect, thus verifying the effective application of virtual piano auxiliary accompaniment technology and digital processing in music singing in this paper.


## 4.1. Analysis of Frequency Response Characteristics of Digital Accompaniment

Based on the resonance principle of the resonance system, this study simulated the effect of the piano sound box with the help of filters, that is, to strengthen or weaken a specific frequency band through filtering, and then generate digital signal spectrum [22]. According to the principle of sound equalizer EQ, the following sound sense characteristics of piano music signals in various frequency bands can be summarized: 8 ~12 KHz, 6 ~ 8 KHz, 1 ~ 6 KHz, 250~1 KHz, 100~250 Hz, and 50~100 Hz. Specific filtering settings in each frequency band can achieve the adjustment of music, for example, a small increase in the energy of 100-250 Hz frequency band can increase the fullness of music, in the piano music signal frequency segment based on the design of band-pass filter. For music signal segmentation implementation of filtering, since the input signal of each bandpass filter is not a series filter, but a sine wave signal output by the string vibration model [23], the weight of the i circuit is ki, the output signal of the i circuit is yin, and the willing signal of the sine wave model is xn. Then the filtered signal yn can be expressed by the following formula:
(3)yn=∑i=1Nkiyinxn.By setting filter parameters according to the characteristics of music signals and writing corresponding programs, the frequency response characteristic curve of filter banks can be obtained as shown in Figure4. Combined in this paper, the piano music is polyphony music harmonic peak method to detect pitch features of music signal, the method of harmonic peaks for harmonic are particularly rich in error, the frequency of the main peak is the largest place often is not necessarily a pitch frequency, and in many cases, it will also play the piano music itself has a harmonic relationship between notes. The original harmonic peak method is improved to transform the extracted pitch into the feature sequence containing pitch.Figure 4
Frequency response characteristic curve of harmonic filter banks.The function of the temperament filter bank is to extract the energy components of the fundamental frequency of 88 notes from the music signal, that is to say, the spectrum of the music signal can get 88 energy outputs after passing through the temperament filter bank, and each energy output represents the energy components of the corresponding fundamental frequency contained in the music signal. According to the figure, in this case, the weights of all frequency bands are 1. Only proper adjustment of the weights of each frequency band can realize the enhancement or attenuation of music energy in a certain frequency band. The distribution of energy in the musical signal on the scale frequency can be obtained through the tonal filter bank.


## 4.2. Digital Vibration Frequency Analysis

Virtual modeling of phase difference physical parameters is carried out in the piano, and the data are constantly modified according to the ideal state until better vibration modes are obtained. Virtual acoustic modeling is used to grasp the vibration modes of mMusical instruments, and the rational cognition of the vibration modes of musical instruments is obtained, so as to provide virtual samples for musical instrument acoustic restoration [24]. Virtual acoustic modeling simulates the acoustic parameters of musical instruments to a certain extent. There are many factors that affect the vibration sound of musical instruments, which cannot be completely covered by virtual modeling technology. Through the repeated analysis of piano vibration modes and frequencies, the vibration modes of piano soundboard are shown in Figure 5.Figure 5
Analysis and comparison diagram of vibration frequency and mode after virtual modeling.The analysis results of vibration frequency and mode after virtual modeling show that the resonance region of this digital design increases. Although the basic mode changes little, the frequency below 100 Hz decreases again, and the frequency response distribution is relatively uniform. From the point of view of vibration mode, the vibration energy is obviously increased and the radiation range is greatly expanded, which can effectively simulate the acoustic quality of piano. For low frequency, high-order frequency has little effect on it. And the high frequencies are reflected in the higher frequencies. Anyhow, this kind of design enlarging radiate range, the vibration displacement degree that raises soundboard reached the effect that enlarging volume, and timbre is even inside the range of the whole frequency band on sound expression.


## 4.3. Simulation Data Set Analysis

The multifundamental frequency estimation method based on the timbre filter bank was analyzed through experiments. The experimental data was from the RAND subset of the database. The experimental evaluation indexes were the recall rate, accuracy rate, andF measure in MIREX standard. Accuracy is measured by the proportion of correct points detected, and recall is measured by the proportion of standard points detected. F measure 1 is a comprehensive measure produced by combining accuracy rate and recall rate. RAND subset contains 300 WAV format audio files; each audio file only has a set of random notes composed of multiple fundamental frequency, so there are a total of 300 groups of notes in the combination; the number of notes in the combination range from 2 to 7, according to the number of notes can be divided into 6 categories; and each category of music is 50. The multifundamental frequency estimation results obtained through experiments are shown in Figure 6. The data set indicators of the timbre filter banks are shown in Figure 7.Figure 6
Cluster analysis of hidden node activation values.Figure 7
Data set index diagram of the timbre filter bank.As can be seen from the figure, with the increase of the number of notes, the recall rate and precision rate will gradually decline, leading to the decline ofF measure, especially when the number of notes is more than 5, the recall rate and precision rate will decrease rapidly. The method performs multifundamental frequency estimation experiments on RAND data, and the final recall rate, accuracy rate, and F measure are 84.07%, 78.71%, and 81.30%, respectively. Compared with other methods, the accuracy rate is slightly lower, but the recall rate and F measure are significantly improved, among which the recall rate is increased by about 6% and F measure by about 3%. From the perspective of comprehensive index F measure, the detection performance of the method in this paper is higher. On the other hand, the feature dimension extracted based on the energy spectrum envelope decomposition method is 890, while the information of harmonic structure is effectively used in this paper, and the extracted feature dimension is only 88. The reduction of feature dimension can save the time of subsequent processing and is conducive to the improvement of calculation efficiency.


## 4.4. Network Rating Analysis of Music Singing Application in 5G Era

In this section, a network scoring platform is constructed to upload the original music and the music after harmony arrangement to the cloud server, so that professional musicians or music lovers can listen to it through the browser and evaluate the effect of harmony arrangement. The network scoring platform includes front-end page display and back-end business logic, and I am responsible for the back-end business logic realization, mainly including request processing, file uploading, and scoring data management. A total of 10 groups of sample music were uploaded, each of which contained one original monophonic music and one music arranged with virtual piano-assisted accompaniment. Considering the large size of WAV files, I cut 30-second music clips from the synthesized WAV music and compressed them into MP3 format and then uploaded them to the cloud server. There are 5 grades in total, with the lowest score being 1 and the highest score being 5. The grades from low to high correspond to the evaluation of low quality, barely acceptable, acceptable, relatively satisfactory, and very satisfactory, respectively. After listening to the music, the audience can evaluate the effect of the arrangement, and they need to fill in the correct invitation code when submitting the score. I invited 100 music lovers to evaluate the effect of harmony arrangement, and the scoring results obtained after averaging the scores of each sample are shown in Figure8.Figure 8
Each sample effect score result diagram.As can be seen from the scoring results, sample 1 got the highest score of 4.6, and sample 9 got the lowest score of only 3.1. The average score of most of the samples was more than 3.5, and the overall average score was about 4, indicating that the listeners were generally satisfied with the effect of virtual piano accompaniment arrangement. When the scale of works was expanded to 50 pieces and the scale of evaluation was expanded to 1000 people, including freshmen and juniors from conservatory of music and nonconservatory of music, four evaluation experiments were conducted on the works considering the psychological changes of people’s evaluation. Sample 1 got the highest score of 4.5, while sample 9 got the lowest score of 3.4. Most of the samples averaged more than 3.8, and the overall average was about 4.2. The effect of virtual piano accompaniment orchestration was evaluated, and most of the songs scored high, indicating that the audients were quite satisfied with the orchestration effect, thus verifying the effective application of virtual piano auxiliary accompaniment technology and digital processing in music singing in this paper.


## 5. Conclusion

With the rapid development of 5G network, traditional singing and performance forms can no longer satisfy the performance. This paper is aimed at introducing 5G technology into automatic accompaniment, focusing on the automation of music singing accompaniment, abstracting and modeling piano accompaniment, and constructing a simulation system of virtual piano automatic accompaniment melody. The piano accompaniment music elements are digitized, and the pitch, length, chord, and rhythm music elements are transformed into a language that can be recognized by the computer. A lot of experiments show that the automatic accompaniment model proposed in this paper can meet the demand of music singing accompaniment. The evaluation results show that the model effectively utilizes the information of harmonic structure, and the extracted feature dimension is only 88. The reduction of feature dimension can save the time of subsequent processing and improve the calculation efficiency. Through the construction of network scoring platform, the original music and the music after harmony arrangement can be uploaded to the cloud server, so that professional music personnel or music lovers can listen to it through the browser, and the evaluation effect is satisfactory. The simulation system can realize the automatic accompaniment of piano effectively. In the process of realization, the computer technology and music are deeply integrated, and the integration of music and computer technology is promoted. In the future research work, it is necessary to further simulate the resonance system and process the digital model of music more accurately.



---

*Source: 4489301-2022-05-11.xml*