<figure><img src="image_1.png" /></figure>

HAL
open science

# A sinusoidal audio analysis and synthesis model based on EMD

Xiao-Ming Li, Chang-Chun Bao, Mao-Shen Jia, Song Wang

► To cite this version:

Xiao-Ming Li, Chang-Chun Bao, Mao-Shen Jia, Song Wang. A sinusoidal audio analysis and synthesis model based on EMD. Second APSIPA Annual Summit and Conference, Dec 2010, Singapore, Singapore. pp.506-509. hal-05452525

HAL Id: hal-05452525

https://hal.science/hal-05452525v1

Submitted on 11 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# A sinusoidal audio analysis and synthesis model based on EMD

Xiao-ming Li, Chang-chun Bao, Mao-shen Jia, Song Wang

Speech and Audio Signal Processing Laboratory, School of Electronic Information and Control Engineering,
Beijing University of Technology, Beijing, China
lixiaoming@emails.bjut.edu.cn, baochch@bjut.edu.cn, jiamaoshen@emails.bjut.edu.cn, s.wang@bipt.edu.cn

***Abstract***— A sinusoidal audio analysis and synthesis model based on Empirical Mode Decomposition (EMD) is proposed in this paper. The EMD is an adaptive method for the analysis of non-stationary signals and it has a unique filtering characteristic. The Intrinsic Mode Functions (IMFs) which derived from EMD fully retain the non-stationarity of original signal. Because the high frequency components of audio are noise-like and less perceptually relevant, the original audio is divided into noise and harmonics in the proposed model by using a combination of different IMFs. Prior to parametric modeling, the audio is first classified to reduce the affect of noise components for sinusoids extraction, and then the matching pursuit algorithm combining with psychoacoustic model and fine frequency searching method is applied for extracting sinusoidal components. Finally, objective measurements of perceived audio quality (PEAQ) are performed to evaluate the effectiveness of this model for the audio synthesis.

***Keywords***-audio analysis and synthesis; sinusoidal model; empirical mode decomposition

## I. INTRODUCTION

A novel method for the analysis of non-stationary signals called Hilbert Huang Transform (HHT) was proposed by N.E.Huang in 1998 [1]. The EMD is used to produce a set of IMFs which are extracted from the original signal, and then the instantaneous frequency of each IMF is computed by Hilbert Transform. It is different from the Fourier analysis that the uniform harmonic components is used to express the signal, the EMD is based on the local features of data and a view of a signal is given in terms of instantaneous frequency. However, it is difficult to get a meaningful instantaneous frequency from a complicated signal, so the signal is decomposed into several suitable basis functions with EMD by extracting the intrinsic time scales.

The IMFs produced by EMD fully retain the non-stationary characteristics of original signal, how to use the superiority of this method in non-stationary signal analysis to process audio signal is a problem. M.Fulton and John.Soraghan proposed a novel method to express IMFs by using a series of amplitude modulation and frequency modulation basis functions, and achieved an excellent synthesis result for piano [2]. Based on the above, the unique filtering characteristic of EMD combining with the perceptual feature of human hearing is used to classify the audio signal in this paper. And the audio is reconstructed effectively by using different parametric models for the classified signal.

In recent years, the model approaches of audio compression using analysis/synthesis techniques have seen increasing interest, and there are a number of audio compression schemes that use sinusoidal modeling [3]. The sinusoidal model can transform the original signal into a series of parameters, and the number of these parameters is far less than the signal length, so the signal can be compressed by transforming the parameters in stead of itself. In this paper, the audio is classified before parametric estimation, and then a typical sine plus noise model based on EMD is used to analyze the classified signals, respectively. So the impact of noise-like components can be reduced effectively and a good synthesis results have been obtained.

The fundamental and details of the algorithm for EMD are first given in section two, and then the filtering characteristics of EMD and the classification method are carefully introduced in section three. The classified signal's modeling is described in section four. In this section, the harmonics are modeled by Perceptual Weighted Matching Pursuit (PWMP) [4] and the noise is modeled by the Noise-fill method of G.722.1C [5]. In the final section, a general discussion on the results and a conclusion are presented.

## II. EMPIRICAL MODE DECOMPOSITION

The instantaneous frequency is widely used in non-stationary signal analysis; it can be conceptually understood as the frequency of a sine wave which is the best local approximation of the signal being analyzed. In theory, the signal can be divided into single-component signals and multi-component signals. Instantaneous frequency is only effective for single-component signals as it has only one frequency at any time. This is the reason why the multi-component signals should be decomposed into a set of single-component signals as well.

By using the EMD, an arbitrary signal can be transformed into a linear combination of a set of IMFs. These IMFs have the same number of zero crossings and extrema or differ at most by one; and the mean value of the envelope defined by the local maxima and local minima is zero. Be restricted by these conditions, the IMFs present a series of intrinsic oscillation mode of data. There is only one mode of oscillation in each cycle which defined by zero point, no complex riding waves exist. As defined above, an IMF is not only a narrow band signal, it can be both amplitude
modulated and frequency modulated [1]. The EMD is designated as a sifting process as follows:

1. Connect the identified local extrema of $x(t)$ by cubic spline line as the upper and lower envelops;

2. Calculate the local mean $m_1(t)$ between the envelops;

3. Compute the difference between the data $x(t)$ and $m_1(t)$, and define it as the first IMF: $x(t) - m_1(t) = h_1(t)$

4. Separate $h_1(t)$ from the data to get a residual $r_1(t)$: $x(t) - h_1(t) = r_1(t)$

5. Repeat steps from 1 to 4 using the residual $r_1(t)$ to extract all IMFs.

Actually, the upper and lower envelops are calculated by interpolation, overshoots and undershoots are common. These will generate new extrema and will shift or exaggerate the existing extrema. So the sifting process has to be repeated more times until it satisfies the following termination condition:

$$S_D = \sum_{t=0}^{T} \left[ \frac{\left| (h_{1(k-1)}(t) - h_{1k}(t)) \right|^2}{h_{1(k-1)}^2(t)} \right] \quad (1)$$

$S_D$ indicates the Standard Deviation of two consecutive sifting results, when the value of $S_D$ between 0.2 and 0.3, the sifting process will stop. An example of original audio signal and its first six IMFs is shown in fig. 1.

<figure><img src="image_3.png" /><figcaption>Figure 1. An example of original audio signal and its first 6 IMFs</figcaption></figure>

### III. AUDIO CLASSIFICATION BASED ON EMD

The signal is decomposed according to the local time scales, so EMD has a unique filtering characteristic.

#### A. The filtering characteristic of EMD

In the traditional signal filtering, the time-domain signal is transformed into frequency domain by Fourier transform. Based on the frequency distribution of the signal, the signal filtering can be achieved by using a filter with particular bandwidth. The sifting process of EMD is finished according to the local time scales, and the IMFs are extracted from local high frequency to local low frequency. So EMD can be designated as a dyadic filter bank with adaptive bandwidth which can filter the signal in temporal domain [6]. An

example of Fourier Spectrum of the first six IMFs of an audio signal decomposed by EMD is shown in figure 2.

<figure><img src="image_4.png" /><figcaption>Figure 2. The fourier spectrum of original audio and the first 6 IMFs</figcaption></figure>

A novel filter can be built by using the unique characteristics of EMD. The EMD filter is nonlinear and adaptive and it works in time domain, so there is no need to set the center frequency, band width and any other parameters. The original signal $x(t)$ decomposed by EMD can be expressed by

$$x(t) = \sum_{i=1}^{N} c_i(t) + r_N(t) \quad (2)$$

The frequency of IMFs is decreasing progressively from high to low, so the signal filtering can be achieved by reconstructing the signal with several selected IMFs [7]. For example, the low-pass filter can be built as:

$$x_l(t) = \sum_{i=k}^{N} c_i(t) + r_N(t) \quad (3)$$

Figure 3 shows an example of Fourier spectrum of the original signal and the output of low-pass adaptive filter.

<figure><img src="image_5.png" /><figcaption>Figure 3. The fourier spectrum of original audio and the output of low-pass adaptive filter</figcaption></figure>

#### B. Classification of Audio Signal

Based on the completeness and filtering characteristic of EMD, the audio signal is classified by the analysis model
based on EMD that proposed in this paper before modeling parameters. The main blocks diagram of the proposed model is given in fig.4.

<figure><img src="image_6.png" /><figcaption>Figure 4. The main blocks diagram of proposed model</figcaption></figure>

For a wide range of audio signals, the high frequency components are less important and the signal components above 8 kHz tend to be noise like [4]. Meanwhile, the human ear is more sensitive to low frequency sound. So after the decomposition, the first several IMFs (the local high frequency components of original signal) are considered as noise, which is extracted from the audio. Then the original signal is divided into two parts by this method: low-frequency part is considered as harmonics and high-frequency part is considered as the noise. In order to obtain the number of IMFs, which is required to approximate the original audio effectively, we compared original audio and the reconstruct audio in three situations that the first IMF, or the first two IMFs, or the first three IMFs are cancelled, respectively. The objective measurements of perceived audio quality (PEAQ) from ITU are used for comparison in terms of the scores of objective difference grade (ODG). The test result of symphonies is shown as figure 5.

<figure><img src="image_7.png" /><figcaption>Figure 5. The audio quality comparison with the cancellation of IMFs</figcaption></figure>

By testing a variety of audio, we got a similar result that the quality of reconstruct signal without the first IMF is close to the original audio and much higher than the quality of G.722.1C at the bit rate of 32kbps. The subjective listening tests also show there is no any perceptual distortion for the reconstructed signal. But the ODG score of the reconstructed signal without the first two IMFs decrease badly, and it will get worse while the more IMFs are cancelled. This is because the extraction of IMFs is based on the local time scales of

original signal, and there is no clear boundary in frequency domain between adjacent IMFs. If too many IMFs are cancelled, the low frequency information of original audio will be destroyed. This will cause a sudden drop in the quality for the reconstruct audio.

After the classification of original audio, different parametric modeling methods are applied. The Perceptual Weighted Matching Pursuit (PWMP) is used to estimate the sinusoidal parameters of harmonics, and the noise components are modeled by using the noise-fill method from G.722.1C. The details of parametric mode will be discussed at the next section.

## IV. PARAMETRIC MODEL

### A. Sinusoidal Model

The sinusoidal model assumes that an underlying structure of audio signal is sinusoids that parameterized by time-varying amplitudes, frequencies and phases.

The estimation of sinusoidal parameters is achieved by Perceptual Weighted Matching Pursuit (PWMP) [3]. Matching pursuit is an analysis-by-synthesis algorithm that uses an over-complete dictionary of signal vectors. The study shows that the dictionary consisted of local cosine basis functions can express the audio signal effectively. So we choose the elements of dictionary as below:

$$h_m = \left\{ h_m[n] = \frac{1}{N} e^{j2\pi \frac{m}{M}n} ; n = 0,1,\dots,N-1 \right\} \quad (4)$$

where $m = 0, 1... M-1$, the elements $h_m$ and its complex conjugate form the over-complex dictionary together. The residual signal at $k$-th iteration is:

$$r_{k+1}[n] = r_k[n] - \partial_k h_{m_k}[n] - \partial_k^* h_{m_k}^*[n] = r_k[n] - \frac{2a_k}{N} \cos\left[2\pi \frac{m_k}{M} n + \varphi_k\right] \quad (5)$$

Equation (5) shows that, in the each iteration, the element extracted from the signal is a cosine with constant amplitude and constant frequency. In order to introduce perceptual characteristics of human hearing into the sinusoidal model, the psychoacoustic model is used to calculate the masking threshold. Then the masking threshold is chosen to determine the weighting sequence of the dictionary.

In this paper, a Sinusoidal Modeling with Perceptual Gradient (SMPG) [8] is adopted to improve the modeling efficiency. For the PWMP method mentioned in literature [4], the masking threshold is calculated based on the original audio and the masking threshold is used as the weighting sequence. The synthetic audio signal is not entirely consistent with the original signal, so the masking threshold of original signal can not reflect the masking effect of synthetic audio exactly. In SMPG, the synthetic signal is first reconstructed with sinusoidal parameters in the each iteration. Then the masking threshold of synthetic signal is computed and fed it back to guide next iteration. By using this method, the signal
vector extracted from the residual is most perceptually relevant with the synthesized signal.

The examples of the location extraction of sinusoids from the same audio frame by PWMP and SMPG are shown in figure 6 (a) and figure 6 (b). The blue line in both figures correspond to the spectrum of the audio, the red line in left figure signify the masking threshold of original audio, and the red line in right figure signify the masking threshold of synthesized audio. Then the sinusoidal location are marked by asterisk (*). The result shows that the sinusoids that extracted by PWMP concentrate at low-frequency region, however, some meaningful high-frequency components can be picked out by SMPG. This is effective to improve the audio quality. After estimating the sinusoidal parameters, the local chirp-z transform is applied to improve the accuracy of parameter estimation [9].

<figure><img src="image_8.png" /><figcaption>(a) The sinusoidal location extracted by PWMP</figcaption></figure>

<caption>(b) The sinusoidal location extracted by SMPG</caption>

<caption>Figure 6. The examples of the location extraction of sinusoids</caption>

## B. Noise Model

The high-frequency components that extracted from the audio are considered as noise and modeled by the Noise-fill method of G.722.1C [4]. First, the noise is transformed with MLT; second, the MLT coefficients are divided into several regions equally and the amplitude envelope of each region is computed as the synthesis parameters; at last, based on the envelope of each region, combining with a certain degree of noise attenuation factor to construct a series of MLT coefficients with random signs and synthesize the noise with IMLT.

## C. Evaluation and Test Results

To verify the effectiveness of the algorithm, we analyze three types of audio: symphonies, songs and rock, and synthesize the audio by using 50, 70 and 90 spectral lines respectively. The test data from the MPEG database include 8 audio segments; the sampling rate is 32 kHz. The objective measurements of perceived audio quality (PEAQ) are computed for comparing the proposed algorithm and G.722.1C. The scores of Objective Difference Grade (ODG) are between 0 and -4. If it is close to 0, the sound quality of testing audio is close to the reference audio. The result is given in table I.

## V. CONCLUSIONS

An analysis and synthesis method for audio that based on EMD is proposed in this paper. It takes full advantage of the

unique filtering characteristic and the superiority of EMD in non-stationary signal processing. By using this method, the non-stationarity of original audio can be fully retained and the synthesized audio quality is comparable to G.722.1C. The acceptable quality is achieved for sinusoidal synthesis by using EMD into parametric audio analysis.

<table><caption>TABLE I. OBJECTIVE QUALITY TEST RESULTS</caption><thead><tr><th rowspan="2"></th><th colspan="3">The ODG Scores</th></tr><tr><th>EMD<br>(50 sinusoids)</th><th>EMD<br>(70 sinusoids)</th><th>EMD<br>(90 sinusoids)</th></tr></thead><tbody><tr><th>symphonies</th><td>-3.781</td><td>-3.691</td><td>-3.642</td></tr><tr><th>songs</th><td>-3.794</td><td>-3.732</td><td>-3.660</td></tr><tr><th>rock</th><td>-3.845</td><td>-3.810</td><td>-3.718</td></tr><tr><th></th><td>G.722.1C<br>16kbps</td><td>G.722.1C<br>24kbps</td><td>G.722.1C<br>32kbps</td></tr><tr><th>symphonies</th><td>-3.773</td><td>-3.674</td><td>-3.530</td></tr><tr><th>songs</th><td>-3.765</td><td>-3.654</td><td>-3.522</td></tr><tr><th>rock</th><td>-3.781</td><td>-3.720</td><td>-3.644</td></tr></tbody></table>

## ACKNOWLEDGEMENT

This work was supported by National Natural Science Foundation of China under Grant 60872027 and Grant 61072089, Beijing Natural Science Foundation under Grant 4082006 and Funding Project for Academic Human Resources Development in Institutions of Higher Learning under the Jurisdiction of Beijing Municipality.

## REFERENCES

[1] N.E.Huang, et al, “The empirical mode composition and the Hilbert spectrum for nonlinear and non-stationary time series analysis,” *Pcoceeding of R.Soc.Lond.A*, vol.454, pp.903-995, 1998.

[2] M.Fulton, John.J.Soraghan, “Modeling the IMF’s produced by the EMD of musical audio for synthesis and manipulation,” *2008 International Conference on Systems, Signals and Image Processing, IWSSIP*, pp.229-232, June 2008.

[3] Tony S.Verma, Teresa H.Y.Meng, “A 6KBPS TO 85 KBPS SCALABLE AUDIO CODER,” *2000 IEEE International Conference on Acoustics, Speech, and Signal Processing*, vol.2, pp 877-880, 2000.

[4] Tony S.Verma, “A Perceptually based audio signal model with application to scalable audio compression,” *PHD thesis*, Stanford University, October 1999.

[5] ITU-T Recommendation G.722.1 Low-complexity coding at 24 and 32 kbit/s for hands-free operation in systems with low frame loss, Geneva, May 2005.

[6] P.Flandrin, G.Rilling, P.Goncalves, “Empirical mode decomposition as a filter bank,” *IEEE Signal Processing Letters*, vol.11, No.2, pp.112-114, 2004.

[7] Zhao Zhidong, Ma Chan, “A novel cancellation method of powerline interference in ECG signal based on EMD and adaptive filter”, *2008 11th International Conference on Communication Technology Proceedings, ICCT*, pp.517-520, 2008.

[8] ZHANG Wen-Yao, XU Gang, WANG Yu-Guo, “A Sinusoidal Modeling Method Based on Matching-Pursuits with Perceptual Gradient,” *Journal of Software*, vol.14, No.3, pp.467-471, 2003.

[9] WANG Song, BAO Chang-Chun, “An audio signal modeling based on composite sinusoids,” *Signal Processing*, vol.25, No.8A, pp135-138, 2009.