# Superwideband Bandwidth Extension Using Normalized MDCT Coefficients for Scalable Speech and Audio Coding

Authors: Young Han Lee; Seung Ho Choi
Journal: Advances in Multimedia
 (2013)

---

## Abstract

A bandwidth extension (BWE) algorithm from wideband to superwideband (SWB) is proposed for a scalable speech/audio codec that uses modified discrete cosine transform (MDCT) coefficients as spectral parameters. The superwideband is first split into several subbands that are represented as gain parameters and normalized MDCT coefficients in the proposed BWE algorithm. We then estimate normalized MDCT coefficients of the wideband to be fetched for the superwideband and quantize the fetch indices. After that, we quantize gain parameters by using relative ratios between adjacent subbands. The proposed BWE algorithm is embedded into a standard superwideband codec, the SWB extension of G.729.1 Annex E, and its bitrate and quality are compared with those of the BWE algorithm already employed in the standard superwideband codec. It is shown from the comparison that the proposed BWE algorithm relatively reduces the bitrate by around 19% with better quality, compared to the BWE algorithm in the SWB extension of G.729.1 Annex E.


---

## Body

## 1. Introduction

In early speech communication services, narrowband codecs having a bandwidth of around 3.4 kHz were commonly used since the available network bandwidth was quite limited. These services could provide sufficient quality for comprehension, but it was generally agreed that they did not satisfy users' increasing expectations for higher sound quality. Due to the advances in network technologies, however, this transmission bandwidth has recently been increased [1–3]. Thus, a great deal of research has been focused on further extending the bandwidth of speech and/or audio signals from narrowband to wideband, superwideband, and audio band [4–6].There are two different kinds of approaches for extending the bandwidth according to whether or not the side information is available, as shown in Figure1. As depicted in Figure 1(a), it is usual to realize bandwidth extension by using the side information that is transmitted from the encoder. On the other hand, it is also possible to extend bandwidth only at the decoder without any side information [7], which is shown in Figure 1(b). In other words, instead of using the side information, artificial bandwidth extension can estimate the higher band signal from the lower band signal by using a pattern recognition algorithm such as hidden Markov models (HMMs) [8] Gaussian mixture models (GMMs) [9] and [10–15]. While artificial bandwidth extension algorithms do not require any additional bits for sending the side information, their performance is somewhat restricted depending on the performance of the pattern recognition algorithm used in the extension, compared with the bandwidth extension using the side information [16–20]. Thus, an issue that arose in this bandwidth extension approach is how to improve the performance of the bandwidth extension algorithm with lower increase of the bitrate for the side information.Classification of bandwidth extension algorithms applied in the MDCT domain: (a) bandwidth extension with the side information and (b) without any side information (artificial bandwidth extension).
(a)
(b)In this paper, we propose a superwideband bandwidth extension (BWE) algorithm using normalized spectral coefficients for scalable speech and audio coding, where speech and audio signals are transformed into the spectral domain via a modified discrete cosine transform (MDCT). To this end, the superwideband is split into several subbands, and MDCT coefficients belonging to each subband are normalized by the subband gain that is obtained by squared sum of each MDCT coefficient. We then estimate normalized MDCT coefficients appropriate for the superwideband in order to fetch them from the wideband. Moreover, to improve the coding efficiency, we incorporate a quantization scheme that deals with the relative ratio of gains between adjacent subbands, resulting in a reduced bitrate for the proposed BWE algorithm.The remainder of this paper is organized as follows. A brief review on superwideband extension algorithm is given in Section2. The proposed superwideband extension algorithm is described in Section 3. Performance evaluation is illustrated in Section 4. Finally, conclusions are given in Section 5.


## 2. Superwideband Extension Algorithm

In this section, we briefly review an existing superwideband (SWB) extension algorithm employed in G.729.1 Annex E [17]. The conventional SWB extension algorithm operates in the MDCT domain performed with a length of 640 samples, and it is comprised of a generic mode and a sinusoidal mode. In the generic mode, the higher band signals are reconstructed by transposing the lower band signals with a gain adjustment. However, since tonal signals, which are generated by an instrument such as a guitar, are characterized as the magnitude and the position of several tones, it is not proper to model them by using the generic mode. Thus, the sinusoidal mode tries to model directly tonal components. In fact, the mode selection is done by estimating the tonality of the input audio signals. Therefore, we are interested in how to improve the BWE algorithm realized as the generic mode in G.729.1 Annex E with the same mode selection and sinusoidal mode.Figure2 shows the structure of the conventional BWE algorithm employed as the generic mode in G.729.1 Annex E. As shown in the figure, the conventional BWE algorithm is composed of two blocks: one is a fetch index search and the other is gain compensation and scaling factor quantization block. As a first step, in order to obtain appropriate MDCT coefficients of the higher band, the maximum cross-correlation lag, dj*, for the jth subband is determined as
(1)dj*=arg maxBjlo≤dj≤BjhiC(dj),0≤j≤M-1,
where
(2)C(dj)=|∑k=0Nj-1XH,j(k)X^L(dj+k)|∑k=0Nj-1X^L2(dj+k)
and M is the total number of subbands. In addition, Nj is the number of MDCT coefficients belonging to the jth subband. In (2), XH,j(k) is the kth higher band MDCT coefficient of the jth subband, and X^L(k) is the kth lower band MDCT coefficient reconstructed by the lower band decoder. In addition, Bjlo and Bjhi are the low and high boundaries for the fetch index search in the jth subband, respectively. The parameters are set as M=4, N0=40, N1=N2=70, and N3=100 [17].Figure 2
Structure of a conventional superwideband extension algorithm employed as the generic mode in G.729.1 Annex E [17].Next, in order to compensate for the mismatch betweenXH,j(k) and X^L(dj*+k) for the jth subband, we compute two different scaling factors in the linear spectral domain as well as in the log spectral domain, α1,j and α2,j, which are defined as
(3)α1,j=∑k=0Nj-1XH,j(k)X^L(dj*+k)∑k=0Nj-1X^L2(dj*+k),α2,j=∑k=0Nj-1(Mj(k)-Mj)Dj(k)∑k=0Nj-1(Mj(k)-Mj)2,
where Mj(k)=log10|α1,jX^L(dj*+k)|, Mj=arg maxkMj(k), and Dj(k)=log10|XH,j(k)|-Mj.


## 3. Proposed SWB Extension Algorithm

The proposed BWE algorithm has three major differences from the SWB extension algorithm described in Section2. First, instead of quantizing the scale factor for each subband, the proposed algorithm quantizes the gain of each subband. Also, the replication of the higher band is done by using the MDCT coefficients normalized by the subband gains. Second, in order to improve the coding efficiency, the numbers of subbands are differently assigned for subband gains and fetch indices, where subband gains and fetch indices are related to the spectral shape and the spectral fine structure, respectively. However, since the threshold in quiet for the higher band is much greater than that for the lower band [21], a detailed representation of the spectral shape is more important than that of the spectral fine structure for the higher band. Thus, it would better to assign more bits to the subband gain quantization than the fetch index quantization. Therefore, we decide to provide more subbands for gain, which is referred to as gain subband, but less for the fetch index, which is referred to as fetch subband. Third, instead of using a maximum correlation criterion, we apply a minimum mean square error (MMSE) criterion for the fetch index search, which can provide better performance to minimize the spectral errors.Figure3 shows the structure of the proposed BWE algorithm. Compared to Figure 2, the proposed BWE algorithm consists of a gain normalization/quantization block and a fetch index search block. In the proposed BWE algorithm, we first estimate the subband gain of the jth gain subband, G(j), using the equation of
(4)G(j)=1Ng,j∑k=0Ng,j-1XH,j2(k),0≤j≤Mg-1,
where Mg is the total number of gain subbands and Ng,j is the number of MDCT coefficients in the jth gain subband. In addition, XH,j(k) is the kth higher band MDCT coefficient in the jth gain subband.Figure 3
Structure of the proposed SWB extension algorithm.In this proposedBWE algorithm, we incorporate the concept of differential quantization to reduce the number of bits for quantizing the subband gains. In other words, we quantize the relative ratios of the subband gains, which corresponds to differential quantization applied in the log spectral domain. This is because the dynamic range of the relative ratio is smaller than that of the subband gain. That is, each subband gain can be quantized as
(5)G^(j)={Qm(G(j)),j=0,G^(j-1)·Qn(G(j)G^(j-1)),1≤j≤Mg-1,
where Qn(x) is an n-bit scalar quantizer for x. Specifically, the gain of the first gain subband should be quantized with a higher bit quantizer than those of the rest of subband gains; that is, m>n. Next, the MDCT coefficients are normalized by using the corresponding quantized gain, such that
(6)X~H,j(k)=XH,j(k)G^(j),
where X~H,j(k) is the kth higher band normalized MDCT coefficient belonging to the jth gain subband.The fetch indices in the proposed BWE algorithm are then obtained based on the MMSE criterion. In other words, a fetch index,dl*, for the lth fetch subband is found by using the equation of
(7)dl*=arg minBllo≤dl≤BlhiE(dl),0≤l≤Mf-1,
where E(dl)=∑k=0Nf,l-1(X~H(k)-X~L(dl+k))2 and X~L(k)=X^L(k)/∑k=0Nf,l-1X^L2(k). In addition, Mf is the total number of fetch subbands, Nf,l is the number of MDCT coefficients belonging to the lth fetch subband, and Bllo and Blhi are the low and high boundaries of the lth fetch subband, respectively.In the proposed BWE algorithm, several combinations are available by varying the settings forMg, Mf, m, and n, whereas parameters such as the sinusoidal coding and envelope shaping parameters are used for the same operation as in [17].


## 4. Performance Evaluation

In this section, we compared the performance of the proposed BWE algorithm with that of the conventional BWE algorithm employed in G.729.1 Annex E [17] in terms of bitrate, spectrogram, and quality. Note here that we only replaced the generic mode of the BWE algorithm in the codec.


### 4.1. Bitrate

The number of bits/frame for the conventional BWE algorithm was 79 bits/frame, and 60 bits/frame out of them were fordj, α1,j, and α2,j (for j=0,…,3) in (1) and (3). Even if the BWE algorithm operated as the generic mode, a single sinusoidal component was always modeled, and its location, sign, and amplitude were quantized with 10 bits/frame. The remaining 9 bits/frame out of the 79 bits/frame were assigned for the mode indicator and the envelope shaping that was applied to reduce the pre- and postecho artifacts. For a more detailed explanation on the bit assignment for the conventional BWE algorithm, refer to the literature [17].For the proposed BWE algorithm, there were a number of combinations for the bit assignment. To this end, we had performed exhaustive experiments by changing the design parameters of the proposed BWE algorithm. Eventually, it was found that by settingMg=14, Mf=2, m=5, and n=2, the proposed BWE algorithm could provide better quality with a lower bitrate than the conventional BWE algorithm. It was also shown from the Table 1 that the proposed BWE algorithm achieved a relative bitrate reduction of around 19%, compared to the conventional BWE algorithm.Table 1
Comparison of bit allocation between the conventional BWE algorithm and the proposed one with a typical set of the design parameters, where the algorithms are realized in the SWB extension of G.729.1 Annex E.
Parameter
ConventionalBWE
Proposed BWE
Bit allocation
Typical:Mg= 14, Mf= 2
Initial gain
m
5
Subband gains
28
n · (Mg-1)
26
Fetch indices
32
7Mf
14
Single sinusoidal/mode indicator/envelop shaping
19
19
19
Total
79
64


### 4.2. Spectrogram Comparison

Figures4(b) and 4(c) show the spectrograms for the original audio signal (Figure 4(a)) when the conventional and proposed BWE algorithms were applied, respectively. Note here that the original signal taken from the sound quality assessment material (SQAM) [22] was sampled at a rate of 32 kHz, but the bandwidth of the decoded signal by G.729.1 Annex E was limited up to 14 kHz. In this paper, the spectrograms were obtained by applying a Blackman window whose length was 1,024 samples with an overlap of 512 samples to the decoded signals.Comparison of spectrograms: (a) audio signal, (b) spectrogram of decoded signal by  the conventional BWE algorithm, and (c) that by the proposed BWE algorithm.
(a)
(b)
(c)As shown in Figure4(b), the spectrogram of decoded signal by the conventional BWE algorithm had horizontal lines at round 9.6 kHz and 11.6 kHz, as denoted by a dotted box, which were caused by the gain mismatch.  This mismatch, however, was mitigated by using the proposed BWE algorithm; thus, there were no horizontal lines in Figure 4(b). These results implied that the proposed BWE algorithm could provide better quality than the conventional BWE algorithm.


### 4.3. Quality Preference Test

As a subjective measure, we conducted an AB preference test between the conventional BWE algorithm and the proposed one. Here, we chose three different signal types for the preference test, including speech, music, and noisy speech signals. Note here that speech signals were taken from the database [6] and others from SQAM [22]. We prepared five files for each signal type, where each file was preprocessed according to the G.729.1/G.718 SWB processing plan [23]. All files were then presented to nine listeners having no auditory disorders. Table 2 shows the preference test results. The proposed BWE showed higher scores for all the signal types than the conventional BWE. In particular, the strong periodicity of music signal resulted in the best preference score. It could be concluded from the table that the listeners preferred the signals decoded by the proposed BWE algorithm for all signals types, rather than those decoded by the conventional BWE algorithm.Table 2
Preference test results.
Signal type
Preference score (%)
Conventional BWE
No difference
Proposed BWE
Speech
8.89
48.89
42.22
Music
11.11
31.11
57.78
Noisy speech
11.11
48.89
40.00
Average
10.37
42.96
46.67


## 4.1. Bitrate

The number of bits/frame for the conventional BWE algorithm was 79 bits/frame, and 60 bits/frame out of them were fordj, α1,j, and α2,j (for j=0,…,3) in (1) and (3). Even if the BWE algorithm operated as the generic mode, a single sinusoidal component was always modeled, and its location, sign, and amplitude were quantized with 10 bits/frame. The remaining 9 bits/frame out of the 79 bits/frame were assigned for the mode indicator and the envelope shaping that was applied to reduce the pre- and postecho artifacts. For a more detailed explanation on the bit assignment for the conventional BWE algorithm, refer to the literature [17].For the proposed BWE algorithm, there were a number of combinations for the bit assignment. To this end, we had performed exhaustive experiments by changing the design parameters of the proposed BWE algorithm. Eventually, it was found that by settingMg=14, Mf=2, m=5, and n=2, the proposed BWE algorithm could provide better quality with a lower bitrate than the conventional BWE algorithm. It was also shown from the Table 1 that the proposed BWE algorithm achieved a relative bitrate reduction of around 19%, compared to the conventional BWE algorithm.Table 1
Comparison of bit allocation between the conventional BWE algorithm and the proposed one with a typical set of the design parameters, where the algorithms are realized in the SWB extension of G.729.1 Annex E.
Parameter
ConventionalBWE
Proposed BWE
Bit allocation
Typical:Mg= 14, Mf= 2
Initial gain
m
5
Subband gains
28
n · (Mg-1)
26
Fetch indices
32
7Mf
14
Single sinusoidal/mode indicator/envelop shaping
19
19
19
Total
79
64


## 4.2. Spectrogram Comparison

Figures4(b) and 4(c) show the spectrograms for the original audio signal (Figure 4(a)) when the conventional and proposed BWE algorithms were applied, respectively. Note here that the original signal taken from the sound quality assessment material (SQAM) [22] was sampled at a rate of 32 kHz, but the bandwidth of the decoded signal by G.729.1 Annex E was limited up to 14 kHz. In this paper, the spectrograms were obtained by applying a Blackman window whose length was 1,024 samples with an overlap of 512 samples to the decoded signals.Comparison of spectrograms: (a) audio signal, (b) spectrogram of decoded signal by  the conventional BWE algorithm, and (c) that by the proposed BWE algorithm.
(a)
(b)
(c)As shown in Figure4(b), the spectrogram of decoded signal by the conventional BWE algorithm had horizontal lines at round 9.6 kHz and 11.6 kHz, as denoted by a dotted box, which were caused by the gain mismatch.  This mismatch, however, was mitigated by using the proposed BWE algorithm; thus, there were no horizontal lines in Figure 4(b). These results implied that the proposed BWE algorithm could provide better quality than the conventional BWE algorithm.


## 4.3. Quality Preference Test

As a subjective measure, we conducted an AB preference test between the conventional BWE algorithm and the proposed one. Here, we chose three different signal types for the preference test, including speech, music, and noisy speech signals. Note here that speech signals were taken from the database [6] and others from SQAM [22]. We prepared five files for each signal type, where each file was preprocessed according to the G.729.1/G.718 SWB processing plan [23]. All files were then presented to nine listeners having no auditory disorders. Table 2 shows the preference test results. The proposed BWE showed higher scores for all the signal types than the conventional BWE. In particular, the strong periodicity of music signal resulted in the best preference score. It could be concluded from the table that the listeners preferred the signals decoded by the proposed BWE algorithm for all signals types, rather than those decoded by the conventional BWE algorithm.Table 2
Preference test results.
Signal type
Preference score (%)
Conventional BWE
No difference
Proposed BWE
Speech
8.89
48.89
42.22
Music
11.11
31.11
57.78
Noisy speech
11.11
48.89
40.00
Average
10.37
42.96
46.67


## 5. Conclusion

In this paper, we proposed a superwideband bandwidth extension algorithm using normalized MDCT coefficients for scalable speech and audio coding. In the proposed algorithm, the gain of each subband is quantized, and the replication of the higher band is done by using the MDCT coefficients normalized by the subband gains. Also, the numbers of subbands are differently assigned according to subband gains and fetch indices. Moreover, we apply a minimum mean square error (MMSE) criterion for the fetch index search, which can provide better performance to minimize the spectral errors. As a result, it was shown from the spectrogram comparison and the AB preference test that the proposed BWE algorithm provided better quality, especially for the music signal than the BWE algorithm currently employed in G.729.1 Annex E, while it achieved a relative bitrate reduction of 18.99%.



---

*Source: 909124-2013-07-08.xml*