Spectral shape estimation from MDCT coefficients

A method, decoder, and program code for controlling a concealment method for a lost audio frame is provided. A first audio frame and a second audio frame of the received audio signal are decoded to obtain modified discrete cosine transform (MDCT) coefficients. Values of a first spectral shape based upon the MDCT coefficients decoded from the first audio frame decoded and values of a second spectral shape based upon MDCT coefficients decoded from the second audio frame decoded are determined, the spectral shapes each comprising a number of sub-bands. The values of the spectral shapes and frame energies of the first audio frame and second audio frame are transformed into representations of FFT based spectral analyses. A transient condition is detected based on the representations of the FFTs. Responsive to detecting the transient condition, the concealment method is modified by selectively adjusting a spectrum magnitude of a substitution frame spectrum.

TECHNICAL FIELD

The invention relates generally to a method of concealing a lost audio frame associated with a received audio signal. The invention also relates to a decoder configured to conceal a lost audio frame associated with a received coded audio signal.

BACKGROUND

Transmission of speech/audio over modern communications channels/networks is mainly done in the digital domain using a speech/audio codec. This may involve taking the analog signal and digitalizing it using sampling and analog to digital converter (ADC) to obtain digital samples. These digital samples may be further grouped into frames that contain samples from a consecutive period of 10-40 ms depending on the application. These frames may then be processed using a compression algorithm, which reduces the number of bits that needs to be transmitted and which may still achieve as high quality as possible. The encoded bit stream is then transmitted as data packets over the digital network to the receiver. In the receiver, the process is reversed. The data packets may first be decoded to recreate the frame with digital samples which may then be inputted to a digital to analog converter (DAC) to recreate the approximation of the input analog signal at the receiver.FIG.1provides an example of a block diagram of an audio transfer using audio encoder and decoder over a network, such as a digital network, using the above-described approach.

When the data packets are transmitted over the network there can be data packets that may either be dropped by the network due to traffic load or dropped as a result of bit errors making the digital data invalid for decoding. When these events happen, the decoder needs to replace the output signal during periods where it is impossible to do the actual decoding. This replacement process is typically called frame/packet loss concealment.FIG.2illustrates a block diagram of a decoder200including packet loss concealment. When a Bad Frame Indicator (BFI) indicates lost or corrupted frame, PLC202may create a signal to replace the lost/corrupted frame. Otherwise, i.e. when BFI does not indicate lost or corrupted frame, the received signal is decoded by a stream decoder204. A frame erasure may be signalled to the decoder by setting the bad frame indicator variable for the current frame active, i.e. BFI=1. The decoded or concealed frame is then input to DAC206to output an analog signal. Frame/packet loss concealment may also be referred to as error concealment unit (ECU).

There are numerous ways of doing packet loss concealment in a decoder. Some examples are replacing the lost frame with silence and repeating the last frame (or decoding of the last frame parameters. Other solutions try to replace the frame with the most likely continuation of the audio signal. For noise like signals, one solution generates noise with a similar spectral structure. For tonal signals, one may first estimate the characteristics of present tones (frequency, amplitude, and phase) and use these parameters to generate a continuation of the tones at the corresponding temporal locations of lost frames.

An example of another embodiment of an ECU is the Phase ECU, originally described in international patent application no. WO2014123470, where the decoder may continuously save a prototype of the decoded signal during normal decoding. This prototype may be used in case of a lost frame. The prototype is spectrally analyzed, and the noise and tonal ECU functions are combined in the spectral domain. The Phase ECU identifies tones and calculates a spectral temporal replacement of related spectral bins. The other bins may be handled as noise and are scrambled to avoid tonal artifacts in these spectral regions. The resulting recreated spectrum is inverse FFT (fast Fourier transform) transformed into time domain and the signal is processed to create a replacement of the lost frame.

Most signals rarely continue over long periods of time. To better handle error burst, the ECUs may use low resolution spectral estimates of the current input signal as a long term background estimate. During longer bursts, the ECU may change the target for the reconstruction over time. Initially the target is to replace the last signal. Over time the target may be changed over to a low resolution background estimate that would make the estimate more noise like, and eventually moved to a target of muting the output.

For packet loss concealment, one needs a low-resolution spectral representation of the current input. In embodiments described in international patent application no. WO2014123471 (see Appendix 1), the spectral representation may be created from two short FFT, each one being a quarter of the prototype frame and used for the transient detection. With short transforms, estimates may have high variance. To obtain a more stable long-term estimate, the average of the two FFTs may be used. In these embodiments, the FFT bins are grouped into sub-bands that approximately represents a critical band in the auditory system.

The long-term estimate may be used as a target spectrum in longer error bursts so that over time, the lost frames may be replaced with a signal that represents the current input background signal.

The drawback with this approach is that the short FFTs add substantial complexity in the first error frame. The first error frame already has high complexity as it used to do the complete PLC chain of spectral analysis, sinusoid location and temporal compensation of sinusoid components before one can do the reconstruction of the lost signal.

In such a case, the two short FFT (located at each side of the overall prototype frame) may also be used for a spectral transient detector (detecting onsets and offsets). Therefore, any replacement must be accurate enough to also be used for that purpose

More information on how the Phase ECU PLC works can be found in international patent application no. WO2014123471 (see Appendix 1) and 3GPP TS 26.447 V15.0.0 clause 5.4.3.5.

SUMMARY

Various embodiments of the inventive concepts that are disclosed herein may arise from the present realization that the short FFTs add substantial complexity in the first error frame. The first error frame already has high complexity as it is used to do the complete PLC chain of spectral analysis, sinusoid location and temporal compensation of sinusoid components before one can do the reconstruction of the lost signal.

According to some embodiments of inventive concepts, methods are provided to control a concealment method for a lost audio frame associated with a received audio signal. In such methods, a first audio frame of the received audio signal is decoded to obtain modified discrete cosine transform, MDCT coefficients. Values of a first spectral shape are determined based upon the MDCT coefficients decoded from the first audio frame decoded, the first spectral shape comprising a number of sub-bands. A second audio frame of the received audio signal is decoded to obtain MDCT coefficients for the second audio frame. Values of a second spectral shape are determined based upon MDCT coefficients decoded from the second audio frame decoded, the second spectral shape comprising the number of sub-bands. The values of the first spectral shape and a first frame energy of the first audio frame are transformed into a first representation of a first fast Fourier transform, FFT, based spectral analysis and the values of the second spectral shape and a second frame energy of the second audio frame are transformed into a second representation of a second FFT spectral analysis. A transient condition is detected based on the first representation of the first FFT and the second representation of a second FFT. Responsive to detecting the transient condition, the concealment method is modified by selectively adjusting a spectrum magnitude of a substitution frame spectrum.

According to some embodiments of inventive concepts, a decoder apparatus is provided to control a concealment method for a lost audio frame associated with a received audio signal. The decoder apparatus decodes a first audio frame of the received audio signal to obtain modified discrete cosine transform, MDCT, coefficients. The decoder apparatus determines values of a first spectral shape based upon MDCT coefficients decoded from the audio frame decoded, the first spectral shape comprising a number of sub-bands. The decoder apparatus decodes a second audio frame of the received audio signal. The decoder apparatus determines values of a second spectral shape based upon MDCT coefficients decoded from the second audio frame decoded, the second spectral shape comprising the number of sub-bands. The decoder apparatus transforms the values of the first spectral shape and a first frame energy of the first audio frame into a first representation of a first fast Fourier transform, FFT, based spectral analysis and transforms the values of the second spectral shape and a second frame energy of the second audio frame into a second representation of a second FFT spectral analysis. The decoder apparatus detects, based on the first representation of the first FFT and the second representation of a second FFT, a transient condition. The decoder apparatus responsive to detecting the transient condition, modifies the concealment method by selectively adjusting a spectrum magnitude of a substitution frame spectrum.

According to some embodiments of inventive concepts, a method by a computer processor for controlling a concealment method for a lost audio frame associated with a received audio signal is provided. The method includes decoding a first audio frame of the received audio signal based on a modified discrete cosine transform, MDCT. The method includes determining values of a first spectral shape based upon decoded MDCT coefficients from the decoded audio frame and storing the calculated values of the first spectral shape in a shapeoldbuffer, the first spectral shape comprising a number of sub-bands. The method includes determining a first frame energy of the audio frame and storing the calculated first frame energy in an E_woldbuffer. The method includes decoding a second audio frame of the received audio signal. The method includes moving the calculated values of the first spectral shape from the shapeoldbuffer to a shapeoldbuffer. The method includes moving the calculated first frame energy from the E_woldbuffer to an E_wooldbuffer. The method includes determining values of a second spectral shape based upon decoded MDCT coefficients from the decoded second audio frame and storing the calculated values of the second spectral shape in the shapeoldbuffer the second spectral shape comprising the number of sub-bands. The method includes determining (915) a second frame energy of the second audio frame and storing the calculated second frame energy in the E_woldbuffer. the method includes transforming the values of the first spectral shape and the first frame energy into a first representation of a first fast Fourier transform, FFT, based spectral analysis and transforming the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis. The method includes detecting, based on the first representation of the first fast FFT and the second representation of a second FFT, a condition that could lead to suboptimal reconstruction quality of a substitution frame for the lost audio frame when the concealment method is used to create the substitution frame for the lost audio frame. The method includes responsive to detecting the condition, modifying (921) the concealment method by selectively adjusting a spectrum magnitude of a substitution frame spectrum.

The advantage of the techniques in various embodiments described below is that one can avoid using the two short FFTs. This is important as the avoidance directly reduces the complexity of the first lost frame and processing overhead of processing the lost frame. In the first lost frame the complexity is high as it involves both a rather long FFT of the prototype frame and an equally long inverse FFT of the reconstructed spectrum.

DETAILED DESCRIPTION

The inventive concepts described herein reduce the complexity of the PLC. These embodiments relate to reducing complexity in embodiments where the approach used for packet concealment is sinusoidal modeling in the frequency domain, with an additional low-resolution background noise model to better handle burst errors. In this case, over longer error bursts, the approach proceeds from the sinusoidal model towards the low-resolution noise model. The low-resolution noise model may be updated during the first error frame based on the saved prototype frame. The techniques described may also be used to determine a high quality (and low complexity) frequency band estimate of the energy decay over time for the signal in various frequency bands, which may be used to model the band energies of the concealed frame.

When the speech/audio compression is done in the frequency domain, there may already be spectral representation available in the frequency domain, most often in the modified discrete cosine transform (MDCT) domain. The coefficient of the available spectral representation in many situations can be used to form an alternative spectral shape to replace the complexity of short FFTs. For example, the spectral shapes of the first frame error can be used to create spectral estimates corresponding to those that would have been generated by the short FFTs.

In embodiments described herein, the available MDCT coefficients may be used to provide a spectral shape while the energy (or level) for the spectral estimate is based on the energy of the windowed prototype frame. However, the inventors came to the realization that using the MDCT coefficients alone for both shape and level has been found to provide insufficient quality estimates for the two short FFTs that are to be replaced.

The advantage of the techniques described below is that one can avoid using the two short FFTs. This is important as the avoidance directly reduces the complexity of the first lost frame. In the first lost frame the complexity is high as it involves both a rather long FFT of the prototype frame and an equally long inverse FFT of the reconstructed spectrum.

While the MDCT coefficients available in the decoder do not provide a stable energy estimate, the coefficients can be used for a spectral shape estimation. To get the level for the spectral estimate, the energy of a windowed prototype frame may be used as this may produce a better estimate of the actual FFT spectrum.

Avoiding the complexity of using the two shorter FFTs may result in a slight difference in both temporal characteristics and spectral characteristics. Such differences are of minor importance for the use in the form of a long-term estimate of the background signal, and the slight differences are also not a major issue for the transient detector energy decay estimation.

The inventive concept of the reuse of MDCT coefficients (or any other spectral domain information available in the normal coded domain) and the transformation into a spectral shape that can be used instead of the two short FFT transforms reduces complexity and processing overhead of processing the lost frame. This also involves how the MDCT coefficients are grouped into a format that approximates the FFT bins as close as possible.

The decoder apparatus may consist of two units or may be part of the PLC illustrated inFIG.2or the decoder apparatus illustrated inFIG.12andFIG.13. The decoder (1201,1301) may update the spectral shape and frame energy during error free operation. The decoder (1201,1301) may use the saved spectral shapes and frame energies during the first frame error to generate the long-term spectral estimate that is to be used during error concealment. A third component of the decoder (1201,1301) may also be used to determine a frequency band decay to be applied in the PLC reconstruction, such as when there is significant drop in energy.

The reuse of MDCT coefficients typically only generates one spectral shape per frame. Having two spectral shapes during the first error frame may be achieved by generating one spectral shape estimate for each good frame and by also saving the spectral shape estimate from the previous good frame. To obtain the correct level of the spectral estimate, the windowed energy of the corresponding PLC-prototype frame may be saved at the end of the good frame processing in a MDCT based decoder. A good frame means a correctly received error free frame, while a bad frame means an erased, i.e. a lost or corrupted, frame.

During the lost frame, the second unit uses the two saved spectral shapes and frame energies to generate two spectral estimates corresponding to the ones that would have been generated by the two short FFTs. This reduces complexity and processor overhead. Based on the saved shapes and energies, the third unit may establish the decay factors to be used for each frequency band, in the PLC reconstruction of the lost frame. After this the normal processing of the Phase ECU is continued as before, see international patent application no. WO2014123471 (Appendix 1) or 3GPP TS 26.447 V15.0.0 clause 5.4.3.5.

The techniques described herein are not limited to using spectral estimation from MDCT as described above. The techniques can be adapted to work with any other spectral estimation technique that is used in a codec.

The following describes the functions of using the MDCT in more detail.

To obtain the MDCT coefficients, the MDCT is taken over a 20 ms window with a 10 ms advance. When using one transform, e.g. MDCT, to make a sub-band estimate of another transform, e.g. FFT. It is important to make the grouping into sub-bands over the correct coefficients. The PLC prototype frame saved after good frames is 16 ms in length and the transient detector sub band analysis module uses two short FFT of length 4 ms—that is one quarter of the PLC prototype frame. The actual length of these items depends on the sampling frequency used and can be from 8 kHz to 48 kHz. These lengths affect the number of spectral bins in each transform. The two short FFT analysis results are used to determine a conversion factor μ as described below.

Spectral Shape History Update in Good Frames

For the transient analysis, the Phase ECU may use a history of the MDCT based spectral shape and MDCT-synthesis windowed energies to build an image of how the input signal has evolved over time. The spectral shape is calculated based on the decoded MDCT coefficients which holds a spectral representation of the decoded signal. The spectral shape consists of sub-bands where the number of sub-bands, Ngrp, depends on the sampling frequency as seen in Table 1.

For good frames, that is when the bad frame indicator indicates the frame is not a bad frame (e.g., BFI=0), the values of spectral shape and frame energy may be updated. These steps are illustrated in the flowchart inFIG.3. Turning toFIG.3, at operation301, a determination is made as to whether BIF=0. Note that the parameters may only be calculated for the current frame. When the frame before was a good frame, the values saved during the last frame may be moved to the buffers designated as second last frames (i.e., a shapeooldbuffer(s)). The spectral shape shapeold(k) from the last frame are moved and saved in a second buffer shapeoold(k) as follows in operation303:
shapeoold(k)=shapeold(k), 0≤k<Ngrp.  (1)

Similarly, in operation305, the last frames energy is moved to a second buffer E_wooldas:
E_woold=E_wold.  (2)

These updates may be followed by calculation of new values of spectral shape shapeold(k) and frame energy E_woldfor the last frame buffers in operations307and309. Table 2 illustrates how the bins of the current MDCT coefficients may be divided among the sub-bands. The table entries in Table 2 show start coefficients of each sub-band for an embodiment that may be used in the methods described in international application WO 2014/123471. Other sub-bands may be used for other embodiments.

It may be desirable to have the sub-band based spectral shape in the range [0, . . . 1]. This may be achieved by first calculating the total magnitude of the MDCT coefficients (q_d(n)) as:

shape_tot=∑n=0NMDCT-1q_d⁢(n)2(3)
Where NMDCTis the number of MDCT coefficients and depends on the sampling frequency such as the sampling frequencies illustrated in Table 3.

TABLE 3Number of MDCT coefficients for different sampling frequencies.fsNMDCT80008016000160240002403200032044100, 48000480

The calculated value for shape_tot may then be used to normalize the spectral shape of each sub-band which may be determined as

shapeold(k)=1shape_tot⁢∑n=grp⁢_⁢bin⁢(k)grp⁢_⁢bin⁢(k+1)-1q_d⁢(n)2,0≤k<Ngrp(4)
which forms the spectral shape estimate for the new value of the last frame. Note that there may be some MDCT coefficients that are not assigned to the spectral shape. This is a result of not using the DC bin in the corresponding short FFTs.

To be able to use the spectral shape during reconstruction, the frame energy may be calculated based on the windowed prototype frame. This may be determined as:

E_wold=∑n=0Lprot-1(wwhr(n)·xprev(n))2(5)
where wwhrmay be (especially for long term background approximation estimation) the long FFT spectral analysis window, xprevis the Phase ECU time domain prototype signal as used to create a substitution for a potentially upcoming lost frame, and Lprotis the number of samples in the xprev, signal which also corresponds to the length of the time window wwhr.

In an alternative embodiment, the overall spectral approximation performance may be balanced between providing a good background estimate and a good estimate for transient offset detection. This balancing may be done in an embodiment by optionally altering the applied wwhrwindow to differ from the long/16 ms FFT spectral analysis window. One possible approach to alter this is to shorten the window and shift the energy estimation window towards the future so that the energy estimation is further time aligned with the energy content of the short (4 ms) FFT windows. This approach also reduces the complexity of energy alignment calculations. For example, E_woldmay be reduced to the windowed energy of the 3*L_prot/4 (12 ms) most recent synthesized samples, or even the L_prot/2 (8 ms) most recent samples. This may balance the spectral approximation between background estimation (targeting the overall spectral period of 16 ms) and transient offset estimation (targeting the last 4 ms)).

Turning toFIG.4, to avoid the use of old values in the secondary buffers after a bad frame or a burst of bad frames, the shapeoold(k) and E_wooldstates of the spectral shape and frame energy may be re-initialized. Therefore, in the case were a good frame BFI=0 (as illustrated by operation401) is preceded by a bad frame BFIprev=1 (as illustrated by operation403) the calculated values are copied to the secondary buffers as described in the first equations (1) and (2) respectively in operations405and407.

Conversion of Spectral Shape into Short FFT Sub-Band Energies

The transient analysis may use the saved spectral shape and frame energies to analyze how the sub-band energies are evolving over time. These values may be used for two things, the first is for sub-band transient detector and the second is for forming a long-term average Ētranthat may be used to adjust sub-band energies during burst errors. These values form a basis for calculating signal modification values that are used during error bursts.

The spectral shapes and frame energies are used to generate the approximations of sub-band energies for the two last error free frames. This is illustrated in the flow chart ofFIG.5when the bad frame indicator indicates a bad frame (i.e. BFI=1) at operation501. Turning toFIG.5, the first frame represents the sub-band frame energies before the last frame and may be generated in operation503by:
Eoold(k)=μ·shapeoold(k)·E_woold, 0≤k<Ngrp(6)

The second sub-frame frame energies are for the last frame and may be generated in operation505by:
Eold(k)=μ·shapeold(k)·Ewold, 0≤k<Ngrp(7)
where μ is a scalar constant that depends on the sampling frequency and handles the conversion of the MDCT based spectral shape to an approximation of an FFT based spectral analysis, Ētran. An example of μ for various ƒsfrequencies is shown in Table 5.

TABLE 5Phase ECU MDCT to FFT spectral shape conversion factor u tablefsμ80001.9906160004.0445240006.0980320008.153344100, 4800012.2603

The conversion factor μ may be calculated off-line and depends on the MDCT-window and the window used in the FFT for which it serves as an approximation for during lost frame reconstruction. To find these coefficients, the PLC should be run with both methods (original FFT analysis and the reduced complexity approximation of the FFT using the MDCT) active to calculate the conversion factor(s). A convenient method for calculating the conversion factor is to use sine waves. One wave may be used in the center of each group interval and the calculation may be started with the coefficient set to one. The correct value may be calculated by comparing the two methods. Note that the bins in Table 4 show the bin grouping for an FFT with an analysis length that is a quarter of the one used for the spectral analysis used by the PLC on the prototype frame, i.e. if the spectral analysis is made using a 16 ms FFT, the bin grouping is for a 4 ms spectral analysis.

FIG.6illustrates an overview of how the framing and the related Frame structure of the MDCT coder is applied for an asymmetrically located MDCT window and with a segment of look ahead zeros—LA_ZEROS. Note that the signal diagram shows that a frame is only decoded up to a point of ¾ of the current frame due to the use of look ahead zeros (LA_ZEROS−⅜ of the frame length) in the MDCT window. The framing affects which part of the current frame is possible to decode and therefore affects the position of the PLC prototype frame that is saved and used in case the next frame is lost.

FIG.6also illustrates the difference in length of the involved transforms used in the embodiment. Even in MDCT with a length twice the length of the encoded frame each spectral point is represented with two coefficients (compare with FFT where an N sample results in N complex numbers—that is 2 N scalar values) where one may be a time reversal of the other.

FIG.7illustrates an overview of how the framing and the related frame structure of the MDCT coder is applied to determine the sub-band energies and the spectral shapes as described above.FIG.7illustrates the current frame and the previous frame being good frames and shows where in relation to the coding process the method ofFIGS.3and9-11may be performed.

FIG.8illustrates a graphical representation of the different spectral representations. The PLC spectral analysis is made on a 16 ms time segment—this results in a inter bin distance of 62.5 Hz. From a N point FFT one gets N/2+1 bins where the start point is 0 Hz and the last is fs/2 (half the sampling frequency). The same applies for the transient analysis where the short FFTs that are to be replaced—the difference is that the time window is 4 ms—this results in inter bin distance of 250 Hz. For the MDCT which is made over a 20 ms time segment—the inter bin distance becomes 100 Hz after grouping the time and time reversed coefficients for M MDCT and length of M MDCT there are M/4 coefficients after grouping. The MDCT does not have a DC or fs/2 coefficients, so the simplest representation is to have a half bin offset as shown inFIG.8.

In an embodiment, these estimates of the spectral estimates for the transient analysis as described above may be used to replace the spectral estimates used in the transient calculation and concealment adaptation as described in international patent application no. WO2014123471 (see Appendix 1). These estimates may also be used in other situations where spectral estimates are used such as in 3GPP TS 25.447 V. 15.0.0.

For example, turning toFIG.9, a decoder (1201,1301) may decode a first audio frame of a received audio signal based on a MDCT in operation901. In operation903, the decoder (1201,1301) may determine values of a first spectral shape based upon MDCT coefficients from the decoded first audio frame and store the determined values of the first spectral shape in a shapeoldbuffer, the first spectral shape comprising a number of sub-bands. In operation905, the decoder (1201,1301) may determine a first frame energy of the first audio frame and store the determined first frame energy in an E_woldbuffer.

In operation907, the decoder (1201,1301) may decode a second audio frame of the received audio signal based on the MDCT. In operation909, the decoder (1201,1301) may move the determined values of the first spectral shape from the shapeoldbuffer to a shapeooldbuffer. Operation909may correspond to operation303ofFIG.3. In operation911, the decoder (1201,1301) may move the determined first frame energy from the E_woldbuffer to an E_wooldbuffer. Operation911may correspond to operation305ofFIG.3.

In operation913, the decoder (1201,1301) may determine values of a second spectral shape based upon decoded MDCT coefficients from the decoded second audio frame and store the determined values of the second spectral shape in the shape old buffer, the second spectral shape comprising the number of sub-bands. In operation915, the decoder (1201,1301) may determine a second frame energy of the second audio frame and store the calculated second frame energy in the E_woldbuffer.

In operation917, the decoder (1201,1301) may transform the values of the first spectral shape and the first frame energy into a first representation of a first fast Fourier transform, FFT, based spectral analysis and transform the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis.

In operation919, the decoder (1201,1301) may detect, based on the transformed values of the first spectral shape and the values of the second spectral shape, a condition that could lead to suboptimal reconstruction quality of a substitution frame for the lost audio frame when the concealment method is used to create the substitution frame.

In operation921, the decoder (1201,1301), responsive to detecting the condition, may modify the concealment method by selectively adjusting a spectrum magnitude of a substitution frame spectrum.

In one embodiment, the spectral estimates describe above may be used to reduce the complexity and processing overhead in the transient calculation and concealment adaptation such as described in international patent application no. WO2014123471 and 3GPP TS 25.447 V. 15.0.0 clause 5.4.3.5. The Eoold(k) and Eold(k) are used to calculate an energy ratio estimate and transient detection may be done using the bins of Eoold(k) and Eold(k). For example, turning toFIG.10, in operation1001, the sub-band energies of Eoold(k) and Eold(k) may be determined as described above. The frequency group selective transient detection can now be based on the band-wise ratio between the respective band energies of the frames associated with Eoold(k) and Eold(k):

Rold⁢\⁢oold,band(k)=Eold(k)Eoold(k).i
Other ratios may be used.

It is to be noted that the interval Ik=[mk−1+1, . . . mk] corresponds to the frequency band

Bk=[mk-1+1Npart·fs,…,mkNpart·fs],
where ƒsdenotes the audio sampling frequency, and Npartcorresponds to the size of the frame. The lowest lower frequency band boundary m0can be set to 0 but may also be set to a DFT index corresponding to a larger frequency in order to mitigate estimation errors that grow with lower frequencies. The highest upper frequency band boundary mKcan be set to Npart/2 but is preferably chosen to correspond to some lower frequency in which a transient still has a significant audible effect.

The ratios may be compared to certain thresholds. For example, a respective upper threshold for (frequency selective) onset detection1003and a respective lower threshold for (frequency selective) offset detection1005may be used. When the energy ratio is above the upper threshold or below the lower threshold, the concealment method may be modified in operation1007. These operations correspond to operation919ofFIG.9.

An example of modifying the concealment method of operation921ofFIG.9is illustrated inFIG.11. In this embodiment of concealment method modification, the magnitude and phase of a substitution frame spectrum is determined. The magnitude is modified by means of scaling with two factors α(m) and β(m) and the phase is modified with an additive phase component ϑ(m). This leads to the calculation of the substitution frame:
Z(m)=α(m)·β(m)·Y(m)·ekj(θ+ϑ(m))1.where Z(m) is the substitution frame spectrum, α(m) is a first magnitude attenuation factor, β(m) is a second magnitude attenuation factor, Y(m) is a protype frame, θkis a phase shift, and ϑ(m) is an additive phase component.

In this embodiment, the number nburstof observed frame losses in a row is determined where a burst loss counter is incremented with one upon each frame loss and reset to zero upon the reception of a valid frame. Magnitude adaptation, in operation1101, is preferably done if the burst loss counter nburstexceeds some threshold thrburst, e.g. thrburst=3 as determined in operation1103. In that case a value smaller than 1 is used for the attenuation factor, e.g. α(m)=0.1. A further adaptation with regards to the magnitude attenuation factor may be done in case a transient has been detected based on that the indicator Rold\oold,band(k) or alternatively Rold\oold(m) or Rold\ooldhave passed a threshold as determined in operation1105. In that case a suitable adaptation action in operation1107is to modify the second magnitude attenuation factor β(m) such that the total attenuation is controlled by the product of the two factors α(m)·β(m).

β(m) may be set in response to an indicated transient. In case an offset is detected the factor β(m) may be chosen to reflect the energy decrease of the offset. A suitable choice is to set β(m) to the detected gain change:
β(m)=√{square root over (Rold\oold,band(k))} form∈Ik, k=1 . . .K

In case an onset is detected it is rather found advantageous to limit the energy increase in the substitution frame. In that case the factor can be set to some fixed value of e.g. 1, meaning that there is no attenuation but not any amplification either.

Examples of the phase dithering in operation1109are in international patent application no. WO2014123471 (see Appendix 1) and in 3GPP_TS_26.447_v.15.0.0_2018_06, clause 5.4.3.5.3 and need not be described herein in detail.

FIG.12is a schematic block diagram of a decoder that may be used according to the embodiments. The decoder1201comprises an input unit1203configured to receive an encoded audio signal.FIG.11illustrates the frame loss concealment by a logical frame loss concealment-unit1205, which indicates that the decoder is configured to implement a concealment of a lost audio frame, according to the above-described embodiments. Further the decoder comprises a controller1207for implementing the embodiments described above, including the operations illustrated inFIGS.3-5and9-11, and/or operations discussed below with respect to respective Example Embodiments.

For example, the controller1207may be configured to determine properties of the previously received and reconstructed audio signal or in the statistical properties of the observed frame losses for which the substitution of a lost frame according to the original, non-adapted Phase ECU method provide relatively reduced quality. In case such a condition is detected, the controller1207may be configured to modify the element of the concealment methods according to which the substitution frame spectrum is calculated by selectively adjusting the phases or the spectrum magnitudes as described above and output the audio frame towards a receiver for playback. The receiver may be a device having a loudspeaker, a loudspeaker device, a phone, etc.

The decoder may be implemented in hardware. There are numerous variants of circuitry elements that can be used and combined to achieve the functions of the units of the decoder. Such variants are encompassed by the embodiments. Particular examples of hardware implementation of the decoder is implementation in digital signal processor (DSP) hardware and integrated circuit technology, including both general-purpose electronic circuitry and application-specific circuitry.

The decoder described herein could alternatively be implemented e.g. as illustrated inFIG.13, i.e. by one or more of a processor1305and adequate software1309with suitable storage or memory1311therefore, in order to reconstruct the audio signal, which includes performing audio frame loss concealment according to the embodiments described herein, as shown inFIGS.3-5and9-11. The incoming encoded audio signal is received by an input (IN)1303, to which the processor1305and the memory1311are connected. The decoded and reconstructed audio signal obtained from the software is outputted from the output (OUT)1307towards a receiver for playback. As discussed herein, operations of the decoder1301may be performed by processor1305. Moreover, modules may be stored in memory1311, and these modules may provide instructions so that when instructions of a module are executed by processor1305, processor1305performs respective operations.

The technology described above may be used e.g. in a receiver, which can be used in a mobile device (e.g. mobile phone, laptop) or a stationary device, such as a personal computer.

It is to be understood that the choice of interacting units or modules, as well as the naming of the units are only for exemplary purpose, and may be configured in a plurality of alternative ways in order to be able to execute the disclosed process actions.

ABBREVIATIONS

At least some of the following abbreviations may be used in this disclosure. If there is an inconsistency between abbreviations, preference should be given to how it is used above. If listed multiple times below, the first listing should be preferred over any subsequent listing(s).

AbbreviationExplanationADCAnalog to Digital ConverterBFIBad Frame IndicatorBFI_prevBad Frame Indicator of previous frameDACDigital to Analog ConverterFFTFast Fourier TransformMDCTModified Discrete Cosine Transform

REFERENCES

[1] International patent application no. WO2014123470[2] International patent application no. WO2014123471[3] 3GPP TS 26.445 V15.1.0 (clauses 5.3.2.2 and 6.2.4.1), hereby incorporated by reference in its entirety[4] 3GPP TS 26.447 V15.0.0 (clause 5.4.3.5), hereby incorporated by reference in its entirety

LISTING OF EXAMPLE EMBODIMENTS

Example Embodiments are discussed below. Reference numbers/letters are provided in parenthesis by way of example/illustration without limiting example embodiments to particular elements indicated by reference numbers/letters.1. A method by a computer processor for controlling a concealment method for a lost audio frame of a received audio signal, the method comprising:decoding (901) a first audio frame of the received audio signal based on a modified discrete cosine transform, MDCT;determining (307-309,903) values of a first spectral shape based upon decoded MDCT coefficients from the decoded audio frame and storing the calculated values of the first spectral shape in a shape old buffer, the first spectral shape comprising a number of sub-bands;determining (905) a first frame energy of the audio frame and storing the calculated first frame energy in an E_woldbuffer;decoding (907) a second audio frame of the received audio signal;moving (303,909) the calculated values of the first spectral shape from the shapeoldbuffer to a shapeooldbuffer;moving (305,911) the calculated first frame energy from the E_woldbuffer to an E_wooldbuffer;determining (307-309,913) values of a second spectral shape based upon decoded MDCT coefficients from the decoded second audio frame and storing the calculated values of the second spectral shape in the shapeoldbuffer the second spectral shape comprising the number of sub-bands;determining (915) a second frame energy of the second audio frame and storing the calculated second frame energy in the E_woldbuffer;transforming (917) the values of the first spectral shape and the first frame energy into a first representation of a first fast Fourier transform, FFT, based spectral analysis and transforming (917) the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis;detecting (919), based on the first representation of the first fast FFT and the second representation of a second FFT, a condition that could lead to suboptimal reconstruction quality of a substitution frame for the lost audio frame when the concealment method is used to create the substitution frame for the lost audio frame; andresponsive to detecting the condition, modifying (921) the concealment method by selectively adjusting a spectrum magnitude of a substitution frame spectrum.2. The method of Embodiment 1 wherein determining the values of the first spectral shape based upon decoded MDCT coefficients comprises:determining (307) a total magnitude of the MDCT coefficients;normalizing each sub-band value of the first spectral shape; andstoring each normalized sub-band value as a value of the values of the first spectral shape.3. The method of Embodiment 2 wherein the total magnitude of the MDCT coefficients is determined in accordance with

shape_tot=∑n=0NMDCT-1q_d⁢(n)2where shape_tot is the total magnitude of the MDCT coefficients, NMDCTis a number of MDCT coefficients and depends on a sampling frequency, and q_d(n) are the MDCT coefficients4. The method of any of Embodiments 2-3 where the normalizing of each sub-band is normalized in accordance with

shapeold(k)=1shape_tot⁢∑n=grp⁢_⁢bin⁢(k)grp⁢_⁢bin⁢(k+1)-1q_d⁢(n)2,0≤k<Ngrpwhere shapeold(k) is a spectral shape of a sub-band (k), shape_tot is the total magnitude of the MDCT coefficients, q_d(n) are the MDCT coefficients, and Ngrpis a number of the MDCT coefficients, grp_bin(k) is a start index for the MDCT coefficients in sub-band(k), and Ngrpis the number sub-bands.5. The method of any of Embodiments 1-4 wherein frame energy of the first frame energy and the second frame energy is determined in accordance with

E_wold=∑n=0Lprot-1(wwhr(n)·xprev(n))2where E_woldis the frame energy, wwhris along FFT spectral analysis window, xprevis a time domain prototype signal used to create a substitution for a potentially upcoming lost frame, and Lprotis a number of samples in the xprevsignal prey6. The method of any of Embodiments 1-5, wherein transforming the values of the first spectral shape and the first frame energy into the first representation of a first fast FFT based spectral analysis and transforming the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis comprises applying a conversion factor to the values of the first spectral shape and the first frame energy and to the values of the second spectral shape and the second frame energy.7. The method of Embodiment 6 wherein the conversion factor depends on a sampling frequency of the decoding.8. The method of any of Embodiments 4-7, further comprising:transforming the values of the first spectral shape and the first frame energy into the first representation of a first fast FFT based spectral analysis and transforming the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis in accordance with
Eoold(k)=μ·shapeoold(k)·E_woold, 0≤k<Ngrp
and
Eold(k)=μ·shapeold(k)·Ewold, 0≤k<Ngrpwhere Eoold(k) is the first representation, μ is the conversion factor, shapeoold(k) is a spectral shape of a sub-band (k) of the first spectral shape, E_wooldis the first frame energy, Eold(k) is the second representation, shapeold(k) is a spectral shape of a sub-band (k) f the second spectral shape, E_woldis the second frame energy, and Ngrpis the number of sub-bands.9. The method of Embodiment 8 further comprising:determining (1105) if a sub-band transient is above a threshold value based on Eoold(k) and Eold(k);responsive to a sub-band transient being above the threshold value, modifying the concealment method by selectively adjusting (1107) the spectrum magnitude of the substitution frame spectrum.10. The method of Embodiment 9 wherein the substitution frame spectrum is calculated according to an expression of
Z(m)=α(m)·β(m)·Y(m)·ekj(θ+ϑ(m))and adjusting the spectrum magnitude comprises adjusting β(m) (1107), where Z(m) is the substitution frame spectrum, α(m) is a first magnitude attenuation factor, β(m) is a second magnitude attenuation factor, Y(m) is a protype frame, θkis a phase shift, and ϑ(m) is an additive phase component.11. The method of any of Embodiments 1-10 further comprising:receiving a bad frame indicator (403,501);responsive to receiving the bad frame indicator, flushing the shapeooldbuffer and the E_wooldenergy buffer;receiving a new audio frame of the received audio signal;determining values of a new spectral shape (503) based upon decoded MDCT coefficients from the decoded new audio frame and storing the calculated values of the new spectral shape in the shapeoldbuffer and the shapeooldbuffer (405), the new spectral shape comprising a number of sub-bands; anddetermining a new frame energy (505) of the audio frame and storing the calculated new frame energy in the E_woldbuffer and the E_wooldbuffer (407).12. A decoder apparatus (1201,1301) adapted to perform operations according to any of Embodiments 1-11.13. A decoder apparatus (1201,1301) configured to control a concealment method for a lost audio frame of a received audio signal, the decoder apparatus configured to:decode a first audio frame of the received audio signal based on a modified discrete cosine transform, MDCT;determine values of a first spectral shape based upon decoded MDCT coefficients from the decoded audio frame and store the calculated values of the first spectral shape in a shape old buffer, the first spectral shape comprising a number of sub-bands;determine a first frame energy of the audio frame and store the calculated first frame energy in an E_woldbuffer;decode a second audio frame of the received audio signal;move the calculated values of the first spectral shape from the shapeoldbuffer to a shapeooldbuffer;move the calculated first frame energy from the E_woldbuffer to a E_wooldbuffer;determine values of a second spectral shape based upon decoded MDCT coefficients from the decoded second audio frame and store the calculated values of the second spectral shape in the shapeoldbuffer the second spectral shape comprising the number of sub-bands;determining a second frame energy of the second audio frame and storing the calculated second frame energy in the E_woldbuffer;transform the values of the first spectral shape and the first frame energy into a first representation of a first fast Fourier transform, FFT, based spectral analysis and transform the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis;detect, based on the first representation of the first fast FFT and the second representation of a second FFT, a condition that could lead to suboptimal reconstruction quality of a substitution frame for the lost audio frame when the concealment method is used to create the substitution frame for the lost audio frame; andresponsive to detecting the condition, modify the concealment method by selectively adjusting a spectrum magnitude of a substitution frame spectrum.14. The decoder apparatus of Embodiment 13, wherein the decoder apparatus is configured to perform the operations of Embodiments 2-11.15. A decoder apparatus (1201,1301) configured to control a concealment method for a lost audio frame of a received audio signal, the decoder apparatus comprising:a processor (1305); anda memory (1311) storing instructions that, when executed by the processor, cause the decoder apparatus (1201,1301) to perform operations comprising:decoding (901) a first audio frame of the received audio signal based on a modified discrete cosine transform, MDCT;determining (903) values of a first spectral shape based upon decoded MDCT coefficients from the decoded audio frame and storing the calculated values of the first spectral shape in a shapeoldbuffer, the first spectral shape comprising a number of sub-bands;determining (905) a first frame energy of the audio frame and storing the calculated first frame energy in an E_woldbuffer;decoding (907) a second audio frame of the received audio signal;moving (303,909) the calculated values of the first spectral shape from the shapeoldbuffer to a shapeooldbuffer;moving (305,911) the calculated first frame energy from the E_woldbuffer to a E_wooldbuffer;determining (307-309,913) values of a second spectral shape based upon decoded MDCT coefficients from the decoded second audio frame and storing the calculated values of the second spectral shape in the shapeoldbuffer the second spectral shape comprising the number of sub-bands;determining (915) a second frame energy of the second audio frame and storing the calculated second frame energy in the E_woldbuffer;transforming (917) the values of the first spectral shape and the first frame energy into a first representation of a first fast Fourier transform, FFT, based spectral analysis and transforming the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis;detecting (919), based on the first representation of the first fast FFT and the second representation of a second FFT, a condition that could lead to suboptimal reconstruction quality of a substitution frame for the lost audio frame when the concealment method is used to create the substitution frame for the lost audio frame; andresponsive to detecting the condition, modifying (921) the concealment method by selectively adjusting a spectrum magnitude of a substitution frame spectrum.16. The decoder apparatus of Embodiment 1 wherein to determine the values of the first spectral shape based upon decoded MDCT coefficients, the instructions comprise further instructions that, when executed by the processor, cause the apparatus to perform operations comprising:determining (307) a total magnitude of the MDCT coefficients;normalizing each sub-band value of the first spectral shape; andstoring each normalized sub-band value as a value of the values of the first spectral shape.17. The decoder apparatus of Embodiment 16 wherein the total magnitude of the MDCT coefficients is determined in accordance with

shape_tot=∑n=0NMDCT-1q_d⁢(n)2where shape_tot is the total magnitude of the MDCT coefficients, NMDCTis a number of MDCT coefficients and depends on a sampling frequency, and q_d(n) are the MDCT coefficients.18. The decoder apparatus of any of Embodiments 16-17 where the normalizing of each sub-band is normalized in accordance with

shapeold(k)=1shape_tot⁢∑n=grp⁢_⁢bin⁢(k)grp⁢_⁢bin⁢(k+1)-1q_d⁢(n)2,0≤k<Ngrpwhere shapeold(k) is a spectral shape of a sub-band (k), shape_tot is the total magnitude of the MDCT coefficients, q_d(n) are the MDCT coefficients, grp_bin(k) is a start index for the MDCT coefficients in sub-band(k), and Ngrpis the number of sub-bands.19. The decoder apparatus of any of Embodiments 15-18 wherein frame energy of the first frame energy and the second frame energy is determined in accordance with

E_wold=∑n=0Lprot-1(wwhr(n)·xprev(n))2where E_woldis the frame energy, wwhris along FFT spectral analysis window, xprevis a time domain prototype signal used to create a substitution for a potentially upcoming lost frame, and Lprotis a number of samples in the xprevsignal.20. The decoder apparatus of any of Embodiments 15-19, wherein to transform the values of the first spectral shape and the first frame energy into the first representation of a first fast FFT based spectral analysis and to transform the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis, the instructions comprise further instructions that, when executed by the processor, cause the apparatus to perform operations comprising:applying a conversion factor to the values of the first spectral shape and the first frame energy and to the values of the second spectral shape and the second frame energy.21. The decoder apparatus of Embodiment 20 wherein the conversion factor depends on a sampling frequency of the decoding.22. The decoder apparatus of any of Embodiments 20-21, further comprising:transforming the values of the first spectral shape and the first frame energy into the first representation of a first fast FFT based spectral analysis and transforming the values of the second spectral shape and the second frame energy into a second representation of a second FFT spectral analysis in accordance with
Eoold(k)=μ·shapeoold(k)·E_woold, 0≤k<Ngrp
and
Eold(k)=μ·shapeold(k)·Ewold, 0≤k<Ngrpwhere Eold(k) is the first representation, μ is the conversion factor, shapeoold(k) is a spectral shape of a sub-band (k) of the first spectral shape, E_wooldis the first frame energy, Eold(k) is the second representation, shapeold(k) is a spectral shape of a sub-band (k) f the second spectral shape, E_woldis the second frame energy, and Ngrpis the number of sub-bands.23. The decoder apparatus of Embodiment 22 wherein the instructions comprise further instructions that, when executed by the processor, cause the apparatus to perform operations further comprising:determining (1105) if a sub-band transient is above a threshold value based on Eoold(k) and Eold(k); andresponsive to a sub-band transient being above the threshold value, modifying the concealment method by selectively adjusting (1107) the spectrum magnitude of the substitution frame spectrum.24. The decoder apparatus of Embodiment 22 wherein the substitution frame spectrum is calculated according to an expression of
Z(m)=α(m)·β(m)·Y(m)·ekj(θk+ϑ(m))and adjusting the spectrum magnitude comprises adjusting β(m) (1107), where Z(m) is the substitution frame spectrum, α(m) is a first magnitude attenuation factor, β(m) is a second magnitude attenuation factor, Y(m) is a protype frame, θkis a phase shift, and ϑ(m) is an additive phase component25. The decoder apparatus of any of Embodiments 1-10 wherein the instructions comprise further instructions that, when executed by the processor, cause the apparatus to perform operations further comprising:receiving a bad frame indicator (403,501);responsive to receiving the bad frame indicator, flushing the shapeooldbuffer and the E_wooldenergy buffer;receiving a new audio frame of the received audio signal;determining values of a new spectral shape (503) based upon decoded MDCT coefficients from the decoded new audio frame and storing the calculated values of the new spectral shape in the shapeoldbuffer and the shapeooldbuffer (405), the new spectral shape comprising a number of sub-bands; anddetermining a new frame energy (505) of the audio frame and storing the calculated new frame energy in the E_woldbuffer and the E_wooldbuffer (407).

ADDITIONAL EXPLANATION