Patent Description:
The possibility of reducing the transmission bandwidth by taking advantage of inactive periods of speech or other noise sources are known in the art. Such schemes generally use some form of detection to distinguish between inactive (or silence) and active (non-silence) phases. During inactive phases, a lower bitrate is achieved by stopping the transmission of the ordinary data stream precisely encoding the recorded signal, and only sending silence insertion description (SID) updates instead. SID updates may be transmitted in a regular interval or when changes in the background noise characteristics are detected. The SID frames may then be used at the decoding side to generate a background noise with characteristics similar to the background noise during the active phases so that the stopping of the transmission of the ordinary data stream encoding the recorded signal does not lead to an unpleasant transition from the active phase to the inactive phase at the recipient's side.

<NPL>, and <CIT> relate to background noise estimation for speech encoders and decoders.

However, there is still a need for further reducing the transmission rate. An increasing number of bitrate consumers, such as an increasing number of mobile phones, and an increasing number of more or less bitrate intensive applications, such as wireless transmission broadcast, require a steady reduction of the consumed bitrate.

On the other hand, the synthesized noise should closely emulate the real noise so that the synthesis is transparent for the users.

Accordingly, it is desirable to provide an audio codec scheme supporting noise generation during inactive phases which enables reducing the transmission bitrate and/or helps increasing the achievable noise generation quality.

An objective of the present invention is to provide an audio codec supporting synthetic noise generation during inactive phases which enables a more realistic noise generation at moderate overhead in terms of, for example, bitrate and/or computational complexity.

The object is achieved by the subject matter of the independent claims of the present application.

In particular, it is a basic idea underlying the present invention that the spectral domain may very efficiently be used in order to parameterize the background noise thereby yielding a background noise synthesis which is more realistic and thus leads to a more transparent active to inactive phase switching. Moreover, it has been found out that parameterizing the background noise in the spectral domain enables separating noise from the useful signal and accordingly, parameterizing the background noise in the spectral domain has an advantage when combined with the aforementioned continuous update of the parametric background noise estimate during the active phases as a better separation between noise and useful signal may be achieved in the spectral domain so that no additional transition from one domain to the other is necessary when combining both advantageous aspects of the present application.

In accordance with specific embodiments valuable bitrate may be saved with maintaining the noise generation quality within inactive phases, by continuously updating the parametric background noise estimate during an active phase so that the noise generation may immediately be started with upon the entrance of an inactive phase following the active phase. For example, the continuous update may be performed at the decoding side, and there is no need to preliminarily provide the decoding side with a coded representation of the background noise during a warm-up phase immediately following the detection of the inactive phase which provision would consume valuable bitrate, since the decoding side has continuously updated the parametric background noise estimate during the active phase and is, thus, prepared at any time to immediately enter the inactive phase with an appropriate noise generation. Likewise, such a warm-up phase may be avoided if the parametric background noise estimate is done at the encoding side. Instead of preliminarily continuing with providing the decoding side with a conventionally coded representation of the background noise upon detecting the entrance of the inactive phase in order to learn the background noise and inform the decoding side after the learning phase accordingly, the encoder is able to provide the decoder with the necessary parametric background noise estimate immediately upon detecting the entrance of the inactive phase by falling back on the parametric background noise estimate continuously updated during the past active phase thereby avoiding the bitrate consuming preliminary further prosecution of supererogatorily encoding the background noise.

Further advantageous details of embodiments of the present invention are the subject of the dependent claims.

Preferred embodiments of the present application are described below with respect to the Figures among which:.

<FIG> shows an audio encoder according to an embodiment of the present invention. The audio encoder of <FIG> comprises a background noise estimator <NUM>, an encoding engine <NUM>, a detector <NUM>, an audio signal input <NUM> and a data stream output <NUM>. Provider <NUM>, encoding engine <NUM> and detector <NUM> have an input connected to audio signal input <NUM>, respectively. Outputs of estimator <NUM> and encoding engine <NUM> are respectively connected to data stream output <NUM> via a switch <NUM>. Switch <NUM>, estimator <NUM> and encoding engine <NUM> have a control input connected to an output of detector <NUM>, respectively.

The encoder <NUM> encodes the input audio signal into a data stream <NUM> during an active phase <NUM> and the detector <NUM> is configured to detect an entrance <NUM> of an inactive phase <NUM> following the active phase <NUM> based on the input signal. The portion of data stream <NUM> output by encoding engine <NUM> is denoted <NUM>.

The background noise estimator <NUM> is configured to determine a parametric background noise estimate based on a spectral decomposition representation of an input audio signal so that the parametric background noise estimate spectrally describes a spectral envelope of a background noise of the input audio signal. The determination may be commenced upon entering the inactive phase <NUM>, i.e. immediately following the time instant <NUM> at which detector <NUM> detects the inactivity. In that case, normal portion <NUM> of data stream <NUM> would slightly extend into the inactive phase, i.e. it would last for another brief period sufficient for background noise estimator <NUM> to learn/estimate the background noise from the input signal which would be, then, be assumed to be solely composed of background noise.

However, the embodiments described below take another line. According to alternative embodiments described further below, the determination may continuously be performed during the active phases to update the estimate for immediate use upon entering the inactive phase.

In any case, the audio encoder <NUM> is configured to encode into the data stream <NUM> the parametric background noise estimate during the inactive phase <NUM> such as by use of SID frames <NUM> and <NUM>.

Thus, although many of the subsequently explained embodiments refer to cases where the noise estimate is continuously performed during the active phases so as to be able to immediately commence noise synthesis this is not necessarily the case and the implementation could be different therefrom. Generally, all the details presented in these advantageous embodiments shall be understood to also explain or disclose embodiments where the respective noise estimate is done in upon detecting the noise estimate, for example.

Thus, the background noise estimator <NUM> may be configured to continuously update the parametric background noise estimate during the active phase <NUM> based on the input audio signal entering the audio encoder <NUM> at input <NUM>. Although <FIG> suggests that the background noise estimator <NUM> may derive the continuous update of the parametric background noise estimate based on the audio signal as input at input <NUM>, this is not necessarily the case. The background noise estimator <NUM> may alternatively or additionally obtain a version of the audio signal from encoding engine <NUM> as illustrated by dashed line <NUM>. In that case, the background noise estimator <NUM> would alternatively or additionally be connected to input <NUM> indirectly via connection line <NUM> and encoding engine <NUM> respectively. In particular, different possibilities exist for background noise estimator <NUM> to continuously update the background noise estimate and some of these possibilities are described further below.

The encoding engine <NUM> is configured to encode the input audio signal arriving at input <NUM> into a data stream during the active phase <NUM>. The active phase shall encompass all times where a useful information is contained within the audio signal such as speech or other useful sound of a noise source. On the other hand, sounds with an almost time-invariant characteristic such as a time-invariance spectrum as caused, for example, by rain or traffic in the background of a speaker, shall be classified as background noise and whenever merely this background noise is present, the respective time period shall be classified as an inactive phase <NUM>. The detector <NUM> is responsible for detecting the entrance of an inactive phase <NUM> following the active phase <NUM> based on the input audio signal at input <NUM>. In other words, the detector <NUM> distinguishes between two phases, namely active phase and inactive phase wherein the detector <NUM> decides as to which phase is currently present. The detector <NUM> informs encoding engine <NUM> about the currently present phase and as already mentioned, encoding engine <NUM> performs the encoding of the input audio signal into the data stream during the active phases <NUM>. Detector <NUM> controls switch <NUM> accordingly so that the data stream output by encoding engine <NUM> is output at output <NUM>. During inactive phases, the encoding engine <NUM> may stop encoding the input audio signal. At least, the data stream outputted at output <NUM> is no longer fed by any data stream possibly output by the encoding engine <NUM>. In addition to that, the encoding engine <NUM> may only perform minimum processing to support the estimator <NUM> with some state variable updates. This action will greatly reduce the computational power. Switch <NUM> is, for example, set such that the output of estimator <NUM> is connected to output <NUM> instead of the encoding engine's output. This way, valuable transmission bitrate for transmitting the bitstream output at output <NUM> is reduced.

In case of the background noise estimator <NUM> being configured to continuously update the parametric background noise estimate during the active phase <NUM> based on the input audio signal <NUM> as already mentioned above, estimator <NUM> is able to insert into the data stream <NUM> output at output <NUM> the parametric background noise estimate as continuously updated during the active phase <NUM> immediately following the transition from the active phase <NUM> to the inactive phase <NUM>, i.e. immediately upon the entrance into the inactive phase <NUM>. Background noise estimator <NUM> may, for example, insert a silence insertion descriptor frame <NUM> into the data stream <NUM> immediately following the end of the active phase <NUM> and immediately following the time instant <NUM> at which the detector <NUM> detected the entrance of the inactive phase <NUM>. In other words, there is no time gap between the detectors detection of the entrance of the inactive phase <NUM> and the insertion of the SID <NUM> necessary due to the background noise estimator's continuous update of the parametric background noise estimate during the active phase <NUM>.

Thus, summarizing the above description of the audio encoder <NUM> of <FIG> in accordance with a preferred option of implementing the embodiment of <FIG>, same may operate as follows. Imagine, for illustration purposes, that an active phase <NUM> is currently present. In this case, the encoding engine <NUM> currently encodes the input audio signal at input <NUM> into the data stream <NUM>. Switch <NUM> connects the output of encoding engine <NUM> to the output <NUM>. Encoding engine <NUM> may use parametric coding and/transform coding in order to encode the input audio signal <NUM> into the data stream. In particular, encoding engine <NUM> may encode the input audio signal in units of frames with each frame encoding one of consecutive - partially mutually overlapping - time intervals of the input audio signal. Encoding engine <NUM> may additionally have the ability to switch between different coding modes between the consecutive frames of the data stream. For example, some frames may be encoded using predictive coding such as CELP coding, and some other frames may be coded using transform coding such as TCX or AAC coding. Reference is made, for example, to USAC and its coding modes as described in ISO/IEC CD <NUM>-<NUM> dated September <NUM>, <NUM>.

The background noise estimator <NUM> continuously updates the parametric background noise estimate during the active phase <NUM>. Accordingly, the background noise estimator <NUM> may be configured to distinguish between a noise component and a useful signal component within the input audio signal in order to determine the parametric background noise estimate merely from the noise component. The background noise estimator <NUM> performs this updating in a spectral domain such as a spectral domain also used for transform coding within encoding engine <NUM>. Moreover, the background noise estimator <NUM> may perform the updating based on an excitation or residual signal obtained as an intermediate result within encoding engine <NUM> during, for example, transform coding a LPC-based filtered version of the input signal rather than the audio signal as entering input <NUM> or as lossy coded into the data stream. By doing so, a large amount of the useful signal component within the input audio signal would already have been removed so that the detection of the noise component is easier for the background noise estimator <NUM>. As the spectral domain, a lapped transform domain such as an MDCT domain, or a filterbank domain such as a complex valued filterbank domain such as an QMF domain may be used.

During the active phase <NUM>, detector <NUM> is also continuously running to detect an entrance of the inactive phase <NUM>. The detector <NUM> may be embodied as a voice/sound activity detector (VAD/SAD) or some other means which decides whether a useful signal component is currently present within the input audio signal or not. A base criterion for detector <NUM> in order to decide whether an active phase <NUM> continues could be checking whether a low-pass filtered power of the input audio signal remains below a certain threshold, assuming that an inactive phase is entered as soon as the threshold is exceeded.

Independent from the exact way the detector <NUM> performs the detection of the entrance of the inactive phase <NUM> following the active phase <NUM>, the detector <NUM> immediately informs the other entities <NUM>, <NUM> and <NUM> of the entrance of the inactive phase <NUM>. In case of the background noise estimator's continuous update of the parametric background noise estimate during the active phase <NUM>, the data stream <NUM> output at output <NUM> may be immediately prevented from being further fed from encoding engine <NUM>. Rather, the background noise estimator <NUM> would, immediately upon being informed of the entrance of the inactive phase <NUM>, insert into the data stream <NUM> the information on the last update of the parametric background noise estimate in the form of the SID frame <NUM>. That is, SID frame <NUM> could immediately follow the last frame of encoding engine which encodes the frame of the audio signal concerning the time interval within which the detector <NUM> detected the inactive phase entrance.

Normally, the background noise does not change very often. In most cases, the background noise tends to be something invariant in time. Accordingly, after the background noise estimator <NUM> inserted SID frame <NUM> immediately after the detector <NUM> detecting the beginning of the inactive phase <NUM>, any data stream transmission may be interrupted so that in this interruption phase <NUM>, the data stream <NUM> does not consume any bitrate or merely a minimum bitrate required for some transmission purposes. In order to maintain a minimum bitrate, background noise estimator <NUM> may intermittently repeat the output of SID <NUM>.

However, despite the tendency of background noise to not change in time, it nevertheless may happen that the background noise changes. For example, imagine a mobile phone user leaving the car so that the background noise changes from motor noise to traffic noise outside the car during the user phoning. In order to track such changes of the background noise, the background noise estimator <NUM> may be configured to continuously survey the background noise even during the inactive phase <NUM>. Whenever the background noise estimator <NUM> determines that the parametric background noise estimate changes by an amount which exceeds some threshold, background estimator <NUM> may insert an updated version of parametric background noise estimate into the data stream <NUM> via another SID <NUM>, whereinafter another interruption phase <NUM> may follow until, for example, another active phase <NUM> starts as detected by detector <NUM> and so forth. Naturally, SID frames revealing the currently updated parametric background noise estimate may alternatively or additionally interspersed within the inactive phases in an intermediate manner independent from changes in the parametric background noise estimate.

Obviously, the data stream <NUM> output by encoding engine <NUM> and indicated in <FIG> by use of hatching, consumes more transmission bitrate than the data stream fragments <NUM> and <NUM> to be transmitted during the inactive phases <NUM> and accordingly the bitrate savings are considerable.

Moreover, in case of the background noise estimator <NUM> being able to immediately start with proceeding to further feed the data stream <NUM> by the above optional continuous estimate update, it is not necessary to preliminarily continue transmitting the data stream <NUM> of encoding engine <NUM> beyond the inactive phase detection point in time <NUM>, thereby further reducing the overall consumed bitrate.

As will be explained in more detail below with regard to more specific embodiments, the encoding engine <NUM> may be configured to, in encoding the input audio signal, predictively code the input audio signal into linear prediction coefficients and an excitation signal with transform coding the excitation signal and coding the linear prediction coefficients into the data stream <NUM> and <NUM>, respectively. One possible implementation is shown in <FIG>. According to <FIG>, the encoding engine <NUM> comprises a transformer <NUM>, a frequency domain noise shaper <NUM> and a quantizer <NUM> which are serially connected in the order of their mentioning between an audio signal input <NUM> and a data stream output <NUM> of encoding engine <NUM>. Further, the encoding engine <NUM> of <FIG> comprises a linear prediction analysis module <NUM> which is configured to determine linear prediction coefficients from the audio signal <NUM> by respective analysis windowing of portions of the audio signal and applying an autocorrelation on the windowed portions, or determine an autocorrelation on the basis of the transforms in the transform domain of the input audio signal as output by transformer <NUM> with using the power spectrum thereof and applying an inverse DFT onto so as to determine the autocorrelation, with subsequently performing LPC estimation based on the autocorrelation such as using a (Wiener-) Levinson-Durbin algorithm.

Based on the linear prediction coefficients determined by the linear prediction analysis module <NUM>, the data stream output at output <NUM> is fed with respective information on the LPCs, and the frequency domain noise shaper is controlled so as to spectrally shape the audio signal's spectrogram in accordance with a transfer function corresponding to the transfer function of a linear prediction analysis filter determined by the linear prediction coefficients output by module <NUM>. A quantization of the LPCs for transmitting them in the data stream may be performed in the LSP/LSF domain and using interpolation so as to reduce the transmission rate compared to the analysis rate in the analyzer <NUM>. Further, the LPC to spectral weighting conversion performed in the FDNS may involve applying a ODFT onto the LPCs and applying the resulting weighting values onto the transformer's spectra as divisor.

Quantizer <NUM> then quantizes the transform coefficients of the spectrally formed (flattened) spectrogram. For example, the transformer <NUM> uses a lapped transform such as an MDCT in order to transfer the audio signal from time domain to spectral domain, thereby obtaining consecutive transforms corresponding to overlapping windowed portions of the input audio signal which are then spectrally formed by the frequency domain noise shaper <NUM> by weighting these transforms in accordance with the LP analysis filter's transfer function.

The shaped spectrogram may be interpreted as an excitation signal and as it is illustrated by dashed arrow <NUM>, the background noise estimator <NUM> may be configured to update the parametric background noise estimate using this excitation signal. Alternatively, as indicated by dashed arrow <NUM>, the background noise estimator <NUM> may use the lapped transform representation as output by transformer <NUM> as a basis for the update directly, i.e. without the frequency domain noise shaping by noise shaper <NUM>.

Further details regarding possible implementation of the elements shown in <FIG> are derivable from the subsequently more detailed embodiments and it is noted that all of these details are individually transferable to the elements of <FIG> and <FIG>.

Before, however, describing these more detailed embodiments, reference is made to <FIG>, which shows that additionally or alternatively, the parametric background noise estimate update may be performed at the decoder side.

The audio decoder <NUM> of <FIG> is configured to decode a data stream entering at an input <NUM> of decoder <NUM> so as to reconstruct therefrom an audio signal to be output at an output <NUM> of decoder <NUM>. The data stream comprises at least an active phase <NUM> followed by an inactive phase <NUM>. Internally, the audio decoder <NUM> comprises a background noise estimator <NUM>, a decoding engine <NUM>, a parametric random generator <NUM> and a background noise generator <NUM>. Decoding engine <NUM> is connected between input <NUM> and output <NUM> and likewise, the serial connection of provider <NUM>, background noise generator <NUM> and parametric random generator <NUM> are connected between input <NUM> and output <NUM>. The decoder <NUM> is configured to reconstruct the audio signal from the data stream during the active phase, so that the audio signal <NUM> as output at output <NUM> comprises noise and useful sound in an appropriate quality.

The background noise estimator <NUM> is configured to determine a parametric background noise estimate based on a spectral decomposition representation of the input audio signal obtained from the data stream so that the parametric background noise estimate spectrally describes the spectral envelope of background noise of the input audio signal. The parametric random generator <NUM> and the background noise generator <NUM> are configured to reconstruct the audio signal during the inactive phase by controlling the parametric random generator during the inactive phase with the parametric background noise estimate.

However, as indicated by dashed lines in <FIG>, the audio decoder <NUM> may not comprise the estimator <NUM>. Rather, the data stream may have, as indicated above, encoded therein a parametric background noise estimate which spectrally describes the spectral envelope of the background noise. In that case, the decoder <NUM> may be configured to reconstruct the audio signal from the data stream during the active phase, while parametric random generator <NUM> and background noise generator <NUM> cooperate so that generator <NUM> synthesizes the audio signal during the inactive phase by controlling the parametric random generator <NUM> during the inactive phase <NUM> depending on the parametric background noise estimate.

If, however, estimator <NUM> is present, decoder <NUM> of <FIG> could be informed on the entrance <NUM> of the inactive phase <NUM> by way of the data stream <NUM> such as by use of a starting inactivity flag. Then, decoder <NUM> could proceed to continue to decode a preliminarily further fed portion <NUM> and background noise estimator could learn/estimate the background noise within that preliminary time following time instant <NUM>. However, in compliance with the above embodiments of <FIG> and <FIG>, it is possible that the background noise estimator <NUM> is configured to continuously update the parametric background noise estimate from the data stream during the active phase.

The background noise estimator <NUM> may not be connected to input <NUM> directly but via the decoding engine <NUM> as illustrated by dashed line <NUM> so as to obtain from the decoding engine <NUM> some reconstructed version of the audio signal. In principle, the background noise estimator <NUM> may be configured to operate very similar to the background noise estimator <NUM>, besides the fact that the background noise estimator <NUM> has merely access to the reconstructible version of the audio signal, i.e. including the loss caused by quantization at the encoding side.

The parametric random generator <NUM> may comprise one or more true or pseudo random number generators, the sequence of values output by which may conform to a statistical distribution which may be parametrically set via the background noise generator <NUM>.

The background noise generator <NUM> is configured to synthesize the audio signal <NUM> during the inactive phase <NUM> by controlling the parametric random generator <NUM> during the inactive phase <NUM> depending on the parametric background noise estimate as obtained from the background noise estimator <NUM>. Although both entities <NUM> and <NUM> are shown to be serially connected, the serial connection should not be interpreted as being limiting. The generators <NUM> and <NUM> could be interlinked. In fact, generator <NUM> could be interpreted to be part of generator <NUM>.

Thus, in accordance with an advantageous implementation of <FIG>, the mode of operation of the audio decoder <NUM> of <FIG> may be as follows. During an active phase <NUM> input <NUM> is continuously provided with a data stream portion <NUM> which is to be processed by decoding engine <NUM> during the active phase <NUM>. The data stream <NUM> entering at input <NUM> then stops the transmission of data stream portion <NUM> dedicated for decoding engine <NUM> at some time instant <NUM>. That is, no further frame of data stream portion is available at time instant <NUM> for decoding by engine <NUM>. The signalization of the entrance of the inactive phase <NUM> may either be the disruption of the transmission of the data stream portion <NUM>, or may be signaled by some information <NUM> arranged immediately at the beginning of the inactive phase <NUM>.

In any case, the entrance of the inactive phase <NUM> occurs very suddenly, but this is not a problem since the background noise estimator <NUM> has continuously updated the parametric background noise estimate during the active phase <NUM> on the basis of the data stream portion <NUM>. Due to this, the background noise estimator <NUM> is able to provide the background noise generator <NUM> with the newest version of the parametric background noise estimate as soon as the inactive phase <NUM> starts at <NUM>. Accordingly, from time instant <NUM> on, decoding engine <NUM> stops outputting any audio signal reconstruction as the decoding engine <NUM> is not further fed with a data stream portion <NUM>, but the parametric random generator <NUM> is controlled by the background noise generator <NUM> in accordance with a parametric background noise estimate such that an emulation of the background noise may be output at output <NUM> immediately following time instant <NUM> so as to gaplessly follow the reconstructed audio signal as output by decoding engine <NUM> up to time instant <NUM>. Crossfading may be used to transit from the last reconstructed frame of the active phase as output by engine <NUM> to the background noise as determined by the recently updated version of the parametric background noise estimate.

As the background noise estimator <NUM> is configured to continuously update the parametric background noise estimate from the data stream <NUM> during the active phase <NUM>, same may be configured to distinguish between a noise component and a useful signal component within the version of the audio signal as reconstructed from the data stream <NUM> in the active phase <NUM> and to determine the parametric background noise estimate merely from the noise component rather than the useful signal component. The way the background noise estimator <NUM> performs this distinguishing/separation corresponds to the way outlined above with respect to the background noise estimator <NUM>. For example, the excitation or residual signal internally reconstructed from the data stream <NUM> within decoding engine <NUM> may be used.

Similar to <FIG>, <FIG> shows a possible implementation for the decoding engine <NUM>. According to <FIG>, the decoding engine <NUM> comprises an input <NUM> for receiving the data stream portion <NUM> and an output <NUM> for outputting the reconstructed audio signal within the active phase <NUM>. Serially connected therebetween, the decoding engine <NUM> comprises a dequantizer <NUM>, a frequency domain noise shaper <NUM> and an inverse transformer <NUM>, which are connected between input <NUM> and output <NUM> in the order of their mentioning. The data stream portion <NUM> arriving at input <NUM> comprises a transform coded version of the excitation signal, i.e. transform coefficient levels representing the same, which are fed to the input of dequantizer <NUM>, as well as information on linear prediction coefficients, which information is fed to the frequency domain noise shaper <NUM>. The dequantizer <NUM> dequantizes the excitation signal's spectral representation and forwards same to the frequency domain noise shaper <NUM> which, in turn, spectrally forms the spectrogram of the excitation signal (along with the flat quantization noise) in accordance with a transfer function which corresponds to a linear prediction synthesis filter, thereby forming the quantization noise. In principle, FDNS <NUM> of <FIG> acts similar to FDNS of <FIG>: LPCs are extracted from the data stream and then subject to LPC to spectral weight conversion by, for example, applying an ODFT onto the extracted LPCs with then applying the resulting spectral weightings onto the dequantized spectra inbound from dequantizer <NUM> as multiplicators. The retransformer <NUM> then transfers the thus obtained audio signal reconstruction from the spectral domain to the time domain and outputs the reconstructed audio signal thus obtained at output <NUM>. A lapped transform may be used by the inverse transformer <NUM> such as by an IMDCT. As illustrated by dashed arrow <NUM>, the excitation signal's spectrogram may be used by the background noise estimator <NUM> for the parametric background noise update. Alternatively, the spectrogram of the audio signal itself may be used as indicated by dashed arrow <NUM>.

With regard to <FIG> and <FIG> it should by noted that these embodiments for an implementation of the encoding/decoding engines are not to be interpreted as restrictive. Alternative embodiments are also feasible. Moreover, the encoding/decoding engines may be of a multi-mode codec type where the parts of <FIG> and <FIG> merely assume responsibility for encoding/decoding frames having a specific frame coding mode associate therewith, whereas other frames are subject to other parts of the encoding/decoding engines not shown in <FIG> and <FIG>. Such another frame coding mode could also be a predictive coding mode using linear prediction coding for example, but with coding in the time-domain rather than using transform coding.

<FIG> shows a more detailed embodiment of the encoder of <FIG>. In particular, the background noise estimator <NUM> is shown in more detail in <FIG> in accordance with a specific embodiment.

In accordance with <FIG>, the background noise estimator <NUM> comprises a transformer <NUM>, an FDNS <NUM>, an LP analysis module <NUM>, a noise estimator <NUM>, a parameter estimator <NUM>, a stationarity measurer <NUM>, and a quantizer <NUM>. Some of the components just-mentioned may be partially or fully co-owned by encoding engine <NUM>. For example, transformer <NUM> and transformer <NUM> of <FIG> may be the same, LP analysis modules <NUM> and <NUM> may be the same, FDNSs <NUM> and <NUM> may be the same and/or quantizers <NUM> and <NUM> may be implemented in one module.

<FIG> also shows a bitstream packager <NUM> which assumes a passive responsibility for the operation of switch <NUM> in <FIG>. In particular, the VAD as the detector <NUM> of encoder of <FIG> is exemplarily called, simply decides as to which path should be taken, either the path of the audio encoding <NUM> or the path of the background noise estimator <NUM>. To be more precise, encoding engine <NUM> and background noise estimator <NUM> are both connected in parallel between input <NUM> and packager <NUM>, wherein within background noise estimator <NUM>, transformer <NUM>, FDNS <NUM>, LP analysis module <NUM>, noise estimator <NUM>, parameter estimator <NUM>, and quantizer <NUM> are serially connected between input <NUM> and packager <NUM> (in the order of their mentioning), while LP analysis module <NUM> is connected between input <NUM> and an LPC input of FDNS module <NUM> and a further input of quantizer <NUM>, respectively, and stationarity measurer <NUM> is additionally connected between LP analysis module <NUM> and a control input of quantizer <NUM>. The bitstream packager <NUM> simply performs the packaging if it receives an input from any of the entities connected to its inputs.

In the case of transmitting zero frames, i.e. during the interruption phase of the inactive phase, the detector <NUM> informs the background noise estimator <NUM>, in particular the quantizer <NUM>, to stop processing and to not send anything to the bitstream packager <NUM>.

In accordance with <FIG>, detector <NUM> may operate in the time and/or transform/spectral domain so as to detect active/inactive phases.

The mode of operation of the encoder of <FIG> is as follows. As will get clear, the encoder of <FIG> is able to improve the quality of comfort noise such as stationary noise in general, such as car noise, babble noise with many talkers, some musical instruments, and in particular those which are rich in harmonics such as rain drops.

In particular, the encoder of <FIG> is to control a random generator at the decoding side so as to excite transform coefficients such that the noise detected at the encoding side is emulated. Accordingly, before discussing the functionality of the encoder of <FIG> further, reference is briefly made to <FIG> showing a possible embodiment for a decoder which would be able to emulate the comfort noise at the decoding side as instructed by the encoder of <FIG>. More generally, <FIG> shows a possible implementation of a decoder fitting to the encoder of <FIG>.

In particular, the decoder of <FIG> comprises a decoding engine <NUM> so as to decode the data stream portion <NUM> during the active phases and a comfort noise generating part <NUM> for generating the comfort noise based on the information <NUM> and <NUM> provided in the data stream concerning the inactive phases <NUM>. The comfort noise generating part <NUM> comprises a parametric random generator <NUM>, an FDNS <NUM> and an inverse transformer (or synthesizer) <NUM>. Modules <NUM> to <NUM> are serially connected to each other so that at the output of synthesizer <NUM>, the comfort noise results, which fills the gap between the reconstructed audio signal as output by the decoding engine <NUM> during the inactive phases <NUM> as discussed with respect to <FIG>. The processors FDNS <NUM> and inverse transformer <NUM> may be part of the decoding engine <NUM>. In particular, they may be the same as FDNS <NUM> and <NUM> in <FIG>, for example
The mode of operation and functionality of the individual modules of <FIG> and <FIG> will become clearer from the following discussion.

In particular, the transformer <NUM> spectrally decomposes the input signal into a spectrogram such as by using a lapped transform. A noise estimator <NUM> is configured to determine noise parameters therefrom. Concurrently, the voice or sound activity detector <NUM> evaluates the features derived from the input signal so as to detect whether a transition from an active phase to an inactive phase or vice versa takes place. These features used by the detector <NUM> may be in the form of transient/onset detector, tonality measurement, and LPC residual measurement. The transient/onset detector may be used to detect attack (sudden increase of energy) or the beginning of active speech in a clean environment or denoised signal; the tonality measurement may be used to distinguish useful background noise such as siren, telephone ringing and music; LPC residual may be used to get an indication of speech presence in the signal. Based on these features, the detector <NUM> can roughly give an information whether the current frame can be classified for example, as speech, silence, music, or noise.

While the noise estimator <NUM> may be responsible for distinguishing the noise within the spectrogram from the useful signal component therein, such as proposed in [R. Martin, Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics, <NUM>], parameter estimator <NUM> may be responsible for statistically analyzing the noise components and determining parameters for each spectral component, for example, based on the noise component.

The noise estimator <NUM> may, for example, be configured to search for local minima in the spectrogram and the parameter estimator <NUM> may be configured to determine the noise statistics at these portions assuming that the minima in the spectrogram are primarily an attribute of the background noise rather than foreground sound.

As an intermediate note it is emphasized that it may also be possible to perform the estimation by noise estimator without the FDNS <NUM> as the minima do also occur in the non-shaped spectrum. Most of the description of <FIG> would remain the same.

Parameter quantizer <NUM>, in turn, may be configured to parameterize the parameters estimated by parameter estimator <NUM>. For example, the parameters may describe a mean amplitude and a first or higher order momentum of a distribution of the spectral values within the spectrogram of the input signal as far as the noise component is concerned. In order to save bitrate, the parameters may be forwarded to the data stream for insertion into the same within SID frames in a spectral resolution lower than the spectral resolution provided by transformer <NUM>.

The stationarity measurer <NUM> may be configured to derive a measure of stationarity for the noise signal. The parameter estimator <NUM> in turn may use the measure of stationarity so as to decide whether or not a parameter update should be initiated by sending another SID frame such as frame <NUM> in <FIG> or to influence the way the parameters are estimated.

Module <NUM> quantizes the parameters calculated by parameter estimator <NUM> and LP analysis <NUM> and signals this to the decoding side. In particular, prior to quantizing, spectral components may be grouped into groups. Such grouping may be selected in accordance with psychoacoustical aspects such as conforming to the bark scale or the like. The detector <NUM> informs the quantizer <NUM> whether the quantization is needed to be performed or not. In case of no quantization is needed, zero frames should follow.

When transferring the description onto a concrete scenario of switching from an active phase to an inactive phase, then the modules of <FIG> act as follows.

During an active phase, encoding engine <NUM> keeps on coding the audio signal via packager into bitstream. The encoding may be performed frame-wise. Each frame of the data stream may represent one time portion/interval of the audio signal. The audio encoder <NUM> may be configured to encode all frames using LPC coding. The audio encoder <NUM> may be configured to encode some frames as described with respect to <FIG>, called TCX frame coding mode, for example. Remaining ones may be encoded using code-excited linear prediction (CELP) coding such as ACELP coding mode, for example. That is, portion <NUM> of the data stream may comprise a continuous update of LPC coefficients using some LPC transmission rate which may be equal to or greater than the frame rate.

In parallel, noise estimator <NUM> inspects the LPC flattened (LPC analysis filtered) spectra so as to identify the minima kmin within the TCX sprectrogram represented by the sequence of these spectra. Of course, these minima may vary in time t, i.e. kmin(t). Nevertheless, the minima may form traces in the spectrogram output by FDNS <NUM>, and thus, for each consecutive spectrum i at time ti, the minima may be associatable with the minima at the preceding and succeeding spectrum, respectively.

The parameter estimator then derives background noise estimate parameters therefrom such as, for example, a central tendency (mean average, median or the like) m and/or dispersion (standard deviation, variance or the like) d for different spectral components or bands. The derivation may involve a statistical analysis of the consecutive spectral coefficients of the spectra of the spectrogram at the minima, thereby yielding m and d for each minimum at kmin. Interpolation along the spectral dimension between the aforementioned spectrum minima may be performed so as to obtain m and d for other predetermined spectral components or bands. The spectral resolution for the derivation and/or interpolation of the central tendency (mean average) and the derivation of the dispersion (standard deviation, variance or the like) may differ.

The just mentioned parameters are continuously updated per spectrum output by FDNS <NUM>, for example.

As soon as detector <NUM> detects the entrance of an inactive phase, detector <NUM> may inform engine <NUM> accordingly so that no further active frames are forwarded to packager <NUM>. However, the quantizer <NUM> outputs the just-mentioned statistical noise parameters in a first SID frame within the inactive phase, instead. The first SID frame may or may not comprise an update of the LPCs. If an LPC update is present, same may be conveyed within the data stream in the SID frame <NUM> in the format used in portion <NUM>, i.e. during active phase, such as using quantization in the LSF/LSP domain, or differently, such as using spectral weightings corresponding to the LPC analysis or LPC synthesis filter's transfer function such as those which would have been applied by FDNS <NUM> within the framework of encoding engine <NUM> in proceeding with an active phase.

During the inactive phase, noise estimator <NUM>, parameter estimator <NUM> and stationarity measurer <NUM> keep on co-operating so as to keep the decoding side updated on changes in the background noise. In particular, measurer <NUM> checks the spectral weighting defined by the LPCs, so as to identify changes and inform the estimator <NUM> when an SID frame should be sent to the decoder. For example, the measurer <NUM> could activate estimator accordingly whenever the afore-mentioned measure of stationarity indicates a degree of fluctuation in the LPCs which exceeds a certain amount. Additionally or alternatively, estimator could be triggered to send the updated parameters an a regular basis. Between these SID update frames <NUM>, nothing would be send in the data streams, i.e. "zero frames".

At the decoder side, during the active phase, the decoding engine <NUM> assumes responsibility for reconstructing the audio signal. As soon as the inactive phase starts, the adaptive parameter random generator <NUM> uses the dequantized random generator parameters sent during the inactive phase within the data stream from parameter quantizer <NUM> to generate random spectral components, thereby forming a random spectrogram which is spectrally formed within the spectral energy processor <NUM> with the synthesizer <NUM> then performing a retransformation from the spectral domain into the time domain. For spectral formation within FDNS <NUM>, either the most recent LPC coefficients from the most recent active frames may be used or the spectral weighting to be applied by FDNS <NUM> may be derived therefrom by extrapolation, or the SID frame <NUM> itself may convey the information. By this measure, at the beginning of the inactive phase, the FDNS <NUM> continues to spectrally weight the inbound spectrum in accordance with a transfer function of an LPC synthesis filter, with the LPS defining the LPC synthesis filter being derived from the active data portion <NUM> or SID frame <NUM>. However, with the beginning of the inactive phase, the spectrum to be shaped by FDNS <NUM> is the randomly generated spectrum rather than a transform coded on as in case of TCX frame coding mode. Moreover, the spectral shaping applied at <NUM> is merely discontinuously updated by use of the SID frames <NUM>. An interpolation or fading could be performed to gradually switch from one spectral shaping definition to the next during the interruption phases <NUM>.

As shown in <FIG>, the adaptive parametric random generator as <NUM> may additionally, optionally, use the dequantized transform coefficients as contained within the most recent portions of the last active phase in the data stream, namely within data stream portion <NUM> immediately before the entrance of the inactive phase. For example, the usage may be thus that a smooth transition is performed from the spectrogram within the active phase to the random spectrogram within the inactive phase.

Briefly referring back to <FIG> and <FIG>, it follows from the embodiments of <FIG> and <FIG> (and the subsequently explained <FIG>) that the parametric background noise estimate as generated within encoder and/or decoder, may comprise statistical information on a distribution of temporally consecutive spectral values for distinct spectral portions such as bark bands or different spectral components. For each such spectral portion, for example, the statistical information may contain a dispersion measure. The dispersion measure would, accordingly, be defined in the spectral information in a spectrally resolved manner, namely sampled at/for the spectral portions. The spectral resolution, i.e. the number of measures for dispersion and central tendency spread along the spectral axis, may differ between, for example, dispersion measure and the optionally present mean or central tendency measure. The statistical information is contained within the SID frames. It may refer to a shaped spectrum such as the LPC analysis filtered (i.e. LPC flattened) spectrum such as shaped MDCT spectrum which enables synthesis at by synthesizing a random spectrum in accordance with the statistical spectrum and de-shaping same in accordance with a LPC synthesis filter's transfer function. In that case, the spectral shaping information may be present within the SID frames, although it may be left away in the first SID frame <NUM>, for example. However, as will be shown later, this statistical information may alternatively refer to a non-shaped spectrum. Moreover, instead of using a real valued spectrum representation such as an MDCT, a complex valued filterbank spectrum such as QMF spectrum of the audio signal may be used. For example, the QMF spectrum of the audio signal in non-shaped from may be used and statistically described by the statistical information in which case there is no spectral shaping other than contained within the statistical information itself.

Similar to the relationship between the embodiment of <FIG> relative to the embodiment of <FIG>, <FIG> shows a possible implementation of the decoder of <FIG>. As is shown by use of the same reference signs as in <FIG>, the decoder of <FIG> may comprise a noise estimator <NUM>, a parameter estimator <NUM> and a stationarity measurer <NUM>, which operate like the same elements in <FIG>, with the noise estimator <NUM> of <FIG>, however, operating on the transmitted and dequantized spectrogram such as <NUM> or <NUM> in <FIG>. The parameter estimator <NUM> then operates like the one discussed in <FIG>. The same applies with regard to the stationarity measurer <NUM>, which operates on the energy and spectral values or LPC data revealing the temporal development of the LPC analysis filter's (or LPC synthesis filter's) spectrum as transmitted and dequantized via/from the data stream during the active phase.

While elements <NUM>, <NUM> and <NUM> act as the background noise estimator <NUM> of <FIG>, the decoder of <FIG> also comprises an adaptive parametric random generator <NUM> and an FDNS <NUM> as well as an inverse transformer <NUM> and they are connected in series to each other like in <FIG>, so as to output the comfort noise at the output of synthesizer <NUM>. Modules <NUM>, <NUM>, and <NUM> act as the backround noise generator <NUM> of <FIG> with module <NUM> assuming responsibility for the functionality of the parametric random generator <NUM>.

The adaptive parametric random generator <NUM> or <NUM> outputs randomly generated spectral components of the spectrogram in accordance with the parameters determined by parameter estimator <NUM> which, in turn, is triggered using the stationarity measure output by stationarity measurer <NUM>. Processor <NUM> then spectrally shapes the thus generated spectrogram with the inverse transformer <NUM> then performing the transition from the spectral domain to the time domain. Note that when during inactive phase <NUM> the decoder is receiving the information <NUM>, the background noise estimator <NUM> is performing an update of the noise estimates followed by some means of interpolation. Otherwise, if zero frames are received, it will simply do processing such as interpolation and/or fading.

Summarizing <FIG>, these embodiments show that it is technically possible to apply a controlled random generator <NUM> to excite the TCX coefficients, which can be real values such in MDCT or complex values as in FFT. It might also be advantageous to apply the random generator <NUM> on groups of coefficients usually achieved through filterbanks.

The random generator <NUM> is preferably controlled such that same models the type of noise as closely as possible. This could be accomplished if the target noise is known in advance. Some applications may allow this. In many realistic applications where a subject may encounter different types of noise, an adaptive method is required as shown in <FIG>. Accordingly, an adaptive parameter random generator <NUM> is used which could be briefly defined as g = f (x), where x = (x<NUM>, x<NUM>,. ) is a set of random generator parameters as provided by parameter estimators <NUM> and <NUM>, respectively.

To make the parameter random generator adaptive, the random generator parameter estimator <NUM> adequately controls the random generator. Bias compensation may be included in order to compensate for the cases where the data is deemed to be statistically insufficient. This is done to generate a statistically matched model of the noise based on the past frames and it will always update the estimated parameters. An example is given where the random generator <NUM> is supposed to generate a Gaussian noise. In this case, for example, only the mean and variance parameters may be needed and a bias can be calculated and applied to those parameters. A more advanced method can handle any type of noise or distribution and the parameters are not necessarily the moments of a distribution.

For the non-stationary noise, it needs to have a stationarity measure and a less adaptive parametric random generator can then be used. The stationarity measure determined by measurer <NUM> can be derived from the spectral shape of the input signal using various methods like, for example, the Itakura distance measure, the Kullback-Leibler distance measure, etc..

To handle the discontinuous nature of noise updates sent through SID frames such as illustrated by <NUM> in <FIG>, additional information is usually being sent such as the energy and spectral shape of the noise. This information is useful for generating the noise in the decoder having a smooth transition even during a period of discontinuity within the inactive phase. Finally, various smoothing or filtering techniques can be applied to help improve the quality of the comfort noise emulator.

As already noted above, <FIG> and <FIG> on the one hand and <FIG> on the other hand belong to different scenarios. In one scenario corresponding to <FIG> and <FIG>, parametric background noise estimation is done in the encoder based on the processed input signal and later on the parameters are transmitted to the decoder. <FIG> corresponds to the other scenario where the decoder can take care of the parametric background noise estimate based on the past received frames within the active phase. The use of a voice/signal activity detector or noise estimator can be beneficial to help extracting noise components even during active speech, for example.

Among the scenarios shown in <FIG>, the scenario of <FIG> may be preferred as this scenario results in a lower bitrate being transmitted. The scenario of <FIG> and <FIG>, however, has the advantage of having a more accurate noise estimate available.

All of the above embodiments could be combined with bandwidth extension techniques such as spectral band replication (SBR), although bandwidth extension in general may be used.

To illustrate this, see <FIG> shows modules by which the encoders of <FIG> and <FIG> could be extended to perform parametric coding with regard to a higher frequency portion of the input signal. In particular, in accordance with <FIG> a time domain input audio signal is spectrally decomposed by an analysis filterbank <NUM> such as a QMF analysis filterbank as shown in <FIG>. The above embodiments of <FIG> and <FIG> would then be applied only onto a lower frequency portion of the spectral decomposition generated by filterbank <NUM>. In order to convey information on the higher frequency portion to the decoder side, parametric coding is also used. To this end, a regular spectral band replication encoder <NUM> is configured to parameterize the higher frequency portion during active phases and feed information thereon in the form of spectral band replication information within the data stream to the decoding side. A switch <NUM> may be provided between the output of QMF filterbank <NUM> and the input of spectral band replication encoder <NUM> to connect the output of filterbank <NUM> with an input of a spectral band replication encoder <NUM> connected in parallel to encoder <NUM> so as to assume responsibility for the bandwidth extension during inactive phases. That is, switch <NUM> may be controlled like switch <NUM> in <FIG>. As will be outlined in more detail below, the spectral band replication encoder module <NUM> may be configured to operate similar to spectral band replication encoder <NUM>: both may be configured to parameterize the spectral envelope of the input audio signal within the higher frequency portion, i.e. the remaining higher frequency portion not subject to core coding by the encoding engine, for example. However, the spectral band replication encoder module <NUM> may use a minimum time/frequency resolution at which the spectral envelope is parameterized and conveyed within the data stream, whereas spectral band replication encoder <NUM> may be configured to adapt the time/frequency resolution to the input audio signal such as depending on the occurrences of transients within the audio signal.

<FIG> shows a possible implementation of the bandwidth extension encoding module <NUM>. A time/frequency grid setter <NUM>, an energy calculator <NUM> and an energy encoder <NUM> are serially connected to each other between an input and an output of encoding module <NUM>. The time/frequency grid setter <NUM> may be configured to set the time/frequency resolution at which the envelope of the higher frequency portion is determined. For example, a minimum allowed time/frequency resolution is continuously used by encoding module <NUM>. The energy calculator <NUM> may then determine the energy of the higher frequency portion of the spectrogram output by filter bank <NUM> within the higher frequency portion in time/frequency tiles corresponding to the time/frequency resolution, and the energy encoder <NUM> may use entropy coding, for example, in order to insert the energies calculated by calculator <NUM> into the data stream <NUM> (see <FIG>) during the inactive phases such as within SID frames, such as SID frame <NUM>.

It should be noted that the bandwidth extension information generated in accordance with the embodiments of <FIG> may also be used in connection with using a decoder in accordance with any of the embodiments outlined above, such as <FIG>, <FIG> and <FIG>.

Thus, <FIG> make it clear that the comfort noise generation as explained with respect to <FIG> may also be used in connection with spectral band replication. For example, the audio encoders and decoders described above may operate in different operating modes, among which some may comprise spectral band replication and some may not. Super wideband operating modes could, for example, involve spectral band replication. In any case, the above embodiments of <FIG> showing examples for generating comfort noise may be combined with bandwidth extension techniques in the manner described with respect to <FIG>. The spectral band replication encoding module <NUM> being responsible for bandwidth extension during inactive phases may be configured to operate on a very low time and frequency resolution. Compared to the regular spectral band replication processing, encoder <NUM> may operate at a different frequency resolution which entails an additional frequency band table with very low frequency resolution along with IIR smoothing filters in the decoder for every comfort noise generating scale factor band which interpolates the energy scale factors applied in the envelope adjuster during the inactive phases. As just mentioned, the time/frequency grid may be configured to correspond to a lowest possible time resolution.

That is, the bandwidth extension coding may be performed differently in the QMF or spectral domain depending on the silence or active phase being present. In the active phase, i.e. during active frames, regular SBR encoding is carried out by the encoder <NUM>, resulting in a normal SBR data stream which accompanies data streams <NUM> and <NUM>, respectively. In inactive phases or during frames classified as SID frames, only information about the spectral envelope, represented as energy scale factors, may be extracted by application of a time/frequency grid which exhibits a very low frequency resolution, and for example the lowest possible time resolution. The resulting scale factors might be efficiently coded by encoder <NUM> and written to the data stream. In zero frames or during interruption phases <NUM>, no side information may be written to the data stream by the spectral band replication encoding module <NUM>, and therefore no energy calculation may be carried out by calculator <NUM>.

In conformity with <FIG>, <FIG> shows a possible extension of the decoder embodiments of <FIG> and <FIG> to bandwidth extension coding techniques. To be more precise, <FIG> shows a possible embodiment of an audio decoder in accordance with the present application. A core decoder <NUM> is connected in parallel to a comfort noise generator, the comfort noise generator being indicated with reference sign <NUM> and comprising, for example, the noise generation module <NUM> or modules <NUM>, <NUM> and <NUM> of <FIG>. A switch <NUM> is shown as distributing the frames within data streams <NUM> and <NUM>, respectively, onto the core decoder <NUM> or comfort noise generator <NUM> depending on the frame type, namely whether the frame concerns or belongs to an active phase, or concerns or belongs to an inactive phase such as SID frames or zero frames concerning interruption phases. The outputs of core decoder <NUM> and comfort noise generator <NUM> are connected to an input of a spectral bandwidth extension decoder <NUM>, the output of which reveals the reconstructed audio signal.

<FIG> shows a more detailed embodiment of a possible implementation of the bandwidth extension decoder <NUM>.

As shown in <FIG>, the bandwidth extension decoder <NUM> in accordance with the embodiment of <FIG> comprises an input <NUM> for receiving the time domain reconstruction of the low frequency portion of the complete audio signal to be reconstructed. It is input <NUM> which connects the bandwidth extension decoder <NUM> with the outputs of the core decoder <NUM> and the comfort noise generator <NUM> so that the time domain input at input <NUM> may either be the reconstructed lower frequency portion of an audio signal comprising both noise and useful component, or the comfort noise generated for bridging the time between the active phases.

As in accordance with the embodiment of <FIG> the bandwidth extension decoder <NUM> is constructed to perform a spectral bandwidth replication, the decoder <NUM> is called SBR decoder in the following. With respect to <FIG>, however, it is emphasized that these embodiments are not restricted to spectral bandwidth replication. Rather, a more general, alternative way of bandwidth extension may be used with regard to these embodiments as well.

Further, the SBR decoder <NUM> of <FIG> comprises a time-domain output <NUM> for outputting the finally reconstructed audio signal, i.e. either in active phases or inactive phases. Between input <NUM> and output <NUM>, the SBR decoder <NUM> comprises - serially connected in the order of their mentioning - a spectral decomposer <NUM> which may be, as shown in <FIG>, an analysis filterbank such as a QMF analysis filterbank, an HF generator <NUM>, an envelope adjuster <NUM> and a spectral-to-time domain converter <NUM> which may be, as shown in <FIG>, embodied as a synthesis filterbank such as a QMF synthesis filterbank.

Modules <NUM> to <NUM> operate as follows. Spectral decomposer <NUM> spectrally decomposes the time domain input signal so as to obtain a reconstructed low frequency portion. The HF generator <NUM> generates a high frequency replica portion based on the reconstructed low frequency portion and the envelope adjuster <NUM> spectrally forms or shapes the high frequency replica using a representation of a spectral envelope of the high frequency portion as conveyed via the SBR data stream portion and provided by modules not yet discussed but shown in <FIG> above the envelope adjuster <NUM>. Thus, envelope adjuster <NUM> adjusts the envelope of the high frequency replica portion in accordance with the time/frequency grid representation of the transmitted high frequency envelope, and forwards the thus obtained high frequency portion to the spectral-to-temporal domain converter <NUM> for a conversion of the whole frequency spectrum, i.e. spectrally formed high frequency portion along with the reconstructed low frequency portion, to a reconstructed time domain signal at output <NUM>.

As already mentioned above with respect to <FIG>, the high frequency portion spectral envelope may be conveyed within the data stream in the form of energy scale factors and the SBR decoder <NUM> comprises an input <NUM> in order to receive this information on the high frequency portions spectral envelope. As shown in <FIG>, in the case of active phases, i.e. active frames present in the data stream during active phases, inputs <NUM> may be directly connected to the spectral envelope input of the envelope adjuster <NUM> via a respective switch <NUM>. However, the SBR decoder <NUM> additionally comprises a scale factor combiner <NUM>, a scale factor data store <NUM>, an interpolation filtering unit <NUM> such as an IIR filtering unit, and a gain adjuster <NUM>. Modules <NUM>, <NUM>, <NUM> and <NUM> are serially connected to each other between <NUM> and the spectral envelope input of envelope adjuster <NUM> with switch <NUM> being connected between gain adjuster <NUM> and envelope adjuster <NUM> and a further switch <NUM> being connected between scale factor data store <NUM> and filtering unit <NUM>. Switch <NUM> is configured to either connect this scale factor data store <NUM> with the input of filtering unit <NUM>, or a scale factor data restorer <NUM>. In case of SID frames during inactive phases - and optionally in cases of active frames for which a very coarse representation of the high frequency portion spectral envelope is acceptable - switches <NUM> and <NUM> connect the sequence of modules <NUM> to <NUM> between input <NUM> and envelope adjuster <NUM>. The scale factor combiner <NUM> adapts the frequency resolution at which the high frequency portions spectral envelope has been transmitted via the data stream to the resolution, which envelope adjuster <NUM> expects receiving and a scale factor data store <NUM> stores the resulting spectral envelope until a next update. The filtering unit <NUM> filters the spectral envelope in time and/or spectral dimension and the gain adjuster <NUM> adapts the gain of the high frequency portion's spectral envelope. To that end, gain adjuster may combine the envelope data as obtained by unit <NUM> with the actual envelope as derivable from the QMF filterbank output. The scale factor data restorer <NUM> reproduces the scale factor data representing the spectral envelope within interruption phases or zero frames as stored by the scale factor store <NUM>.

Thus, at the decoder side the following processing may be carried out. In active frames or during active phases, regular spectral band replication processing may be applied. During these active periods, the scale factors from the data stream, which are typically available for a higher number of scale factor bands as compared to comfort noise generating processing, are converted to the comfort noise generating frequency resolution by the scale factor combiner <NUM>. The scale factor combiner combines the scale factors for the higher frequency resolution to result in a number of scale factors compliant to CNG by exploiting common frequency band borders of the different frequency band tables. The resulting scale factor values at the output of the scale factor combining unit <NUM> are stored for the reuse in zero frames and later reproduction by restorer <NUM> and are subsequently used for updating the filtering unit <NUM> for the CNG operating mode. In SID frames, a modified SBR data stream reader is applied which extracts the scale factor information from the data stream. The remaining configuration of the SBR processing is initialized with predefined values, the time/frequency grid is initialized to the same time/frequency resolution used in the encoder. The extracted scale factors are fed into filtering unit <NUM>, where, for example, one IIR smoothing filter interpolates the progression of the energy for one low resolution scale factor band over time. In case of zero frames, no payload is read from the bitstream and the SBR configuration including the time/frequency grid is the same as is used in SID frames. In zero frames, the smoothing filters in filtering unit <NUM> are fed with a scale factor value output from the scale factor combining unit <NUM> which have been stored in the last frame containing valid scale factor information. In case the current frame is classified as an inactive frame or SID frame, the comfort noise is generated in TCX domain and transformed back to the time domain. Subsequently, the time domain signal containing the comfort noise is fed into the QMF analysis filterbank <NUM> of the SBR module <NUM>. In QMF domain, bandwidth extension of the comfort noise is performed by means of copy-up transposition within HF generator <NUM> and finally the spectral envelope of the artificially created high frequency part is adjusted by application of energy scale factor information in the envelope adjuster <NUM>. These energy scale factors are obtained by the output of the filtering unit <NUM> and are scaled by the gain adjustment unit <NUM> prior to application in the envelope adjuster <NUM>. In this gain adjustment unit <NUM>, a gain value for scaling the scale factors is calculated and applied in order to compensate for huge energy differences at the border between the low frequency portion and the high frequency content of the signal. The embodiments described above are commonly used in the embodiments of <FIG> and <FIG>. <FIG> shows an embodiment of an audio encoder according to an embodiment of the present application, and <FIG> shows an embodiment of an audio decoder. Details disclosed with regard to these figures shall equally apply to the previously mentioned elements individually.

The audio encoder of <FIG> comprises a QMF analysis filterbank <NUM> for spectrally decomposing an input audio signal. A detector <NUM> and a noise estimator <NUM> are connected to an output of QMF analysis filterbank <NUM>. Noise estimator <NUM> assumes responsibility for the functionality of background noise estimator <NUM>. During active phases, the QMF spectra from QMF analysis filterbank are processed by a parallel connection of a spectral band replication parameter estimator <NUM> followed by some SBR encoder <NUM> on the one hand, and a concatenation of a QMF synthesis filterbank <NUM> followed by a core encoder <NUM> on the other hand. Both parallel paths are connected to a respective input of bitstream packager <NUM>. In case of outputting SID frames, SID frame encoder <NUM> receives the data from the noise estimator <NUM> and outputs the SID frames to bitstream packager <NUM>.

The spectral bandwidth extension data output by estimator <NUM> describe the spectral envelope of the high frequency portion of the spectrogram or spectrum output by the QMF analysis filterbank <NUM>, which is then encoded, such as by entropy coding, by SBR encoder <NUM>. Data stream multiplexer <NUM> inserts the spectral bandwidth extension data in active phases into the data stream output at an output <NUM> of the multiplexer <NUM>.

Detector <NUM> detects whether currently an active or inactive phase is active. Based on this detection, an active frame, an SID frame or a zero frame, i.e. inactive frame, is to currently be output. In other words, module <NUM> decides whether an active phase or an inactive phase is active and if the inactive phase is active, whether or not an SID frame is to be output. The decisions are indicated in <FIG> using I for zero frames, A for active frames, and S for SID frames. A frames which correspond to time intervals of the input signal where the active phase is present are also forwarded to the concatenation of the QMF synthesis filterbank <NUM> and the core encoder <NUM>. The QMF synthesis filterbank <NUM> has a lower frequency resolution or operates at a lower number of QMF subbands when compared to QMF analysis filterbank <NUM> so as to achieve by way of the subband number ratio a corresponding downsampling rate in transferring the active frame portions of the input signal to the time domain again. In particular, the QMF synthesis filterbank <NUM> is applied to the lower frequency portions or lower frequency subbands of the QMF analysis filterbank spectrogram within the active frames. The core coder <NUM> thus receives a downsampled version of the input signal, which thus covers merely a lower frequency portion of the original input signal input into QMF analysis filterbank <NUM>. The remaining higher frequency portion is parametrically coded by modules <NUM> and <NUM>.

SID frames (or, to be more precise, the information to be conveyed by same) are forwarded to SID encoder <NUM>, which assumes responsibility for the functionalities of module <NUM> of <FIG>, for example. The only difference: module <NUM> operates on the spectrum of input signal directly - without LPC shaping. Moreover, as the QMF analysis filtering is used, the operation of module <NUM> is independent from the frame mode chosen by the core coder or the spectral bandwidth extension option being applied or not. The functionalities of module <NUM> and <NUM> of <FIG> may be implemented within module <NUM>.

Multiplexer <NUM> multiplexes the respective encoded information into the data stream at output <NUM>.

The audio decoder of <FIG> is able to operate on a data stream as output by the encoder of <FIG>. That is, a module <NUM> is configured to receive the data stream and to classify the frames within the data stream into active frames, SID frames and zero frames, i.e. a lack of any frame in the data stream, for example. Active frames are forwarded to a concatenation of a core decoder <NUM>, a QMF analysis filterbank <NUM> and a spectral bandwidth extension module <NUM>. Optionally, a noise estimator <NUM> is connected to QMF analysis filterbank's output. The noise estimator <NUM> may operate like, and may assume responsibility for the functionalities of, the background noise estimator <NUM> of <FIG>, for example, with the exception that the noise estimator operates on the un-shaped spectra rather than the excitation spectra. The concatenation of modules <NUM>, <NUM> and <NUM> is connected to an input of a QMF synthesis filterbank <NUM>. SID frames are forwarded to an SID frame decoder <NUM> which assumes responsibility for the functionality of the background noise generator <NUM> of <FIG>, for example. A comfort noise generating parameter updater <NUM> is fed by the information from decoder <NUM> and noise estimator <NUM> with this updater <NUM> steering the random generator <NUM>, which assumes responsibility for the parametric random generators functionality of <FIG>. As inactive or zero frames are missing, they do not have to be forwarded anywhere, but they trigger another random generation cycle of random generator <NUM>. The output of random generator <NUM> is connected to QMF synthesis filterbank <NUM>, the output of which reveals the reconstructed audio signal in silence and active phases in time domain.

Thus, during active phases, the core decoder <NUM> reconstructs the low-frequency portion of the audio signal including both noise and useful signal components. The QMF analysis filterbank <NUM> spectrally decomposes the reconstructed signal and the spectral bandwidth extension module <NUM> uses spectral bandwidth extension information within the data stream and active frames, respectively, in order to add the high frequency portion. The noise estimator <NUM>, if present, performs the noise estimation based on a spectrum portion as reconstructed by the core decoder, i.e. the low frequency portion. In inactive phases, the SID frames convey information parametrically describing the background noise estimate derived by the noise estimation <NUM> at the encoder side. The parameter updater <NUM> may primarily use the encoder information in order to update its parametric background noise estimate, using the information provided by the noise estimator <NUM> primarily as a fallback position in case of transmission loss concerning SID frames. The QMF synthesis filterbank <NUM> converts the spectrally decomposed signal as output by the spectral band replication module <NUM> in active phases and the comfort noise generated signal spectrum in the time domain. Thus, <FIG> and <FIG> make it clear that a QMF filterbank framework may be used as a basis for QMF-based comfort noise generation. The QMF framework provides a convenient way to resample the input signal down to a core-coder sampling rate in the encoder, or to upsample the core-decoder output signal of core decoder <NUM> at the decoder side using the QMF synthesis filterbank <NUM>. At the same time, the QMF framework can also be used in combination with bandwidth extension to extract and process the high frequency components of the signal which are left over by the core coder and core decoder modules <NUM> and <NUM>. Accordingly, the QMF filterbank can offer a common framework for various signal processing tools. In accordance with the embodiments of <FIG> and <FIG>, comfort noise generation is successfully included into this framework.

In particular, in accordance with the embodiments of <FIG> and <FIG>, it may be seen that it is possible to generate comfort noise at the decoder side after the QMF analysis, but before the QMF synthesis by applying a random generator <NUM> to excite the real and imaginary parts of each QMF coefficient of the QMF synthesis filterbank <NUM>, for example. The amplitude of the random sequences are, for example, individually computed in each QMF band such that the spectrum of the generated comfort noise resembles the spectrum of the actual input background noise signal. This can be achieved in each QMF band using a noise estimator after the QMF analysis at the encoding side. These parameters can then be transmitted through the SID frames to update the amplitude of the random sequences applied in each QMF band at the decoder side.

Ideally, note that the noise estimation <NUM> applied at the encoder side should be able to operate during both inactive (i.e., noise-only) and active periods (typically containing noisy speech) so that the comfort noise parameters can be updated immediately at the end of each active period. In addition, noise estimation might be used at the decoder side as well. Since noise-only frames are discarded in a DTX-based coding/decoding system, the noise estimation at the decoder side is favorably able to operate on noisy speech contents. The advantage of performing the noise estimation at the decoder side, in addition to the encoder side, is that the spectral shape of the comfort noise can be updated even when the packet transmission from the encoder to the decoder fails for the first SID frame(s) following a period of activity.

The noise estimation should be able to accurately and rapidly follow variations of the background noise's spectral content and ideally it should be able to perform during both active and inactive frames, as stated above. One way to achieve these goals is to track the minima taken in each band by the power spectrum using a sliding window of finite length, as proposed in [R. Martin, Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics, <NUM>]. The idea behind it is that the power of a noisyspeech spectrum frequently decays to the power of the background noise, e.g., between words or syllables. Tracking the minimum of the power spectrum provides therefore an estimate of the noise floor in each band, even during speech activity. However, these noise floors are underestimated in general. Furthermore, they do not allow to capture quick fluctuations of the spectral powers, especially sudden energy increases.

Nevertheless, the noise floor computed as described above in each band provides very useful side-information to apply a second stage of noise estimation. In fact, we can expect the power of a noisy spectrum to be close to the estimated noise floor during inactivity, whereas the spectral power will be far above the noise floor during activity. The noise floors computed separately in each band can hence be used as rough activity detectors for each band. Based on this knowledge, the background noise power can be easily estimated as a recursively smoothed version of the power spectrum as follows:
<MAT>
where σx<NUM>(m,k) denotes the power spectral density of the input signal at the frame m and band k, σN<NUM>(m,k) refers the noise power estimate, and β(m,k) is a forgetting factor (necessarily between <NUM> and <NUM>) controlling the amount of smoothing for each band and each frame separately. Using the noise floor information to reflect the activity status, it should take a small value during inactive periods (i.e., when the power spectrum is close to the noise floor), whereas a high value should be chosen to apply more smoothing (ideally keeping σN<NUM>(m,k) constant) during active frames. To achieve this, a soft decision may be made by computing the forgetting factors as follows:
<MAT>
where σNF<NUM> is the noise floor power and a is a control parameter. A higher value for a results in larger forgetting factors and hence causes overall more smoothing.

Thus, a Comfort Noise Generation (CNG) concept has been described where the artificial noise is produced at the decoder side in a transform domain. The above embodiments can be applied in combination with virtually any type of spectro-temporal analysis tool (i.e., a transform or filterbank) decomposing a time-domain signal into multiple spectral bands.

Again, it should be noted that the use of the spectral domain alone provides a more precise estimate of the background noise and achieves advantages without using the above possibility of continuously updating the estimate during active phases.

Claim 1:
Audio encoder comprising
a background noise estimator (<NUM>) configured to determine a parametric background noise estimate based on a spectral decomposition representation of an input audio signal so that the parametric background noise estimate spectrally describes a spectral envelope of a background noise of the input audio signal;
an encoder (<NUM>) for encoding the input audio signal into a data stream during the active phase; and
a detector (<NUM>) configured to detect an entrance of an inactive phase following the active phase based on the input signal,
wherein the audio encoder is configured to encode into the data stream the parametric background noise estimate in the inactive phase,
characterised in that
the background noise estimator is configured to identify local minima in the spectral decomposition representation of the input audio signal and to estimate the spectral envelope of the background noise of the input audio signal using interpolation between the identified local minima as supporting points.