Patent Description:
The present invention generally relates to digital audio coding and more precisely to decoding techniques for audio signals containing components of different characters.

A widespread class of coding method for audio signals containing speech or singing includes code excited linear prediction (CELP) applied in time alternation with different coding methods, including frequency-domain coding methods especially adapted for music or methods of a general nature, to account for variations in character between successive time periods of the audio signal. For example, a simplified Moving Pictures Experts Group (MPEG) Unified Speech and Audio Coding (USAC; see standard ISO/IEC <NUM>-<NUM>) decoder is operable in at least three decoding modes, Advanced Audio Coding (AAC; see standard ISO/IEC <NUM>-<NUM>), algebraic CELP (ACELP) and transform-coded excitation (TCX), as shown in the upper portion of accompanying <FIG>.

The various embodiments of CELP are adapted to the properties of the human organs of speech and, possibly, to the human auditory sense. As used in this application, CELP will refer to all possible embodiments and variants, including but not limited to ACELP, wide- and narrow-band CELP, SB-CELP (sub-band CELP), low- and high-rate CELP, RCELP (relaxed CELP), LD-CELP (low-delay CELP), CS-CELP (conjugate-structure CELP), CS-ACELP (conjugate-structure ACELP), PSI-CELP (pitch-synchronous innovation CELP) and VSELP (vector sum excited linear prediction). The principles of CELP are discussed by <NPL>, and some of its applications are described in references <NUM>-<NUM> cited in <NPL>. As further detailed in the former paper, a CELP decoder (or, analogously, a CELP speech synthesizer) may include a pitch predictor, which restores the periodic component of an encoded speech signal, and an pulse codebook, from which an innovation sequence is added. The pitch predictor may in turn include a long-delay predictor for restoring the pitch and a short-delay predictor for restoring formants by spectral envelope shaping. In this context, the pitch is generally understood as the fundamental frequency of the tonal sound component produced by the vocal chords and further coloured by resonating portions of the vocal tract. This frequency together with its harmonics will dominate speech or singing. Generally speaking, CELP methods are best suited for processing solo or one-part singing, for which the pitch frequency is well-defined and relatively easy to determine.

To improve the perceived quality of CELP-coded speech, it is common practice to combine it with post filtering (or pitch enhancement by another term). <CIT> and section II of the paper by Chen and Gersho disclose desirable properties of such post filters, namely their ability to suppress noise components located between the harmonics of the detected voice pitch (long-term portion; see section IV). It is believed that an important portion of this noise stems from the spectral envelope shaping. The long-term portion of a simple post filter may be designed to have the following transfer function: <MAT> where T is an estimated pitch period in terms of number of samples and α is a gain of the post filter, as shown in <FIG> and <FIG>. In a manner similar to a comb filter, such a filter attenuates frequencies <NUM>/(2T), <NUM>/(2T), <NUM>/(2T),. , which are located midway between harmonics of the pitch frequency, and adjacent frequencies. The attenuation depends on the value of the gain α. Slightly more sophisticated post filters apply this attenuation only to low frequencies - hence the commonly used term bass post filter - where the noise is most perceptible. This can be expressed by cascading the transfer function HE described above and a low-pass filter HLP. Thus, the post-processed decoded SE provided by the post filter will be given, in the transform domain, by <MAT> where <MAT> and S is the decoded signal which is supplied as input to the post filter. <FIG> shows an embodiment of a post filter with these characteristics, which is further discussed in section <NUM>. <NUM> of the Technical Specification ETSI TS <NUM><NUM>, version <NUM>. <NUM>, release <NUM>. As this figure suggests, the pitch information is encoded as a parameter in the bit stream signal and is retrieved by a pitch tracking module communicatively connected to the long-term prediction filter carrying out the operations expressed by PLT.

The long-term portion described in the previous paragraph may be used alone. Alternatively, it is arranged in series with a noise-shaping filter that preserves components in frequency intervals corresponding to the formants and attenuates noise in other spectral regions (short-term portion; see section III), that is, in the 'spectral valleys' of the formant envelope. As another possible variation, this filter aggregate is further supplemented by a gradual high-pass-type filter to reduce a perceived deterioration due to spectral tilt of the short-term portion.

Document <NPL>, and <CIT> disclose audio processing systems with selective application of post-filtering.

Audio signals containing a mixture of components of different origins - e.g., tonal, non-tonal, vocal, instrumental, non-musical - are not always reproduced by available digital coding technologies in a satisfactory manner. It has more precisely been noted that available technologies are deficient in handling such non-homogeneous audio material, generally favouring one of the components to the detriment of the other. In particular, music containing singing accompanied by one or more instruments or choir parts which has been encoded by methods of the nature described above, will often be decoded with perceptible artefacts spoiling part of the listening experience.

In order to mitigate at least some of the drawbacks outlined in the previous section, it is an object of the present invention to provide methods and devices adapted for audio encoding and decoding of signals containing a mixture of components of different origins. As particular objects, the invention seeks to provide such methods and devices that are suitable from the point of view of coding efficiency or (perceived) reproduction fidelity or both.

The invention achieves at least one of these objects by providing an a decoder system, a decoding method and a computer program product for carrying out the method, as defined in the independent claims. The dependent claims define embodiments of the invention.

The inventors have realized that some artefacts perceived in decoded audio signals of non-homogeneous origin derive from an inappropriate switching between several coding modes of which at least one includes post filtering at the decoder and at least one does not. More precisely, available post filters remove not only interharmonic noise (and, where applicable, noise in spectral valleys) but also signal components representing instrumental or vocal accompaniment and other material of a 'desirable' nature. The fact that the just noticeable difference in spectral valleys may be as large as <NUM> dB (as noted by <NPL>) may have been taken as a justification by many designers to filter these frequency bands severely. The quality degradation by the interharmonic (and spectral-valley) attenuation itself may however be less important than that of the switching occasions. When the post filter is switched on, the background of a singing voice sounds suddenly muffled, and when the filter is deactivated, the background instantly becomes more sonorous. If the switching takes place frequently, due to the nature of the audio signal or to the configuration of the coding device, there will be a switching artefact. As one example, a USAC decoder may be operable either in an ACELP mode combined with post filtering or in a TCX mode without post filtering. The ACELP mode is used in episodes where a dominant vocal component is present. Thus, the switching into the ACELP mode may be triggered by the onset of singing, such as at the beginning of a new musical phrase, at the beginning of a new verse, or simply after an episode where the accompaniment is deemed to drown the singing voice in the sense that the vocal component is no longer prominent. Experiments have confirmed that an alternative solution, or rather circumvention of the problem, by which TCX coding is used throughout (and the ACELP mode is disabled) does not remedy the problem, as reverb-like artefacts appear.

It is noted that the methods and apparatus disclosed in this section may be applied, after appropriate modifications within the skilled person's abilities including routine experimentation, to coding of signals having several components, possibly corresponding to different channels, such as stereo channels. Throughout the present application, pitch enhancement and post filtering are used as synonyms. It is further noted that AAC is discussed as a representative example of frequency-domain coding methods. Indeed, applying the invention to a decoder or encoder operable in a frequency-domain coding mode other than AAC will only require small modifications, if any, within the skilled person's abilities. Similarly, TCX is mentioned as an example of weighted linear prediction transform coding and of transform coding in general.

Features from two or more embodiments described hereinabove can be combined, unless they are clearly complementary, in further embodiments. The fact that two features are recited in different claims does not preclude that they can be combined to advantage. Likewise, further embodiments can also be provided by the omission of certain features that are not necessary or not essential for the desired purpose.

Embodiments of the present invention will now be described with reference to the accompanying drawings, on which:.

<FIG> is a schematic drawing of a decoder system <NUM> according to an embodiment of the invention, having as its input a bit stream signal and as its output an audio signal. As in the conventional decoders shown in <FIG>, a post filter <NUM> is arranged downstream of a decoding module <NUM> but can be switched into or out of the decoding path by operating a switch <NUM>. The post filter is enabled in the switch position shown in the figure. It would be disabled if the switch was set in the opposite position, whereby the signal from the decoding module <NUM> would instead be conducted over the bypass line <NUM>. As an inventive contribution, the switch <NUM> is controllable by post filtering information contained in the bit stream signal, so that post filtering may be applied and removed irrespectively of the current status of the decoding module <NUM>. Because a post filter <NUM> operates at some delay - for example, the post filter shown in <FIG> will introduce a delay amounting to at least the pitch period T - a compensation delay module <NUM> is arranged on the bypass line <NUM> to maintain the modules in a synchronized condition at switching. The delay module <NUM> delays the signal by the same period as the post filter <NUM> would, but does not otherwise process the signal. To minimize the change-over time, the compensation delay module <NUM> receives the same signal as the post filter <NUM> at all times. In an alternative embodiment where the post filter <NUM> is replaced by a zero-delay post filter (e.g., a causal filter, such as a filter with two taps, independent of future signal values), the compensation delay module <NUM> can be omitted.

<FIG> illustrates a further development according to the teachings of the invention of the triple-mode decoder system <NUM> of <FIG>. An ACELP decoding module <NUM> is arranged in parallel with a TCX decoding module <NUM> and an AAC decoding module <NUM>. In series with the ACELP decoding module <NUM> is arranged a post filter <NUM> for attenuating noise, particularly noise located between harmonics of a pitch frequency directly or indirectly derivable from the bit stream signal for which the decoder system <NUM> is adapted. The bit stream signal also encodes post filtering information governing the positions of an upper switch <NUM> operable to switch the post filter <NUM> out of the processing path and replace it with a compensation delay <NUM> like in <FIG>. A lower switch <NUM> is used for switching between different decoding modes. With this structure, the position of the upper switch <NUM> is immaterial when one of the TCX or AAC modules <NUM>, <NUM> is used; hence, the post filtering information does not necessary indicate this position except in the ACELP mode. Whatever decoding mode is currently used, the signal is supplied from the downstream connection point of the lower switch <NUM> to a spectral band replication (SBR) module <NUM>, which outputs an audio signal. The skilled person will realize that the drawing is of a conceptual nature, as is clear notably from the switches which are shown schematically as separate physical entities with movable contacting means. In a possible realistic implementation of the decoder system, the switches as well as the other modules will be embodied by computer-readable instructions.

<FIG> and <FIG> are also block diagrams of two triple-mode decoder systems operable in an ACELP, TCX or frequency-domain decoding mode. With reference to the latter figure, which shows an embodiment of the invention, a bit stream signal is supplied to an input point <NUM>, which is in turn permanently connected via respective branches to the three decoding modules <NUM>, <NUM>, <NUM>. The input point <NUM> also has a connecting branch <NUM> (not present in the conventional decoding system of <FIG>) to a pitch enhancement module <NUM>, which acts as a post filter of the general type described above. As is common practice in the art, a first transition windowing module <NUM> is arranged downstream of the ACELP and TCX modules <NUM>, <NUM>, to carry out transitions between the decoding modules. A second transition module <NUM> is arranged downstream of the frequency-domain decoding module <NUM> and the first transition windowing module <NUM>, to carry out transition between the two super-modes. Further a SBR module <NUM> is provided immediately upstream of the output point <NUM>. Clearly, the bit stream signal is supplied directly (or after demultiplexing, as appropriate) to all three decoding modules <NUM>, <NUM>, <NUM> and to the pitch enhancement module <NUM>. Information contained in the bit stream controls what decoding module is to be active. By the invention however, the pitch enhancement module <NUM> performs an analogous self actuation , which responsive to post filtering information in the bit stream may act as a post filter or simply as a pass-through. This may for instance be realized through the provision of a control section (not shown) in the pitch enhancement module <NUM>, by means of which the post filtering action can be turned on or off. The pitch enhancement module <NUM> is always in its pass-through mode when the decoder system operates in the frequency-domain or TCX decoding mode, wherein strictly speaking no post filtering information is necessary. It is understood that modules not forming part of the inventive contribution and whose presence is obvious to the skilled person, e.g., a demultiplexer, have been omitted from <FIG> and other similar drawings to increase clarity.

As a variation, the decoder system of <FIG> may be equipped with a control module (not shown) for deciding whether post filtering is to be applied using an analysis-by-synthesis approach. Such control module is communicatively connected to the pitch enhancement module <NUM> and to the ACELP module <NUM>, from which it extracts an intermediate decoded signal si_DEC(n) representing an intermediate stage in the decoding process, preferably one corresponding to the excitation of the signal. The detection module has the necessary information to simulate the action of the pitch enhancement module <NUM>, as defined by the transfer functions PLT(z) and HLP(z) (cf. Background section and <FIG>), or equivalently their filter impulse responses pLT(z) and hLP(n). As follows by the discussion in the Background section, the component to be subtracted at post filtering can be estimated by an approximate difference signal sAD(n) which is proportional to [(si_DEC * pLT) * hLP](n), where * denotes discrete convolution. This is an approximation of the true difference between the original audio signal and the post-filtered decoded signal, namely <MAT> where α is the post filter gain. By studying the total energy, low-band energy, tonality, actual magnitude spectrum or past magnitude spectra of this signal, as disclosed in the Summary section and the claims, the control section may find a basis for the decision whether to activate or deactivate the pitch enhancement module <NUM>.

<FIG> shows an encoder system <NUM> compatible with a decoder according to an embodiment of the invention. The encoder system <NUM> is adapted to process digital audio signals, which are generally obtained by capturing a sound wave by a microphone and transducing the wave into an analog electric signal. The electric signal is then sampled into a digital signal susceptible to be provided, in a suitable format, to the encoder system <NUM>. The system generally consists of an encoding module <NUM>, a decision module <NUM> and a multiplexer <NUM>. By virtue of switches <NUM>, <NUM> (symbolically represented), the encoding module <NUM> is operable in either a CELP, a TCX or an AAC mode, by selectively activating modules <NUM>, <NUM>, <NUM>. The decision module <NUM> applies one or more predefined criteria to decide whether a bit stream signal produced by the encoder system <NUM> to encode an audio signal. For this purpose, the decision module <NUM> may examine the audio signal directly or may receive data from the encoding module <NUM> via a connection line <NUM>. A signal indicative of the decision taken by the decision module <NUM> is provided, together with the encoded audio signal from the encoding module <NUM>, to a multiplexer <NUM>, which concatenates the signals into a bit stream constituting the output of the encoder system <NUM>.

Preferably, the decision module <NUM> bases its decision on an approximate difference signal computed from an intermediate decoded signal si_DEC, which can be subtracted from the encoding module <NUM>. The intermediate decoded signal represents an intermediate stage in the decoding process, as discussed in preceding paragraphs, but may be extracted from a corresponding stage of the encoding process. However, in the encoder system <NUM> the original audio signal sORIG is available so that, advantageously, the approximate difference signal is formed as: <MAT> The approximation resides in the fact that the intermediate decoded signal is used in lieu of the final decoded signal. This enables an appraisal of the nature of the component that a post filter would remove at decoding, and by applying one of the criteria discussed in the Summary section, the decision module <NUM> will be able to take a decision whether to disable post filtering.

As a variation to this, the decision module <NUM> may use the original signal in place of an intermediate decoded signal, so that the approximate difference signal will be [(si_DEC * pLT) * hLP](n). This is likely to be a less faithful approximation but on the other hand makes the presence of a connection line <NUM> between the decision module <NUM> and the encoding module <NUM> optional.

In such other variations of this embodiment where the decision module <NUM> studies the audio signal directly, one or more of the following criteria may be applied:.

In all the described variations of the encoder structure shown in <FIG> - that is, irrespectively of the basis of the detection criterion - the decision section <NUM> may be enabled to decide on a gradual onset or gradual removal of post filtering, so as to achieve smooth transitions. The gradual onset and removal may be controlled by adjusting the post filter gain.

<FIG> shows a conventional decoder operable in a frequency-decoding mode and a CELP decoding mode depending on the bit stream signal supplied to the decoder. Post filtering is applied whenever the CELP decoding mode is selected. An improvement of this decoder is illustrated in <FIG>, which shows an decoder <NUM> according to an embodiment of the invention. This decoder is operable not only in a frequency-domain-based decoding mode, wherein the frequency-domain decoding module <NUM> is active, and a filtered CELP decoding mode, wherein the CELP decoding module <NUM> and the post filter <NUM> are active, but also in an unfiltered CELP mode, in which the CELP module <NUM> supplies its signal to a compensation delay module <NUM> via a bypass line <NUM>. A switch <NUM> controls what decoding mode is currently used responsive to post filtering information contained in the bit stream signal provided to the decoder <NUM>. In this decoder and that of <FIG>, the last processing step is effected by an SBR module <NUM>, from which the final audio signal is output.

<FIG> shows a post filter <NUM> suitable to be arranged downstream of a decoder <NUM>. The filter <NUM> includes a post filtering module <NUM>, which is enabled or disabled by a control module (not shown), notably a binary or non-binary gain controller, in response to a post filtering signal received from a decision module <NUM> within the post filter <NUM>. The decision module performs one or more tests on the signal obtained from the decoder to arrive at a decision whether the post filtering module <NUM> is to be active or inactive. The decision may be taken along the lines of the functionality of the decision module <NUM> in <FIG>, which uses the original signal and/or an intermediate decoded signal to predict the action of the post filter. The decision of the decision module <NUM> may also be based on similar information as the decision modules uses in those embodiments where an intermediate decoded signal is formed. As one example, the decision module <NUM> may estimate a pitch frequency (unless this is readily extractable from the bit stream signal) and compute the energy content in the signal below the pitch frequency and between its harmonics. If this energy content is significant, it probably represents a relevant signal component rather than noise, which motivates a decision to disable the post filtering module <NUM>.

A <NUM>-person listening test has been carried out, during which music samples encoded and decoded according to the invention were compared with reference samples containing the same music coded while applying post filtering in the conventional fashion but maintaining all other parameters unchanged. The results confirm a perceived quality improvement.

Further embodiments of the present invention will become apparent to a person skilled in the art after reading the description above. Even though the present description and drawings disclose embodiments and examples, the invention is not restricted to these specific examples. Numerous modifications and variations can be made without departing from the scope of the present invention, which is defined by the accompanying claims.

Claim 1:
A method of decoding a bit stream signal as an audio time signal, including
the steps of:
decoding a bit stream signal as a preliminary audio time signal according to a coding mode selected from a plurality of coding modes, wherein the plurality of coding modes includes at least a first coding mode which includes a post-filtering step and at least a second coding mode which does not include the post-filtering step,
wherein the post-filtering step applies a pitch-enhancement filter to the preliminary audio time signal, thereby obtaining an audio time signal, wherein the post-filtering is applied only to low frequencies, and
wherein the post-filtering step is selectively omitted responsive to post-filtering information encoded in the bit stream signal, the post-filtering information being indicative of an encoder-side decision of whether or not to omit the post-filtering step, whereby the post-filtering step is selectively omitted in the first coding mode.