Patent Description:
In the present disclosure and the appended claims:.

Historically, conversational telephony has been implemented with handsets having only one transducer to output sound only to one of the user's ears. In the last decade, users have started to use their portable handset in conjunction with a headphone to receive the sound over their two ears mainly to listen to music but also, sometimes, to listen to speech. Nevertheless, when a portable handset is used to transmit and receive conversational speech, the content is still mono but presented to the user's two ears when a headphone is used.

With the newest 3GPP speech coding standard as described in Reference [<NUM>], the quality of the coded sound, for example speech and/or audio that is transmitted and received through a portable handset has been significantly improved. The next natural step is to transmit stereo information such that the receiver gets as close as possible to a real life audio scene that is captured at the other end of the communication link.

In audio codecs, for example as described in Reference [<NUM>], transmission of stereo information is normally used.

For conversational speech codecs, mono signal is the norm. When a stereo signal is transmitted, the bit-rate often needs to be doubled since both the left and right channels of the stereo signal are coded using a mono codec. This works well in most scenarios, but presents the drawbacks of doubling the bit-rate and failing to exploit any potential redundancy between the two channels (left and right channels of the stereo signal). Furthermore, to keep the overall bit-rate at a reasonable level, a very low bit-rate for each channel is used, thus affecting the overall sound quality. To reduce the bit-rate, efficient stereo coding techniques have been developed and used. As non-limitative examples, the use of three stereo coding techniques that can be efficiently used at low bit-rates is discussed in the following paragraphs.

A first stereo coding technique is called parametric stereo. Parametric stereo coding encodes two, left and right channels as a mono signal using a common mono codec plus a certain amount of stereo side information (corresponding to stereo parameters) which represents a stereo image. The two input, left and right channels are down-mixed into a mono signal, and the stereo parameters are then computed usually in transform domain, for example in the Discrete Fourier Transform (DFT) domain, and are related to so-called binaural or inter-channel cues. The binaural cues (Reference [<NUM>]) comprise Interaural Level Difference (ILD), Interaural Time Difference (ITD) and Interaural Correlation (IC). Depending on the signal characteristics, stereo scene configuration, etc., some or all binaural cues are coded and transmitted to the decoder. Information about what binaural cues are coded and transmitted is sent as signaling information, which is usually part of the stereo side information. A particular binaural cue can be also quantized using different coding techniques which results in a variable number of bits being used. Then, in addition to the quantized binaural cues, the stereo side information may contain, usually at medium and higher bit-rates, a quantized residual signal that results from the down-mixing. The residual signal can be coded using an entropy coding technique, e.g. an arithmetic coder. Parametric stereo coding with stereo parameters computed in a transform domain will be referred to in the present disclosure as "DFT stereo" coding.

Another stereo coding technique is a technique operating in time-domain (TD). This stereo coding technique mixes the two input, left and right channels into so-called primary channel and secondary channel. For example, following the method as described in Reference [<NUM>], time-domain mixing can be based on a mixing ratio, which determines respective contributions of the two input, left and right channels upon production of the primary channel and the secondary channel. The mixing ratio is derived from several metrics, e.g. normalized correlations of the input left and right channels with respect to a mono signal version or a long-term correlation difference between the two input left and right channels. The primary channel can be coded by a common mono codec while the secondary channel can be coded by a lower bit-rate codec. The secondary channel coding may exploit coherence between the primary and secondary channels and might re-use some parameters from the primary channel. Time-domain stereo coding will be referred to in the present disclosure as "TD stereo" coding. In general, TD stereo coding is most efficient at lower and medium bit-rates for coding speech signals.

A third stereo coding technique is a technique operating in the Modified Discrete Cosine Transform (MDCT) domain. It is based on joint coding of both the left and right channels while computing global ILD and Mid/Side (M/S) processing in whitened spectral domain. This third stereo coding technique uses several tools adapted from TCX (Transform Coded eXcitation) coding in MPEG (Moving Picture Experts Group) codecs as described for example in References [<NUM>] and [<NUM>]. These tools may include TCX core coding, TCX LTP (Long-Term Prediction) analysis, TCX noise filling, Frequency-Domain Noise Shaping (FDNS), stereophonic Intelligent Gap Filling (IGF), and/or adaptive bit allocation between channels. In general, this third stereo coding technique is efficient to encode all kinds of audio content at medium and high bit-rates. The MDCT-domain stereo coding technique will be referred to in the present disclosure as "MDCT stereo coding". In general, MDCT stereo coding is most efficient at medium and high bit-rates for coding general audio signals.

In recent years, stereo coding was further extended to multichannel coding. There exist several techniques to provide multichannel coding but the fundamental core of all these techniques is often based on single or multiple instance(s) of mono or stereo coding techniques. Thus, the present disclosure presents switching between stereo coding modes that can be part of multichannel coding techniques such as Metadata-Assisted Spatial Audio (MASA) as described for example in Reference [<NUM>]. In the MASA approach, the MASA metadata (e.g. direction, energy ratio, spread coherence, distance, surround coherence, all in several time-frequency slots) are generated in a MASA analyzer, quantized, coded, and passed into the bit-stream while MASA audio channel(s) are treated as (multi-)mono or (multi-)stereo transport signals coded by the core coder(s). At the MASA decoder, MASA metadata then guide the decoding and rendering process to recreate an output spatial sound. <CIT> discloses a multichannel audio encoder configured to switch between a linear prediction domain encoder and a frequency domain encoder as well as a corresponding decoder.

The present disclosure provides stereo sound signal encoding and decoding devices and methods as defined in the appended claims.

The foregoing and other objects, advantages and features of the stereo encoding and decoding devices and methods will become more apparent upon reading of the following non-restrictive description of illustrative embodiments thereof, given by way of example only with reference to the accompanying drawings.

As mentioned hereinabove, the present disclosure relates to stereo sound encoding, in particular but not exclusively to switching between stereo coding modes in a sound, including speech and/or audio, codec capable in particular but not exclusively of producing a good stereo quality for example in a complex audio scene at low bit-rate and low delay. In the present disclosure, a complex audio scene includes situations, for example but not exclusively, in which (a) the correlation between the sound signals that are recorded by the microphones is low, (b) there is an important fluctuation of the background noise, and/or (c) an interfering talker is present. Non-limitative examples of complex audio scenes comprise a large anechoic conference room with an A/B microphones configuration, a small echoic room with binaural microphones, and a small echoic room with a mono/side microphones set-up. All these room configurations could include fluctuating background noise and/or interfering talkers.

<FIG> is a schematic block diagram of a stereo sound processing and communication system <NUM> depicting a possible context of implementation of the IVAS stereo encoding device and method and IVAS stereo decoding device and method.

The stereo sound processing and communication system <NUM> of <FIG> supports transmission of a stereo sound signal across a communication link <NUM>. The communication link <NUM> may comprise, for example, a wire or an optical fiber link. Alternatively, the communication link <NUM> may comprise at least in part a radio frequency link. The radio frequency link often supports multiple, simultaneous communications requiring shared bandwidth resources such as may be found with cellular telephony. Although not shown, the communication link <NUM> may be replaced by a storage device in a single device implementation of the system <NUM> that records and stores the coded stereo sound signal for later playback.

Still referring to <FIG>, for example a pair of microphones <NUM> and <NUM> produces left <NUM> and right <NUM> channels of an original analog stereo sound signal. As indicated in the foregoing description, the sound signal may comprise, in particular but not exclusively, speech and/or audio.

The left <NUM> and right <NUM> channels of the original analog sound signal are supplied to an analog-to-digital (A/D) converter <NUM> for converting them into left <NUM> and right <NUM> channels of an original digital stereo sound signal. The left <NUM> and right <NUM> channels of the original digital stereo sound signal may also be recorded and supplied from a storage device (not shown).

A stereo sound encoder <NUM> codes the left <NUM> and right <NUM> channels of the original digital stereo sound signal thereby producing a set of coding parameters that are multiplexed under the form of a bit-stream <NUM> delivered to an optional error-correcting encoder <NUM>. The optional error-correcting encoder <NUM>, when present, adds redundancy to the binary representation of the coding parameters in the bit-stream <NUM> before transmitting the resulting bit-stream <NUM> over the communication link <NUM>.

On the receiver side, an optional error-correcting decoder <NUM> utilizes the above mentioned redundant information in the received digital bit-stream <NUM> to detect and correct errors that may have occurred during transmission over the communication link <NUM>, producing a bit-stream <NUM> with received coding parameters. A stereo sound decoder <NUM> converts the received coding parameters in the bit-stream <NUM> for creating synthesized left <NUM> and right <NUM> channels of the digital stereo sound signal. The left <NUM> and right <NUM> channels of the digital stereo sound signal reconstructed in the stereo sound decoder <NUM> are converted to synthesized left <NUM> and right <NUM> channels of the analog stereo sound signal in a digital-to-analog (D/A) converter <NUM>.

The synthesized left <NUM> and right <NUM> channels of the analog stereo sound signal are respectively played back in a pair of loudspeaker units, or binaural headphones, <NUM> and <NUM>. Alternatively, the left <NUM> and right <NUM> channels of the digital stereo sound signal from the stereo sound decoder <NUM> may also be supplied to and recorded in a storage device (not shown).

For example, (a) the left channel of <FIG> may be implemented by the left channel of <FIG>, (b) the right channel of <FIG> may be implemented by the right channel of <FIG>, (c) the stereo sound encoder <NUM> of <FIG> may be implemented by the IVAS stereo encoding device of <FIG>, and (d) the stereo sound decoder <NUM> of <FIG> may be implemented by the IVAS stereo decoding device of <FIG>.

<FIG> is a high-level block diagram illustrating concurrently the IVAS stereo encoding device <NUM> and the corresponding IVAS stereo encoding method <NUM>, <FIG> is a block diagram illustrating concurrently the FD stereo encoder <NUM> of the IVAS stereo encoding device <NUM> of <FIG> and the corresponding FD stereo encoding method <NUM>, <FIG> is a block diagram illustrating concurrently the TD stereo encoder <NUM> of the IVAS stereo encoding device <NUM> of <FIG> and the corresponding TD stereo encoding method <NUM>, and <FIG> is a block diagram illustrating concurrently the MDCT stereo encoder <NUM> of the IVAS stereo encoding device <NUM> of <FIG> and the corresponding MDCT stereo encoding method <NUM>.

In the illustrative, non-limitative implementation of <FIG>, the framework of the IVAS stereo encoding device <NUM> (and correspondingly the IVAS stereo decoding device <NUM> of <FIG>) is based on a modified version of the Enhanced Voice Services (EVS) codec (See Reference [<NUM>]). Specifically, the EVS codec is extended to code (and decode) stereo and multi-channels, and address Immersive Voice and Audio Services (IVAS). For that reason, the encoding device <NUM> and method <NUM> are referred to as IVAS stereo encoding device and method in the present disclosure. In the described exemplary implementation, the IVAS stereo encoding device <NUM> and method <NUM> use, as a non-limitative example, three stereo coding modes: a Frequency-Domain (FD) stereo mode based on DFT (Discrete Fourier Transform), referred to in the present disclosure as "DFT stereo mode", a Time-Domain (TD) stereo mode, referred to in the present disclosure as "TD stereo mode", and a joint stereo coding mode based on the Modified Discrete Cosine Transform (MDCT) stereo mode, referred to in the present disclosure as "MDCT stereo mode". It should be kept in mind that other codec structures may be used as a basis for the framework of the IVAS stereo encoding device <NUM> (and correspondingly the IVAS stereo decoding device <NUM>).

Stereo mode switching in the IVAS codec (IVAS stereo encoding device <NUM> and IVAS stereo decoding device <NUM>) refers, in the described, non-limitative implementation, to switching between the DFT, TD and MDCT stereo modes.

The following nomenclature is used in the present disclosure and the accompanying figures: small letters indicate time-domain signals, capital letters indicate transform-domain signals, I/L stands for left channel, r/R stands for right channel, m/M stands for mid-channel, s/S stands for side channel, PCh stands for primary channel, and SCh stands for secondary channel. Also, in the figures, numbers without unit correspond to a number of samples at a <NUM> sampling rate.

Differences exist between (a) the DFT stereo encoder <NUM> and encoding method <NUM>, (b) the TD stereo encoder <NUM> and encoding method <NUM>, and (c) the MDCT stereo encoder <NUM> and encoding method <NUM>. Some of these differences are summarized in the following paragraphs and at least some of them will be better explained in the following description.

The IVAS stereo encoding device <NUM> and encoding method <NUM> performs operations such as buffering one <NUM>-ms frame (as well known in the art, the stereo sound signal is processed in successive frames of given duration containing a given number of sound signal samples) of stereo input signal (left and right channels), few classification steps, down-mixing, pre-processing and actual coding. A <NUM> look-ahead is available and used mainly for analysis, classification and OverLap-Add (OLA) operations used in transform-domain such as in a Transform Coded eXcitation (TCX) core, a High Quality (HQ) core, and Frequency-Domain BandWidth-Extension (FD-BWE). These operations are described in Reference [<NUM>], Clauses <NUM> and <NUM>.

The look-ahead is shorter in the IVAS stereo encoding device <NUM> and encoding method <NUM> compared to the non-modified EVS encoder by <NUM> (corresponding to a Finite Impulse Response (FIR) filter resampling delay (See Reference [<NUM>], Clause <NUM>. This has an impact on the procedure of resampling the down-processed signal (down-mixed signal for TD and DFT stereo modes) in every frame:.

The resampling in the DFT stereo encoder <NUM>, the TD stereo encoder <NUM> and the MDCT stereo encoder <NUM>, is done from the input sampling rate (usually <NUM>, <NUM>, or <NUM>) to the internal sampling rate(s) (usually <NUM>, <NUM>, <NUM>, or <NUM>). The resampled signal(s) is then used in the pre-processing and the core encoding.

Also, the look-ahead contains a part of down-processed signal (down-mixed signal for TD and DFT stereo modes) signal that is not accurate but rather extrapolated or estimated which also has an impact on the resampling process. The inaccuracy of the look-ahead down-processed signal (down-mixed signal for TD and DFT stereo modes) depends on the current stereo coding mode:.

The redressed/extrapolated signal part in the look-ahead is not subject to actual coding but used for analysis and classification. Consequently, the redressed/extrapolated, signal part in the look-ahead is re-computed in the next frame and the resulting down-processed signal (down-mixed signal for TD and DFT stereo modes) is then used for actual coding. The length of the re-computed signal depends on the stereo mode and coding processing:.

It is noted that the lengths of the redressed, respectively extrapolated signal part in the look-ahead are mentioned here as an illustration while any other lengths can be implemented in general.

Additional information regarding the DFT stereo encoder <NUM> and encoding method <NUM> may be found in References [<NUM>] and [<NUM>]. Additional information regarding the TD stereo encoder <NUM> and encoding method <NUM> may be found in Reference [<NUM>]. And additional information regarding the MDCT stereo encoder <NUM> and encoding method <NUM> may be found in References [<NUM>] and [<NUM>].

The following Table I lists in a sequential order processing operations for each frame depending on the current stereo coding mode (See also <FIG>).

The IVAS stereo encoding method <NUM> comprises an operation (not shown) of controlling switching between the DFT, TD and MDCT stereo modes. To perform the switching controlling operation, the IVAS stereo encoding device <NUM> comprises a controller (not shown) of switching between the DFT, TD and MDCT stereo modes. Switching between the DFT and TD stereo modes in the IVAS stereo encoding device <NUM> and coding method <NUM> involves the use of the stereo mode switching controller (not shown) to maintain continuity of the following input signals <NUM>) to <NUM>) to enable adequate processing of these signals in the IVAS stereo encoding device <NUM> and method <NUM>:.

While it is straightforward to maintain the continuity for signal <NUM>) above, it is challenging for signals <NUM>) - <NUM>) due to several aspects, for example a different down-mixing, a different length of the re-computed part of the look-ahead, use of Inter-Channel Alignment (ICA) in the TD stereo mode only, etc..

The operation (not shown) of controlling switching between the DFT, TD and MDCT stereo modes comprises an operation <NUM> of stereo classification and stereo mode selection, for example as described in Reference [<NUM>]. To perform the operation <NUM>, the controller (not shown) of switching between the DFT, TD and MDCT stereo modes comprises a stereo classifier and stereo mode selector <NUM>.

Switching between the TD stereo mode, the DFT stereo mode, and the MDCT stereo mode is responsive to the stereo mode selection. Stereo classification (Reference [<NUM>]) is conducted in response to the left l and right r channels of the input stereo signal, and/or requested coded bit-rate. Stereo mode selection (Reference [<NUM>]) consists of choosing one of the DFT, TD, and MDCT stereo modes based on stereo classification.

The stereo classifier and stereo mode selector <NUM> produces stereo mode signaling <NUM> for identifying the selected stereo coding mode.

The operation (not shown) of controlling switching between the DFT, TD and MDCT stereo modes comprises an operation of memory allocation (not shown). To perform the operation of memory allocation, the controller of switching between the DFT, TD and MDCT stereo modes (not shown) dynamically allocates/deallocates static memory data structures to/from the DFT, TD and MDCT stereo modes depending on the current stereo mode. Such memory allocation keeps the static memory impact of the IVAS stereo encoding device <NUM> as low as possible by maintaining only those data structures that are employed in the current frame.

For example, in a first DFT stereo frame following a TD stereo frame, the data structures related to the TD stereo mode (for example TD stereo data handling, second core-encoder data structure) are freed (deallocated) and the data structures related to the DFT stereo mode (for example DFT stereo data structure) are instead allocated and initialized. It is noted that the deallocation of the further unused data structures is done first, followed by the allocation of newly used data structures. This order of operations is important to not increase the static memory impact at any point of the encoding.

A summary of main static memory data structures as used in the various stereo modes is shown in Table II.

An example implementation of the memory allocation/deallocation encoder module in the C source code is shown below. <IMG>
<IMG>
<IMG>
<IMG>
<IMG>
<IMG>.

The TD stereo mode may consist of two sub-modes. One is a so-called normal TD stereo sub-mode for which the TD stereo mixing ratio is higher than <NUM> and lower than <NUM>. The other is a so-called LRTD stereo sub-mode for which the TD stereo mixing ratio is either <NUM> or <NUM>; thus, LRTD is an extreme case of the TD stereo mode where the TD down-mixing actually does not mix the content of the time-domain left l and right r channels to form primary PCh and secondary SCh channels but get them directly from the channels l and r.

When the two sub-modes (normal and LRTD) of the TD stereo mode are available, the stereo mode switching operation (not shown) comprises a TD stereo mode setting (not show). To perform the TD stereo mode setting, forming part of the memory allocation, the stereo mode switching controller (not shown) of the IVAS stereo encoding device <NUM> allocates/deallocates certain static memory data structures when switching between the normal TD stereo mode and the LRTD stereo mode. For example, an IC-BWE data structure is allocated only in frames using the normal TD stereo mode (See Table II) while several data structures (BWEs and Complex Low Delay Filter Bank (CLDFB) for secondary channel SCh) are allocated only in frames using the LRTD stereo mode (See Table II). An example implementation of the memory allocation/deallocation encoder module in the C source code is shown below:
<IMG>
<IMG>.

Mostly, only the normal TD stereo mode (for simplicity referred further only as the TD stereo mode) will be described in detail in the present disclosure. The LRTD stereo mode is mentioned as a possible implementation.

The stereo mode switching controlling operation (not shown) comprises an operation of stereo switching updates (not shown). To perform this stereo switching updates operation, the stereo mode switching controller (not shown) updates long-term parameters and updates or resets past buffer memories.

Upon switching from the DFT stereo mode to the TD stereo mode, the stereo mode switching controller (not shown) resets TD stereo and ICA static memory data structures. These data structures store the parameters and memories of the TD stereo analysis and weighted down-mixing (<NUM> in <FIG>), respectively of the ICA algorithm (<NUM> in <FIG>). Then the stereo mode switching controller (not shown) sets a TD stereo past frame mixing ratio index according to the normal TD stereo mode or LRTD stereo mode. As a non-limitative illustrative example:.

Upon switching from the TD stereo mode to the DFT stereo mode, the stereo mode switching controller (not shown) resets the DFT stereo data structure. This DFT stereo data structure stores parameters and memories related to the DFT stereo processing and down-mixing module (<NUM> in <FIG>).

Also, the stereo mode switching controller (not shown) transfers some stereo-related parameters between data structures. As an example, parameters related to time shift and energy between the channels l and r, namely a side gain (or ILD parameter) and ITD parameter of the DFT stereo mode are used to update a target gain and correlation lags (ICA parameters <NUM>) of the TD stereo mode and vice versa. These target gain and correlation lags are further described in next Section <NUM>. <NUM> of the present disclosure.

Updates/resets related to the core-encoders (See <FIG> and <FIG>) are described later in Section <NUM> of the present disclosure. An example implementation of the handling of some memories in the encoder is shown below. <IMG>
<IMG>
<IMG>
<IMG>
<IMG>.

In TD stereo frames, the stereo mode switching controlling operation (not shown) comprises a temporal Inter-Channel Alignment (ICA) operation <NUM>. To perform operation <NUM>, the stereo mode switching controller (not shown) comprises an ICA encoder <NUM> to time-align the channels l and r of the input stereo signal and then scale the channel r.

As described in the foregoing description, before TD down-mixing, ICA is performed using ITD synchronization between the two input channels l and r in the time-domain. This is achieved by delaying one of the input channels (l or r) and by extrapolating a missing part of the down-mixed signal corresponding to the length of the ITD delay; a maximum value of the ITD delay is <NUM>. The time alignment, i.e. the ICA time shift, is applied first and alters the most part of the current TD stereo frame. The extrapolated part of the look-ahead down-mixed signal is recomputed and thus temporally adjusted in the next frame based on the ITD estimated in that next frame.

When no stereo mode switching is anticipated, the <NUM> long extrapolated signal is re-computed in the ICA encoder <NUM>. However, when stereo mode switching may happen, namely switching from the DFT stereo mode to the TD stereo mode, a longer signal is subject to re-computation. The length then corresponds to the length of the DFT stereo redressed signal plus the FIR resampling delay, i.e. <NUM> + <NUM> = <NUM>. Section <NUM> explains these features in more detail.

Another purpose of the ICA encoder <NUM> is the scaling of the input channel r. The scaling gain, i.e. the above mentioned the target gain, is estimated as a logarithm ratio of the l and r channels energies smoothed with the previous frame target gain at every frame regardless of the DFT or TD stereo mode being used. The target gain estimated in the current frame (<NUM>) is applied to the last <NUM> of the current input channel r while the first <NUM> of the current channel r is scaled by a combination of the previous and current frame target gains in a fade-in / fade-out manner.

The ICA encoder <NUM> produces ICA parameters <NUM> such as the ITD delay, the target gain and a target channel index.

The stereo mode switching controlling operation (not shown) comprises an operation <NUM> of detecting time-domain transient in the channel l from the ICA encoder <NUM>. To perform operation <NUM>, the stereo mode switching controller (not shown) comprises a detector <NUM> to detect time-domain transient in the channel I.

In the same manner, the stereo mode switching controlling operation (not shown) comprises an operation <NUM> of detecting time-domain transient in the channel r from the ICA encoder <NUM>. To perform operation <NUM>, the stereo mode switching controller (not shown) comprises a detector <NUM> to detect time-domain transient in the channel r.

Time-domain transient detection in the time-domain channels l and r is a pre-processing step that enables detection and, therefore proper processing and encoding of such transients in the transform-domain core encoding modules (TCX core, HQ core, FD-BWE).

Further information regarding the time-domain transient detectors <NUM> and <NUM> and the time-domain transient detection operations <NUM> and <NUM> can be found, for example, in Reference [<NUM>], Clause <NUM>.

To perform stereo encoder configurations, the IVAS stereo encoding device <NUM> sets parameters of the stereo encoders <NUM>, <NUM> and <NUM>. For example, a nominal bit-rate for the core-encoders is set.

Referring to <FIG>, the DFT stereo encoding method <NUM> comprises an operation <NUM> for applying a DFT transform to the channel l from the time-domain transient detector <NUM> of <FIG>. To perform operation <NUM>, the DFT stereo encoder <NUM> comprises a calculator <NUM> of the DFT transform of the channel l (DFT analysis) to produce a channel L in DFT domain.

The DFT stereo encoding method <NUM> also comprises an operation <NUM> for applying a DFT transform to the channel r from the time-domain transient detector <NUM> of <FIG>. To perform operation <NUM>, the DFT stereo encoder <NUM> comprises a calculator <NUM> of the DFT transform of the channel r (DFT analysis) to produce a channel R in DFT domain.

The DFT stereo encoding method <NUM> further comprises an operation <NUM> of stereo processing and down-mixing in DFT domain. To perform operation <NUM>, the DFT stereo encoder <NUM> comprises a stereo processor and down-mixer <NUM> to produce side information on a side channel S. Down-mixing of the channels L and R also produces a residual signal on the side channel S. The side information and the residual signal from side channel S are coded, for example, using a coding operation <NUM> and a corresponding encoder <NUM>, and then multiplexed in an output bit-stream <NUM> of the DFT stereo encoder <NUM>. The stereo processor and down-mixer <NUM> also down-mixes the left L and right R channels from the DFT calculators <NUM> and <NUM> to produce mid-channel M in DFT domain. Further information regarding the operation <NUM> of stereo processing and down-mixing, the stereo processor and down-mixer <NUM>, the mid-channel M and the side information and residual signal from side channel S can be found, for example, in Reference [<NUM>].

In an inverse DFT (IDFT) synthesis operation <NUM> of the DFT stereo encoding method <NUM>, a calculator <NUM> of the DFT stereo encoder <NUM> calculates the IDFT transform m of the mid-channel M at the sampling rate of the input stereo signal, for example <NUM>. In the same manner, in an inverse DFT (IDFT) synthesis operation <NUM> of the DFT stereo encoding method <NUM>, a calculator <NUM> of the DFT stereo encoder <NUM> calculates the IDFT transform m the channel M at the internal sampling rate.

Referring to <FIG>, the TD stereo encoding method <NUM> comprises an operation <NUM> of time domain analysis and weighted down-mixing in TD domain. To perform operation <NUM>, the TD stereo encoder <NUM> comprises a time domain analyzer and down-mixer <NUM> to calculate stereo side parameters <NUM> such as a sub-mode flag, mixing ratio index, or linear prediction reuse flag, which are multiplexed in an output bit-stream <NUM> of the TD stereo encoder <NUM>. The time domain analyzer and down-mixer <NUM> also performs weighted down-mixing of the channels l and r from the detectors <NUM> and <NUM> (<FIG>) to produce the primary channel PCh and secondary channel SCh using an estimated mixing ratio, in alignment with the ICA scaling. Further information regarding the time-domain analyzer and down-mixer <NUM> and the operation <NUM> can be found, for example, in Reference [<NUM>].

Down-mixing using the current frame mixing ratio is performed for example on the last <NUM> of the current frame of the input channels l and r while the first <NUM> of the current frame is down-mixed using a combination of the previous and current frame mixing ratios in a fade-in / fade-out manner to smooth the transition from one channel to the other. The two channels (primary channel PCh and secondary channel SCh) sampled at the stereo input channel sampling rate, for example <NUM>, are resampled using FIR decimation filters to their representations at <NUM>, and at the internal sampling rate.

In the TD stereo mode, it is not only the stereo input signal of the current frame which is down-mixed. Also, stored down-mixed signals that correspond to the previous frame are down-mixed again. The length of the previous signal subject to this re-computation corresponds to the length of the time-shifted signal re-computed in the ICA module, i.e. <NUM> + <NUM> = <NUM>.

In the IVAS codec (IVAS stereo encoding device <NUM> and IVAS stereo decoding device <NUM>), there is a restructuration of the traditional pre-processing such that some classification decisions are done on the codec overall bit-rate while other decisions are done depending on the core-encoding bit-rate. Consequently, the traditional pre-processing, as used for example in the EVS codec (Reference [<NUM>]), is split into two parts to ensure that the best possible codec configuration is used in each processed frame. Thus, the codec configuration can change from frame to frame while certain changes of configuration can be made as fast as possible, for example those based on signal activity or signal class. On the other hand, some changes in codec configuration should not happen too often, for example selection of coded audio bandwidth, selection of internal sampling rate or bit-budget distribution between low-band and high-band coding; too frequent changes in such codec configuration can lead to unstable coded signal quality or even audible artifacts.

The first part of the pre-processing, the front pre-processing, may include pre-processing and classification modules such as resampling at the pre-processing sampling rate, spectral analysis, Band-Width Detection (BWD), Sound Activity Detection (SAD), Linear Prediction (LP) analysis, open-loop pitch search, signal classification, speech/music classification. It is noted that the decisions in the front pre-processing depend exclusively on the overall codec bit-rate. Further information regarding the operations performed during the above described pre-processing can be found, for example, in Reference [<NUM>].

In the DFT stereo mode (DFT stereo encoder <NUM> of <FIG>), front pre-processing is performed by a front pre-processor <NUM> and the corresponding front pre-processing operation <NUM> on the mid-channel m in time domain at the internal sampling rate from IDFT calculator <NUM>.

In the TD stereo mode, the front pre-processing is performed by (a) a front pre-processor <NUM> and the corresponding front pre-processing operation <NUM> on the primary channel PCh from the time domain analyzer and down-mixer <NUM>, and (b) a front pre-processor <NUM> and the corresponding front pre-processing operation <NUM> on the secondary channel SCh from the time domain analyzer and down-mixer <NUM>.

In the MDCT stereo mode, the front pre-processing is performed by a front pre-processor <NUM> and the corresponding front pre-processing operation <NUM> on the input left channel l from the time domain transient detector <NUM> (<FIG>), and (b) a front pre-processor <NUM> and the corresponding front pre-processing operation <NUM> on the input right channel r from the time domain transient detector <NUM> (<FIG>).

Configurations of the core-encoder(s) is made on the basis of the codec overall bit-rate and front pre-processing.

Specifically, in the DFT stereo encoder <NUM> and the corresponding DFT stereo encoding method <NUM> (<FIG>), a core-encoder configurator <NUM> and the corresponding core-encoder configuration operation <NUM> are responsive to the mid-channel m in time domain from the IDFT calculator <NUM> and the output from the front pre-processor <NUM> to configure the core-encoder <NUM> and corresponding core-encoding operation <NUM>. The core-encoder configurator <NUM> is responsible for example of setting the internal sampling rate and/or modifying the core-encoder type classification. Further information regarding the core-encoder configuration in the DFT domain can be found, for example, in References [<NUM>] and [<NUM>].

In the TD stereo encoder <NUM> and the corresponding TD stereo encoding method <NUM> (<FIG>), a core-encoders configurator <NUM> and the corresponding core-encoders configuration operation <NUM> are responsive to the front pre-processed primary channel PCh and secondary channel SCh from the front pre-processors <NUM> and <NUM>, respectively, to perform configuration of the core-encoder <NUM> and corresponding core-encoding operation <NUM> of the primary channel PCh and the core-encoder <NUM> and corresponding core-encoding operation <NUM> of the secondary channel SCh. The core-encoder configurator <NUM> is responsible for example of setting the internal sampling rate and/or modifying the core-encoder type classification. Further information regarding core-encoders configuration in the TD domain can be found, for example, in References [<NUM>] and [<NUM>].

The DFT encoding method <NUM> comprises an operation <NUM> of further pre-processing. To perform operation <NUM>, a so-called further pre-processor <NUM> of the DFT stereo encoder <NUM> conducts a second part of the pre-processing that may include classification, core selection, pre-processing at encoding internal sampling rate, etc. The decisions in the front pre-processor <NUM> depend on the core-encoding bit-rate which usually fluctuates during a session. Additional information regarding the operations performed during such further pre-processing in DFT domain can be found, for example, in Reference [<NUM>].

The TD encoding method <NUM> comprises an operation <NUM> of further pre-processing. To perform operation <NUM>, a so-called further pre-processor <NUM> of the TD stereo encoder <NUM> conducts, prior to core-encoding the primary channel PCh, a second part of the pre-processing that may include classification, core selection, pre-processing at encoding internal sampling rate, etc. The decisions in the further pre-processor <NUM> depend on the core-encoding bit-rate which usually fluctuates during a session.

Also, the TD encoding method <NUM> comprises an operation <NUM> of further pre-processing. To perform operation <NUM>, the TD stereo encoder <NUM> comprises a so-called further pre-processor <NUM> to conduct, prior to core-encoding the secondary channel SCh, a second part of the pre-processing that may include classification, core selection, pre-processing at encoding internal sampling rate, etc. The decisions in the further pre-processor <NUM> depend on the core-encoding bit-rate which usually fluctuates during a session.

Additional information regarding such further pre-processing in the TD domain can be found, for example, in Reference [<NUM>].

The MDCT encoding method <NUM> comprises an operation <NUM> of further pre-processing of the left channel l. To perform operation <NUM>, a so-called further pre-processor <NUM> of the MDCT stereo encoder <NUM> conducts a second part of the pre-processing of the left channel l that may include classification, core selection, pre-processing at encoding internal sampling rate, etc., prior to an operation <NUM> of joint core-encoding of the left channel l and the right channel r performed by the joint core-encoder <NUM> of the MDCT stereo encoder <NUM>.

The MDCT encoding method <NUM> comprises an operation <NUM> of further pre-processing of the right channel r. To perform operation <NUM>, a so-called further pre-processor <NUM> of the MDCT stereo encoder <NUM> conducts a second part of the pre-processing of the left channel l that may include classification, core selection, pre-processing at encoding internal sampling rate, etc., prior to the operation <NUM> of joint core-encoding of the left channel l and the right channel r performed by the joint core-encoder <NUM> of the MDCT stereo encoder <NUM>.

Additional information regarding such further pre-processing in the MDCT domain can be found, for example, in Reference [<NUM>].

In general, the core-encoder <NUM> in the DFT stereo encoder <NUM> (performing the core-encoding operation <NUM>) and the core-encoders <NUM> (performing the core-encoding operation <NUM>) and <NUM> (performing the core-encoding operation <NUM>) in the TD stereo encoder <NUM> can be any variable bit-rate mono codec. In the illustrative implementation of the present disclosure, the EVS codec (See Reference [<NUM>]) with fluctuating bit-rate capability (See Reference [<NUM>]) is used. Of course, other suitable codecs may be possibly considered and implemented. In the MDCT stereo encoder <NUM>, the joint core-encoder <NUM> is employed which can be in general a stereo coding module with stereophonic tools that processes and quantizes the l and r channels in a joint manner.

Finally, common stereo updates are performed. Further information regarding common stereo updates may be found, for example, in Reference [<NUM>].

Referring to <FIG> and <FIG>, the stereo mode signaling <NUM> from the stereo classifier and stereo mode selector <NUM>, a bit-stream <NUM> from the side information, residual signal encoder <NUM>, and a bit-stream <NUM> from the core-encoder <NUM> are multiplexed to form the DFT stereo encoder bit stream <NUM> (then forming an output bit-stream <NUM> of the IVAS stereo encoding device <NUM> (<FIG>)).

Referring to <FIG> and <FIG>, the stereo mode signaling <NUM> from the stereo classifier and stereo mode selector <NUM>, the side parameters <NUM> from the time-domain analyzer and down-mixer <NUM>, the ICA parameters <NUM> from the ICA encoder <NUM>, a bit-stream <NUM> from the core-encoder <NUM> and a bit-stream <NUM> from the core-encoder <NUM> are multiplexed to form the TD stereo encoder bit-stream <NUM> (then forming the output bit-stream <NUM> of the IVAS stereo encoding device <NUM> (<FIG>)).

Referring to <FIG> and <FIG>, the stereo mode signaling <NUM> from the stereo classifier and stereo mode selector <NUM>, and a bit-stream <NUM> from the joint core-encoder <NUM> are multiplexed to form the MDCT stereo encoder bit-stream <NUM> (then forming the output bit-stream <NUM> of the IVAS stereo encoding device <NUM> (<FIG>)).

Switching from the TD stereo mode (TD stereo encoder <NUM>) to the DFT stereo mode (DFT stereo encoder <NUM>) is relatively straightforward as illustrated in <FIG>.

Specifically, <FIG> is a flow chart illustrating processing operations in the IVAS stereo encoding device <NUM> and method <NUM> upon switching from the TD stereo mode to the DFT stereo mode. As can be seen, <FIG> shows two frames of stereo input signal, i.e. a TD stereo frame <NUM> followed by a DFT stereo frame <NUM>, with different processing operations and related time instances when switching from the TD stereo mode to the DFT stereo mode.

A sufficiently long look-ahead is available, resampling is done in the DFT domain (thus no FIR decimation filter memory handling), and there is a transition from two core-encoders <NUM> and <NUM> in the last TD stereo frame <NUM> to one core-encoder <NUM> in the first DFT stereo frame <NUM>.

The following operations performed upon switching from the TD stereo mode (TD stereo encoder <NUM>) to the DFT stereo mode (DFT stereo encoder <NUM>) are performed by the above mentioned stereo mode switching controller (not shown) in response to the stereo mode selection.

The instance A) of <FIG> refers to an update of the DFT analysis memory, specifically the DFT stereo OLA analysis memory as part of the DFT stereo data structure which is subject to windowing prior to the DFT calculating operations <NUM> and <NUM>. This update is done by the stereo mode switching controller (not shown) before the Inter-Channel Alignment (ICA) (See <NUM> in <FIG>) and comprises storing samples related to the last <NUM> of the current TD stereo frame <NUM> of the channels l and r of the input stereo signal. This update is done every TD stereo frame in both channels l and r. Further information regarding the DFT analysis memory may be found, for example, in References [<NUM>] and [<NUM>].

The instance B) of <FIG> refers to an update of the DFT synthesis memory, specifically the OLA synthesis memory as part of the DFT stereo data structure which results from windowing after the IDFT calculating operations <NUM> and <NUM>, upon switching from the TD stereo mode to the DFT stereo mode. The stereo mode switching controller (not shown) performs this update in the first DFT stereo frame <NUM> following the TD stereo frame <NUM> and uses, for this update, the TD stereo memories as part of the TD stereo data structure and used for the TD stereo processing corresponding to the down-mixed primary channel PCh. Further information regarding the DFT synthesis memory may be found, for example, in References [<NUM>] and [<NUM>], and further information regarding the TD stereo memories may be found, for example, in Reference [<NUM>].

Starting with the first DFT stereo frame <NUM>, certain TD stereo related data structures, for example the TD stereo data structure (as used in the TD stereo encoder <NUM>) and a data structure of the core-encoder <NUM> related to the secondary channel SCh, are no longer needed and, therefore, are de-allocated, i.e. freed by the stereo mode switching controller (not shown).

In the DFT stereo frame <NUM> following the TD stereo frame <NUM>, the stereo mode switching controller (not shown) continues the core-encoding operation <NUM> in the core-encoder <NUM> of the DFT stereo encoder <NUM> with memories of the primary PCh channel core-encoder <NUM> (e.g. synthesis memory, pre-emphasis memory, past signals and parameters, etc.) in the preceding TD stereo frame <NUM> while controlling time instance differences between the TD and DFT stereo modes to ensure continuity of several core-encoder buffers, e.g. pre-emphasized input signal buffers, HB input buffers, etc. which are later used in the low-band encoder, resp. the FD-BWE high-band encoder. Further information regarding the core-encoding operation <NUM>, memories of the PCh channel core-encoder <NUM>, pre-emphasized input signal buffers, HB input buffers, etc. may be found, for example, in Reference [<NUM>].

Switching from the DFT stereo mode to the TD stereo mode is more complicated than switching from the TD stereo mode to the DFT stereo mode, due to the more complex structure of the TD stereo encoder <NUM>. The following operations performed upon switching from the DFT stereo mode (DFT stereo encoder <NUM>) to the TD stereo mode (TD stereo encoder <NUM>) are performed by the stereo mode switching controller (not shown) in response to the stereo mode selection.

<FIG> is a flow chart illustrating processing operations in the IVAS stereo encoding device <NUM> and method <NUM> upon switching from the DFT stereo mode to the TD stereo mode. In particular, <FIG> shows two frames of the stereo input signal, i.e. a DFT stereo frame <NUM> followed by a TD stereo frame <NUM>, at different processing operations with related time instances when switching from the DFT stereo mode to the TD stereo mode.

The instance A) of <FIG> refers to the update of the FIR resampling filter memory (as employed in the FIR resampling from the input stereo signal sampling rate to the <NUM> sampling rate and to the internal core-encoder sampling rate) used in the primary channel PCh of the TD stereo coding mode. The stereo mode switching controller (not shown) performs this update in every DFT stereo frame using the down-mixed mid-channel m and corresponds to a <NUM> x <NUM> long segment <NUM> before the last <NUM> long segment in the DFT stereo frame <NUM> (See <NUM>), thereby ensuring continuity of the FIR resampling memory for the primary channel PCh.

Since the side channel s (<FIG>) of the DFT stereo encoding method <NUM> is not available though it is used at, for example, the <NUM> sampling rate, at the input stereo signal sampling rate and at the internal sampling rate, the stereo mode switching controller (not shown) populates the FIR resampling filter memory of the down-mixed secondary channel SCh differently. In order to reconstruct the full length of the down-mixed signal at the internal sampling rate for the core-encoder <NUM>, a <NUM> segment (See <NUM>) of the down-mixed signal of the previous frame is recomputed in the TD stereo frame <NUM>. Thus, the update of the down-mixed secondary channel SCh FIR resampling filter memory corresponds to a <NUM> x <NUM> long segment <NUM> of the down-mixed mid-channel m before the last <NUM> long segment (See <NUM>); this is done in the first TD stereo frame <NUM> after switching from the preceding DFT stereo frame <NUM>. The secondary channel SCh FIR resampling filter memory update is referred to by instance C) in <FIG>. As can be seen, the stereo mode switching controller (not shown) re-computes in the TD stereo frame a length (See <NUM>) of the down-mixed signal which is longer in the secondary channel SCh with respect to the recomputed length of the down-mixed signal in the primary channel PCh (See <NUM>).

Instance B) in <FIG> relates to updating (re-computation) of the primary PCh and secondary SCh channels in the first TD stereo frame <NUM> following the DFT stereo frame <NUM>. The operations of instance B) as performed by the stereo mode switching controller (not shown) are illustrated in more detail in <FIG>. As mentioned in the foregoing description, <FIG> is a flow chart illustrating processing operations upon switching from the DFT stereo mode to the TD stereo mode.

Referring to <FIG>, in an operation <NUM>, the stereo mode switching controller (not shown) recalculates the ICA memory as used in the ICA analysis and computation (See operation <NUM> in <FIG>) and later as input signal for the pre-processing and core-encoders (See operations <NUM>-<NUM> and <NUM>-<NUM>) of length of <NUM> (as discussed in Sections <NUM>. <NUM>-<NUM>. <NUM> of the present disclosure) of the channels l and r corresponding to the previous DFT stereo frame <NUM>.

Thus, in operations <NUM> and <NUM>, the stereo mode switching controller (not shown) recalculates the primary PCh and secondary SCh channels of the DFT stereo frame <NUM> by down-mixing the ICA-processed channels l and r using a stereo mixing ratio of that frame <NUM>.

For the secondary channel SCh, the length (See <NUM>) of the past segment to be recalculated by the stereo mode switching controller (not shown) in operation <NUM> is <NUM> although a segment of length of only <NUM> (See <NUM>) is recalculated when there is no stereo coding mode switching. For the primary channel PCh (See operation <NUM>), the length of the segment to be recalculated by the stereo mode switching controller (not shown) using the TD stereo mixing ratio of the past frame <NUM> is always <NUM> (See <NUM>). This ensures continuity of the primary PCh and secondary SCh channels.

A continuous down-mixed signal is employed when switching from mid-channel m of the DFT stereo frame <NUM> to the primary channel PCh of the TD stereo frame <NUM>. For that purpose, the stereo mode switching controller (not shown) cross-fades (<NUM>) the <NUM> long segment (See <NUM>) of the DFT mid-channel m with the recalculated primary channel PCh (<NUM>) of the DFT stereo frame <NUM> in order to smooth the transition and to equalize for different down-mix signal energy between the DFT stereo mode and the TD stereo mode. The reconstruction of the secondary channel SCh in operation <NUM> uses the mixing ratio of the frame <NUM> while no further smoothing is applied because the secondary channel SCh from the DFT stereo frame <NUM> is not available.

Core-encoding in the first TD stereo frame <NUM> following the DFT stereo frame <NUM> then continues with resampling of the down-mixed signals using the FIR filters, pre-emphasizing these signals, computation of HB signals, etc. Further information regarding these operations may be found, for example, in Reference [<NUM>].

With respect to the pre-emphasis filter implemented as a first-order high-pass filter used to emphasize higher frequencies of the input signal (See Reference [<NUM>], Clause <NUM>. <NUM>), the stereo mode switching controller (not shown) stores two values of the pre-emphasis filter memory in every DFT stereo frame. These memory values correspond to time instances based on different re-computation length of the DFT and TD stereo modes. This mechanism ensures an optimal re-computation of the pre-emphasis signal in the channel m respectively the primary channel PCh with a minimal signal length. For the secondary channel SCh of the TD stereo mode, the pre-emphasis filter memory is set to zero before the first TD stereo frame is processed.

Starting with the first TD stereo frame <NUM> following the DFT stereo frame <NUM>, certain DFT stereo related data structures (e.g. DFT stereo data structure mentioned herein above) are not needed, so they are deallocated/freed by the stereo mode switching controller (not shown). On the other hand, a second instance of the core-encoder data structure is allocated and initialized for the core-encoding (operation <NUM>) of the secondary channel SCh. The majority of the secondary channel SCh core-encoder data structures are reset though some of them are estimated for smoother switching transitions. For example, the previous excitation buffer (adaptive codebook of the ACELP core), previous LSF parameters and LSP parameters (See Reference [<NUM>]) of the secondary channel SCh are populated from their counterparts in the primary channel PCh. Reset or estimation of the secondary channel SCh previous buffers may be a source of a number of artifacts. While many of such artifacts are significantly suppressed in smoothing-based processes at the decoder, few of them might remain a source of subjective artifacts.

Switching from the TD stereo mode to the MDCT stereo mode is relatively straightforward because both these stereo modes handle two input channels and employ two core-encoder instances. The main obstacle is to maintain the correct phase of the input left and right channels.

In order to maintain the correct phase of the input left and right channels of the stereo sound signal, the stereo mode switching controller (not shown) alters TD stereo down-mixing. In the last TD stereo frame before the first MDCT stereo frame, the TD stereo mixing ratio is set to β = <NUM> and an opposite-phase down-mixing of the left and right channels of the stereo sound signal is implemented using, for example, the following formula for the TD stereo down-mixing: <MAT> <MAT> where PCh(i) is the TD primary channel, SCh(i) is the TD secondary channel, l(i) is the left channel, r(i) is the right channel, β is the TD stereo mixing ratio, and i is the discrete time index.

In turn, this means that the TD stereo primary channel PCh(i) is identical to the MDCT stereo past left channel lpast(i) and the TD stereo secondary channel SCh(i) is identical to the MDCT stereo past right channel rpast(i) where i is the discrete time index. For completeness, it is noted that the stereo mode switching controller (not shown) may use in the last TD stereo frame a default TD stereo down-mixing using for example the following formula: <MAT> <MAT>.

Next, in usual (no stereo mode switching) MDCT stereo processing, the front pre-processing (front pre-processors <NUM> and <NUM> and front pre-processing operations <NUM> and <NUM>) does not recompute the look-ahead of the left l and right r channels of the stereo sound signal except for its last <NUM> long segment. However, in practice, the look-ahead of the length of <NUM> + <NUM> is subject to re-computation at the internal sampling rate (<NUM> in this non-limitative illustrative implementation). Thus, no specific handling is needed to maintain the continuity of input signals at the input sampling rate.

Then, in usual (no stereo mode switching) MDCT stereo processing, the further pre-processing (further pre-processors <NUM> and <NUM> and front pre-processing operations <NUM> and <NUM>) does not recompute the look-ahead of the left l and right r channels of the stereo sound signal except of its last <NUM> long segment. In contrast with the front pre-processing, the input signals (left l and right r channels of the stereo sound signal) at the internal sampling rate (<NUM> in this non-limitative illustrative implementation) of a length of only <NUM> are recomputed in the further pre-processing.

In other words:
The MDCT stereo encoder <NUM> comprises (a) front pre-processors <NUM> and <NUM> which, in the second MDCT stereo mode, recompute the look-ahead of first duration of the left l and right r channels of the stereo sound signal at the internal sampling rate, and (b) further pre-processors which, in the second MDCT stereo mode, recompute a last segment of given duration of the look-ahead of the left l and right r channels of the stereo sound signal at the internal sampling rate, wherein the first and second durations are different.

The MDCT stereo coding operation <NUM> comprises, in the second MDCT stereo mode, (a) recomputing the look-ahead of first duration of the left l and right r channels of the stereo sound signal at the internal sampling rate, and (b) recomputing a last segment of given duration of the look-ahead of the left l and right r channels of the stereo sound signal at the internal sampling rate, wherein the first and second durations are different.

Similarly to the switching from the TD stereo mode to the MDCT stereo mode, two input channels are always available and two core-encoder instances are always employed in this scenario. The main obstacle is again to maintain the correct phase of the input left and right channels. Thus, in the first TD stereo frame after the last MDCT stereo frame, the stereo mode switching controller (not shown) sets the TD stereo mixing ratio to β = <NUM> and alters TD stereo down-mixing by using the opposite-phase mixing scheme similarly as described in Section <NUM>.

Another specific about the switching from the MDCT stereo mode to the TD stereo mode is that the stereo mode switching controller (not shown) properly reconstructs in the first TD frame the past segment of input channels of the stereo sound signal at the internal sampling rate. Thus, a part of the look-ahead corresponding to <NUM> - <NUM> = <NUM> is reconstructed (resampled and pre-emphasized) in the first TD stereo frame.

A mechanism similar to the switching from the DFT stereo mode to the TD stereo mode as described above is used in this scenario, wherein the primary PCh and secondary SCh channels of the TD stereo mode are replaced by the left l and right r channels of the MDCT stereo mode.

A mechanism similar to the switching from the TD stereo mode to the DFT stereo mode as described above is used in this scenario, wherein the primary PCh and secondary SCh channels of the TD stereo mode are replaced by the left l and right r channels of the MDCT stereo mode.

<FIG> is a high-level block diagram illustrating concurrently an IVAS stereo decoding device <NUM> and the corresponding decoding method <NUM>, wherein the IVAS stereo decoding device <NUM> comprises a DFT stereo decoder <NUM> and the corresponding DFT stereo decoding method <NUM>, a TD stereo decoder <NUM> and the corresponding TD stereo decoding method <NUM>, and a MDCT stereo decoder <NUM> and the corresponding MDCT stereo decoding method <NUM>. For simplicity, only DFT, TD and MDCT stereo modes are shown and described; however, it is within the scope of the present disclosure to use and implement other types of stereo modes.

The IVAS stereo decoding device <NUM> and corresponding decoding method <NUM> receive a bit-stream <NUM> transmitted from the IVAS stereo encoding device <NUM>. Generally speaking, the IVAS stereo decoding device <NUM> and corresponding decoding method <NUM> decodes, from the bit-stream <NUM>, successive frames of a coded stereo signal, for example <NUM>-ms long frames as in the case of the EVS codec, performs an up-mixing of the decoded frames, and finally produces a stereo output signal including channels I and r.

Core-decoding, performed at the internal sampling rate, is basically the same regardless of the actual stereo mode; however, core-decoding is done once (mid-channel m) for a DFT stereo frame and twice for a TD stereo frame (primary PCh and secondary SCh channels) or for a MDCT stereo frame (left l and right r channels). An issue is to maintain (update) memories of the secondary channel SCh of a TD stereo frame when switching from a DFT stereo frame to a TD stereo frame, resp. to maintain (update) memories of the r channel of a MDCT stereo frame when switching from a DFT stereo frame to a MDCT stereo frame.

Moreover, further decoding operations after core-decoding strongly depend on the actual stereo mode which consequently complicates switching between the stereo modes. The most fundamental differences are the following:.

DFT stereo decoder <NUM> and decoding method <NUM>:.

TD stereo decoder <NUM> and decoding method <NUM>: (Further information regarding the TD stereo decoder may be found, for example, in Reference [<NUM>]).

MDCT stereo decoder <NUM> and decoding method <NUM>:.

The different operations during decoding, mainly the DFT "vs" TD domain processing, and the different delay schemes between the DFT stereo mode and the TD stereo mode are carefully taken into consideration in the herein below described procedure for switching between the DFT and TD stereo modes.

The following Table III lists in a sequential order the processing operations in the IVAS stereo decoding device <NUM> for each frame depending on the current DFT, TD or MDCT stereo mode (See also <FIG>).

The IVAS stereo decoding method <NUM> comprises an operation (not shown) of controlling switching between the DFT, TD and MDCT stereo modes. To perform the switching controlling operation, the IVAS stereo decoding device <NUM> comprises a controller (not shown) of switching between the DFT, TD and MDCT stereo modes. Switching between the DFT, TD and MDCT stereo modes in the IVAS stereo decoding device <NUM> and decoding method <NUM> involves the use of the stereo mode switching controller (not shown) to maintain continuity of the following several decoder signals and memories <NUM>) to <NUM>) to enable adequate processing of these signals and use of said memories in the IVAS stereo decoding device <NUM> and method <NUM>:.

While it is relatively straightforward to maintain the continuity for one channel (mid-channel m in the DFT stereo mode, respectively primary channel PCh in the TD stereo mode or l channel in the MDCT stereo mode) in item <NUM>) above, it is challenging for the secondary channel SCh in item <NUM>) above and also for signals/memories in items <NUM>) - <NUM>) due to several aspects, for example completely missing past signal and memories of the secondary channel SCh, a different down-mixing, a different default delay between DFT stereo mode and TD stereo mode, etc. Also, a shorter decoder delay (<NUM>) when compared to the encoder delay (<NUM>) further complicates the decoding process.

The IVAS stereo decoding method <NUM> starts with reading (not shown) the stereo mode and audio bandwidth information from the transmitted bit-stream <NUM>. Based on the currently read stereo mode, the related decoding operations are performed for each particular stereo mode (see Table III) while memories and buffers of the other stereo modes are maintained.

Similarly as the IVAS stereo encoding device <NUM>, in a memory allocation operation (not shown), the stereo mode switching controller (not shown) dynamically allocates/deallocates data structures (static memory) depending on the current stereo mode. The stereo mode switching controller (not shown) keeps the static memory impact of the codec as low as possible by maintaining only those parts of the static memory that are used in the current frame. Reference is made to Table II for summary of data structures allocated in a particular stereo mode.

In addition, a LRTD stereo sub-mode flag is read by the stereo mode switching controller (not shown) to distinguish between the normal TD stereo mode and the LRTD stereo mode. Based on the sub-mode flag, the stereo mode switching controller (not shown) allocates/deallocates related data structures within the TD stereo mode as shown in Table II.

Similarly as the IVAS stereo encoding device <NUM>, the stereo mode switching controller (not shown) handles memories in case of switching from one the DFT, TD, and MDCT stereo modes to another stereo mode. This keeps updated long-term parameters and updates or resets past buffer memories.

Upon receiving a first DFT stereo frame following a TD stereo frame or MDCT stereo frame, the stereo mode switching controller (not shown) performs an operation of resetting the DFT stereo data structure (already defined in relation to the DFT stereo encoder <NUM>). Upon receiving a first TD stereo frame following a DFT or MDCT stereo frame, the stereo mode switching controller performs an operation of resetting the TD stereo data structure (already described in relation to the TD stereo decoder <NUM>). Finally, upon receiving a first MDCT stereo frame following a DFT or TD stereo frame, the stereo mode switching controller (not shown) performs an operation of resetting the MDCT stereo data structure. Again, upon switching from one the DFT and TD stereo modes to the other stereo mode, the stereo mode switching controller (not shown) performs an operation of transferring some stereo-related parameters between data structures as described in relation to the IVAS stereo encoding device <NUM> (See above Section <NUM>.

Updates/resets related to the secondary channel SCh of core-decoding are described in Section <NUM>.

Also, further information about the operations of stereo decoder configuration, core-decoder configuration, TD stereo decoder configuration, core-decoding, core switching in DFT domain, core-switching in TD domain in Table III may be found, for example, in References [<NUM>] and [<NUM>].

The stereo mode switching controller (not shown) maintains or updates the DFT OLA memories in each TD or MDCT stereo frame (See "Update of DFT stereo mode overlap memories", "Update MDCT stereo TCX overlap buffer" and "Reset / update of DFT stereo overlap memories" of Table III). In this manner, updated DFT OLA memories are available for a next DFT stereo frame. The actual maintaining/updating mechanism and related memory buffers are described later in Section <NUM> of the present disclosure. An example implementation of updating of the DFT stereo OLA memories performed in TD or MDCT stereo frames in the C source code is given below. <IMG>
<IMG>
<IMG>.

The DFT decoding method <NUM> comprises an operation <NUM> of core decoding the mid-channel m. To perform operation <NUM>, a core-decoder <NUM> decodes in response to the received bit-stream <NUM> the mid-channel m in time domain. The core-decoder <NUM> (performing the core-decoding operation <NUM>) in the DFT stereo decoder <NUM> can be any variable bit-rate mono codec. In the illustrative implementation of the present disclosure, the EVS codec (See Reference [<NUM>]) with fluctuating bit-rate capability (See Reference [<NUM>]) is used. Of course, other suitable codecs may be possibly considered and implemented.

In a DFT calculating operation <NUM> of the DFT decoding method <NUM> (DFT analysis of Table III), a calculator <NUM> computes the DFT of the mid-channel m to recover mid-channel M in the DFT domain.

The DFT decoding method <NUM> also comprises an operation <NUM> of decoding stereo side information and residual signal S (residual decoding of Table III). To perform operation <NUM>, a decoder <NUM> is responsive to the bit-stream <NUM> to recover the stereo side information and residual signal S.

In a DFT stereo decoding (DFT stereo decoding of Table III) and up-mixing (up-mixing in DFT domain of Table III) operation <NUM>, a DFT stereo decoder and up-mixer <NUM> produces the channels L and R in the DFT domain in response to the mid-channel M and the side information and residual signal S. Generally speaking, the DFT stereo decoding and up-mixing operation <NUM> is the inverse to the DFT stereo processing and down-mixing operation <NUM> of <FIG>.

In IDFT calculating operation <NUM> (DFT synthesis of Table III), a calculator <NUM> calculates the IDFT of channel L to recover channel l in time domain. Likewise, in IDFT calculating operation <NUM> (DFT synthesis of Table III), a calculator <NUM> calculates the IDFT of channel R to recover channel r in time domain.

The TD decoding method <NUM> comprises an operation <NUM> of core-decoding the primary channel PCh. To perform operation <NUM>, a core-decoder <NUM> decodes in response to the received bit-stream <NUM> the primary channel PCh.

The TD decoding method <NUM> also comprises an operation <NUM> of core-decoding the secondary channel SCh. To perform operation <NUM>, a core-decoder <NUM> decodes in response to the received bit-stream <NUM> the secondary channel SCh.

Again, the core-decoder <NUM> (performing the core-decoding operation <NUM> in the TD stereo decoder <NUM>) and the core-decoder <NUM> (performing the core-decoding operation <NUM> in the TD stereo decoder <NUM>) can be any variable bit-rate mono codec. In the illustrative implementation of the present disclosure, the EVS codec (See Reference [<NUM>]) with fluctuating bit-rate capability (See Reference [<NUM>]) is used. Of course, other suitable codecs may be possibly considered and implemented.

In a time domain (TD) up-mixing operation <NUM> (up-mixing in TD domain of Table III), an up-mixer <NUM> receives and up-mixes the primary PCh and secondary SCh channels to recover the time-domain channels l and r of the stereo signal based on the TD stereo mixing factor.

The MDCT decoding method <NUM> comprises an operation <NUM> of joint core-decoding (joint stereo decoding of Table III) the left channel l and the right channel r. To perform operation <NUM>, a joint core-decoder <NUM> decodes in response to the received bit-stream <NUM> the left channel l and the right channel r. It is noted that no up-mixing operation is performed and no up-mixer is employed in the MDCT stereo mode.

To perform a stereo synthesis time synchronization (synthesis synchronization of Table III) and stereo switching operation <NUM>, the stereo mode switching controller (not shown) comprises a time synchronizer and stereo switch <NUM> to receive the channels l and r from the DFT stereo decoder <NUM>, the TD stereo decoder <NUM> or the MDCT stereo decoder <NUM> and to synchronize the up-mixed output stereo channels l and r. The time synchronizer and stereo switch <NUM> delays the up-mixed output stereo channels l and r to match the codec overall delay value and handles transitions between the DFT stereo output channels, the TD stereo output channels and the MDCT stereo output channels.

By default, in the DFT stereo mode, the time synchronizer and stereo switch <NUM> introduces a delay of <NUM> at the DFT stereo decoder <NUM>. In order to match the codec overall delay of <NUM> (frame length of <NUM>, encoder delay of <NUM>, decoder delay of <NUM>), a delay synchronization of <NUM> is applied by the time synchronizer and stereo switch <NUM>. In case of the TD or MDCT stereo mode, the time synchronizer and stereo switch <NUM> applies a delay consisting of the <NUM> resampling delay and the <NUM> delay used for synchronization between the LB and HB synthesis and to match the overall codec delay of <NUM>.

After time synchronization and stereo switching (See the synthesis time synchronization and stereo switching operation <NUM> and time synchronizer and stereo switch <NUM> of <FIG>) are performed, the HB synthesis (from BWE or IC-BWE) is added to the core synthesis (IC-BWE, addition of HB synthesis of Table III; See also in <FIG> BWE or IC-BWE calculation operation <NUM> and BWE or IC-BWE calculator <NUM>) and ICA decoding (ICA decoder - temporal adjustment of Table III which desynchronize two output channels l and r) is performed before the final stereo synthesis of the channels l and r is outputted from the IVAS stereo decoding device <NUM> (See temporal ICA operation <NUM> and corresponding ICA decoder <NUM>). These operations <NUM> and <NUM> are skipped in the MDCT stereo mode.

Finally, as shown in Table III, common stereo updates are performed.

Further information regarding the elements, operations and signals mentioned in section <NUM> and <NUM> may be found, for example, in References [<NUM>] and [<NUM>].

The mechanism of switching from the TD stereo mode to the DFT stereo mode at the IVAS stereo decoding device <NUM> is complicated by the fact that the decoding steps between these two stereo modes are fundamentally different (see above Section <NUM> for details) including a transition from two core-decoders <NUM> and <NUM> in the last TD stereo frame to one core-decoder <NUM> in the first DFT stereo frame.

<FIG> is a flow chart illustrating processing operations in the IVAS stereo decoding device <NUM> and method <NUM> upon switching from the TD stereo mode to the DFT stereo mode. Specifically, <FIG> shows two frames of the decoded stereo signal at different processing operations with related time instances when switching from a TD stereo frame <NUM> to a DFT stereo frame <NUM>.

First, the core-decoders <NUM> and <NUM> of the TD stereo decoder <NUM> are used for both the primary PCh and secondary SCh channels and each output the corresponding decoded core synthesis at the internal sampling rate. In the TD stereo frame <NUM>, the decoded core synthesis from the two core-decoders <NUM> and <NUM> is used to update the DFT stereo OLA memory buffers (one memory buffer per channel, i.e. two OLA memory buffers in total; See above described DFT OLA analysis and synthesis memories). These OLA memory buffers are updated in every TD stereo frame to be up-to-date in case the next frame is a DFT stereo frame.

The instance A) of <FIG> refers to, upon receiving a first DFT stereo frame <NUM> following a TD stereo frame <NUM>, an operation (not shown) of updating the DFT stereo analysis memories (these are used in the OLA part of the windowing in the previous and current frame before the DFT calculating operation <NUM>) at the internal sampling rate, input_mem_LB[], using the stereo mode switching controller (not shown). For that purpose, a number Lovl of last samples <NUM> of the TD stereo synthesis at the internal sampling rate of the primary channel PCh and the secondary channel SCh in the TD stereo frame <NUM> are used by the stereo mode switching controller (not shown) to update the DFT stereo analysis memories of the DFT stereo mid-channel m and the side channel s, respectively. The length of the overlap segment <NUM>, Lovl, corresponds to the <NUM> long overlap part of the DFT analysis window <NUM>, e.g. Lovl = <NUM> samples at a <NUM> internal sampling rate.

Similarly, the stereo mode switching controller (not shown) updates the DFT stereo Bass Post-Filter (BPF) analysis memory (which is used in the OLA part of the windowing in the previous and current frame before the DFT calculating operation <NUM>) of the mid-channel m at the internal sampling rate, input_mem_BPF[], using Lovl last samples of the BPF error signal (See Reference [<NUM>], Clause <NUM>. <NUM>) of the TD primary channel PCh. Moreover, the DFT stereo Full Band (FB) analysis memory (this memory is used in the OLA part of the windowing in the previous and current frame before the DFT calculating operation <NUM>) of the mid-channel m at the output stereo signal sampling rate, input_mem[], is updated using the <NUM> last samples of the TD stereo PCh HB synthesis (ACELP core) respectively PCh TCX synthesis. The DFT stereo BPF and FB analysis memories are not employed for the side information channel s, so that these memories are not updated using the secondary channel SCh core synthesis.

Next, in the TD stereo frame <NUM>, the decoded ACELP core synthesis (primary PCh and secondary SCh channels) at the internal sampling rate is resampled using CLDFB-domain filtering which introduces a delay of <NUM>. In case of the TCX/HQ core frame, a compensation delay of <NUM> is used to synchronize the core synthesis between different cores. Then the TCX-LTP post-filter is applied to both core channels PCh and SCh.

At the next operation, the primary PCh and secondary SCh channels of the TD stereo synthesis at the output stereo signal sampling rate from the TD stereo frame <NUM> are subject to TD stereo up-mixing (combination of the primary PCh and secondary SCh channels using the TD stereo mixing ratio in TD up-mixer <NUM> (See Reference [<NUM>]) resulting in up-mixed stereo channels l and r in the time-domain. Since the up-mixing operation <NUM> is performed in the time-domain, it introduces no up-mixing delay.

Then, the left l and right r up-mixed channels of the TD stereo frame <NUM> from the up-mixer <NUM> of the TD stereo decoder <NUM> are used in an operation (not shown) of updating the DFT stereo synthesis memories (these are used in the OLA part of the windowing in the previous and current frame after the IDFT calculating operation <NUM>). Again, this update is done in every TD stereo frame by the stereo mode switching controller (not shown) in case the next frame is a DFT stereo frame. Instance B) of <FIG> depicts that the number of available last samples of the TD stereo left l and right r channels synthesis is insufficient to be used for a straightforward update of the DFT stereo synthesis memories. The <NUM> long DFT stereo synthesis memories are thus reconstructed in two segments using approximations. The first segment corresponds to the (<NUM> - <NUM>) ms long signal that is available (that is the up-mixed synthesis at the output stereo signal sampling rate) while the second segment corresponds to the remaining <NUM> long signal that is not available due to the core-decoder resampling delay.

Specifically, the DFT stereo synthesis memories are updated by the stereo mode switching controller (not shown) using the following sub-operations as illustrated in <FIG> is a flow chart illustrating the instance B) of <FIG>, comprising updating DFT stereo synthesis memories in a TD stereo frame on the decoder side:.

Finally, the up-mixed reconstructed synthesis <NUM> of the TD stereo frame <NUM> is aligned, i.e. delayed by <NUM> in the time synchronizer and stereo switch <NUM> in order to match the codec overall delay. Specifically:.

<FIG> is a flow chart illustrating an instance C) of <FIG>, comprising smoothing the output stereo synthesis in the first DFT stereo frame <NUM> following stereo mode switching, on the decoder side.

Referring to <FIG>, once the DFT stereo synthesis is aligned and synchronized to the codec overall delay in the first DFT stereo frame <NUM>, the stereo mode switching controller (not shown) performs a cross-fading operation <NUM> between the TD stereo aligned and synchronized synthesis <NUM> (from operation <NUM>) and the DFT stereo aligned and synchronized synthesis <NUM> (from operation <NUM>) to smooth the switching transition. The cross-fading is performed on a <NUM> long segment <NUM> starting after a <NUM> delay <NUM> at the beginning of both output channels l and r (all signals are at the output stereo signal sampling rate). This instance corresponds to instance C) in <FIG>.

Decoding then continues regardless of the current stereo mode with the IC-BWE calculator <NUM>, the ICA decoder <NUM> and common stereo decoder updates.

The fundamentally different decoding operations between the DFT stereo mode and the TD stereo mode and the presence of two core-decoders <NUM> and <NUM> in the TD stereo decoder <NUM> makes switching from the DFT stereo mode to the TD stereo mode in the IVAS stereo decoding device <NUM> challenging. <FIG> is a flow chart illustrating processing operations in the IVAS stereo decoding device <NUM> and method <NUM> upon switching from the DFT stereo mode to the TD stereo mode. Specifically, <FIG> shows two frames of decoded stereo signal at different processing operations with related time instances upon switching from a DFT stereo frame <NUM> to a TD stereo frame <NUM>.

Core-decoding may use a same processing regardless of the actual stereo mode with two exceptions.

First exception: In DFT stereo frames, resampling from the internal sampling rate to the output stereo signal sampling rate is performed in the DFT domain but the CLDFB resampling is run in parallel in order to maintain/update CLDFB analysis and synthesis memories in case the next frame is a TD stereo frame.

Second exception: Then, the BPF (Bass Post-Filter) (a low-frequency pitch enhancement procedure, see Reference [<NUM>], Clause <NUM>. <NUM>) is applied in the DFT domain in DFT stereo frames while the BPF analysis and computation of error signal is done in the time-domain regardless of the stereo mode.

Otherwise, all internal states and memories of the core-decoder are simply continuous and well maintained when switching from the DFT mid-channel m to the TD primary channel PCh.

In the DFT stereo frame <NUM>, decoding then continues with core-decoding (<NUM>) of mid-channel m, calculation (<NUM>) of the DFT transform of the mid-channel m in the time domain to obtain mid-channel M in the DFT domain, and stereo decoding and up-mixing (<NUM>) of channels M and S into channels L and R in the DFT domain including decoding (<NUM>) of the residual signal. The DFT domain analysis and synthesis introduces an OLA delay of <NUM>. The synthesis transitions are then handled in the time synchronizer and stereo switch <NUM>.

Upon switching from the DFT stereo frame <NUM> to the TD stereo frame <NUM>, the fact that there is only one core-decoder <NUM> in the DFT stereo decoder <NUM> makes core-decoding of the TD secondary channel SCh complicated because the internal states and memories of the second core-decoder <NUM> of the TD stereo decoder <NUM> are not continuously maintained (on the contrary, the internal states and memories of the first core-decoder <NUM> are continuously maintained using the internal states and memories of the core-decoder <NUM> of the DFT stereo decoder <NUM>). The memories of the second core-decoder <NUM> are thus usually reset in the stereo mode switching updates (See Table III) by the stereo mode switching controller (not shown). There are however few exceptions where the primary channel SCh memory is populated with the memory of certain PCh buffers, for example previous excitation, previous LSF parameters and previous LSP parameters. In any case, the synthesis at the beginning of the first TD secondary channel SCh frame after switching from the DFT stereo frame <NUM> to the TD stereo frame <NUM> consequently suffers from an imperfect reconstruction. Accordingly, while the synthesis from the first core-decoder <NUM> is well and smoothly decoded during stereo mode switching, the limited-quality synthesis from the second core decoder <NUM> introduces discontinuities during the stereo up-mixing and final synthesis (<NUM>). These discontinuities are suppressed by employing the DFT stereo OLA memories during the first TD stereo output synthesis reconstruction as described later.

The stereo mode switching controller (not shown) suppresses possible discontinuities and differences between the DFT stereo and the TD stereo up-mixed channels by a simple equalization of the signal energy. If the ICA target gain, gICA, is lower than <NUM>, the channel l, yL(i), after the up-mixing (<NUM>) and before the time synchronization (<NUM>) is altered in the first TD stereo frame <NUM> after stereo mode switching using the following relation: <MAT> where Leq is the length of the signals to equalize which corresponds in the IVAS stereo decoding device <NUM> to a <NUM> long segment (which corresponds for example to Leq = <NUM> samples at a <NUM> output stereo signal sampling rate). Then, the value of the gain factor α is obtained using the following relation: <MAT>.

Referring to <FIG>, the instance A) relates to a missing part <NUM> of the TD stereo up-mixed synchronized synthesis (from operation <NUM>) of the TD stereo frame <NUM> corresponding to a previous DFT stereo up-mixed synchronization synthesis memory from DFT stereo frame <NUM>. This memory of length of (<NUM> - <NUM>) ms is not available when switching from the DFT stereo frame <NUM> to the TD stereo frame <NUM> except for its first <NUM> long segment <NUM>.

<FIG> is a flow chart illustrating the instance A) of <FIG>, comprising updating the TD stereo up-mixed synchronization synthesis memory in a first TD stereo frame following switching from the DFT stereo mode to the TD stereo mode, on the decoder side.

Referring to both <FIG> and <FIG>, the stereo mode switching controller (not shown) reconstructs the <NUM> <NUM> of the TD stereo up-mixed synchronized synthesis using the following operations (a) to (e) for both the left l and right r channels:.

Switching from the TD stereo mode to the MDCT stereo mode is relatively straightforward because both these stereo modes handle two transport channels and employ two core-decoder instances.

As an opposite-phase down-mixing scheme was employed in the TD stereo encoder <NUM>, the stereo mode switching controller (not shown) similarly alters the TD stereo channel up-mixing to maintain the correct phase of the left and right channels of the stereo sound signal in the last TD stereo frame before the first MDCT stereo frame. Specifically, the stereo mode switching controller (not shown) sets the mixing ratio β = <NUM> and implements an opposite-phase up-mixing (inverse to opposite-phase down-mixing employed in the TD stereo encoder <NUM>) of the TD stereo primary channel PCh(i) and TD stereo secondary channel SCh(i) to calculate the MDCT stereo past left channel lpast(i) and the MDCT stereo past right channel rpast(i). Consequently, the TD stereo primary channel PCh(i) is identical to the MDCT stereo past left channel lpast(i) and the TD stereo secondary channel SCh(i) signal is identical to the MDCT stereo past right channel rpast(i).

Similarly to the switching from the TD stereo mode to the MDCT stereo mode, two transport channels are available and two core-decoder instances are employed in this scenario. In order to maintain the correct phase of the left and right channels of the stereo sound signal, the TD stereo mixing ratio is set to <NUM> and the opposite-phase up-mixing scheme is used again by the stereo mode switching controller (not shown) in the first TD stereo frame after the last MDCT stereo frame.

A mechanism similar to the decoder-side switching from the DFT stereo mode to the TD stereo mode is used in this scenario, wherein the primary PCh and secondary SCh channels of the TD stereo mode are replaced by the left l and right r channels of the MDCT stereo mode.

A mechanism similar to the decoder-side switching from the TD stereo mode to the DFT stereo mode is used in this scenario, wherein the primary PCh and secondary SCh channels of the TD stereo mode are replaced by the left l and right r channels of the MDCT stereo mode.

Finally, the decoding continues regardless of the current stereo mode with the IC-BWE decoding <NUM> (skipped in the the MDCT stereo mode), adding of the HB synthesis (skipped in the MDCT stereo mode), temporal ICA alignment <NUM> (skipped in the MDCT stereo mode) and common stereo decoder updates.

<FIG> is a simplified block diagram of an example configuration of hardware components forming each of the above described IVAS stereo encoding device <NUM> and IVAS stereo decoding device <NUM>.

Each of the IVAS stereo encoding device <NUM> and IVAS stereo decoding device <NUM> may be implemented as a part of a mobile terminal, as a part of a portable media player, or in any similar device. Each of the IVAS stereo encoding device <NUM> and IVAS stereo decoding device <NUM> (identified as <NUM> in <FIG>) comprises an input <NUM>, an output <NUM>, a processor <NUM> and a memory <NUM>.

The input <NUM> is configured to receive the left l and right r channels of the input stereo sound signal in digital or analog form in the case of the IVAS stereo encoding device <NUM>, or the bit-stream <NUM> in the case of the IVAS stereo decoding device <NUM>. The output <NUM> is configured to supply the multiplexed bit stream <NUM> in the case of the IVAS stereo encoding device <NUM> or the decoded left channel l and right channel r in the case of the IVAS stereo decoding device <NUM>. The input <NUM> and the output <NUM> may be implemented in a common module, for example a serial input/output device.

The processor <NUM> is operatively connected to the input <NUM>, to the output <NUM>, and to the memory <NUM>. The processor <NUM> is realized as one or more processors for executing code instructions in support of the functions of the various elements and operations of the above described IVAS stereo encoding device <NUM>, IVAS stereo encoding method <NUM>, IVAS stereo decoding device <NUM> and IVAS stereo decoding method <NUM> as shown in the accompanying figures and/or as described in the present disclosure.

The memory <NUM> may comprise a non-transient memory for storing code instructions executable by the processor <NUM>, specifically, a processor-readable memory storing non-transitory instructions that, when executed, cause a processor to implement the elements and operations of the IVAS stereo encoding device <NUM>, IVAS stereo encoding method <NUM>, IVAS stereo decoding device <NUM> and IVAS stereo decoding method <NUM>. The memory <NUM> may also comprise a random access memory or buffer(s) to store intermediate processing data from the various functions performed by the processor <NUM>.

Those of ordinary skill in the art will realize that the description of the IVAS stereo encoding device <NUM>, IVAS stereo encoding method <NUM>, IVAS stereo decoding device <NUM> and IVAS stereo decoding method <NUM> are illustrative only and are not intended to be in any way limiting. Other embodiments will readily suggest themselves to such persons with ordinary skill in the art having the benefit of the present disclosure. Furthermore, the disclosed IVAS stereo encoding device <NUM>, IVAS stereo encoding method <NUM>, IVAS stereo decoding device <NUM> and IVAS stereo decoding method <NUM> may be customized to offer valuable solutions to existing needs and problems of encoding and decoding stereo sound.

In the interest of clarity, not all of the routine features of the implementations of the IVAS stereo encoding device <NUM>, IVAS stereo encoding method <NUM>, IVAS stereo decoding device <NUM> and IVAS stereo decoding method <NUM> are shown and described. It will, of course, be appreciated that in the development of any such actual implementation of the IVAS stereo encoding device <NUM>, IVAS stereo encoding method <NUM>, IVAS stereo decoding device <NUM> and IVAS stereo decoding method <NUM>, numerous implementation-specific decisions may need to be made in order to achieve the developer's specific goals, such as compliance with application-, system-, network- and business-related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. Moreover, it will be appreciated that a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of engineering for those of ordinary skill in the field of sound processing having the benefit of the present disclosure.

In accordance with the present disclosure, the elements, processing operations, and/or data structures described herein may be implemented using various types of operating systems, computing platforms, network devices, computer programs, and/or general purpose machines. In addition, those of ordinary skill in the art will recognize that devices of a less general purpose nature, such as hardwired devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or the like, may also be used. Where a method comprising a series of operations and sub-operations is implemented by a processor, computer or a machine and those operations and sub-operations may be stored as a series of non-transitory code instructions readable by the processor, computer or machine, they may be stored on a tangible and/or non-transient medium.

Elements and processing operations of the IVAS stereo encoding device <NUM>, IVAS stereo encoding method <NUM>, IVAS stereo decoding device <NUM> and IVAS stereo decoding method <NUM> as described herein may comprise software, firmware, hardware, or any combination(s) of software, firmware, or hardware suitable for the purposes described herein.

In the IVAS stereo encoding method <NUM> and IVAS stereo decoding method <NUM> as described herein, the various processing operations and sub-operations may be performed in various orders and some of the processing operations and sub-operations may be optional.

Claim 1:
A device for encoding a stereo sound signal, comprising:
a first stereo encoder of the stereo sound signal using a first stereo mode operating in time domain, TD, wherein the first TD stereo mode, in TD frames of the stereo sound signal, (a) produces a first down-mixed signal and (b) uses first data structures and memories;
a second stereo encoder of the stereo sound signal using a second stereo mode operating in frequency domain, FD, wherein the second FD stereo mode, in FD frames of the stereo sound signal, (a) produces a second down-mixed signal and (b) uses second data structures and memories;
a controller configured to switch between (i) the first TD stereo mode and first stereo encoder, and (ii) the second FD stereo mode and second stereo encoder to code the stereo sound signal in time domain or frequency domain;
characterised in that, upon switching from one of the first TD and second FD stereo modes to the other of the first TD and second FD stereo modes, the stereo mode switching controller is configured to recalculate at least one length of down-mixed signal in a current frame of the stereo sound signal, wherein the recalculated down-mixed signal length in the first TD stereo mode is different from the recalculated down-mixed signal length in the second FD stereo mode.