Patent Description:
The MPEG-<NUM> Enhanced Low Delay AAC (AAC-ELD) usually operates at sample rates up to <NUM>, which results in an algorithmic delay of <NUM>. For some applications, e.g. lip-sync transmission of audio, an even lower delay is desirable. AAC-ELD already provides such an option by operating at higher sample rates, e.g. <NUM>, and therefore provides operation modes with even lower delay, e.g. <NUM>. However, this operation mode comes along with an unnecessary high complexity due to the high sample rate.

The solution to this problem is to apply a downscaled version of the filter bank and therefore, to render the audio signal at a lower sample rate, e.g. <NUM> instead of <NUM>. The downscaling operation is already part of AAC-ELD as it is inherited from the MPEG-<NUM> AAC-LD codec, which serves as a basis for AAC-ELD.

The question which remains, however, is how to find the downscaled version of a specific filter bank. That is, the only uncertainty is the way the window coefficients are derived whilst enabling clear conformance testing of the downscaled operation modes of the AAC-ELD decoder.

In the following the principles of the down-scaled operation mode of the AAC-(E)LD codecs are described.

The downscaled operation mode or AAC-LD is described for AAC-LD in ISO/IEC <NUM>-<NUM>:<NUM> in section <NUM>. <NUM> "Adaptation to systems using lower sampling rates" as follows:.

Please note that AAC-LD works with a standard MDCT framework and two window shapes, i.e. sine-window and low-overlap-window. Both windows are fully described by formulas and therefore, window coefficients for any transformation lengths can be determined.

Compared to AAC-LD, the AAC-ELD codec shows two major differences:.

The IMDCT algorithm using the low delay MDCT window is described in <NUM>. <NUM> in [<NUM>], which is very similar to the standard IMDCT version using e.g. the sine window. The coefficients of the low delay MDCT windows (<NUM> and <NUM> samples frame size) are given in Table <NUM>. <NUM> and <NUM>. <NUM> in [<NUM>]. Please note that the coefficients cannot be determined by a formula, as the coefficients are the result of an optimization algorithm. <FIG> shows a plot of the window shape for frame size <NUM>.

In case the low delay SBR (LD-SBR) tool is used in conjunction with the AAC-ELD coder, the filter banks of the LD-SBR module are downscaled as well. This ensures that the SBR module operates with the same frequency resolution and therefore, no more adaptions are required.

Thus, the above description reveals that there is a need for downscaling decoding operations such as, for example, downscaling a decoding at an AAC-ELD. It would be feasible to find out the coefficients for the downscaled synthesis window function anew, but this is a cumbersome task, necessitates additional storage for storing the downscaled version and renders a conformity check between the non-downscaled decoding and the downscaled decoding more complicated or, from another perspective, does not comply with the manner of downscaling requested in the AAC-ELD, for example. Depending on the downscale ratio, i.e. the ratio between the original sampling rate and the downscaled sampling rate, one could derive the downscaled synthesis window function simply by downsampling, i.e. picking out every second, third,. window coefficient of the original synthesis window function, but this procedure does not result in a sufficient conformity of the non-downscaled decoding and downscaled decoding, respectively. Using more sophisticated decimating procedures applied to the synthesis window function, lead to unacceptable deviations from the original synthesis window function shape. Therefore, there is a need in the art for an improved downscaled decoding concept.

The scientific publication <NPL>, describes to directly decode a high-fidelity audio bit-stream signal into a lower-sampled audio signal without first decoding and then down-sampling to the target sampling rate.

It is an object of the present invention to provide an audio decoding scheme which allows for an improved downscaled decoding.

This object is achieved by the subject matter of the independent claim.

The present invention is based on the finding that a downscaled version of an audio decoding procedure may more effectively and/or at improved compliance maintenance be achieved if the synthesis window used for downscaled audio decoding is a downsampled version of a reference synthesis window involved in the non-downscaled audio decoding procedure by downsampling by the downsampling factor by which the downsampled sampling rate and the original sampling rate deviate, and downsampled using a segmental interpolation in segments of <NUM>/<NUM> of the frame length. Advantageous aspects of the present application are the subject of dependent claims. Preferred embodiments of the present application are described below with respect to the figures, among which:.

The following description starts with an illustration of an embodiment for downscaled decoding with respect to the AAC-ELD codec. That is, the following description starts with an embodiment which could form a downscaled mode for AAC-ELD. This description concurrently forms a kind of explanation of the motivation underlying the embodiments of the present application. Later on, this description is generalized, thereby leading to a description of an audio decoder and audio decoding method in accordance with an embodiment of the present application.

As described in the introductory portion of the specification of the present application, AAC-ELD uses low delay MDCT windows. In order to generate downscaled versions thereof, i.e. downscaled low delay windows, the subsequently explained proposal for forming a downscaled mode for AAC-ELD uses a segmental spline interpolation algorithm which maintains the perfect reconstruction property (PR) of the LD-MDCT window with a very high precision. Therefore, the algorithm allows the generation of window coefficients in the direct form, as described in ISO/IEC <NUM>-<NUM>:<NUM>, as well as in the lifting form, as described in [<NUM>], in a compatible way. This means both implementations generate 16bit-conform output.

The interpolation of Low Delay MDCT window is performed as follows.

In general a spline interpolation is to be used for generating the downscaled window coefficients to maintain the frequency response and mostly the perfect reconstruction property (around 170dB SNR). The interpolation needs to be constraint in certain segments to maintain the perfect reconstruction property. For the window coefficients c covering the DCT kernel of the transformation (see also <FIG>, c(<NUM>). c(<NUM>)), the following constraint is required, <MAT> where N denotes the frame size. Some implementation may use different signs to optimize the complexity, here, denoted by sgn. The requirement in (<NUM>) can be illustrated by <FIG>. It should be recalled that simply in even in case of F=<NUM>, i.e. halfening the sample rate, leaving-out every second window coefficient of the reference synthesis window to obtain the downscaled synthesis window does not fulfil the requirement.

The coefficients c(<NUM>). c(<NUM>N - <NUM>) are listed along the diamond shape. The N/<NUM> zeros in the window coefficients, which are responsible for the delay reduction of the filter bank, are marked using a bold arrow. <FIG> shows the dependencies of the coefficients caused by the folding involved in the MDCT and also the points where the interpolation needs to be constraint in order to avoid any undesired dependencies.

The second constraint is not only required for the segment containing the zeros but also for the other segments. Knowing that some coefficients in the DCT kernel were not determined by the optimization algorithm but were determined by formula (<NUM>) to enable PR, several discontinuities in the window shape can be explained, e.g. around c(<NUM>+<NUM>) in <FIG>. In order to minimize the PR error, the interpolation needs to stop at such points, which appear in a N/<NUM> grid.

Due to that reason, the segment size of N/<NUM> is chosen for the segmental spline interpolation to generate the downscaled window coefficients. The source window coefficients are always given by the coefficients used for N = <NUM>, also for downscaling operations resulting in frame sizes of N = <NUM> or N = <NUM>. The basic algorithm is outlined very briefly in the following as MATLAB code:
<IMG>.

As the spline function may not be fully deterministic, the complete algorithm is exactly specified in the following section, which may be included into ISO/IEC <NUM>-<NUM>:<NUM>, in order to form an improved downscaled mode in AAC-ELD.

In other words, the following section provides a proposal as to how the above-outlined idea could be applied to ER AAC ELD, i.e. as to how a low-complex decoder could decode a ER AAC ELD bitstream coded at a first data rate at a second data rate lower than the first data rate. It is emphasized however, that the definition of N as used in the following adheres to the standard. Here, N corresponds to the length of the DCT kernel whereas hereinabove, in the claims, and the subsequently described generalized embodiments, N corresponds to the frame length, namely the mutual overlap length of the DCT kernels, i.e. the half of the DCT kernel length. Accordingly, while N was indicated to be <NUM> hereinabove, for example, it is indicated to be <NUM> in the following.

The following paragraphs are proposed for inclusion to <NUM>-<NUM>:<NUM> [<NUM>] via Amendment. The numbered references below refer to that document [<NUM>].

For certain applications, ER AAC LD can change the playout sample rate in order to avoid additional resampling steps (see <NUM>. ER AAC ELD can apply similar downscaling steps using the Low Delay MDCT window and the LD-SBR tool. In case AAC-ELD operates with the LD-SBR tool, the downscaling factor is limited to multiples of <NUM>. Without LD-SBR, the downscaled frame size needs to be an integer number.

The LD-MDCT window wLD for N=<NUM> is downscaled by a factor F using a segmental spline interpolation. The number of leading zeros in the window coefficients, i.e. N/<NUM>, determines the segment size. The downscaled window coefficients wLD_d are used for the inverse MDCT as described in <NUM>. <NUM> but with a downscaled window length Nd = N / F. Please note that the algorithm is also able to generate downscaled lifting coefficients of the LD-MDCT. <IMG>
<IMG>.

In case the Low Delay SBR tool is used in conjunction with ELD, this tool can be downscaled to lower sample rates, at least for downscaling factors of a multiple of <NUM>. The downscale factor F controls the number of bands used for the CLDFB analysis and synthesis filter bank. The following two paragraphs describe a downscaled CLDFB analysis and synthesis filter bank, see also <NUM>.

In the equation, exp( ) denotes the complex exponential function and j is the imaginary unit.

Please note that setting F = <NUM> provides the downsampled synthesis filter bank according to <NUM>. Therefore, to process a downsampled LD-SBR bit stream with an additional downscale factor F, F needs to be multiplied by <NUM>.

The downscaling of the CLDFB can be applied for the real valued versions of the low power SBR mode as well. For illustration, please also consider <NUM>.

For the downscaled real-valued analysis and synthesis filter bank, follow the description in <NUM>. <NUM> and <NUM>. <NUM> and exchange the exp() modulator in M by a cos() modulator.

This subclause describes the Low Delay MDCT filter bank utilized in the AAC ELD encoder. The core MDCT algorithm is mostly unchanged, but with a longer window, such that n is now running from -N to N-<NUM> (rather than from <NUM> to N-<NUM>)
The spectral coefficient, Xi,k, are defined as follows: <MAT> where:.

The window length N (based on the sine window) is <NUM> or <NUM>.

The window length of the low-delay window is <NUM>*N. The windowing is extended to the past in the following way: <MAT> for n=-N,. ,N-<NUM>, with the synthesis window w used as the analysis window by inverting the order.

The synthesis filter bank is modified compared to the standard IMDCT algorithm using a sine window in order to adopt a low-delay filter bank. The core IMDCT algorithm is mostly unchanged, but with a longer window, such that n is now running up to 2N-<NUM> (rather than up to N-<NUM>). <MAT> where:.

The windowing and overlap-add is conducted in the following way:
The length N window is replaced by a length 2N window with more overlap in the past, and less overlap to the future (N/<NUM> values are actually zero).

Windowing for the Low Delay Window: <MAT>.

Where the window now has a length of 2N, hence n=<NUM>,.

Overlap and add: <MAT> for <NUM><=n<N/<NUM>.

Here, the paragraphs proposed for being included into <NUM>-<NUM>:<NUM> via amendment end.

Naturally, the above description of a possible downscaled mode for AAC-ELD merely represents one embodiment of the present application and several modifications are feasible. Generally, embodiments of the present application are not restricted to an audio decoder performing a downscaled version of AAC-ELD decoding. In other words, embodiments of the present application may, for instance, be derived by forming an audio decoder capable of performing the inverse transformation process in a downscaled manner only without supporting or using the various AAC-ELD specific further tasks such as, for instance, the scale factor-based transmission of the spectral envelope, TNS (temporal noise shaping) filtering, spectral band replication (SBR) or the like.

Subsequently, a more general embodiment for an audio decoder is described. The above-outlined example for an AAC-ELD audio decoder supporting the described downscaled mode could thus represent an implementation of the subsequently described audio decoder. In particular, the subsequently explained decoder is shown in <FIG> while <FIG> illustrates the steps performed by the decoder of <FIG>.

The audio decoder of <FIG>, which is generally indicated using reference sign <NUM>, comprises a receiver <NUM>, a grabber <NUM>, a spectral-to-time modulator <NUM>, a windower <NUM> and a time domain aliasing canceler <NUM>, all of which are connected in series to each other in the order of their mentioning. The interaction and functionality of blocks <NUM> to <NUM> of audio decoder <NUM> are described in the following with respect to <FIG>. As described at the end of the description of the present application, blocks <NUM> to <NUM> may be implemented in software, programmable hardware or hardware such as in the form of a computer program, an FPGA or appropriately programmed computer, programmed microprocessor or application specific integrated circuit with the blocks <NUM> to <NUM> representing respective subroutines, circuit paths or the like.

In a manner outlined in more details below, the audio decoder <NUM> of <FIG> is configured to, - and the elements of the audio decoder <NUM> are configured to appropriately cooperate - in order to decode an audio signal <NUM> from a data stream <NUM> with a noteworthiness that audio decoder <NUM> decodes signal <NUM> at a sampling rate being <NUM>/Fth of the sampling rate at which the audio signal <NUM> has been transform coded into data stream <NUM> at the encoding side. F may, for instance, be any rational number greater than one. The audio decoder may be configured to operate at different or varying downscaling factors F or at a fixed one. Alternatives are described in more detail below.

The manner in which the audio signal <NUM> is transform coded at the encoding or original sampling rate into the data stream is illustrated in <FIG> in the upper half. At <NUM> <FIG> illustrates the spectral coefficients using small boxes or squares <NUM> arranged in a spectrotemporal manner along a time axis <NUM> which runs horizontally in <FIG>, and a frequency axis <NUM> which runs vertically in <FIG>, respectively. The spectral coefficients <NUM> are transmitted within data stream <NUM>. The manner in which the spectral coefficients <NUM> have been obtained, and thus the manner via which the spectral coefficients <NUM> represent the audio signal <NUM>, is illustrated in <FIG>at <NUM>, which illustrates for a portion of time axis <NUM> how the spectral coefficients <NUM> belonging to, or representing the respective time portion, have been obtained from the audio signal.

In particular, coefficients <NUM> as transmitted within data stream <NUM> are coefficients of a lapped transform of the audio signal <NUM> so that the audio signal <NUM>, sampled at the original or encoding sampling rate, is partitioned into immediately temporally consecutive and nonoverlapping frames of a predetermined length N, wherein N spectral coefficients are transmitted in data stream <NUM> for each frame <NUM>. That is, transform coefficients <NUM> are obtained from the audio signal <NUM> using a critically sampled lapped transform. In the spectrotemporal spectrogram representation <NUM>, each column of the temporal sequence of columns of spectral coefficients <NUM> corresponds to a respective one of frames <NUM> of the sequence of frames. The N spectral coefficients <NUM> are obtained for the corresponding frame <NUM> by a spectrally decomposing transform or time-to-spectral modulation, the modulation functions of which temporally extend, however, not only across the frame <NUM> to which the resulting spectral coefficients <NUM> belong, but also across E + <NUM> previous frames, wherein E may be any integer or any even numbered integer greater than zero. That is, the spectral coefficients <NUM> of one column of the spectrogram at <NUM> which belonged to a certain frame <NUM> are obtained by applying a transform onto a transform window, which in addition the respective frame comprises E + <NUM> frames lying in the past relative to the current frame. The spectral decomposition of the samples of the audio signal within this transform window <NUM>, which is illustrated in <FIG> for the column of transform coefficients <NUM> belonging to the middle frame <NUM> of the portion shown at <NUM> is achieved using a low delay unimodal analysis window function <NUM> using which the spectral samples within the transform window <NUM> are weighted prior to subjecting same to an MDCT or MDST or other spectral decomposition transform. In order to lower the encoder-side delay, the analysis window <NUM> comprises a zero-interval <NUM> at the temporal leading end thereof so that the encoder does not need to await the corresponding portion of newest samples within the current frame <NUM> so as to compute the spectral coefficients <NUM> for this current frame <NUM>. That is, within the zero-interval <NUM> the low delay window function <NUM> is zero or has zero window coefficients so that the co-located audio samples of the current frame <NUM> do not, owing to the window weighting <NUM>, contribute to the transform coefficients <NUM> transmitted for that frame and a data stream <NUM>. That is, summarizing the above, transform coefficients <NUM> belonging to a current frame <NUM> are obtained by windowing and spectral decomposition of samples of the audio signal within a transform window <NUM> which comprises the current frame as well as temporally preceding frames and which temporally overlaps with the corresponding transform windows used for determining the spectral coefficients <NUM> belonging to temporally neighboring frames.

Before resuming the description of the audio decoder <NUM>, it should be noted that the description of the transmission of the spectral coefficients <NUM> within the data stream <NUM> as provided so far has been simplified with respect to the manner in which the spectral coefficients <NUM> are quantized or coded into data stream <NUM> and/or the manner in which the audio signal <NUM> has been pre-processed before subjecting the audio signal to the lapped transform. For example, the audio encoder having transform coded audio signal <NUM> into data stream <NUM> may be controlled via a psychoacoustic model or may use a psychoacoustic model to keep the quantization noise and quantizing the spectral coefficients <NUM> unperceivable for the hearer and/or below a masking threshold function, thereby determining scale factors for spectral bands using which the quantized and transmitted spectral coefficients <NUM> are scaled. The scale factors would also be signaled in data stream <NUM>. Alternatively, the audio encoder may have been a TCX (transform coded excitation) type of encoder. Then, the audio signal would have had subject to a linear prediction analysis filtering before forming the spectrotemporal representation <NUM> of spectral coefficients <NUM> by applying the lapped transform onto the excitation signal, i.e. the linear prediction residual signal. For example, the linear prediction coefficients could be signaled in data stream <NUM> as well, and a spectral uniform quantization could be applied in order to obtain the spectral coefficients <NUM>.

Furthermore, the description brought forward so far has also been simplified with respect to the frame length of frames <NUM> and/or with respect to the low delay window function <NUM>. In fact, the audio signal <NUM> may have been coded into data stream <NUM> in a manner using varying frame sizes and/or different windows <NUM>. However, the description brought forward in the following concentrates on one window <NUM> and one frame length, although the subsequent description may easily be extended to a case where the entropy encoder changes these parameters during coding the audio signal into the data stream.

Returning back to the audio decoder <NUM> of <FIG> and its description, receiver <NUM> receives data stream <NUM> and receives thereby, for each frame <NUM>, N spectral coefficients <NUM>, i.e. a respective column of coefficients <NUM> shown in <FIG>. It should be recalled that the temporal length of the frames <NUM>, measured in samples of the original or encoding sampling rate, is N as indicated in <FIG> at <NUM>, but the audio decoder <NUM> of <FIG> is configured to decode the audio signal <NUM> at a reduced sampling rate. The audio decoder <NUM> supports, for example, merely this downscaled decoding functionality described in the following. Alternatively, audio decoder <NUM> would be able to reconstruct the audio signal at the original or encoding sampling rate, but may be switched between the downscaled decoding mode and a non-downscaled decoding mode with the downscaled decoding mode coinciding with the audio decoder's <NUM> mode of operation as subsequently explained. For example, audio encoder <NUM> could be switched to a downscaled decoding mode in the case of a low battery level, reduced reproduction environment capabilities or the like. Whenever the situation changes the audio decoder <NUM> could, for instance, switch back from the downscaled decoding mode to the non-downscaled one. In any case, in accordance with the downscaled decoding process of decoder <NUM> as described in the following, the audio signal <NUM> is reconstructed at a sampling rate at which frames <NUM> have, at the reduced sampling rate, a lower length measured in samples of this reduced sampling rate, namely a length of N/F samples at the reduced sampling rate.

The output of receiver <NUM> is the sequence of N spectral coefficients, namely one set of N spectral coefficients, i.e. one column in <FIG>, per frame <NUM>. It already turned out from the above brief description of the transform coding process for forming data stream <NUM> that receiver <NUM> may apply various tasks in obtaining the N spectral coefficients per frame <NUM>. Receiver <NUM> uses entropy decoding in order to read the spectral coefficients <NUM> from the data stream <NUM>. Receiver <NUM> also spectrally shapes the spectral coefficients read from the data stream with scale factors provided in the data stream and/or scale factors derived by linear prediction coefficients conveyed within data stream <NUM>. For example, receiver <NUM> may obtain scale factors from the data stream <NUM>, namely on a per frame and per subband basis, and use these scale factors in order to scale the scale factors conveyed within the data stream <NUM>. Alternatively, receiver <NUM> may derive scale factors from linear prediction coefficients conveyed within the data stream <NUM>, for each frame <NUM>, and use these scale factors in order to scale the transmitted spectral coefficients <NUM>. Optionally, receiver <NUM> may perform gap filling in order to synthetically fill zero-quantized portions within the sets of N spectral coefficients <NUM> per frame. Additionally or alternatively, receiver <NUM> may apply a TNS-synthesis filter onto a transmitted TNS filter coefficient per frame to assist the reconstruction of the spectral coefficients <NUM> from the data stream with the TNS coefficients also being transmitted within the data stream <NUM>. The just outlined possible tasks of receiver <NUM> shall be understood as a non-exclusive list of possible measures and receiver <NUM> may perform further or other tasks in connection with the reading of the spectral coefficients <NUM> from data stream <NUM>.

Grabber <NUM> thus receives from receiver <NUM> the spectrogram <NUM> of spectral coefficients <NUM> and grabs, for each frame <NUM>, a low frequency fraction <NUM> of the N spectral coefficients of the respective frame <NUM>, namely the N/F lowest-frequency spectral coefficients.

That is, spectral-to-time modulator <NUM> receives from grabber <NUM> a stream or sequence <NUM> of N/F spectral coefficients <NUM> per frame <NUM>, corresponding to a low-frequency slice out of the spectrogram <NUM>, spectrally registered to the lowest frequency spectral coefficients illustrated using index "<NUM>" in <FIG>, and extending till the spectral coefficients of index N/F - <NUM>.

The spectral-to-time modulator <NUM> subjects, for each frame <NUM>, the corresponding low-frequency fraction <NUM> of spectral coefficients <NUM> to an inverse transform <NUM> having modulation functions of length (E + <NUM>) · N/F temporally extending over the respective frame and E + <NUM> previous frames as illustrated at <NUM> in <FIG>, thereby obtaining a temporal portion of length (E + <NUM>) · N/F, i.e. a not-yet windowed time segment <NUM>. That is, the spectral-to-time modulator may obtain a temporal time segment of (E + <NUM>) · N/F samples of reduced sampling rate by weighting and summing modulation functions of the same length using, for instance, the first formulae of the proposed replacement section A. <NUM> indicated above. The newest N/F samples of time segment <NUM> belong to the current frame <NUM>. The modulation functions may, as indicated, be cosine functions in case of the inverse transform being an inverse MDCT, or sine functions in case of the inverse transform being an inverse MDCT, for instance.

Thus, windower <NUM> receives, for each frame, a temporal portion <NUM>, the N/F samples at the leading end thereof temporally corresponding to the respective frame while the other samples of the respective temporal portion <NUM> belong to the corresponding temporally preceding frames. Windower <NUM> windows, for each frame <NUM>, the temporal portion <NUM> using a unimodal synthesis window <NUM> of length (E + <NUM>) · N/F comprising a zero-portion <NUM> of length <NUM>/<NUM> · N/F at a leading end thereof, i.e. <NUM>/F · N/F zero-valued window coefficients, and having a peak <NUM> within its temporal interval succeeding, temporally, the zero-portion <NUM>, i.e. the temporal interval of temporal portion <NUM> not covered by the zero-portion <NUM>. The latter temporal interval may be called the non-zero portion of window <NUM> and has a length of <NUM>/<NUM> · N/F measured in samples of the reduced sampling rate, i.e. <NUM>/<NUM> · N/F window coefficients. The windower <NUM> weights, for instance, the temporal portion <NUM> using window <NUM>. This weighting or multiplying <NUM> of each temporal portion <NUM> with window <NUM> results in a windowed temporal portion <NUM>, one for each frame <NUM>, and coinciding with the respective temporal portion <NUM> as far as the temporal coverage is concerned. In the above proposed section A. <NUM>, the windowing processing which may be used by window <NUM> is described by the formulae relating zi,n to xi,n, where xi,n corresponds to the aforementioned temporal portions <NUM> not yet windowed and zi,n corresponds to the windowed temporal portions <NUM> with i indexing the sequence of frames/windows, and n indexing, within each temporal portion <NUM>/<NUM>, the samples or values of the respective portions <NUM>/<NUM> in accordance with a reduced sampling rate.

Thus, the time domain aliasing canceler <NUM> receives from windower <NUM> a sequence of windowed temporal portions <NUM>, namely one per frame <NUM>. Canceler <NUM> subjects the windowed temporal portions <NUM> of frames <NUM> to an overlap-add process <NUM> by registering each windowed temporal portion <NUM> with its leading N/F values to coincide with the corresponding frame <NUM>. By this measure, a trailing-end fraction of length (E + <NUM>)/(E + <NUM>) of the windowed temporal portion <NUM> of a current frame, i.e. the remainder having length (E + <NUM>)· N/F, overlaps with a corresponding equally long leading end of the temporal portion of the immediately preceding frame. In formulae, the time domain aliasing canceler <NUM> may operate as shown in the last formula of the above proposed version of section A. <NUM>, where outi,n corresponds to the audio samples of the reconstructed audio signal <NUM> at the reduced sampling rate.

The processes of windowing <NUM> and overlap-adding <NUM> as performed by windower <NUM> and time domain aliasing canceler <NUM> are illustrated in more detail below with respect to <FIG> uses both the nomenclature applied in the above-proposed section A. <NUM> and the reference signs applied in <FIG> and <FIG>. x<NUM>,<NUM> to x<NUM>,(E+<NUM>)·N/F-<NUM> represents the <NUM>th temporal portion <NUM> obtained by the spatial-to-temporal-modulator <NUM> for the <NUM>th frame <NUM>. The first index of x indexes the frames <NUM> along the temporal order, and the second index of x orders the samples of the temporal along the temporal order, the inter-sample pitch belonging to the reduced sample rate. Then, in <FIG>, w<NUM> to w(E+<NUM>)·N/F-<NUM> indicate the window coefficients of window <NUM>. Like the second index of x, i.e. the temporal portion <NUM> as output by modulator <NUM>, the index of w is such that index <NUM> corresponds to the oldest and index (E + <NUM>) · N/F - <NUM> corresponds to the newest sample value when the window <NUM> is applied to the respective temporal portion <NUM>. Windower <NUM> windows the temporal portion <NUM> using window <NUM> to obtain the windowed temporal portion <NUM> so that z<NUM>,<NUM> to z<NUM>,(E+<NUM>)·N/F-<NUM>, which denotes the windowed temporal portion <NUM> for the <NUM>th frame, is obtained according to z<NUM>,<NUM> = x<NUM>,<NUM> · w<NUM>,. , z<NUM>,(E+<NUM>)·N/F-<NUM> = x<NUM>,(E+<NUM>)·N/F-<NUM> · w(E+<NUM>)·N/F-<NUM>. The indices of z have the same meaning as for x. In this manner, modulator <NUM> and windower <NUM> act for each frame indexed by the first index of x and z. Canceler <NUM> sums up E + <NUM> windowed temporal portions <NUM> of E + <NUM> immediately consecutive frames with offsetting the samples of the windowed temporal portions <NUM> relative to each other by one frame, i.e. by the number of samples per frame <NUM>, namely N/F, so as to obtain the samples u of one current frame, here u-(E+<NUM>),<NUM>. u-(E+<NUM>),N/F-<NUM>). Here, again, the first index of u indicates the frame number and the second index orders the samples of this frame along the temporal order. The canceller joins the reconstructed frames thus obtained so that the samples of the reconstructed audio signal <NUM> within the consecutive frames <NUM> follow each other according to u-(E+<NUM>),<NUM>. u-(E+<NUM>),N/F-<NUM>, u-E,<NUM>,. u-E,N/F-<NUM>, u-(E-<NUM>),<NUM>,. the canceler <NUM> computes each sample of the audio signal <NUM> within the -(E+<NUM>)th frame according to u-(E+<NUM>),<NUM> = z<NUM>,<NUM> + z-<NUM>,N/F +. z-(E+<NUM>),(E+<NUM>)·N/F,. , u-(E+<NUM>)·N/F-<NUM> = z<NUM>,N/F-<NUM> + z-<NUM>,<NUM>·N/F-<NUM> +. + z-(E+<NUM>),(E+<NUM>)·N/F-<NUM>, i.e. summing up (e+<NUM>) addends per samples u of the current frame.

<FIG> illustrates a possible exploitation of the fact that, among the just windowed samples contributing to the audio samples u of frame -(E + <NUM>), the ones corresponding to, or having been windowed using, the zero-portion <NUM> of window <NUM>, namely z-(E+<NUM>),(E+<NUM>/<NUM>)·N/F. z-(E+<NUM>),(E+<NUM>)·N/F-<NUM> are zero valued. Thus, instead of obtaining all N/F samples within the -(E+<NUM>)th frame <NUM> of the audio signal u using E+<NUM> addends, canceler <NUM> may compute the leading end quarter thereof, namely u-(E+<NUM>),(E+<NUM>/<NUM>)·N/F. u-(E+<NUM>),(E+<NUM>)·N/F-<NUM> merely using E+<NUM> addends according to u-(E+<NUM>),(E+<NUM>/<NUM>)·N/F = z<NUM>,<NUM>/<NUM>·N/F + z-<NUM>,<NUM>/<NUM>·N/F +. + z-E,(E+<NUM>/<NUM>)·N/F,. , u-(E+<NUM>),(E+<NUM>)·N/F-<NUM> = z<NUM>,N/F-<NUM> + z-<NUM>,<NUM>·N/F-<NUM> +. + z-E,(E+<NUM>)·N/F-<NUM>. In this manner, the windower could even leave out, effectively, the performance of the weighting <NUM> with respect to the zero-portion <NUM>. Samples u-(E+<NUM>),(E+<NUM>/<NUM>)·N/F. u-(E+<NUM>),(E+<NUM>)·N/F-<NUM> of current -(E+<NUM>)th frame would, thus, be obtained using E+<NUM> addends only, while u-(E+<NUM>),(E+<NUM>)·N/F. u-(E+<NUM>),(E+<NUM>/<NUM>)·N/F-<NUM> would be obtained using E+<NUM> addends.

Thus, in the manner outlined above, the audio decoder <NUM> of <FIG> reproduces, in a downscaled manner, the audio signal coded into data stream <NUM>. To this end, the audio decoder <NUM> uses a window function <NUM> which is itself a downsampled version of a reference synthesis window of length (E+<NUM>)·N. As explained with respect to <FIG>, this downsampled version, i.e. window <NUM>, is obtained by downsampling the reference synthesis window by a factor of F, i.e. the downsampling factor, using a segmental interpolation, namely in segments of length <NUM>/<NUM>·N when measured in the not yet downscaled regime, in segments of length <NUM>/<NUM>·N/F in the downsampled regime, in segments of quarters of a frame length of frames <NUM>, measured temporally and expressed independently from the sampling rate. In <NUM> · (E+<NUM>) the interpolation is, thus, performed, thus yielding <NUM> · (E+<NUM>) times <NUM>/<NUM>·N/F long segments which, concatenated, represent the downsampled version of the reference synthesis window of length (E+<NUM>)·N. See <FIG> for illustration. <FIG> shows the synthesis window <NUM> which is unimodal and used by the audio decoder <NUM> in accordance with a downsampled audio decoding procedure underneath the reference synthesis window <NUM> which his of length (E+<NUM>)·N. That is, by the downsampling procedure <NUM> leading from the reference synthesis window <NUM> to the synthesis window <NUM> actually used by the audio decoder <NUM> for downsampled decoding, the number of window coefficients is reduced by a factor of F. In <FIG>, the nomenclature of <FIG> and <FIG> has been adhered to, i.e. w is used in order to denote the downsampled version window <NUM>, while w' has been used to denote the window coefficients of the reference synthesis window <NUM>.

As just mentioned, in order to perform the downsampling <NUM>, the reference synthesis window <NUM> is processed in segments <NUM> of equal length. In number, there are (E+<NUM>)·<NUM> such segments <NUM>. Measured in the original sampling rate, i.e. in the number of window coefficients of the reference synthesis window <NUM>, each segment <NUM> is <NUM>/<NUM> · N window coefficients w' long, and measured in the reduced or downsampled sampling rate, each segment <NUM> is <NUM>/<NUM>·N/F window coefficients w long.

Naturally, it would be possible to perform the downsampling <NUM> for each downsampled window coefficient wi coinciding accidentally with any of the window coefficients <MAT> of the reference synthesis window <NUM> by simply setting <MAT> with the sample time of wi coinciding with that of <MAT>, and/or by linearly interpolating any window coefficients wi residing, temporally, between two window coefficients <MAT> and <MAT> by linear interpolation, but this procedure would result in a poor approximation of the reference synthesis window <NUM>, i.e. the synthesis window <NUM> used by audio decoder <NUM> for the downsampled decoding would represent a poor approximation of the reference synthesis window <NUM>, thereby not fulfilling the request for guaranteeing conformance testing of the downscaled decoding relative to the non-downscaled decoding of the audio signal from data stream <NUM>. Thus, the downsampling <NUM> involves an interpolation procedure according to which the majority of the window coefficients wi of the downsampled window <NUM>, namely the ones positioned offset from the borders of segments <NUM>, depend by way of the downsampling procedure <NUM> on more than two window coefficients w' of the reference window <NUM>. In particular, while the majority of the window coefficients wi of the downsampled window <NUM> depend on more than two window coefficients <MAT> of the reference window <NUM> in order to increase the quality of the interpolation/downsampling result, i.e. the approximation quality, for every window coefficient wi of the downsampled version <NUM> it holds true that same does not depend in window coefficients <MAT> belonging to different segments <NUM>. Rather, the downsampling procedure <NUM> is a segmental interpolation procedure.

The synthesis window <NUM> is a concatenation of spline functions of length <NUM>/<NUM> · N/F. Cubic spline functions are used. Such an example has been outlined above in section A. <NUM> where the outer for-next loop sequentially looped over segments <NUM> wherein, in each segment <NUM>, the downsampling or interpolation <NUM> involved a mathematical combination of consecutive window coefficients w' within the current segment <NUM> at, for example, the first for next clause in the section "calculate vector r needed to calculate the coefficients c". The interpolation applied in segments, may, however, also be chosen differently. That is, the interpolation is not restricted to splines or cubic splines. Rather, linear interpolation or any other interpolation method may be used as well. In any case, the segmental implementation of the interpolation would cause the computation of samples of the downscaled synthesis window, i.e. the outmost samples of the segments of the downscaled synthesis window, neighboring another segment, to not depend on window coefficients of the reference synthesis window residing in different segments.

It may be that windower <NUM> obtains the downsampled synthesis window <NUM> from a storage where the window coefficients wi of this downsampled synthesis window <NUM> have been stored after having been obtained using the downsampling <NUM>. Alternatively, as illustrated in <FIG>, the audio decoder <NUM> may comprise a segmental downsampler <NUM> performing the downsampling <NUM> of <FIG> on the basis of the reference synthesis window <NUM>.

It should be noted that the audio decoder <NUM> of <FIG> may be configured to support merely one fixed downsampling factor F or may support different values. In that case, the audio decoder <NUM> may be responsive to an input value for F as illustrated in <FIG> at <NUM>. The grabber <NUM>, for instance, may be responsive to this value F in order to grab, as mentioned above, the N/F spectral values per frame spectrum. In a like manner, the optional segmental downsampler <NUM> may also be responsive to this value of F an operate as indicated above. The S/T modulator <NUM> may be responsive to F either in order to, for example, computationally derive downscaled/downsampled versions of the modulation functions, downscaled/downsampled relative to the ones used in not-downscaled operation mode where the reconstruction leads to the full audio sample rate.

Naturally, the modulator <NUM> would also be responsive to F input <NUM>, as modulator <NUM> would use appropriately downsampled versions of the modulation functions and the same holds true for the windower <NUM> and canceler <NUM> with respect to an adaptation of the actual length of the frames in the reduced or downsampled sampling rate.

For example, F may lie between <NUM> and <NUM>, both inclusively.

It should be noted that the decoder of <FIG> and <FIG> or any modification thereof outlined herein, may be implemented so as to perform the spectral-to-time transition using a lifting implementation of the Low Delay MDCT as taught in, for example, <CIT>.

<FIG> illustrates an implementation of the decoder using the lifting concept. The S/T modulator <NUM> performs exemplarily an inverse DCT-IV and is shown as followed by a block representing the concatenation of the windower <NUM> and the time domain aliasing canceller <NUM>. In the example of <FIG> and in the invention E is <NUM>, i.e. E=<NUM>.

The modulator <NUM> comprises an inverse type-iv discrete cosine transform frequency/time converter. Instead of outputing sequences of (E+<NUM>)N/F long temporal portions <NUM>, it merely outputs temporal portions <NUM> of length <NUM>·N/F, all derived from the sequence of N/F long spectra <NUM>, these shortened portions <NUM> corresponding to the DCT kernel, i.e. the <NUM>·N/F newest samples of the erstwhile described portions.

The windower <NUM> acts as described previously and generates a windowed temporal portion <NUM> for each temporal portion <NUM>, but it operates merely on the DCT kernel. To this end, windower <NUM> uses window function ωi with i=<NUM>. 2N/F-<NUM>, having the kernel size. The relationship between wi with i=<NUM>. (E+<NUM>)·N/F-<NUM> is described later, just as the relationship between the subsequently mentioned lifting coefficients and wi with i=<NUM>. (E+<NUM>)·N/F-<NUM> is.

Using the nomenclature applied above, the process described so far yields: <MAT> with redefining M = N/F, so that M corresponds to the frame size expressed in the downscaled domain and using the nomenclature of <FIG>, wherein, however, zk,n and xk,n shall contain merely the samples of the windowed temporal portion and the not-yet windowed temporal portion within the DCT kernel having size <NUM>·M and temporally corresponding to samples E·N/F. (E+<NUM>)·N/F-<NUM> in <FIG>. That is, n is an integer indicating a sample index and ωn is a real-valued window function coefficient corresponding to the sample index n.

The overlap/add process of the canceller <NUM> operates in a manner different compared to the above description. It generates intermediate temporal portions mk(<NUM>),. mk(M-<NUM>) based on the equation or expression <MAT>.

In the implementation of <FIG>, the apparatus further comprises a lifter <NUM> which may be interpreted as a part of the modulator <NUM> and windower <NUM> since the lifter <NUM> compensates the fact the modulator and the windower restricted their processing to the DCT kernel instead of processing the extension of the modulation functions and the synthesis window beyond the kernel towards the past which extension was introduced to compensate for the zero portion <NUM>. The lifter <NUM> produces, using a framework of the delayers and multipliers <NUM> and adders <NUM>, the finally reconstructed temporal portions or frames of length M in pairs of immediately consecutive frames based on the equation or expression <MAT> and <MAT> wherein In with n = <NUM>. M-<NUM> are real-valued lifting coefficients related to the downscaled synthesis window in a manner described in more detail below.

In other words, for the extended overlap of E frames into the past, only M additional multiplier-add operations are required, as can be seen in the framework of the lifter <NUM>. These additional operations are sometimes also referred to as "zero-delay matrices". Sometimes these operations are also known as "lifting steps". The efficient implementation shown in <FIG> may under some circumstances be more efficient as a straightforward implementation. To be more precise, depending on the concrete implementation, such a more efficient implementation might result in saving M operations, as in the case of a straightforward implementation for M operations, it might be advisable to implement, as the implementation shown in Fig. <NUM>, requires in principle, <NUM> operations in the framework of the module <NUM> and M operations in the framework of the lifter <NUM>.

As to the dependency of ωn with n=<NUM>. <NUM>-<NUM> and In with n = <NUM>. M-<NUM> on the synthesis window wi with i = <NUM>. (E+<NUM>)M-<NUM> (it is recalled that here E=<NUM>), the following formulae describe the relationship between them with displacing, however, the subscript indices used so far into the parenthesis following the respective variable: <MAT> <MAT> <MAT> <MAT> <MAT> <MAT> <MAT> <MAT> <MAT>.

Please note that the window wi contains the peak values on the right side in this formulation, i.e. between the indices <NUM> and <NUM> - <NUM>. The above formulae relate coefficients In with n = <NUM>. M-<NUM> and ωn n = <NUM>,. ,<NUM>-<NUM> to the coefficients wn with n = <NUM>. (E+<NUM>)M-<NUM> of the downscaled synthesis window. As can be seen, In with n = <NUM>. M-<NUM> actually merely depend on ¾ of the coefficients of the downsampled synthesis window, namely on wn with n = <NUM>. (E+<NUM>)M-<NUM>, while ωn n = <NUM>,. ,<NUM>-<NUM> depend on all wn with n = <NUM>. (E+<NUM>)M-<NUM>.

As stated above, it might be that windower <NUM> obtains the downsampled synthesis window <NUM> wn with n = <NUM>. (E+<NUM>)M-<NUM> from a storage where the window coefficients wi of this downsampled synthesis window <NUM> have been stored after having been obtained using the downsampling <NUM>, and from where same are read to compute coefficients In with n = <NUM>. M-<NUM> and ωn n = <NUM>,. ,<NUM>-<NUM> using the above relation, but alternatively, winder <NUM> may retrieve the coefficients In with n = <NUM>. M-<NUM> and ωn n = <NUM>,. ,<NUM>-<NUM>, thus computed from the pre-downsampled synthesis window, from the storage directly. Alternatively, as stated above, the audio decoder <NUM> may comprise the segmental downsampler <NUM> performing the downsampling <NUM> of <FIG> on the basis of the reference synthesis window <NUM>, thereby yielding wn with n = <NUM>. (E+<NUM>)M-<NUM> on the basis of which the windower <NUM> computes coefficients In with n = <NUM>. M-<NUM> and ωn n = <NUM>,. ,<NUM>-<NUM> using above relation/formulae. Even using the lifting implementation, more than one value for F may be supported.

Briefly summarizing the lifting implementation, same results in an audio decoder <NUM> configured to decode an audio signal <NUM> at a first sampling rate from a data stream <NUM> into which the audio signal is transform coded at a second sampling rate, the first sampling rate being <NUM>/Fth of the second sampling rate, the audio decoder <NUM> comprising the receiver <NUM> which receives, per frame of length N of the audio signal, N spectral coefficients <NUM>, the grabber <NUM> which grabs-out for each frame, a low-frequency fraction of length N/F out of the N spectral coefficients <NUM>, a spectral-to-time modulator <NUM> configured to subject, for each frame <NUM>, the low-frequency fraction to an inverse transform having modulation functions of length <NUM>·N/F temporally extending over the respective frame and a previous frame so as to obtain a temporal portion of length <NUM>·N/F, and a windower <NUM> which windows, for each frame <NUM>, the temporal portion xk,n according to zk,n = ωn · xk,n for n = <NUM>,. ,<NUM>-<NUM> so as to obtain a windowed temporal portion zk,n with with n = <NUM>. <NUM>-<NUM>. The time domain aliasing canceler <NUM> generates intermediate temporal portions mk(<NUM>),. mk(M-<NUM>) according to mk,n = zk,n + zk-<NUM>,n+M for n = <NUM>,. Finally, the lifter <NUM> computes frames uk,n of the audio signal with n = <NUM>. M-<NUM> according to uk,n= mk,n + ln-M/<NUM> · mk-<NUM>,M-<NUM>-n for n = M/<NUM>,. ,M-<NUM>, and uk,n = mk,n + lM-<NUM>-n · outk-<NUM>,M-<NUM>-n for n=<NUM>,. ,M/<NUM>-<NUM>, wherein In with n = <NUM>. M-<NUM> are lifting coefficients, wherein the inverse transform is an inverse MDCT or inverse MDST, and wherein ln with n = <NUM>. M-<NUM> and ωn n = <NUM>,. ,<NUM>-<NUM> depend on coefficients wn with n = <NUM>. (E+<NUM>)M-<NUM> of a synthesis window, and the synthesis window is a downsampled version of a reference synthesis window of length <NUM> · N, downsampled by a factor of F by a segmental interpolation in segments of length <NUM>/<NUM> · N.

It already turned out from the above discussion of a proposal for an extension of AAC-ELD with respect to a downscaled decoding mode that the audio decoder of <FIG> may be accompanied with a low delay SBR tool. The following outlines, for instance, how the AAC-ELD coder extended to support the above-proposed downscaled operating mode, would operate when using the low delay SBR tool. As already mentioned in the introductory portion of the specification of the present application, in case the low delay SBR tool is used in connection with the AAC-ELD coder, the filter banks of the low delay SBR module are downscaled as well. This ensures that the SBR module operates with the same frequency resolution and therefore no more adaptations are required. <FIG> outlines the signal path of the AAC-ELD decoder operating at <NUM>, with frame size of <NUM> samples, in downsampled SBR mode and with a downscaling factor F of <NUM>.

In <FIG>, the bitstream arriving as processed by a sequence of blocks, namely an AAC decoder, an inverse LD-MDCT block, a CLDFB analysis block, an SBR decoder and a CLDFB synthesis block (CLDFB = complex low delay filter bank). The bitstream equals the data stream <NUM> discussed previously with respect to <FIG>, but is additionally accompanied by parametric SBR data assisting the spectral shaping of a spectral replicate of a spectral extension band extending the spectra frequency of the audio signal obtained by the downscaled audio decoding at the output of the inverse low delay MDCT block, the spectral shaping being performed by the SBR decoder. In particular, the AAC decoder retrieves all of the necessary syntax elements by appropriate parsing and entropy decoding. The AAC decoder may partially coincide with the receiver <NUM> of the audio decoder <NUM> which, in <FIG>, is embodied by the inverse low delay MDCT block. In <FIG>, F is exemplarily equal to <NUM>. That is, the inverse low delay MDCT block of <FIG> outputs, as an example for the reconstructed audio signal <NUM> of <FIG>, a <NUM> time signal downsampled at half the rate at which the audio signal was originally coded into the arriving bitstream. The CLDFB analysis block subdivides this <NUM> time signal, i.e. the audio signal obtained by downscaled audio decoding, into N bands, here N = <NUM>, and the SBR decoder computes re-shaping coefficients for these bands, re-shapes the N bands accordingly - controlled via the SBR data in the input bitstream arriving at the input of the AAC decoder, and the CLDFB synthesis block re-transitions from spectral domain to time domain with obtaining, thereby, a high frequency extension signal to be added to the original decoded audio signals output by the inverse low delay MDCT block.

Please note, that the standard operation of SBR utilizes a <NUM> band CLDFB. The interpolation algorithm for the <NUM> band CLDFB window coefficients ci<NUM> is already given in <NUM>. <NUM> in [<NUM>], <MAT> where c<NUM> are the window coefficients of the <NUM> band window given in Table <NUM>. <NUM> in [<NUM>]. This formula can be further generalized to define window coefficients for a lower number of bands B as well <MAT> where F denotes the downscaling factor being F = <NUM>/B. With this definition of the window coefficients, the CLDFB analysis and synthesis filter bank can be completely described as outlined in the above example of section A.

Claim 1:
Method for decoding an audio signal (<NUM>) at a first sampling rate from a data stream (<NUM>) into which the audio signal is transform coded at a second sampling rate, the first sampling rate being <NUM>/Fth of the second sampling rate, the method comprising:
receiving, per frame of length N of the audio signal, N spectral coefficients (<NUM>);
grabbing-out for each frame, a low-frequency fraction of length N/F out of the N spectral coefficients (<NUM>);
performing a spectral-to-time modulation by subjecting, for each frame (<NUM>), the low-frequency fraction to an inverse transform having modulation functions of length (E + <NUM>) · N/F temporally extending over the respective frame and E + <NUM> previous frames so as to obtain a temporal portion of length (E + <NUM>) · N/F;
windowing, for each frame (<NUM>), the temporal portion using a synthesis window of length (E +<NUM> ) · N/F comprising a zero-portion of length <NUM>/<NUM>·N/F at a leading end thereof and having a peak within a temporal interval of the synthesis window, the temporal interval succeeding the zero-portion and having length <NUM>/<NUM> · N/F so that the windower obtains a windowed temporal portion of length (E + <NUM>) · N/F; and
performing a time domain aliasing cancellation by subjecting the windowed temporal portion of the frames to an overlap-add process so that a trailing-end fraction of length (E + <NUM>)/(E + <NUM>) of the windowed temporal portion of a current frame overlaps a leading end of length (E + <NUM>)/(E + <NUM>) of the windowed temporal portion of a preceding frame,
wherein the inverse transform is an inverse MDCT or inverse MDST, and
wherein the synthesis window is a downsampled version of a reference synthesis window of length (E + <NUM>) · N, downsampled by a factor of F by a segmental interpolation in segments of length <NUM>/<NUM> · N,
wherein the synthesis window is a concatenation of cubic spline functions of length <NUM>/<NUM> · N/F,
wherein E = <NUM>, and
wherein entropy decoding is used in order to read the spectral coefficients from the data stream and the method comprises spectrally shaping the spectral coefficients with scale factors provided in the data stream or scale factors derived by linear prediction coefficients conveyed within data stream (<NUM>).