Patent Description:
The present invention relates to coding of audio signals, and in particular to high frequency reconstruction methods including a frequency domain harmonic transposer.

HFR technologies, such as the Spectral Band Replication (SBR) technology, allow to significantly improve the coding efficiency of traditional perceptual audio codecs. In combination with MPEG-<NUM> Advanced Audio Coding (AAC), HFR technologies form very efficient audio codecs, which are already in use within the XM Satellite Radio system and Digital Radio Mondiale, and also standardized within 3GPP, DVD Forum and others. The combination of AAC and SBR is called aacPlus. It is part of the MPEG-<NUM> standard where it is referred to as the High Efficiency AAC Profile (HE-AAC). In general, HFR technology can be combined with any perceptual audio codec in a back and forward compatible way, thus offering the possibility to upgrade already established broadcasting systems like the MPEG Layer-<NUM> used in the Eureka DAB system. HFR transposition methods can also be combined with speech codecs to allow wide band speech at ultra low bit rates.

The basic idea behind HFR is the observation that usually a strong correlation between the characteristics of the high frequency range of a signal and the characteristics of the low frequency range of the same signal is present. Thus, a good approximation for a representation of the original input high frequency range of a signal can be achieved by a signal transposition from the low frequency range to the high frequency range.

This concept of transposition was established in <CIT> which is incorporated by reference, as a method to recreate a high frequency band from a lower frequency band of an audio signal. A substantial saving in bit-rate can be obtained by using this concept in audio coding and/or speech coding. In the following, reference will be made to audio coding, but it should be noted that the described methods and systems are equally applicable to speech coding and in unified speech and audio coding (USAC).

In a HFR based audio coding system, a low bandwidth signal is presented to a core waveform coder for encoding, and higher frequencies are regenerated at the decoder side using transposition of the low bandwidth signal and additional side information, which is typically encoded at very low bit-rates and which describes the target spectral shape. For low bit-rates, where the bandwidth of the core coded signal is narrow, it becomes increasingly important to reproduce or synthesize a high band, i.e. the high frequency range of the audio signal, with perceptually pleasant characteristics.

One of the underlying problems that exist with harmonic HFR methods are the opposing constraints of an intended high frequency resolution in order to get a high quality transposition for stationary sounds, and the time response of the system for transient or percussive sounds. In other words, while the use of a high frequency resolution is beneficial for the transposition of stationary signals, such high frequency resolution typically requires large window sizes which are detrimental when dealing with transient portions of a signal. One approach to deal with this problem may be to adaptively change the windows of the transposer, e.g. by using window-switching, as a function of input signal characteristics. Typically long windows will be used for stationary portions of a signal, in order to achieve high frequency resolution, while short windows will be used for transient portions of the signal, in order to implement a good transient response, i.e. a good temporal resolution, of the transposer. However, this approach has the drawback that signal analysis measures such as transient detection or the like have to be incorporated into the transposition system. Such signal analysis measures often involve a decision step, e.g. a decision on the presence of a transient, which triggers a switching of signal processing. Furthermore, such measures typically affect the reliability of the system and they may introduce signal artifacts when switching the signal processing, e.g. when switching between window sizes.

In order to reach improved audio quality and in order to synthesize the required bandwidth of the high band signal, harmonic HFR methods typically employ several orders of transposition. In order to implement a plurality of transpositions of different transposition order, prior art solutions require a plurality of filter banks either in the analysis stage or the synthesis stage or in both stages. Typically, a different filter bank is required for each different transposition order. Moreover, in situations where the core waveform coder operates at a lower sampling rate than the sampling rate of the final output signal, there is typically an additional need to convert the core signal to the sampling rate of the output signal, and this upsampling of the core signal is usually achieved by adding yet another filter bank. All in all, the computationally complexity increases significantly with an increasing number of different transposition orders.

The present document addresses the aforementioned problems regarding the transient performance of harmonic transposition and regarding the computational complexity. As a result, improved harmonic transposition is achieved at a low additional complexity.

According to an aspect, a system according to independent claim <NUM> is proposed. According to further aspects, methods according to independent claims <NUM> and <NUM>, as well as storage media according to independent claims <NUM> and <NUM> are proposed.

It should be noted that the methods and systems including its preferred embodiments as outlined in the present patent application may be used stand-alone or in combination with the other methods and systems disclosed in this document.

The present invention will now be described by way of illustrative examples, not limiting the scope of the invention, with reference to the accompanying drawings, in which:.

The below-described embodiments are merely illustrative for the principles of the present invention for oversampling in a combined transposer filter bank. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

<FIG> illustrates the operation of a frequency domain (FD) harmonic transposer <NUM>. In a basic form, a Tth order harmonic transposer is a unit that shifts all signal components H(f) of the input signal, i.e. a subband of the signal in the frequency domain, to H(Tf). the frequency component H(f) of the input signal is shifted to a T times higher frequency. In order to implement such transposition in the frequency domain, an analysis filter bank <NUM> transforms the input signal from the time-domain to the frequency domain and outputs complex subbands or subband signals, also referred to as the analysis subbands or analysis subband signals. The analysis filter bank typically comprises an analysis transform, e.g. an FFT, DFT or wavelet transform, and a sliding analysis window. The analysis subband signals are submitted to nonlinear processing <NUM> modifying the phase and/or the amplitude according to the chosen transposition order T. Typically, the nonlinear processing outputs a number of subband signals which is equal to the number of input subband signals, i.e. equal to the number of analysis subband signals. The modified subbands or subband signals, which are also referred to as the synthesis subbands or synthesis subband signals, are fed to a synthesis filter bank <NUM> which transforms the subband signals from the frequency domain into the time domain and outputs the transposed time domain signal. The synthesis filter bank <NUM> typically comprises an inverse transform, e.g. an inverse FFT, inverse DFT or inverse wavelet transform, in combination with a sliding synthesis window.

Typically, each filter bank has a physical frequency resolution Δf measured in Hertz and a physical time stride parameter Δt measured in seconds, wherein the physical frequency resolution Δf is usually associated with the frequency resolution of the transform function and the physical time stride parameter Δt is usually associated with the time interval between succeeding window functions. These two parameters, i.e. the frequency resolution and the time stride, define the discrete-time parameters of the filter bank given the chosen sampling rate. By choosing the physical time stride parameters, i.e. the time stride parameter measured in time units e.g. seconds, of the analysis and synthesis filter banks to be identical, an output signal of the transposer <NUM> may be obtained which has the same sampling rate as the input signal. Furthermore, by omitting the nonlinear processing <NUM> a perfect reconstruction of the input signal at the output may be achieved. This requires a careful design of the analysis and synthesis filter banks. On the other hand, if the output sampling rate is chosen to be different from the input sampling rate, a sampling rate conversion may be obtained. This mode of operation may be necessary in the case where the desired bandwidth of the output signal y is larger than half of sampling rate of the input signal x , i.e. when the desired output bandwidth exceeds the Nyqvist frequency of the input signal.

<FIG> illustrates the operation of a multiple transposer or multiple transposer system <NUM> comprising several harmonic transposers <NUM>-<NUM>,. , <NUM>-P of different orders. The input signal which is to be transposed is passed to a bank of P individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. The individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P perform a harmonic transposition of the input signal as outlined in the context of <FIG>. Typically, each of the individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P performs a harmonic transposition of a different transposition order T. By way of example, transposer <NUM>-<NUM> may perform a transposition of order T=<NUM>, transposer <NUM>-<NUM> may perform a transposition of order T=<NUM>,. , and transposer <NUM>-P may perform a transposition of order T=P. However, in generic terms, any of the transposers <NUM>-<NUM>,. , <NUM>-P may perform a harmonic transposition of an arbitrary transposition order T. The contributions, i.e. the output signals of the individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P may be summed in the combiner <NUM> to yield the combined transposer output.

It should be noted that each transposer <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P requires an analysis and a synthesis filter bank as depicted in <FIG>. Moreover, the usual implementation of the individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P will typically change the sampling rate of the processed input signal by different amounts. By way of example, the sampling rate of the output signal of the transposer <NUM>-P may be T times higher than the sampling rate of the input signal to the transposer <NUM>-P, wherein T is the transposition order applied by the transposer <NUM>-P. This may be due to a bandwidth expansion factor of T used within the transposer <NUM>-P, i.e. due to the use of a synthesis filter bank which has T times more subchannels than the analysis filter bank. By doing this the sampling rate and the Nyqvist frequency is increased by a factor T. As a consequence, the individual time domain signals may need to be resampled in order to allow for a combining of the different output signals in the combiner <NUM>. The resampling of the time domain signals can be carried out on the input side or on the output side of each individual transposer <NUM>-<NUM>, <NUM>-<NUM>,.

<FIG> illustrates an exemplary configuration of a multiple harmonic transposer or multiple transposer system <NUM> performing several orders of transposition and using a common analysis filter bank <NUM>. A starting point for the design of the multiple transposer <NUM> may be to design the individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P of <FIG> such that the analysis filter banks (reference sign <NUM> in <FIG>) of all transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P are identical and can be replaced by a single analysis filter bank <NUM>. As a consequence, the time domain input signal is transformed into a single set of frequency domain subband signals, i.e. a single set of analysis subband signals. These subband signals are submitted to different nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P for different orders of transposition. As outlined above in the context of <FIG> each nonlinear processing unit performs a modification of the phase and/or amplitude of the subband signals and this modification differs for different orders of transposition. Subsequently, the differently modified subband signals or subbands have to be submitted to different synthesis filter banks <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P corresponding to the different nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. As an outcome, P differently transposed time domain output signals are obtained which are summed in the combiner <NUM> to yield the combined transposer output.

It should be noted that if the synthesis filter banks <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P corresponding to the different transposition orders operate at different sampling rates, e.g. by using different degrees of bandwidth expansion, the time domain output signals of the different synthesis filter banks <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P need to be differently resampled in order to align the P output signals to a common time grid, prior to their summation in combiner <NUM>.

<FIG> illustrates an example operation of a multiple harmonic transposer <NUM> using several orders of transposition, while using a common synthesis filter bank <NUM>. The starting point for the design of such a multiple transposer <NUM> may be the design of the individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P of <FIG> such that the synthesis filter banks of all transposers are identical and can be replaced by a single synthesis filter bank <NUM>. It should be noted that in an analogous manner as in the situation shown in <FIG>, the nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P are different for each transposition order. Furthermore, the analysis filter banks <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P are different for the different transposition orders. As such, a set of P analysis filter banks <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P determines P sets of analysis subband signals. These P sets of analysis subband signals are submitted to corresponding nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P to yield P sets of modified subband signals. These P sets of subband signals may be combined in the frequency domain in the combiner <NUM> to yield a combined set of subband signals as an input to the single synthesis filter bank <NUM>. This combination in combiner <NUM> may comprise the feeding of differently processed subband signals into different subband ranges and/or the superposing of contributions of subband signals to overlapping subband ranges. In other words, different analysis subband signals which have been processed with different transposition orders may cover overlapping frequency ranges. By way of example, a second order transposer may transpose the analysis subband [2A,2B] to the subband range [4A,4B]. At the same time, a fourth order transposer may transpose the analysis subband [A,B] to the same subband range [4A,4B]. In such cases, the superposing contributions may be combined, e.g. added and/or averaged, by the combiner <NUM>. The time domain output signal of the multiple transposer <NUM> is obtained from the common synthesis filter bank <NUM>. In a similar manner as outlined above, if the analysis filter banks <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P operate at different sampling rates, the time domain signals input to the different analysis filter banks <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P may need to be resampled in order to align the output signals of the different nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P to the same time grid.

<FIG> illustrates the operation of a multiple harmonic transposer <NUM> using several orders of transposition and comprising a single common analysis filter bank <NUM> and a single common synthesis filter bank <NUM>. In this case, the individual transposers <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P of <FIG> should be designed such that both, the analysis filter banks and the synthesis filter banks of all the P harmonic transposers are identical. If the condition of identical analysis and synthesis filter banks for the different P harmonic transposers is met, then the identical filter banks can be replaced by a single analysis filter bank <NUM> and a single synthesis filter bank <NUM>. The advanced nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P output different contributions to partly overlapping frequency ranges that are combined in the combiner <NUM> to yield a combined input to the respective subbands of the synthesis filter bank <NUM>. Similarly to the multiple harmonic transposer <NUM> depicted in <FIG>, the combination in the combiner <NUM> may comprise the feeding of the different output signals of the plurality of nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P into different subband ranges, and the superposing of multiple contributing outputs to overlapping subband ranges.

As already indicated above, the nonlinear processing <NUM> typically provides a number of subbands at its output which corresponds to the number of subbands at the input. The non-linear processing <NUM> typically modifies the phase and/or the amplitude of the subband or the subband signal according to the underlying transposition order T. By way of example a subband at the input is converted to a subband at the output with T times higher frequency, i.e. a subband at the input to the nonlinear processing <NUM>, i.e. the analysis subband, <MAT> may be transposed to a subband at the output of the nonlinear processing <NUM>, i.e. the synthesis subband, <MAT>, wherein k is a subband index number and Δf if the frequency resolution of the analysis filter bank. In order to allow for the use of common analysis filter banks <NUM> and common synthesis filter banks <NUM>, one or more of the advanced processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P may be configured to provide a number of output subbands which may be different from the number of input subbands.

In the following, the principles of advanced nonlinear processing in the nonlinear processing units <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P will be outlined. For this purpose, it is assumed that.

Furthermore, it is assumed that the filter banks are evenly stacked, i.e. the subband with index zero is centered around the zero frequency, such that the analysis filter bank center frequencies are given by kΔf where the analysis subband index k = <NUM>,. , KA -<NUM> and KA is the number of subbands of the analysis filter bank. The synthesis filter bank center frequencies are given by kQΔf where the synthesis subband index n = <NUM>,. , NS -<NUM> and NS is the number of subbands of the synthesis filter bank.

When performing a conventional transposition of integer order T ≥ <NUM> as shown in <FIG>, the resolution factor Q is selected as Q = T and the nonlinearly processed analysis subband k is mapped into the synthesis subband with the same index n = k. The nonlinear processing <NUM> typically comprises multiplying the phase of a subband or subband signal by the factor T. for each sample of the filter bank subbands one may write <MAT> where θA(k) is the phase of a (complex) sample of the analysis subband k and θS(k) is the phase of a (complex) sample of the synthesis subband k. The magnitude or amplitude of a sample of the subband may be kept unmodified or may be increased or decreased by a constant gain factor. Due to the fact that T is an integer, the operation of equation (<NUM>) is independent of the definition of the phase angle.

In conventional multiple transposers, the resolution factor Q of an analysis/synthesis filter bank is selected to be equal to the transposition order T of the respective transposer, i.e. Q = T. In this case, the frequency resolution of the synthesis filter bank is TΔf and therefore depends on the transposition order T. Consequently, it is necessary to use different filter banks for different transposition orders T either in the analysis or synthesis stage. This is due to the fact that the transposition order T defines the quotient of physical frequency resolutions, i.e. the quotient of the frequency resolution Δf of the analysis filter bank and the frequency resolution TΔf of the synthesis filter bank.

In order to be able to use a common analysis filter bank <NUM> and a common synthesis filter bank <NUM> for a plurality of different transposition orders T, it is proposed to set the frequency resolution of the synthesis filter bank <NUM> to QΔf, i.e. it is proposed to make the frequency resolution of the synthesis filter bank <NUM> independent of the transposition order T. Then the question arises of how to implement a transposition of order T when the resolution factor Q, i.e. the quotient Q of the physical frequency resolution of the analysis and synthesis filter bank, does not necessarily obey the relation Q = T.

As outlined above, a principle of harmonic transposition is that the input to the synthesis filter bank subband n with center frequency nQΔf is determined from an analysis subband at a T times lower center frequency, i.e. at the center frequency nQΔf /T. The center frequencies of the analysis subbands are identified through the analysis subband index k as kΔf. Both expressions for the center frequency of the analysis subband index, i.e. nQΔf / T and kΔf, may be set equal. Taking into account that the index n is an integer value, the expression <MAT> is a rational number which can be expressed as the sum of an integer analysis subband index k and a remainder r ∈ {<NUM>,<NUM> / T,<NUM> / T,. , (T - <NUM>) / T} such that <MAT>.

As such, it may be stipulated that the input to a synthesis subband with synthesis subband index n may be derived, using a transposition of order T, from the analysis subband with the index k given by equation (<NUM>). In view of the fact that <MAT> is a rational number, the remainder r may be unequal to <NUM> and the value k+r may be greater than the analysis subband index k and smaller than the analysis subband index k+<NUM>, i.e. k ≤ k + r ≤ k + <NUM>. Consequently, the input to a synthesis subband with synthesis subband index n should be derived, using a transposition of order T, from the analysis subbands with the analysis subband index k and k+<NUM>, wherein k is given by equation (<NUM>). In other words, the input of a synthesis subband may be derived from two consecutive analysis subbands.

As an outcome of the above, the advanced nonlinear processing performed in a nonlinear processing unit <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P may comprise the step of considering two neighboring analysis subbands with index k and k +<NUM> in order to provide the output for synthesis subband n. For a transposition order T, the phase modification performed by the nonlinear processing unit <NUM>-<NUM>, <NUM>-<NUM>,. , <NUM>-P may for example be defined by the linear interpolation rule, <MAT> where θA(k) is the phase of a sample of the analysis subband k, θA(k + <NUM>) is the phase of a sample of the analysis subband k+<NUM>, and θS(n) is the phase of a sample of the synthesis subband n. If the remainder r is close to zero, i.e. if the value k+r is close to k, then the main contribution of the phase of the synthesis subband sample is derived from the phase of the analysis subband sample of subband k. On the other hand, if the remainder r is close to one, i.e. if the value k+r is close to k+<NUM>, then the main contribution of the phase of the synthesis subband sample is derived from the phase of the analysis subband sample of subband k+<NUM>. It should be noted that the phase multipliers T(<NUM> - r) and Tr are both integers such that the phase modifications of equation (<NUM>) are well defined and independent of the definition of the phase angle.

Concerning the magnitudes of the subband samples, the following geometrical mean value may be selected for the determination of the magnitude of the synthesis subband samples, <MAT> where as(n) denotes the magnitude of a sample of the synthesis subband n, aA(k) denotes the magnitude of a sample of the analysis subband k and aA(k + <NUM>) denotes the magnitude of a sample of the analysis subband k+<NUM>. It should be noted that other interpolation rules for the phase and/or the magnitude may be contemplated.

For the case of an oddly stacked filter bank where the analysis filter bank center frequencies are given by <MAT> with k =<NUM>,. , KA -<NUM> and the synthesis filter bank center frequencies are given by <MAT> with n = <NUM>,. , NS -<NUM>, an corresponding equation to equation (<NUM>) may be derived by equating the transposed synthesis filter bank center frequency <MAT> and the analysis filter bank center frequency <MAT>. Assuming an integer index k and a remainder r ∈ [<NUM>,<NUM>[ the following equation for oddly stacked filter banks can be derived: <MAT>.

The skilled person will appreciate that if T - Q, i.e. the difference between the transposition order and the resolution factor, is even, T(<NUM> - r) and Tr are both integers and the interpolation rules of equations (<NUM>) and (<NUM>) can be used.

The mapping of analysis subbands into synthesis subbands is illustrated in <FIG> shows four diagrams for different transposition orders T=<NUM> to T=<NUM>. Each diagram illustrates how the source bins <NUM>, i.e. the analysis subbands, are mapped into target bins <NUM>, i.e. synthesis subbands. For ease of illustration, it is assumed that the resolution factor Q is equal to one. In other words, <FIG> illustrates the mapping of analysis subband signals to synthesis subband signals using Eq.(<NUM>) and (<NUM>). In the illustrated example the analysis/synthesis filter bank is evenly stacked, with Q = <NUM> and the maximum transposition order T = <NUM>.

In the illustrated case, equation (<NUM>) may be written as <MAT>. Consequently, for a transposition order T=<NUM>, an analysis subband with an index k is mapped to a corresponding synthesis subband n and the remainder r is always zero. This can be seen in <FIG> where for example source bin <NUM> is mapped one to one to a target bin <NUM>.

In case of transposition order T=<NUM>, the remainder r takes on the values <NUM> and ½ and a source bin is mapped to a plurality of target bins. When reversing the perspective, it may be stated that each target bin <NUM>, <NUM> receives a contribution from up to two source bins. This can be seen in <FIG>, where the target bin <NUM> receives a contribution from source bins <NUM> and <NUM>. However, the target bin <NUM> receives a contribution from source bin <NUM> only. If it is assumed that target bin <NUM> has an even index n, e.g. n=<NUM>, then equation (<NUM>) specifies that target bin <NUM> receives a contribution from the source bin <NUM> with an index k=n/<NUM>, e.g. k=<NUM>. The remainder r is zero, i.e. there is no contribution from the source bin <NUM> with index k+<NUM>, e.g. k+<NUM>=<NUM>. This changes for target bin <NUM> with an uneven index n, e.g. n=<NUM>. In this case, equation (<NUM>) specifies that target bin <NUM> receives contributions from the source bin <NUM> (index k=<NUM>) and source bin <NUM> (index k+<NUM>=<NUM>). This applies in a similar manner to higher transposition orders T, e.g. T=<NUM> and T=<NUM>, as shown in <FIG>.

A further interpretation of the above advanced nonlinear processing may be as follows. The advanced nonlinear processing may be understood as a combination of a transposition of a given order T into intermediate subband signals on an intermediate frequency grid TΔf , and a subsequent mapping of the intermediate subband signals to a frequency grid defined by a common synthesis filter bank, i.e. by a frequency grid QΔf. In order to illustrate this interpretation, reference is made again to <FIG>. However, for this illustration, the source bins <NUM> are considered to be intermediate subbands derived from the analysis subbands using an order of transposition T. These intermediate subbands have a frequency grid given by TΔf. In order to generate synthesis subband signals on a pre-defined frequency grid QΔf given by the target bins <NUM>, the source bins <NUM>, i.e. the intermediate subbands having the frequency grid TΔf, need to be mapped onto the pre-defined frequency grid QΔf. This can be performed by determining a target bin <NUM>, i.e. a synthesis subband signal on the frequency grid QΔf, by interpolating one or two source bins <NUM>, i.e. intermediate subband signals on the frequency grid TΔf. In a preferred embodiment, linear interpolation is used, wherein the weights of the interpolation are inversely proportional to the difference between the center frequency of the target bin <NUM> and the corresponding source bin <NUM>. By way of example, if the difference is zero, then the weight is <NUM>, and if the difference is TΔf then the weight is <NUM>.

In summary, a nonlinear processing method has been described which allows the determination of contributions to a synthesis subband by means of transposition of several analysis subbands. The nonlinear processing method enables the use of single common analysis and synthesis subband filter banks for different transposition orders, thereby significantly reducing the computational complexity of multiple harmonic transposers.

<FIG> illustrate example analysis / synthesis filter banks using a M = <NUM> point FFT/DFT (Fast Fourier Transform or Discrete Fourier Transform) for multiple transposition orders of T = <NUM>,<NUM>,<NUM>. <FIG> illustrates the conventional case of a multiple harmonic transposer <NUM> using a common analysis filter bank <NUM> and separate synthesis filter banks <NUM>, <NUM>, <NUM> for each transposition factor T = <NUM>,<NUM>,<NUM>. <FIG> shows the analysis windows vA <NUM> and the synthesis windows vS <NUM>, <NUM>, <NUM> applied at the analysis filter bank <NUM> and the synthesis filter banks <NUM>, <NUM>, <NUM>, respectively. In the illustrated example, the analysis window vA <NUM> has a length LA = <NUM> which is equal to the size M of the FFT or DFT of the analysis/synthesis filter banks <NUM>, <NUM>, <NUM>, <NUM>. In a similar manner, the synthesis windows vS <NUM>, <NUM>, <NUM> have a length of LS = <NUM> which is equal to the size M of the FFT or DFT.

<FIG> also illustrates the hop size ΔsA employed by the analysis filter bank <NUM> and the hop size ΔsS employed the synthesis filter banks <NUM>, <NUM>, <NUM>, respectively. The hop size Δs corresponds to the number of data samples by which the respective window <NUM>, <NUM>, <NUM>, <NUM> is moved between successive transformation steps. The hop size Δs relates to the physical time stride Δt via the sampling rate of the underlying signal, i.e. Δs = fsΔt, wherein fs is the sampling rate.

It can be seen that the analysis window <NUM> is moved by a hop size <NUM> of <NUM> samples. The synthesis window <NUM> corresponding to a transposition of order T=<NUM> is moved by a hop size <NUM> of <NUM> samples, i.e. a hop size <NUM> which is twice the hop size <NUM> of the analysis window <NUM>. As outlined above, this leads to a time stretch of the signal by the factor T=<NUM>. Alternatively, if a T=<NUM> times higher sampling rate is assumed, the difference between the analysis hop size <NUM> and the synthesis hop size <NUM> leads to a harmonic transposition of order T=<NUM>. a time stretch by an order T may be converted into a harmonic transposition by performing a sampling rate conversion of order T.

In a similar manner, it can be seen that the synthesis hop size <NUM> associated with the harmonic transposer of order T=<NUM> is T=<NUM> times higher than the analysis hop size <NUM>, and the synthesis hop size <NUM> associated with the harmonic transposer of order T=<NUM> is T=<NUM> times higher than the analysis hop size <NUM>. In order to align the sampling rates of the <NUM>rd order transposer and the <NUM>th order transposer with the output sampling rate of the <NUM>nd order transposer, the <NUM>rd order transposer and the <NUM>th order transposer comprise a factor <NUM>/<NUM> - downsampler <NUM> and a factor <NUM> - downsampler <NUM>, respectively. In general terms, the Tth order transposer would comprise a factor T/<NUM> - downsampler, if an output sampling rate is requested, which is <NUM> times higher than the input sampling rate. no downsampling is required for the harmonic transposer of order T=<NUM>.

Finally, <FIG> illustrates the separate phase modification units <NUM>, <NUM>, <NUM> for the transposition order T=<NUM>, <NUM>, <NUM>, respectively. These phase modification units <NUM>, <NUM>, <NUM> perform a multiplication of the phase of the respective subband signals by the transposition order T=<NUM>, <NUM>, <NUM>, respectively (see Equation (<NUM>)).

An efficient combined filter bank structure for the transposer can be obtained by limiting the multiple transposer of <FIG> to a single analysis filter bank <NUM> and a single synthesis filter bank <NUM>. The <NUM>rd and <NUM>th order harmonics are then produced in a non-linear processing unit <NUM> within a <NUM>nd order filter bank as depicted in <FIG> shows an analysis filter bank comprising a <NUM> point forward FFT unit <NUM> and an analysis window <NUM> which is applied on the input signal x with an analysis hop size <NUM>. The synthesis filter bank comprises a <NUM> point inverse FFT unit <NUM> and a synthesis window <NUM> which is applied with a synthesis hop size <NUM>. In the illustrated example the synthesis hop size <NUM> is twice the analysis hop size <NUM>. Furthermore, the sampling rate of the output signal y is assumed to be twice the sampling rate of the input signal x.

The analysis / synthesis filter bank of <FIG> comprises a single analysis filter bank and a single synthesis filter bank. By using advanced nonlinear processing <NUM> in accordance to the methods outlined in the context of <FIG> and <FIG>, i.e. the advanced non-linear processing performed in the units <NUM>-<NUM>,. ,<NUM>-P, this analysis / synthesis filter bank may be used to provide a multiple transposer, i.e. a harmonic transposer for a plurality of transposition orders T.

As has been outlined in the context of <FIG> and <FIG>, the one-to-one mapping of analysis subbands to corresponding synthesis subbands involving a multiplication of the phase of the subband signals by the respective transposition order T, may be generalized to interpolation rules (see Equations (<NUM>) and (<NUM>)) involving one or more subband signals. It has been outlined that if the physical spacing QΔf of the synthesis filter bank subbands is Q times the physical spacing Δf of the analysis filter bank, the input to the synthesis band with index n is obtained from the analysis bands with indices k and k + <NUM>. The relationship between the indexes n and k is given by Equation (<NUM>) or (<NUM>), depending on whether the filter banks are evenly or unevenly stacked. A geometrical interpolation for the magnitudes is applied with powers <NUM>- r and r (Equation (<NUM>)) and the phases are linearly combined with weights T(<NUM> - r) and Tr (Equation (<NUM>)). For the illustrated case where Q = <NUM>, the phase mappings for each transposition factor are illustrated graphically in <FIG>.

In a similar manner to the case of Q=<NUM> illustrated in <FIG>, a target subband or target bin <NUM> receives contributions from up to two source subbands or source bins <NUM>. In the case T=Q=<NUM>, each phase modified source bin <NUM> is assigned to a corresponding target bin <NUM>. For higher transposition orders T>Q, a target bin <NUM> may be obtained from one corresponding phase modified source bin <NUM>. This is the case if the remainder r obtained from Equation (<NUM>) or (<NUM>) is zero. Otherwise, a target bin <NUM> is obtained by interpolating two phase modified source bins <NUM> and <NUM>.

The above mentioned non-linear processing is performed in the multiple transposer unit <NUM> which determines target bins <NUM> for the different orders of transposition T=<NUM>, <NUM>, <NUM> using advanced non-linear processing units <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>. Subsequently, corresponding target bins <NUM> are combined in a combiner unit <NUM> to yield a single set of synthesis subband signals which are fed to the synthesis filter bank. As outlined above, the combiner unit <NUM> is configured to combine a plurality of contributions in overlapping frequency ranges from the output of the different non-linear processing units <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>.

In the following, the harmonic transposition of transient signals using harmonic transposers is outlined. In this context, it should be noted that harmonic transposition of order T using analysis/synthesis filter banks may be interpreted as time stretching of an underlying signal by an integer transposition factor T followed by a downsampling and/or sample rate conversion. The time stretching is performed such that frequencies of sinusoids which compose the input signal are maintained. Such time stretching may be performed using the analysis / synthesis filter bank in combination with intermediate modification of the phases of the subband signals based on the transposition order T. As outlined above, the analysis filter bank may be a windowed DFT filter bank with analysis window vA and the synthesis filter bank may be a windowed inverse DFT filter bank with synthesis window vs. Such analysis/synthesis transform is also referred to as short-time Fourier Transform (STFT).

A short-time Fourier transform is performed on a time-domain input signal x to obtain a succession of overlapped spectral frames. In order to minimize possible side-band effects, appropriate analysis/synthesis windows, e.g. Gaussian windows, cosine windows, Hamming windows, Hann windows, rectangular windows, Bartlett windows, Blackman windows, and others, should be selected. The time delay at which every spectral frame is picked up from the input signal x is referred to as the hop size Δs or physical time stride Δt. The STFT of the input signal x is referred to as the analysis stage and leads to a frequency domain representation of the input signal x. The frequency domain representation comprises a plurality of subband signals, wherein each subband signal represents a certain frequency component of the input signal.

For the purpose of time-stretching of the input signal, each subband signal may be time-stretched, e.g. by delaying the subband signal samples. This may be achieved by using a synthesis hop-size which is greater than the analysis hop-size. The time domain signal may be rebuilt by performing an inverse (Fast) Fourier transform on all frames followed by a successive accumulation of the frames. This operation of the synthesis stage is referred to as overlap-add operation. The resulting output signal is a time-stretched version of the input signal comprising the same frequency components as the input signal. In other words, the resulting output signal has the same spectral composition as the input signal, but it is slower than the input signal i.e. its progression is stretched in time.

The transposition to higher frequencies may then be obtained subsequently, or in an integrated manner, through downsampling of the stretched signals or by performing a sample-rate conversion of the time stretched output signal. As a result the transposed signal has the length in time of the initial signal, but comprises frequency components which are shifted upwards by a pre-defined transposition factor.

In view of the above, the harmonic transposition of transient signals using harmonic transposers is described by considering as a starting point the time stretching of a prototype transient signal, i.e. a discrete time Dirac pulse at time instant t = t<NUM>, <MAT>.

The Fourier transform of such a Dirac pulse has unit magnitude and a linear phase with a slope proportional to t<NUM> : <MAT> wherein <MAT> is the center frequency of the mth subband signal of the STFT analysis and M is the size of the discrete Fourier transform (DFT). Such Fourier transform can be considered as the analysis stage of the analysis filter bank described above, wherein a flat analysis window vA of infinite duration is used. In order to generate an output signal y which is time-stretched by a factor T, i.e. a Dirac pulse δ(t - Tt<NUM>) at the time instant t = Tt<NUM>, the phase of the analysis subband signals should be multiplied by the factor T in order to obtain the synthesis subband signal Y(Ωm) = exp(-jΩmTt<NUM>) which yields the desired Dirac pulse δ(t - Tt<NUM>) as an output of an inverse Fourier Transform.

However, it should be noted that the above considerations refer to an analysis/synthesis stage using analysis and synthesis windows of infinite lengths. Indeed, a theoretical transposer with a window of infinite duration would give the correct stretch of a Dirac pulse δ(t-t<NUM>). For a finite duration windowed analysis, the situation is scrambled by the fact that each analysis block is to be interpreted as one period interval of a periodic signal with a period equal to the size of the DFT.

This is illustrated in <FIG> which shows the analysis and synthesis <NUM> of a Dirac pulse δ(t-t<NUM>). The upper part of <FIG> shows the input to the analysis stage <NUM> and the lower part of <FIG> shows the output of the synthesis stage <NUM>. The upper and lower graphs represent the time domain. The stylized analysis window <NUM> and synthesis window <NUM> are depicted as triangular (Bartlett) windows. The input pulse δ(t - t<NUM>)<NUM> at time instant t = t<NUM> is depicted on the top graph <NUM> as a vertical arrow. It is assumed that the DFT transform block is of size M = L = LA = LS, i.e. the size of the DFT transform is chosen to be equal to the size of the windows. The phase multiplication of the subband signals by the factor T will produce the DFT analysis of a Dirac pulse δ(t - Tt<NUM>) at t = Tt<NUM>, however, of a Dirac pulse periodized to a Dirac pulse train with period L. This is due to the finite length of the applied window and Fourier Transform. The periodized pulse train with period L is depicted by the dashed arrows <NUM>, <NUM> on the lower graph.

In a real-world system, the pulse train actually contains a few pulses only (depending on the transposition factor), one main pulse, i.e. the wanted term, a few pre-pulses and a few post-pulses, i.e. the unwanted terms. The pre- and post-pulses emerge because the DFT is periodic (with L). When a pulse is located within an analysis window, so that the complex phase gets wrapped when multiplied by T (i.e. the pulse is shifted outside the end of the window and wraps back to the beginning), an unwanted pulse emerges within the synthesis window. The unwanted pulses may have, or may not have, the same polarity as the input pulse, depending on the location in the analysis window and the transposition factor.

In the example of <FIG>, the synthesis windowing uses a finite window vS <NUM>. The finite synthesis window <NUM> picks the desired pulse δ(t - Tt<NUM>) at t = Tt<NUM> which is depicted as a solid arrow <NUM> and cancels the other unwanted contributions which are shown as dashed arrows <NUM>, <NUM>.

As the analysis and synthesis stage move along the time axis according to the hop factor Δs or the time stride Δt, the pulse δ(t - t<NUM>) <NUM> will have another position relative to the center of the respective analysis window <NUM>. As outlined above, the operation to achieve time-stretching consists in moving the pulse <NUM> to T times its position relative to the center of the window. As long as this position is within the window <NUM>, this time-stretch operation guarantees that all contributions add up to a single time stretched synthesized pulse δ(t - Tt<NUM>) at <MAT>.

However, a problem occurs for the situation of <FIG>, where the pulse δ(t - t<NUM>) <NUM> moves further out towards the edge of the DFT block. <FIG> illustrates a similar analysis/synthesis configuration <NUM> as <FIG>. The upper graph <NUM> shows the input to the analysis stage and the analysis window <NUM>, and the lower graph <NUM> illustrates the output of the synthesis stage and the synthesis window <NUM>. When time-stretching the input Dirac pulse <NUM> by a factor T, the time stretched Dirac pulse <NUM>, i.e. δ(t - Tt<NUM>), comes to lie outside the synthesis window <NUM>. At the same time, another Dirac pulse <NUM> of the pulse train, i.e. δ(t - Tt<NUM> + L) at time instant t = Tt<NUM> - L, is picked up by the synthesis window. In other words, the input Dirac pulse <NUM> is not delayed to a T times later time instant, but it is moved forward to a time instant that lies before the input Dirac pulse <NUM>. The final effect on the audio signal is the occurrence of a pre-echo at a time distance of the scale of the rather long transposer windows, i.e. at a time instant t = Tt<NUM> - L which is L - (T -<NUM>)t<NUM> earlier than the input Dirac pulse <NUM>.

The principle of the solution to this problem is described in reference to <FIG> illustrates an analysis/synthesis scenario <NUM> similar to <FIG>. The upper graph <NUM> shows the input to the analysis stage with the analysis window <NUM>, and the lower graph <NUM> shows the output of the synthesis stage with the synthesis window <NUM>. The DFT size is adapted so as to avoid pre-echoes. This may be achieved by setting the size M of the DFT such that no unwanted Dirac pulse images from the resulting pulse train are picked up by the synthesis window. The size of the DFT transform <NUM> is increased to M = FL, where L is the length of the window function <NUM> and the factor F is a frequency domain oversampling factor. In other words, the size of the DFT transform <NUM> is selected to be larger than the window size <NUM>. In particular, the size of the DFT transform <NUM> may be selected to be larger than the window size <NUM> of the synthesis window. Due to the increased length <NUM> of the DFT transform, the period of the pulse train comprising the Dirac pulses <NUM>, <NUM> is FL. By selecting a sufficiently large value of F, i.e. by selecting a sufficiently large frequency domain oversampling factor, undesired contributions to the pulse stretch can be cancelled. This is shown in <FIG>, where the Dirac pulse <NUM> at time instant t = Tt<NUM> - FL lies outside the synthesis window <NUM>. Therefore, the Dirac pulse <NUM> is not picked up by the synthesis window <NUM> and by consequence, pre-echoes can be avoided.

It should be noted that in a preferred embodiment the synthesis window and the analysis window have equal "nominal" lengths (measured in the number of samples). However, when using implicit resampling of the output signal by discarding or inserting samples in the frequency bands of the transform or filter bank, the synthesis window size (measured in the number of samples) will typically be different from the analysis size, depending on the resampling and/or transposition factor.

The minimum value of F, i.e. the minimum frequency domain oversampling factor, can be deduced from <FIG>. The condition for not picking up undesired Dirac pulse images may be formulated as follows: For any input pulse δ(t - t<NUM>) at position <MAT>, i.e. for any input pulse comprised within the analysis window <NUM>, the undesired image δ(t - Tt<NUM> + FL) at time instant t = Tt<NUM> - FL must be located to the left of the left edge of the synthesis window at <MAT>. In an equivalent manner, the condition <MAT> must be met, which leads to the rule <MAT>.

As can be seen from formula (<NUM>), the minimum frequency domain oversampling factor F is a function of the transposition order T. More specifically, the minimum frequency domain oversampling factor F is proportional to the transposition order T.

By repeating the line of thinking above for the case where the analysis and synthesis windows have different lengths one obtains a more general formula. Let LA and Ls be the lengths of the analysis and synthesis windows (measured in the number of samples), respectively, and let M be the DFT size employed. The general rule extending formula (<NUM>) is then <MAT>.

That this rule indeed is an extension of (<NUM>) can be verified by inserting M = FL, and LA = LS = L in (<NUM>) and dividing by L on both side of the resulting equation.

The above analysis is performed for a rather special model of a transient, i.e. a Dirac pulse. However, the reasoning can be extended to show that when using the above described time-stretching and/or harmonic transposition scheme, input signals which have a near flat spectral envelope and which vanish outside a time interval [a, b] will be stretched to output signals which are small outside the interval [Ta, Tb]. It can also be verified, by studying spectrograms of real audio and/or speech signals, that pre-echoes disappear in the stretched or transposed signals when the above described rule for selecting an appropriate frequency domain oversampling factor is respected. A more quantitative analysis also reveals that pre-echoes are still reduced when using frequency domain oversampling factors which are slightly inferior to the value imposed by the condition of formula (<NUM>) or (<NUM>). This is due to the fact that typical window functions vS are small near their edges, thereby attenuating undesired pre-echoes which are positioned near the edges of the window functions.

In summary, a way to improve the transient response of frequency domain harmonic transposers, or time-stretchers, has been described by introducing an oversampled transform, where the amount of oversampling is a function of the transposition factor chosen. The improved transient response of the transposer is obtained by means of frequency domain oversampling.

In the multiple transposer of <FIG>, frequency domain oversampling may be implemented by using DFT kernels <NUM>, <NUM>, <NUM>, <NUM> of length <NUM>F and by zero padding the analysis and synthesis windows symmetrically to that length. It should be noted that for complexity reasons, it is beneficial to keep the amount of oversampling low. If formula (<NUM>) is applied to the multiple transposer of <FIG>, an oversampling factor F=<NUM> should be applied to cover all the transposition factors T=<NUM>, <NUM>, <NUM>. However, it can be shown that the use of F=<NUM> already leads to a significant quality improvement for real audio signals.

In the following, the use of frequency domain oversampling in the context of combined analysis / synthesis filter banks, such as described in the context of <FIG> or <FIG>, is described.

In general, for a combined transposition filter bank where the physical spacing QΔf of the synthesis filter bank subbands is Q times the physical spacing Δf of the analysis filter bank and where the physical analysis window duration DA (measured in units of time, e.g. seconds) is also Q times that of the synthesis filter bank, DA = QDS, the analysis for a Dirac pulse as above will apply for all transposition factors T = Q, Q + <NUM>, Q + <NUM>,. as if T = Q. In other words, the rule for the degree of frequency domain oversampling required in a combined transposition filter bank is given by <MAT>.

In particular, it should be noted that for T > Q, the frequency domain oversampling factor <MAT> is sufficient, while still ensuring the suppression of artifacts on transient signals caused by harmonic transposition of order T. using the above oversampling rules for the combined filter bank, it can be seen that even when using higher transposition orders T>Q, it is not required to further increase the oversampling factor F. As indicated by equation (6b), it is sufficient in the combined filter bank implementation of <FIG> to use an oversampling factor F=<NUM> in order to avoid the occurrence of pre-echoes. This value is lower than the oversampling factor F=<NUM> required for the multiple transposer of <FIG>. Consequently, the complexity of performing frequency domain oversampling in order to improve the transient performance of multiple harmonic transposers can be further reduced when using a combined analysis/synthesis filter bank (instead of separate analysis and/or synthesis filter banks for the different transposition orders).

In a more general scenario, the physical time durations of the analysis and synthesis windows DA and DS, respectively, may be arbitrarily selected. Then the physical spacing Δf of the analysis filter bank subbands should satisfy <MAT>.

in order to avoid the described artifacts caused by harmonic transposition. It should be noted that the duration of a window D typically differs from the length of a window L. Whereas the length of a window L corresponds to the number of signal samples covered by the window, the duration of the window D corresponds to the time interval of the signal covered by the window. As illustrated in <FIG>, the windows <NUM>, <NUM>, <NUM>, <NUM> have an equal length of L = <NUM> samples. However, the duration DA of the analysis window <NUM> is T times the duration DS of the synthesis window <NUM>, <NUM>, <NUM>, wherein T is the respective transposition order and the resolution factor of the respective synthesis filter bank. In a similar manner, the duration DA of the analysis window <NUM> in <FIG> is Q times the duration DS of the synthesis window <NUM>, wherein Q is the resolution factor of the synthesis filter bank. The duration of a window D is related to the length of the window L via the sampling frequency fs, i.e. notably <MAT>. In a similar manner, the frequency resolution of a transform Δf is related to the number of points or length M of the transform via the sampling frequency fs, i.e. notably <MAT>. Furthermore, the physical time stride Δt of a filter bank is related to the hop size Δs of the filter bank via the sampling frequency fs, i.e. notably <MAT>.

Using the above relations, equation (6b) may be written as <MAT> i.e. the product of the frequency resolution and the window length of the analysis filter bank and/or the frequency resolution and the window length of the synthesis filter bank should be selected to be smaller or equal to <MAT>. For T > Q, the product ΔfDA and/or QΔfDs may be selected to be greater than <MAT>, thereby reducing the computational complexity of the filter banks.

In the present document, various methods for performing harmonic transposition of signals, preferably audio and/or speech signals, have been described. Particular emphasis has been put on the computational complexity of multiple harmonic transposers. In this context, a multiple transposer has been described, which is configured to perform multiple orders of transposition using a combined analysis/synthesis filter bank, i.e. a filter bank comprising a single analysis filter bank and a single synthesis filter bank. A multiple tranposer using a combined analysis/synthesis filter bank has reduced computational complexity compared to a conventional multiple transposer. Furthermore, frequency domain oversampling has been described in the context of combined analysis/synthesis filter banks. Frequency domain oversampling may be used to reduce or remove artifacts caused on transient signals by harmonic transposition. It has been shown that frequency domain oversampling can be implemented at reduced computational complexity within combined analysis/synthesis filter banks, compared to conventional multiple transposer implementations.

While specific embodiments of the present invention and applications of the invention have been described herein, it will be apparent to those of ordinary skill in the art that many variations on the embodiments and applications described herein are possible.

Claim 1:
A system for generating an output signal comprising a high frequency component from an input audio signal comprising a low frequency component using a transposition order T, comprising:
an analysis window unit configured to apply an analysis window (<NUM>) of a length of LA samples, thereby extracting a frame of the input signal;
an analysis transformation unit of order M (<NUM>) and having a frequency resolution Δf configured to transform the LA samples into M complex coefficients;
a nonlinear processing unit (<NUM>,<NUM>, <NUM>), configured to alter the phase of the complex coefficients by using the transposition order T;
a synthesis transformation unit (<NUM>) of order M and having a frequency resolution QΔf, configured to transform the altered coefficients into M time domain samples; wherein Q is a frequency resolution factor smaller than or equal to the transposition order T; and
a synthesis window unit configured to apply a synthesis window (<NUM>) of a length of Ls samples to the M time domain samples, thereby generating a frame of the output signal;
wherein LA is equal to LS and M is greater than or equal to (QLA+Ls)/<NUM>.