Source: http://www.google.com/patents/US7917369?dq=5,884,271
Timestamp: 2016-12-08 08:01:12
Document Index: 750806011

Matched Legal Cases: ['art 3', 'art 3', 'art 3', 'art 7', 'art 7', 'art 7', 'art 7', 'art 3', 'art 3', 'art 7', 'art 7', 'Application No. 03', 'Application No. 03']

Patent US7917369 - Quality improvement techniques in an audio encoder - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsAn audio encoder implements multi-channel coding decision, band truncation, multi-channel rematrixing, and header reduction techniques to improve quality and coding efficiency. In the multi-channel coding decision technique, the audio encoder dynamically selects between joint and independent coding of...http://www.google.com/patents/US7917369?utm_source=gb-gplus-sharePatent US7917369 - Quality improvement techniques in an audio encoderAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS7917369 B2Publication typeGrantApplication numberUS 11/737,072Publication dateMar 29, 2011Filing dateApr 18, 2007Priority dateDec 14, 2001Fee statusPaidAlso published asUS7240001, US8554569, US8805696, US9443525, US20030115041, US20070185706, US20090326962, US20140039884, US20140316788Publication number11737072, 737072, US 7917369 B2, US 7917369B2, US-B2-7917369, US7917369 B2, US7917369B2InventorsWei-ge Chen, Naveen Thumpudi, Ming-Chieh LeeOriginal AssigneeMicrosoft CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (131), Non-Patent Citations (122), Referenced by (9), Classifications (13), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetQuality improvement techniques in an audio encoder
US 7917369 B2Abstract
An audio encoder implements multi-channel coding decision, band truncation, multi-channel rematrixing, and header reduction techniques to improve quality and coding efficiency. In the multi-channel coding decision technique, the audio encoder dynamically selects between joint and independent coding of a multi-channel audio signal via an open-loop decision based upon (a) energy separation between the coding channels, and (b) the disparity between excitation patterns of the separate input channels. In the band truncation technique, the audio encoder performs open-loop band truncation at a cut-off frequency based on a target perceptual quality measure. In multi-channel rematrixing technique, the audio encoder suppresses certain coefficients of a difference channel by scaling according to a scale factor, which is based on current average levels of perceptual quality, current rate control buffer fullness, coding mode, and the amount of channel separation in the source. In the header reduction technique, the audio encoder selectively modifies the quantization step size of zeroed quantization bands so as to encode in fewer frame header bits.
1. In a transform-based audio encoder, a method of improved band truncation, the method comprising:
performing a frequency transform on each of multiple input channels of a multi-channel input audio signal to produce multiple input channels of transform domain coefficients;
performing a multi-channel transformation on the multiple input channels of transform domain coefficients for the multi-channel input audio signal to produce a plurality of joint coding channels;
on a channel-by-channel basis, selectively suppressing at least one of the joint coding channels as a function of at least quality of reproduction, rate control buffer fullness, and channel separation, wherein the selectively suppressing comprises scaling the at least one joint coding channel by a scaling factor having a value varying based on a current average level of quality, current rate control buffer fullness and amount of channel separation; and
for at least one of the joint coding channels:
selecting as an open-loop process a first portion of the transform domain coefficients for band truncation as a function of a target quality measurement;
suppressing the selected first portion of the transform domain coefficients from encoding in a compressed audio data stream;
measuring an achieved quality measurement of the input audio signal encoded with the selected first portion of the transform domain coefficients suppressed;
selecting as a closed-loop process a second portion of the transform domain coefficients for second band truncation as a function of the achieved quality measurement; and
suppressing the selected second portion of the transform domain coefficients from encoding in a second compressed audio data stream.
2. The method of claim 1 wherein the target quality measurement is a target noise-to-excitation ratio for the input audio signal.
3. In a transform-based audio encoder, a method of encoding a multi-channel audio input signal, the method comprising:
performing a frequency transform on each of multiple input channels of the multi-channel audio input signal to produce multiple input channels of frequency coefficient data;
performing a multi-channel transformation on the multiple input channels of frequency coefficient data for the multi-channel audio input signal to produce a plurality of joint coding channels;
encoding the frequency coefficient data for the multi-channel audio input signal with said selective suppression of said at least one joint coding channel.
4. The method of claim 3 further comprising measuring the current average level of quality as a noise-to-excitation ratio for a portion of the multi-channel audio input signal.
5. The method of claim 3 wherein the selectively suppressing the at least one joint coding channel is also a function of a rate setting of the transform-based audio encoder.
This application is a continuation of U.S. patent application Ser. No. 10/016,918, entitled, “QUALITY IMPROVEMENT TECHNIQUES IN AN AUDIO ENCODER,” filed Dec. 14, 2001 now U.S. Pat. No. 7,240,001, the disclosure of which is hereby incorporated by reference in its entirety. The following U.S. patent applications relate to the present application: U.S. patent application Ser. No. 10/017,694, entitled, “QUALITY AND RATE CONTROL STRATEGY FOR DIGITAL AUDIO,” filed Dec. 14, 2001, now U.S. Pat. No. 7,027,982, the disclosure of which is hereby incorporated by reference; U.S. patent application Ser. No. 10/017,861, entitled, “TECHNIQUES FOR MEASUREMENT OF PERCEPTUAL AUDIO QUALITY,” filed Dec. 14, 2001, now U.S. Pat. No. 7,146,313, the disclosure of which is hereby incorporated by reference; U.S. patent application Ser. No. 10/017,702, entitled, “QUANTIZATION MATRICES FOR DIGITAL AUDIO,” filed Dec. 14, 2001, now U.S. Pat. No. 6,934,677, the disclosure of which is hereby incorporated by reference; and U.S. patent application Ser. No. 10/020,708, entitled, “ADAPTIVE WINDOW-SIZE SELECTION IN TRANSFORM CODING,” filed Dec. 14, 2001, now U.S. Pat. No. 7,460,993, the disclosure of which is hereby incorporated by reference.
The present invention relates to techniques for improving sound quality of an audio codec (encoder/decoder).
The digital transmission and storage of audio signals are increasingly based on data reduction algorithms, which are adapted to the properties of the human auditory system and particularly rely on masking effects. Such algorithms do not mainly aim at minimizing the distortions but rather attempt to handle these distortions in a way that they are perceived as little as possible.
To understand these audio encoding techniques, it helps to understand how audio information is represented in a computer and how humans perceive audio.
Sample depth (or precision) indicates the range of numbers used to represent a sample. The more values possible for the sample, the higher the quality is because the number can capture more subtle variations in amplitude. For example, an 8-bit sample has 256 possible values, while a 16-bit sample has 65,536 possible values.
Mono and stereo are two common channel modes for audio. In mono mode, audio information is present in one channel. In stereo mode, audio information is present two channels usually labeled the left and right channels. Other modes with more channels, such as 5-channel surround sound, are also possible. Table 1 shows several formats of audio with different quality levels, along with corresponding raw bit rate costs.
Bit rates for different quality audio information
As Table 1 shows, the cost of high quality audio information such as CD audio is high bit rate. High quality audio information consumes large amounts of computer storage and transmission capacity.
Compression (also called encoding or coding) decreases the cost of storing and transmitting audio information by converting the information into a lower bit rate form. Compression can be lossless (in which quality does not suffer) or lossy (in which quality suffers). Decompression (also called decoding) extracts a reconstructed version of the original information from the compressed form.
Quantization is a conventional lossy compression technique. There are many different kinds of quantization including uniform and non-uniform quantization, scalar and vector quantization, and adaptive and non-adaptive quantization. Quantization maps ranges of input values to single values. For example, with uniform, scalar quantization by a factor of 3.0, a sample with a value anywhere between −1.5 and 1.499 is mapped to 0, a sample with a value anywhere between 1.5 and 4.499 is mapped to 1, etc. To reconstruct the sample, the quantized value is multiplied by the quantization factor, but the reconstruction is imprecise. Continuing the example started above, the quantized value 1 reconstructs to 1×3=3; it is impossible to determine where the original sample value was in the range 1.5 to 4.499. Quantization causes a loss in fidelity of the reconstructed value compared to the original value. Quantization can dramatically improve the effectiveness of subsequent lossless compression, however, thereby reducing bit rate.
An audio encoder can use various techniques to provide the best possible quality for a given bit rate, including transform coding, rate control, and modeling human perception of audio. As a result of these techniques, an audio signal can be more heavily quantized at selected frequencies or times to decrease bit rate, yet the increased quantization will not significantly degrade perceived quality for a listener.
Transform coding techniques convert information into a form that makes it easier to separate perceptually important information from perceptually unimportant information. The less important information can then be quantized heavily, while the more important information is preserved, so as to provide the best perceived quality for a given bit rate. Transform coding techniques typically convert information into the frequency (or spectral) domain. For example, a transform coder converts a time series of audio samples into frequency coefficients. Transform coding techniques include Discrete Cosine Transform [“DCT”], Modulated Lapped Transform [“MLT”], and Fast Fourier Transform [“FFT”]. In practice, the input to a transform coder is partitioned into blocks, and each block is transform coded. Blocks may have varying or fixed sizes, and may or may not overlap with an adjacent block. After transform coding, a frequency range of coefficients may be grouped for the purpose of quantization, in which case each coefficient is quantized like the others in the group, and the frequency range is called a quantization band. For more information about transform coding and MLT in particular, see Gibson et al., Digital Compression for Multimedia, “Chapter 7: Frequency Domain Coding,” Morgan Kaufman Publishers, Inc., pp. 227-262 (1998); U.S. Pat. No. 6,115,689 to Malvar; H. S. Malvar, Signal Processing with Lapped Transforms, Artech House, Norwood, Mass., 1992; or Seymour Schlein, “The Modulated Lapped Transform, Its Time-Varying Forms, and Its Application to Audio Coding Standards,” IEEE Transactions on Speech and Audio Processing, Vol. 5, No. 4, pp. 359-66, July 1997.
With rate control, an encoder adjusts quantization to regulate bit rate. For audio information at a constant quality, complex information typically has a higher bit rate (is less compressible) than simple information. So, if the complexity of audio information changes in a signal, the bit rate may change. In addition, changes in transmission capacity (such as those due to Internet traffic) affect available bit rate in some applications. The encoder can decrease bit rate by increasing quantization, and vice versa. Because the relation between degree of quantization and bit rate is complex and hard to predict in advance, the encoder can try different degrees of quantization to get the best quality possible for some bit rate, which is an example of a quantization loop.
Typically, an auditory model considers the range of human hearing and critical bands. Humans can hear sounds ranging from roughly 20 Hz to 20 kHz, and are most sensitive to sounds in the 2-4 kHz range. The human nervous system integrates sub-ranges of frequencies. For this reason, an auditory model may organize and process audio information by critical bands. For example, one critical band scale groups frequencies into 24 critical bands with upper cut-off frequencies (in Hz) at 100, 200, 300, 400, 510, 630, 770, 920, 1080, 1270, 1480, 1720, 2000, 2320, 2700, 3150, 3700, 4400, 5300, 6400, 7700, 9500, 12000, and 15500. Different auditory models use a different number of critical bands (e.g., 25, 32, 55, or 109) and/or different cutoff frequencies for the critical bands. Bark bands are a well-known example of critical bands.
TABLE 2 Various factors that relate to perception of audio Factor Relation to Perception of an Audio Signal outer and middle Generally, the outer and middle ear attenuate higher ear transfer frequency information and pass middle frequency information. Noise is less audible in higher frequencies than middle frequencies. noise in the Noise present in the auditory nerve, together with noise auditory nerve from the flow of blood, increases for low frequency information. Noise is less audible in lower frequencies than middle frequencies. perceptual Depending on the frequency of the audio signal, hair frequency scales cells at different positions in the inner ear react, which affects the pitch that a human perceives. Critical bands relate frequency to pitch. Excitation Hair cells typically respond several milliseconds after the onset of the audio signal at a frequency. After exposure, hair cells and neural processes need time to recover full sensitivity. Moreover, loud signals are processed faster than quiet signals. Noise can be masked when the ear will not sense it. Detection Humans are better at detecting changes in loudness for quieter signals than louder signals. Noise can be masked in quieter signals. simultaneous For a masker and maskee present at the same time, the masking maskee is masked at the frequency of the masker but also at frequencies above and below the masker. The amount of masking depends on the masker and maskee structures and the masker frequency. temporal The masker has a masking effect before and after than masking the masker itself. Generally, forward masking is more pronounced than backward masking. The masking effect diminishes further away from the masker in time. loudness Perceived loudness of a signal depends on frequency, duration, and sound pressure level. The components of a signal partially mask each other, and noise can be masked as a result. cognitive Cognitive effects influence perceptual audio quality. processing Abrupt changes in quality are objectionable. Different components of an audio signal are important in different applications (e.g., speech vs. music). III. Measuring Audio Quality
D=(u−q(u)Q)2 (1)
M [ k , n ] = E [ k , n ] 10 m [ k ] 10 ( 2 ) for m[k]=3.0 if k*res≦12 and m[k]=k*res if k*res>12, where k is the critical band, res is the resolution of the band scale in terms of Bark bands, n is the frame, and E[k,n] is the excitation pattern.
NMR local [ n ] = 10 * log 10 1 Z ∑ k = 0 Z - 1 P noise [ k , n ] M [ k , n ] ( 3 ) where n is the frame number, Z is the number of critical bands per frame, Pnoise[k,n] is the noise pattern, and M[k,n] is the masking threshold. NMR can also be calculated for a whole signal as a combination of NMR values for frames.
Microsoft Corporation's Windows Media Audio version 7.0 [“WMA7”] partially addresses some of the problems with implementing quality measurement in an audio encoder. In WMA7, the encoder may jointly code the left and right channels of stereo mode audio into a sum channel and a difference channel. The sum channel is the averages of the left and right channels; the difference channel is the differences between the left and right channels divided by two. The encoder calculates a noise signal for each of the sum channel and the difference channel, where the noise signal is the difference between the original channel and the reconstructed channel. The encoder then calculates the maximum Noise to Excitation Ratio [“NER”] of all quantization bands in the sum channel and difference channel:
NER max ofalld = max ( max d ( F Diff [ d ] E Diff [ d ] ) , max d ( F Sum [ d ] E Sum [ d ] ) ) ( 4 ) where d is the quantization band number, maxd is the maximum value across all d, and EDiff[d], ESum[d], FDiff[d], and FSum[d] are the excitation pattern for the difference channel, the excitation pattern for the sum channel, the noise pattern of the difference channel, and the noise pattern of the sum channel, respectively, for quantization bands. In WMA7, calculating an excitation or noise pattern includes squaring values to determine energies, and then, for each quantization band, adding the energies of the coefficients within that quantization band. If WMA7 does not use jointly coded channels, the same equation is used to measure the quality of left and right channels. That is,
WMA7 works in real time and measures audio quality for input with rates other than 48 kHz. WMA7 uses a MLT with variable transform block sizes, and measures audio quality using the same frequency coefficients used in compression. WMA7 does not address several of the problems of ITU-R BS 1387, however, and WMA7 has several other shortcomings as well, each of which decreases the accuracy of the measurement of perceptual audio quality. First, although the quality measurement of WMA7 is simple enough to be used in a quantization loop of the audio encoder, it does not adequately correlate with actual human perception. As a result, changes in quality in order to keep constant bit rate can be dramatic and perceptible. Second, the NER of WMA7 measures perceptible degradation compared to the excitation pattern of the original information (as opposed to reconstructed information), which can inaccurately estimate perceptible degradation for a listener of the reconstructed signal. Third, the NER of WMA7 fails to adequately weight NER on a per-band basis, which limits its usefulness and adaptability. Fourth, although WMA7 works with variable-size transform blocks, WMA7 is unable perform operations such as temporal masking between blocks due to the variable sizes. Fifth, WMA7 measures quality with respect to excitation and noise patterns for quantization bands, which are not necessarily related to a model of human perception with critical bands, and which can be different in different variable-size blocks, preventing comparisons of results. Sixth, WMA7 measures the maximum NER for all quantization bands of a channel, which can inappropriately ignore the contribution of NER s for other quantization bands. Seventh, WMA7 applies the same quality measurement techniques whether independently or jointly coded channels are used, which ignores differences between the two channel modes.
In MP3, the encoder incorporates a psychoacoustic model to calculate Signal to Mask Ratios [“SMRs”] for frequency ranges called threshold calculation partitions. In a path separate from the rest of the encoder, the encoder processes the original audio information according to the psychoacoustic model The psychoacoustic model uses a different frequency transform than the rest of the encoder (FFT vs. hybrid polyphase/MDCT filter bank) and uses separate computations for energy and other parameters. In the psychoacoustic model, the MP3 encoder processes blocks of frequency coefficients according to the threshold calculation partitions, which have sub-Bark band resolution (e.g., 62 partitions for a long block of 48 kHz input). The encoder calculates a SMR for each partition. The encoder converts the SMRs for the partitions into SMRs for scale factor bands. A scale factor band is a range of frequency coefficients for which the encoder calculates a weight called a scale factor. The number of scale factor bands depends on sampling rate and block size (e.g., 21 scale factor bands for a long block of 48 kHz input). The encoder later converts the SMRs for the scale factor bands into allowed distortion thresholds for the scale factor bands.
For additional information about MP3 and AAC, see the MP3 standard (“ISO/IEC 11172-3, Information Technology—Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1.5 Mbit/s—Part 3: Audio”) and the MC standard.
Although MP3 encoding has achieved widespread adoption, it is unsuitable for some applications (for example, real-time audio streaming at very low to mid bit rates) for several reasons. First, calculating SMRs and allowed distortion thresholds with MP3's psychoacoustic model occurs outside of the quantization loops. The psychoacoustic model is too complex for some applications, and cannot be integrated into a quantization loop for such applications. At the same time, as the psychoacoustic model is outside of the quantization loops, it works with original audio information (as opposed to reconstructed audio information), which can lead to inaccurate estimation of perceptible degradation for a listener of the reconstructed signal at lower bit rates. Second, the MP3 encoder fails to adequately weight SMRs and allowed distortion thresholds on a per-band basis, which limits the usefulness and adaptability of the MP3 encoder. Third, computing SMRs and allowed distortion thresholds in separate tracks for long blocks and short blocks prevents or complicates operations such as temporal spreading or comparing measures for blocks of different sizes. Fourth, the MP3 encoder does not adequately exploit differences between independently coded channels and jointly coded channels when calculating SMRs and allowed distortion thresholds.
Embodiments of an audio encoder are described herein that digitally encode audio signals with improved audio quality.
In a first audio encoding technique, an audio encoder dynamically selects between joint and independent coding of a multi-channel audio signal using an open-loop selection decision based upon (a) energy separation between the coding channels, and (b) the disparity between excitation patterns of the separate input channels.
In a second audio encoding technique, an audio encoder performs band truncation to suppress a few higher frequency transform coefficients, so as to permit better coding of surviving coefficients. In one implementation, the audio encoder determines a cut-off frequency as a function of a perceptual quality measure (e.g., a noise-to-excitation ratio (“NER”) of the input signal). This way, if the content being compressed is not complex, less of such filtering is performed.
In a third audio encoding technique, an audio encoder performs channel re-matrixing when jointly encoding a multi-channel audio signal. In one implementation, the audio encoder suppresses certain coefficients of a difference channel by scaling according to a scale factor, which is based on (a) current average levels of perceptual quality, (b) current rate control buffer fullness, (c) coding mode (e.g., bit rate and sample rate settings, etc.), and (d) the amount of channel separation in the source. For example, if the current average perceptual quality measure indicates poor reproduction, the scale factor is varied to cause severe suppression of the difference channel in re-matrixing. Similar severe re-matrixing is performed as the rate control buffer approaches fullness. Conversely, if the two channels of the input audio signal significantly differ, the scale factor is varied so that little or no re-matrixing takes place.
In a fourth audio encoding technique, an audio encoder reduces the size of a quantization matrix in the encoded audio signal. The quantization matrix encodes quantizer step size of quantization bands of an encoded channel in the encoded audio signal. In one implementation, the quantization matrix is differentially encoded for successive frames of the audio signal. At certain (e.g., lower) coding rates, particular quantization bands may be quantized to all zeroes (e.g., due to quantization or band truncation). In such cases, the audio encoder reduces the bits needed to differentially encode the quantization matrices of successive frames by modifying the quantization step size of bands that are quantized to zero, so as to be differentially encoded using fewer bits. For example, the various bands that are quantized to zero may initially have various quantization step sizes. Via this technique, the audio encoder may adjust the quantization step sizes of these bands to be identical so that they may be differentially encoded in the quantization matrix using fewer bits.
FIG. 2 is a block diagram of a suitable computing environment for an audio encoder incorporating quality enhancement techniques described herein.
FIGS. 3 and 4 are a block diagram of an audio encoder and decoder in which quality enhancement techniques described herein are incorporated.
FIG. 5 is a flow diagram of joint channel coding in the audio encoder of FIG. 3.
FIG. 6 is a flow diagram of independent channel coding in the audio encoder of FIG. 3.
FIG. 7 is a flow chart of a multi-channel coding decision process in the audio encoder of FIG. 3.
FIG. 8 is a graph of cutoff frequency for band truncation as a function of a perceptual quality measure in the audio encoder of FIG. 3.
FIG. 9 is a data flow diagram of a pre-encoding band truncation process based on a target quality measure in the audio encoder of FIG. 3.
FIG. 10 is a data flow diagram of a multi-channel rematrixing process in the audio encoder of FIG. 3.
FIG. 11 is a flow chart of a quantization step-size modification process for header bit reduction in the audio encoder of FIG. 3.
FIG. 12 is a graph of an example of quantization step-size modification to reduce header bits.
FIG. 13 is a chart showing a mapping of quantization bands to critical bands according to the illustrative embodiment.
FIGS. 14 a-14 d are diagrams showing computation of NER in an audio encoder according to the illustrative embodiment.
FIG. 15 is a flowchart showing a technique for measuring the quality of a normalized block of audio information according to the illustrative embodiment.
FIG. 16 is a graph of an outer/middle ear transfer function according to the illustrative embodiment.
FIG. 17 is a flowchart showing a technique for computing an effective masking measure according to the illustrative embodiment.
FIG. 18 is a flowchart showing a technique for computing a band-weighted quality measure according to the illustrative embodiment.
FIG. 19 is a graph showing a set of perceptual weights for critical band according to the illustrative embodiment.
FIG. 20 is a flowchart showing a technique for measuring audio quality in a coding channel mode-dependent manner according to the illustrative embodiment.
The following detailed description addresses embodiments of an audio encoder that implements various audio quality improvements. The audio encoder incorporates an improved multi-channel coding decision based on energy separation and excitation pattern disparity between channels. The audio encoder further performs band truncation at a cut-off frequency based on a perceptual quality measure. The audio encoder also performs multi-channel rematrixing with suppression based on (a) current average levels of perceptual quality, (b) current rate control buffer fullness, (c) coding mode (e.g., bit rate and sample rate settings, etc.), and (d) the amount of channel separation in the source. The audio encoder also adjusts step size of zero-quantized quantization bands for efficient coding of the quantization matrix, such as in frame headers.
The invention can be described in the general context of computer-executable instructions, such as those included in program modules, being executed in a computing environment on a target real or virtual processor. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Computer executable instructions for program modules may be executed within a local or distributed computing environment.
The generalized audio encoder (300) includes a frequency transformer (31 0), a multi-channel transformer (320), a perception modeler (330), a weighter (340), a quantizer (350), an entropy encoder (360), a rate/quality controller (370), and a bitstream multiplexer [“MUX”] (380).
The perception modeler (330) models properties of the human auditory system to improve the quality of the reconstructed audio signal for a given bit rate. The perception modeler (330) computes the excitation pattern of a variable-size block of frequency coefficients. First, the perception modeler (330) normalizes the size and amplitude scale of the block. This enables subsequent temporal smearing and establishes a consistent scale for quality measures. Optionally, the perception modeler (330) attenuates the coefficients at certain frequencies to model the outer/middle ear transfer function. The perception modeler (330) computes the energy of the coefficients in the block and aggregates the energies by 25 critical bands. Alternatively, the perception modeler (330) uses another number of critical bands (e.g., 55 or 109). The frequency ranges for the critical bands are implementation-dependent, and numerous options are well known. For example, see ITU-R BS 1387 or a reference mentioned therein. The perception modeler (330) processes the band energies to account for simultaneous and temporal masking. In alternative embodiments, the perception modeler (330) processes the audio data according to a different auditory model, such as one described or mentioned in ITU-R BS 1387.
The quantizer (350) quantizes the output of the weighter (340), producing quantized coefficient data to the entropy encoder (360) and side information including quantization step size to the MUX (380). Quantization introduces irreversible loss of information, but also allows the encoder (300) to regulate the bit rate of the output bitstream (395) in conjunction with the rate/quality controller (370). In FIG. 3, the quantizer (350) is an adaptive, uniform scalar quantizer. The quantizer (350) applies the same quantization step size to each frequency coefficient, but the quantization step size itself can change from one iteration to the next to affect the bit rate of the entropy encoder (360) output. In alternative embodiments, the quantizer is a non-uniform quantizer, a vector quantizer, and/or a non-adaptive quantizer.
The rate/quality controller (370) works with the quantizer (350) to regulate the bit rate and quality of the output of the encoder (300). The rate/quality controller (370) receives information from other modules of the encoder (300). In one implementation, the rate/quality controller (370) receives estimates of future complexity from the frequency transformer (310), sampling rate, block size information, the excitation pattern of original audio data from the perception modeler (330), weighting factors from the weighter (340), a block of quantized audio information in some form (e.g., quantized, reconstructed, or encoded), and buffer status information from the MUX (380). The rate/quality controller (370) can include an inverse quantizer, an inverse weighter, an inverse multi-channel transformer, and, potentially, an entropy decoder and other modules, to reconstruct the audio data from a quantized form.
The rate/quality controller (370) processes the information to determine a desired quantization step size given current conditions and outputs the quantization step size to the quantizer (350). The rate/quality controller (370) then measures the quality of a block of reconstructed audio data as quantized with the quantization step size, as described below. Using the measured quality as well as bit rate information, the rate/quality controller (370) adjusts the quantization step size with the goal of satisfying bit rate and quality constraints, both instantaneous and long-term. In alternative embodiments, the rate/quality controller (370) applies works with different or additional information, or applies different techniques to regulate quality and bit rate.
In conjunction with the rate/quality controller (370), the encoder (300) can apply noise substitution, band truncation, and/or multi-channel rematrixing to a block of audio data. At low and mid-bit rates, the audio encoder (300) can use noise substitution to convey information in certain bands. In band truncation, if the measured quality for a block indicates poor quality, the encoder (300) can completely eliminate the coefficients in certain (usually higher frequency) bands to improve the overall quality in the remaining bands. In multi-channel rematrixing, for low bit rate, multi-channel audio data in jointly coded channels, the encoder (300) can suppress information in certain channels (e.g., the difference channel) to improve the quality of the remaining channel(s) (e.g., the sum channel).
The MUX (380) includes a virtual buffer that stores the bitstream (395) to be output by the encoder (300). The virtual buffer stores a pre-determined duration of audio information (e.g., 5 seconds for streaming audio) in order to smooth over short-term fluctuations in bit rate due to complexity changes in the audio. The virtual buffer then outputs data at a relatively constant bit rate. The current fullness of the buffer, the rate of change of fullness of the buffer, and other characteristics of the buffer can be used by the rate/quality controller (370) to regulate quality and bit rate.
The DEMUX (410) parses information in the bitstream (405) and sends information to the modules of the decoder (400). The DEMUX (410) includes one or more buffers to compensate for short-term variations in bit rate due to fluctuations in complexity of the audio, network jitter, and/or other factors.
III. Multi-Channel Coding Decision
As described above, the audio encoder 300 (FIG. 3) can dynamically decide between encoding a multiple channel input audio signal in a joint channel coding mode or an independent channel coding mode, such as on a block-by-block or other basis, for improved compression efficiency. In joint channel coding 500 (FIG. 5), the audio encoder applies a multi-channel transformation 510 on multiple channels of the input signal to produce coding channels, which are then transform encoded (erg., via frequency transform, quantization, and entropy encoding processes described above). An example of a multi-channel transformation is the conversion of left and right stereo channels into sum and difference channels using the equations (1) and (2) given above. In alternative embodiments, the joint coding can be performed on other multiple channel input signals, such as 5.1 channel surround sound, etc. Various alternative multi-channel transformations can be used to combine input channel signals into coding channels for the joint channel coding of such other multiple channel signals. By contrast, the audio encoder 300 separately transform encodes the individual channels of a multiple channel input signal in independent channel coding 600 (FIG. 6).
FIG. 7 shows one implementation of a multi-channel coding decision process 700 performed in the audio encoder 300 (FIG. 3) to decide the channel coding mode (joint channel coding 500 or independent channel coding 600). In this implementation, the multi-channel coding decision process 700 is an open-loop decision, which generally is less computationally expensive. In this open-loop decision process 700, the decision between channel coding modes is made based on: (a) energy separation between the coding channels, and (b) the disparity between excitation patterns of the individual input channels. This latter basis (excitation pattern disparity) for the multi-channel coding decision is beneficial in audio encoders in which the quantization matrices are forced to be the same for both coding channels when performing joint channel coding. If the aggregate excitation pattern used in generating the quantization matrix is severely mismatched with the excitation patterns of either of the coding channels, then the joint channel coding 500 in such audio encoders would produce a severe coding efficiency penalty. The excitation pattern of the audio signal is discussed in the section below, entitled, “Measuring Audio Quality.”
In the illustrated process 700, the audio encoder 300 decides the channel coding mode on a block basis. In other words, the process 700 is performed per input signal block as indicated at decision 770. Alternatively, the channel coding decision can be made on other bases.
At a first action 710 in the process 700, the audio encoder 300 measures the energy separation between the coding channels with and without the multi-channel transformation 510. At decision 720, the audio encoder 300 then determines whether the energy separation of the coding channels with the multi-channel transformation is greater than that without the transformation. In the case of two stereo channels (left and right), the audio encoder can determine the energy is greater with the transformation if the following relation evaluates to true:
Max ( σ l , σ r ) Min ( σ l , σ r ) < Max ( σ s , σ d ) Min ( σ s , σ d ) ( 8 ) where σl, σr, σs, and σd, refer to standard deviation in left, right, sum and difference channels, respectively, in either the time or frequency (transform) domain. If either denominator is zero, that corresponding ratio is taken to be a large value, e.g. infinity.
If the energy separation is greater with the multi-channel transformation at decision 720, the audio encoder 300 proceeds to also measure the disparity between excitation patterns of the individual input channels at action 730. In one implementation, the disparity in excitation patterns between the input channels is measured using the following calculation:
Max b { E [ b ] of left channel E [ b ] of right channel , E [ b ] of right channel E [ b ] of left channel } ( 9 ) where E[b] refers to the excitation pattern computed for critical band b.
In a second implementation, the audio encoder 300 uses a ratio between the expected noise-to-excitation ratio (NER) of the two input channels as a measure of the disparity. The measurement of NER is discussed in more detail below in the section entitled, “Measuring Audio Quality.” For joint coding mode, for a given channel c, the expected NER is given as:
NER Expected = ∑ b W [ b ] ( E ~ [ b ] ) 2 β E [ b ] ( 10 ) where {tilde over (E)}[b] is the aggregate excitation pattern of the input channels at critical band b, E[b] is the excitation pattern of channel c at critical band b, and W[b] is the weighting used in the NER computation described below in the section entitled, “Measuring Audio Quality.” In one implementation, based on experimentation, β=0.25. Alternatively, other calculations measuring disparity in the excitation patterns of the input channels can be used.
At decision 740, the audio encoder compares the measurement of the input channel excitation pattern disparity to a pre-determined threshold. In one implementation example, the threshold rule is that the ratio of the expected NER of the two channels exceeds 2.0, and the smaller expected NER is greater than 0.001. Other threshold values or rules can be used in alternative implementations of the audio encoder.
If the disparity measurement does not exceed the threshold, the audio encoder 300 decides to use joint channel coding 500 (FIG. 5) for the block as indicated at action 750. Otherwise, if the disparity measurement exceeds the threshold, the audio encoder 300 decides against joint channel coding and instead uses independent channel coding 600 (FIG. 6).
The process 700 then continues with the next block of the input signal as indicated at decision 770.
IV. Band Truncation
In audio encoding, a general rule of thumb can be expressed that “coding lower frequencies well” produces better sounding reconstructed audio than “coding all frequencies poorly.” The audio encoder 300 (FIG. 3) performs a band truncation process that applies this rule. In this band truncation process, the audio encoder eliminates a few higher frequency coefficients from the transform coefficients that are coded into the compressed audio stream. In other words, the audio encoder zeroes out or otherwise does not code the value of the eliminated transform coefficients. This permits the surviving transform coefficients to be coded at a higher resolution at a given coding bit rate. More specifically, the audio encoder 300 suppresses transform coefficients for frequencies above a cut-off frequency that is a function of the achieved perceptual audio quality (e.g., the NER value calculated as described below in the section entitled, “Measuring Audio Quality”).
FIG. 8 shows a graph 800 of one example of the cut-off frequency of the band truncation process as a function of the achieved NER value, where the cut-off frequency decreases (eliminating more transform coefficients from coding) as the NER value increases. In some audio encoders, the function relating cut-off frequency to NER value is coding mode dependent. Alternatively, various other functions relating the cut-off frequency of band truncation to an achieved quality measurement can be used. In another example, 20% of transform coefficients are truncated if the NER value is greater than or equal to 0.5 for an 8 KHz audio source and 8 Kbps bit rate of compressed audio.
FIG. 9 shows an improved band truncation process 810 in the audio encoder 300 (FIG. 3). In the improved band truncation process 810, the audio encoder 300 performs a first-pass band truncation as an open-loop computation based on a target NER for the audio signal, then performs a second band truncation as a closed-loop computation based on the achieved NER after compression of the audio signal with the first-pass band truncation.
The improved band truncation process 810 utilizes a combination of audio encoder components, including a target NER setting 820, a band truncation component 830, encoding component 840, and quality measurement component 850. The target NER setting 820 provides the target NER for the audio signal to the band truncation component 830, which then performs the first-pass band truncation on the input audio signal using the cut-off frequency yielded from the target NER by the function shown in the graph 800 of FIG. 8. The encoding component 840 performs encoding and decoding of the first-pass band truncated audio signal as described above with reference to the generalized encoder 300 (FIG. 3) and decoder 400 (FIG. 4), including frequency transform, quantization and inverse transform. The quality measurement component 850 then calculates the achieved NER for the now reconstructed audio signal as described below in the section entitled, “Measuring Audio Quality.” The quality measurement component 850 provides feedback of the achieved NER to the band truncation component 830, which then performs the second-pass band truncation on the input audio signal using the cut-off frequency yielded from the achieved NER by the function shown in graph 800. The encoding component then performs final encoding of the input audio signal with the second-pass band truncation to produce the compressed audio signal stream 860. The illustrated improved band truncation process 810 is performed on a block basis on the input audio signal, but alternatively can be performed on other bases.
The improved band truncation process 810 provides the benefit of yielding a more accurate achieved NER quality measure in the audio encoder 300, such as for use in closed-loop band truncation, and multi-channel re-matrixing, among other purposes.
V. Multi-Channel Rematrixing
FIG. 10 shows a multi-channel rematrixing process 900. When compressing a multi-channel audio signal at very low rates, the distortion (e.g., quantization noise) introduced in each channel can have a significant impact on the “stereo-image” upon play-back. The multi-channel re-matrixing process 900 can reduce the impact of audio compression on the stereo image of a multi-channel audio signal, as well as improve the joint-channel coding efficiency, by selectively suppressing certain coding channels in joint channel coding 500 (FIG. 5).
In one implementation of the multi-channel re-matrixing process 900, the audio encoder 300 (FIG. 3) includes a channel suppressor component 910 following the multi-channel transformation 510. The audio encoder 300 calculates suppression parameters 920 for the multi-channel re-matrixing process 900. Based on the suppression parameters, the channel suppressor component 910 selectively suppresses certain of the coding channels. Upon later application of an inverse multi-channel transformation 930 (e.g., in the audio decoder 400 of FIG. 4 for playback), this multi-channel re-matrixing process 900 produces re-matrixed multi-channel audio data with reduced impact of the distortion from compression on the stereo-image.
In one embodiment, the suppression parameters 920 include a scaling factor (ρ) whose value is based on: (a) current average levels of a perceptual audio quality measure (e.g., the NER described in more detail below in the section entitled, “Measuring Audio Quality”), (b) current rate control buffer fullness, (c) the coding mode (e.g., the bit rate and sample rate settings, etc. of the audio encoder), and (d) the amount of channel separation in the source. More specifically, if the current average level of quality indicates poor reproduction, the value of the scaling factor (ρ) is made much smaller than unity so as to produce severe re-matrixing of the multi-channel audio signal. A similar measure is taken if the rate control buffer is close to being full. On the other hand, if the two channels in the input data are significantly different, the scaling factor (ρ) is made closer to unity, so that little or no re-matrixing takes place.
In the case of two-channel stereo audio signal for example, the audio encoder 300 (FIG. 3) produces the sum and difference coding channels using the equations (6) and (7) with the multi-channel transformation 510 as described above. The coding channel suppression 910 can be described as scaling the difference channel by the scaling factor (ρ) in the following equation:
{tilde over (x)} d [n]=ρ·x d [n] (11)
The scaling factor (ρ) in this illustrated embodiment for two-channel stereo audio is calculated as follows. If the sample rate is greater than 32 KHz and the bit rate is greater than 32 Kbps, then the scaling factor (ρ) is set equal to 1.0. For other combinations of sample and and bit rates, the audio encoder 300 first calculates the energy separation of the channels. The energy separation of left and right stereo channels is computed as:
sep = Max ( σ l , σ r ) Min ( σ l , σ r ) ( 12 ) whose value is taken as a large quantity (>100) if the denominator is zero.
The audio encoder 300 then determines the scaling factor from the following tables (13-15), dependent on the perceptual quality measure (NER) and coefficient index (B) which are described in more detail below in the section entitled, “Measuring Audio Quality.” If (sep<5), the scaling factor (ρ) is given as follows:
ρ = { 6 / 16 ( NER > 2 ) OR ( B F > 0.9 ) 7 / 16 ( NER > 1.75 ) OR ( B F > 0.9 ) 8 / 16 ( NER > 1.5 ) OR ( B F > 0.85 ) 9 / 16 ( NER > 1.25 ) OR ( B F > 0.85 ) 10 / 16 ( NER > 1.0 ) OR ( B F > 0.85 ) 11 / 16 ( NER > 0.75 ) OR ( B F > 0.8 ) 12 / 16 ( NER > 0.5 ) OR ( B F > 0.75 ) 13 / 16 ( NER > 0.25 ) 14 / 16 ( NER > 0.1 ) 16 / 16 Otherwise ( 13 ) If (5≦sep<100), the scaling factor (ρ) is given as follows:
ρ = { 8 / 16 ( NER > 2.5 ) OR ( B F > 0.95 ) 9 / 16 ( NER > 2.25 ) OR ( B F > 0.9 ) 10 / 16 ( NER > 2 ) OR ( B F > 0.9 ) 10 / 16 ( NER > 1.75 ) OR ( B F > 0.9 ) 11 / 16 ( NER > 1.5 ) OR ( B F > 0.85 ) 11 / 16 ( NER > 1.25 ) OR ( B F > 0.85 ) 12 / 16 ( NER > 1.0 ) OR ( B F > 0.85 ) 13 / 16 ( NER > 0.75 ) OR ( B F > 0.8 ) 14 / 16 ( NER > 0.5 ) OR ( B F > 0.75 ) 15 / 16 ( NER > 0.25 ) 16 / 16 Otherwise ( 14 ) If (100≦sep), the scaling factor (ρ) is given as follows:
Finally, the-matrixed channels can then be obtained (e.g., in the inverse multi-channel information 930) through the following equations:
{tilde over (x)} l [n]=x s [n]+{tilde over (x)} d [n] (16)
{tilde over (x)} l [n]=x s [n]−{tilde over (x)} d [n] (17)
VI. Quantizer Step-Size Modification For Header Reduction
FIG. 11 shows a header reduction process 1100 to further improve coding efficiency in the audio encoder 300 (FIG. 3). In the audio encoder 300, a quantization matrix containing quantizer step size information for each quantization band of each coding channel is normally sent for every frame of coded data in the compressed audio data stream. These quantization matrices are differentially encoded (e.g., similar to differential pulse code modulation) in a header of each frame within the compressed audio stream produced by the audio encoder. The quantization matrix is described in further detail in the related patent application, entitled “Quantization Matrices For Digital Audio,” which is incorporated herein by reference above.
Generally at lower coding rates, the audio encoder 300 quantizes certain quantization band coefficients to all zeroes, such as due to quantization or due to the band truncation process described above. In such case, the quantization step size for the zeroed quantization band is not needed by the decoder to decode the compressed audio signal stream.
The header reduction process 1100 reduces the size of the header by selectively modifying the quantization step size of quantization band coefficients that are quantized, so that such quantization step sizes will differentially encode using fewer bits in the header. More specifically, at action 1110 in the header reduction process 1100, the audio encoder 300 identifies which quantization bands are quantized to zero, either due to band truncation or because the value of the coefficient for that band is sufficiently small to quantize to zero. At action 1120, the audio encoder 300 modifies the quantization step size of the identified quantization bands to values that will be encoded in fewer bits in the header.
FIG. 12 shows a graph 1200 of an example of quantization step-size modification for header reduction via the header reduction process 1100. The values of the original quantization step sizes of the quantization bands for this frame of the audio signal is shown by the line labeled “quant. step before bit reduction” in graph 1200. In this example, quantization bands numbered 2 through 20 are quantized to zero (as indicated by the “band required” line of the graph 1200). The header reduction process 1100 therefore modifies the quantization step sizes for these bands to values (e.g., the value of quantization band numbered 21 in this example) that will be differentially encoded in the header using fewer bits. The modified values are depicted in the graph 1200 by the line labeled “quant. step after bit reduction.” The particular modification of the quantization step sizes that will yield fewer bits in the header is dependent on the particular form of encoding used. Accordingly, the header reduction process 1100 modifies the value of the quantization step sizes of the zeroed quantization band coefficients to a value that will encode in fewer bits for the particular form of quantization step encoding employed by the audio encoder (whether differential encoding or otherwise).
V. Measuring Audio Quality
FIG. 13 shows an example of a mapping (1300) between quantization bands and critical bands. The critical bands are determined by an auditory model, while the quantization bands are determined by the encoder for efficient representation of the quantization matrix. The number of quantization bands can be different (typically less) than the number of critical bands, and the band boundaries can be different as well in one implementation, the number of quantization bands relates to block size. For a block of 2048 frequency coefficients, the number of quantization bands is 25, and each quantization band maps to one of 25 critical bands of the same frequency range. For a block of the 64 frequency coefficients, the number of quantization bands is 13, and some quantization bands map to multiple critical bands.
FIGS. 14 a-14 d show techniques for computing one particular type of quality measure—Noise to Excitation Ratio [“NER”]. FIG. 14 a shows a technique (1400) for computing NER of a block by critical bands for a single channel. The overall quality measure for the block is a weighted sum of NER s of individual critical bands. FIGS. 14 b and 14 c show additional detail for several stages of the technique (1400). FIG. 14 d shows a technique (701) for computing NER of a block by quantization bands.
The inputs to the techniques (1400) and (1401) include the original frequency coefficients X[k] for the block, the reconstructed coefficients {circumflex over (X)}[k] (inverse quantized, inverse weighted, and inverse multi-channel transformed if needed), and one or more weight arrays. The one or more weight arrays can indicate 1) the relative importance of different bands to perception, 2) whether bands are truncated, and/or 3) whether bands are noise-substituted. The one or more weight arrays can be in separate arrays (e.g., W[b], Z[b], G[b]), in a single aggregate array, or in some other combination. FIGS. 14 b and 14 c show other inputs such as transform block size (i.e., current window/sub-frame size), maximum block size (i.e., largest time window/frame size), sampling rate, and the number and positions of critical bands.
With reference to FIG. 14 a, the encoder computes (1410) the excitation pattern E[b] for the original frequency coefficients X[k] and computes (1430) the excitation pattern Ê[b] for the reconstructed frequency coefficients {circumflex over (X)}[k] for a block of audio information. The encoder computes the excitations pattern Ê[b] with the same coefficients that are used in compression, using the sampling rate and block sizes used in compression, which makes the process more flexible than the process for computing excitation patterns described in ITU-R BS 1387. In addition, several steps from ITU-R BS 1387 are eliminated (e.g., the adding of internal noise) or simplified to reduce complexity with only a little loss of accuracy.
FIG. 14 b shows in greater detail the stage of computing (1410) the excitation pattern E[b] for the original frequency coefficients X[k] in a variable-size transform block. To compute (1430) Ê[b], the input is {circumflex over (X)}[k] instead of X[k], and the process is analogous.
First, the encoder normalizes (1412) the block of frequency coefficients X[k],0≦k< (subframe_size/2) for a sub-frame, taking as inputs the current sub-frame size and the maximum sub-frame size (if not pre-determined in the encoder). The encoder normalizes the size of the block to a standard size by interpolating values between frequency coefficients up to the largest time window/sub-frame size. For example, the encoder uses a zero-order hold technique (i.e., coefficient repetition):
Y[k]=αX[k′] (18),
k ′ = floor ( k ρ ) , ( 19 ) ρ = max_subframe _size subframe_size , ( 20 ) where Y[k] is the normalized block with interpolated frequency coefficient values, α is an amplitude scaling factor described below, and k′ is an index in the block of frequency coefficients. The index k′ depends on the interpolation factor ρ, which is the ratio of the largest sub-frame size to the current sub-frame size. If the current sub-frame size is 1024 coefficients and the maximum size is 4096 coefficients, ρ is 4, and for every coefficient from 0-511 in the current transform block (which has a size of 0≦k< (subframe_size/2)), the normalized block Y[k] includes four consecutive values. Alternatively, the encoder uses other linear or non-linear interpolation techniques to normalize block size.
The scaling factor a compensates for changes in amplitude scale that relate to sub-frame size. In one implementation, the scaling factor is:
α = c subframe_size , ( 21 ) where c is a constant with a value determined experimentally, for example, c=1.0. Alternatively, other scaling factors can be used to normalize block amplitude scale.
FIG. 15 shows a technique (1500) for measuring the audio quality of normalized, variable-size blocks in a broader context than FIGS. 14 a through 14 d. A tool such as an audio encoder gets (1510) a first variable-size block and normalizes (1520) the variable-size block. The variable-size block is, for example, a variable-size transform block of frequency coefficients. The normalization can include block size normalization as well as amplitude scale normalization, and enables comparisons and operations between different variable-size blocks.
Next, the tool computes (1530) a quality measure for the normalized block. For example, the tool computes NER for the block.
If the tool determines (1540) that there are no more blocks to measure quality for, the technique ends. Otherwise, the tool gets (1550) the next block and repeats the process. For the sake of simplicity, FIG. 15 does not show repeated computation of the quality measure (as in a quantization loop) or other ways in which the technique (1500) can be used in conjunction with other techniques.
Returning to FIG. 14 b, after normalizing (1412) the block, the encoder optionally applies (1414) an outer/middle ear transfer function to the normalized block.
Y[k]←A[k]·Y[k] (22).
Modeling the effects of the outer and middle ear on perception, the function A[k] generally preserves coefficients at lower and middle frequencies and attenuates coefficients at higher frequencies. FIG. 16 shows an example of a transfer function (1600) used in one implementation. Alternatively, a transfer function of another shape is used. The application of the transfer function is optional. In particular, for high bit rate applications, the encoder preserves fidelity at higher frequencies by not applying the transfer function.
The encoder next computes (1416) the band energies for the block, taking as inputs the normalized block of frequency coefficients Y[k], the number and positions of the bands, the maximum sub-frame size, and the sampling rate. (Alternatively, one or more of the band inputs, size, or sampling rate is predetermined.) Using the normalized block Y[k], the energy within each critical band b is accumulated:
E [ b ] = ∑ k ∈ B [ b ] Y 2 [ k ] , ( 23 ) where B[b] is a set of coefficient indices that represent frequencies within critical band b. For example, if the critical band b spans the frequency range [ƒl, ƒh), the set B[b] can be given as:
So, if the sampling rate is 44.1 kHz and the maximum sub-frame size is 4096 samples, the coefficient indices 38 through 47 (of 0 to 2047) fall within a critical band that runs from 400 up to but not including 510. The frequency ranges [ƒl, ƒh) for the critical bands are implementation-dependent, and numerous options are well known. For example, see ITU-R BS 1387, the MP3 standard, or references mentioned therein.
Next, also in optional stages, the encoder smears the energies of the critical bands in frequency smearing (1418) between critical bands in the block and temporal smearing (1420) from block to block. The normalization of block sizes facilitates and simplifies temporal smearing between variable-size transform blocks. The frequency smearing (1418) and temporal smearing (1420) are also implementation-dependent, and numerous options are well known. For example, see ITU-R BS 1387, the MP3 standard, or references mentioned therein. The encoder outputs the excitation pattern E[b] for the block.
Returning to FIG. 14 a, from the excitation patterns E[b] and Ê[b] for the original and the reconstructed frequency coefficients, respectively, the encoder computes (1450) an effective excitation pattern {tilde over (E)}[b]. For example, the encoder finds the minimum excitation on a band by band basis between E[b] and Ê[b]:
{tilde over (E)}[b]=Min(E[b],Ê[b]) (25).
Alternatively the encoder uses another formula to determine the effective excitation pattern. Excitation in the reconstructed signal can be more than or less the excitation in the original signal due to the effects of quantization. Using the effective excitation pattern {tilde over (E)}[b] rather than the excitation pattern E[b] for the original signal ensures that the masking component is present at reconstruction. For example, if the original frequency coefficients in a band are heavily quantized, the masking component that is supposed to be in that band might not be present in the reconstructed signal, making noise audible rather than inaudible. On the other hand, if the excitation at a band in the reconstructed signal is much greater than the excitation at that band in the original signal, the excess excitation in the reconstructed signal may itself be due to noise, and should not be factored into later NER calculations.
FIG. 17 shows a technique (1700) for computing an effective masking measure in a broader context than FIGS. 7 a through 7 d. A tool such as an audio encoder computes (1710) an original audio masking measure. For example, the tool computes an excitation pattern for a block of original frequency coefficients.
Alternatively, the tool computes another type of masking measure (e.g., masking threshold), measures something other than blocks (e.g., channels, entire signals), and/or measures another type of information.
The tool computes (1720) a reconstructed audio masking measure of the same general format as the original audio masking measure.
Next, the tool computes (1730) an effective masking measure based at least in part upon the original audio masking measure and the reconstructed audio masking measure. For example, the tool finds the minimum of two excitation patterns. Alternatively, the tool uses another technique to determine the effective excitation masking measure. For the sake of simplicity, FIG. 17 does not show repeated computation of the effective masking measure (as in a quantization loop) or other ways in which the technique (1700) can be used in conjunction with other techniques.
Returning to FIG. 14 a, the encoder computes (1470) the noise pattern F[b] from the difference between the original frequency coefficients and the reconstructed frequency coefficients. Alternatively, the encoder computes the noise pattern F[b] from the difference between time series of original and reconstructed audio samples. The computing of the noise pattern F[b] uses some of the steps used in computing excitation patterns. FIG. 14 c shows in greater detail the stage of computing (1470) the noise pattern F[b].
First, the encoder computes (1472) the differences between a block of original frequency coefficients X[k] and a block of reconstructed frequency coefficients {circumflex over (X)}[k] for 0≦k< (subframe_size/2). The encoder normalizes (1474) the block of differences, taking as inputs the current sub-frame size and the maximum sub-frame size (if not pre-determined in the encoder). The encoder normalizes the size of the block to a standard size by interpolating values between frequency coefficients up to the largest time window/sub-frame size. For example, the encoder uses a zero-order hold technique (i.e., coefficient repetition):
DY[k]=α(X[k′]−{circumflex over (X)}[k′]) (26),
After normalizing (1474) the block, the encoder optionally applies (1476) an outer/middle ear transfer function to the normalized block.
DY[k]←A[k]·DY[k] (27),
where A[k] is a transfer function as shown, for example, in FIG. 16.
The encoder next computes (1478) the band energies for the block, taking as inputs the normalized block of frequency coefficient differences DY[k], the number and positions of the bands, the maximum sub-frame size, and the sampling rate. (Alternatively, one or more of the band inputs, size, or sampling rate is predetermined.) Using the normalized block of frequency coefficient differences DY[k], the energy within each critical band b is accumulated:
F [ b ] = ∑ k ∈ B [ b ] DY 2 [ k ] , ( 28 ) where B[b] is a set of coefficient indices that represent frequencies within critical band b as described in Equation 13. As the noise pattern F[b] represents a masked signal rather than a masking signal, the encoder does not smear the noise patterns of critical bands for simultaneous or temporal masking.
FIG. 18 shows a technique (1800) for computing a band-weighted quality measure for a block in a broader context than FIGS. 14 a through 14 d. A tool such as an audio encoder gets (1810) a first block of spectral information and determines (1820) band weights for the block. For example, the tool computes a set of perceptual weights, a set of weights indicating which bands are noise-substituted, a set of weights indicating which bands are truncated, and/or another set of weights for another weighting factor. Alternatively, the tool receives the band weights from another module. Within an encoding session, the band weights for one block can be different than the band weights for another block in terms of the weights themselves or the number of bands.
The tool then computes (1830) a band-weighted quality measure. For example, the tool computes a band-weighted NER. The tool determines (1840) if there are more blocks. If so, the tool gets (1850) the next block and determines (1820) band weights for the next block. For the sake of simplicity, FIG. 18 does not show different ways to combine sets of band weights, repeated computation of the quality measure for the block (as in a quantization loop), or other ways in which the technique (1800) can be used in conjunction with other techniques.
With reference to FIG. 14 a, a perceptual weight array W[b] accounts for the relative importance of different bands to the perceived quality of the reconstructed audio. In general, bands for middle frequencies are more important to perceived quality than bands for low or high frequencies. FIG. 19 shows an example of a set of perceptual weights (1900) for critical bands for NER computation. The middle critical bands are given higher weights than the lower and higher critical bands. The perceptual weight array W[b] can vary in terms of amplitudes from block to block within an encoding session; the weights can be different for different patterns of audio information (e.g., different excitation patterns), different applications (e.g., speech coding, music coding), different sampling rates (e.g., 8 kHz, 96 kHz), different bitrates of coding, or different levels of audibility of target listeners (e.g., playback at 40 dB, 96 dB). The perceptual weight array W[b] can also change in response to user input (e.g., a user adjusting weights based on the user's preferences).
In one implementation, the encoder can use noise substitution (rather than quantization of spectral information) to parametrically convey audio information for a band in low and mid-bit rate coding. The encoder considers the audio pattern (e.g., harmonic, tonal) in deciding whether noise substitution is more efficient than sending quantized spectral information. Typically, the encoder starts using noise substitution for higher bands and does not use noise substitution at all for certain bands. When the generated noise pattern for a band is combined with other audio information to reconstruct audio samples, the audibility of the noise is comparable to the audibility of the noise associated with an actual noise pattern.
For this reason, the encoder of FIG. 14 a does not factor the generated noise patterns of the noise-substituted bands into the NER. The array G[b] indicates which critical bands are noise-substituted in the block with a weight of 1 for each noise-substituted band and a weight of 0 for each other band. The encoder uses the array G[b] to skip noise-substituted bands when computing NER. Alternatively, the array G[b] includes a weight of 0 for noise-substituted bands and 1 for all other bands, and the encoder multiplies the NER by the weight 0 for noise-substituted bands; or, the encoder uses another technique to account for noise substitution in quality measurement.
An encoder typically uses noise substitution with respect to quantization bands. The encoder of FIG. 14 a measures quality for critical bands, however, so the encoder maps noise-substituted quantization bands to critical bands. For example, suppose the spectrum of noise-substituted quantization band d overlaps (partially or completely) the spectrum of critical bands blowd through bhighd. The entries G[blowd] through G[bhighd] are set to indicate noise-substituted bands. Alternatively, the encoder uses another linear or non-linear technique to map noise-substituted quantization bands to critical bands.
For multi-channel audio, the encoder computes NER for each channel separately. If the multi-channel audio is in independently coded channels, the encoder can use a different array G[b] for each channel. On the other hand, if the multi-channel audio is in jointly coded channels, the encoder uses an identical array G[b] for all reconstructed channels that are jointly coded. If any of the jointly coded channels has a noise-substituted band, when the jointly coded channels are transformed into independently coded channels, each independently coded channel will have noise from the generated noise pattern for that band. Accordingly, the encoder uses the same array G[b] for all reconstructed channels, and the encoder includes fewer arrays G[b] in the output bitstream, lowering overall bit rate.
More generally, FIG. 20 shows a technique (2000) for measuring audio quality in a channel mode-dependent manner. A tool such as an audio encoder optionally applies (2010) a multi-channel transform to multi-channel audio. For example, a tool that works with stereo mode audio optionally outputs the stereo audio in independently coded channels or in jointly coded channels.
The tool determines (2020) the channel mode of the multi-channel audio and then measures quality in a channel mode-dependent manner. If the audio is in independently coded channels, the tool measures (2030) quality using a technique for independently coded channels, and if the audio is in jointly coded channels, the tool measures (2040) quality using a technique for jointly coded channels. For example, the tool uses a different band weighting technique depending on the channel mode. Alternatively, the tool uses a different technique for measuring noise, excitation, masking capacity, or other pattern in the audio depending on the channel mode.
While FIG. 20 shows two modes, other numbers of modes are possible. For the sake of simplicity, FIG. 20 does not show repeated computation of the quality measure for the block (as in a quantization loop), or other ways in which the technique (2000) can be used in conjunction with other techniques.
When the encoder truncates a band, the encoder does not factor the quality measurement for the truncated band into the NER. With reference to FIG. 14 a, the array Z[b] indicates which bands are truncated in the block with a weighting pattern such as one described above for the array G[b]. When the encoder measures quality for critical bands, the encoder maps truncated quantization bands to critical bands using a mapping technique such as one described above for the array G[b]. When the encoder measures quality of multi-channel audio in jointly coded channels, the encoder can use the same array Z[b] for all reconstructed channels.
With reference to FIG. 14 a, the encoder next computes (790) band-weighted NER for the block. For the critical bands of the block, the encoder computes the ratio of the noise pattern F[b] to the effective excitation pattern {tilde over (E)}[b]. The encoder weights the ratio with band weights to determine the band-weighted NER for a block of a channel c:
Instead of a single set of band weights representing one kind of weighting factor or an aggregation of all weighting factors, the encoder can work with multiple sets of band weights. For example, FIG. 14 a shows three sets of band weights W[b], G[b], and Z[b], and the equation for NER[c] is:
For multi-channel audio, the encoder can compute an overall NER from NER[c] of each of the multiple channels. In one implementation, the encoder computes overall NER as the maximum distortion over all channels:
Instead of measuring audio quality of a block by critical bands, the encoder can measure audio quality of a block by quantization bands, as shown in FIG. 14 d. The encoder computes (1410, 1430) the excitation patterns E[b] and Ê[b], computes (1450) the effective excitation pattern {tilde over (E)}[b], and computes (1470) the noise pattern F[b] as in FIG. 14 a. At some point before computing (791) the band-weighted NER, however, the encoder converts all patterns for critical bands into patterns for quantization bands. For example, the encoder converts (780) the effective excitation pattern {tilde over (E)}[b] for critical bands into an effective excitation pattern {tilde over (E)}[d] for quantization bands. Alternatively, the encoder converts from critical bands to quantization bands at some other point, for example, after computing the excitation patterns. In one implementation, the encoder creates {tilde over (E)}[d] by weighting {tilde over (E)}[b] according to proportion of spectral overlap (i.e., overlap of frequency ranges) of the critical bands and the quantization bands. Alternatively, the encoder uses another linear or non-linear weighting techniques for the band conversion.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4251688Jan 15, 1979Feb 17, 1981Ana Maria FurnerAudio-digital processing system for demultiplexing stereophonic/quadriphonic input audio signals into 4-to-72 output audio signalsUS4538234 *Sep 2, 1982Aug 27, 1985Nippon Telegraph & Telephone Public CorporationAdaptive predictive processing systemUS4953196 *May 11, 1988Aug 28, 1990Ricoh Company, Ltd.Image transmission systemUS5079547Feb 27, 1991Jan 7, 1992Victor Company Of Japan, Ltd.Method of orthogonal transform coding/decodingUS5260980Aug 20, 1991Nov 9, 1993Sony CorporationDigital signal encoderUS5388181Sep 29, 1993Feb 7, 1995Anderson; David J.Digital audio compression systemUS5455888Dec 4, 1992Oct 3, 1995Northern Telecom LimitedSpeech bandwidth extension method and apparatusUS5473727Nov 1, 1993Dec 5, 1995Sony CorporationVoice encoding method and voice decoding methodUS5524054Jun 21, 1994Jun 4, 1996Deutsche Thomson-Brandt GmbhMethod for generating a multi-channel audio decoder matrixUS5539829Jun 7, 1995Jul 23, 1996U.S. Philips CorporationSubband coded digital transmission system using some composite signalsUS5581653Aug 31, 1993Dec 3, 1996Dolby Laboratories Licensing CorporationLow bit-rate high-resolution spectral envelope coding for audio encoder and decoderUS5627938Sep 22, 1994May 6, 1997Lucent Technologies Inc.Rate loop processor for perceptual encoder/decoderUS5629780Dec 19, 1994May 13, 1997The United States Of America As Represented By The Administrator Of The National Aeronautics And Space AdministrationImage data compression having minimum perceptual errorUS5632003 *Nov 1, 1993May 20, 1997Dolby Laboratories Licensing CorporationComputationally efficient adaptive bit allocation for coding method and apparatusUS5636324Mar 31, 1995Jun 3, 1997Matsushita Electric Industrial Co., Ltd.Apparatus and method for stereo audio encoding of digital audio signal dataUS5661755Oct 20, 1995Aug 26, 1997U. S. Philips CorporationEncoding and decoding of a wideband digital information signalUS5661823Sep 24, 1993Aug 26, 1997Kabushiki Kaisha ToshibaImage data processing apparatus that automatically sets a data compression rateUS5682152Mar 19, 1996Oct 28, 1997Johnson-Grace CompanyData compression using adaptive bit allocation and hybrid lossless entropy encodingUS5684920Mar 13, 1995Nov 4, 1997Nippon Telegraph And TelephoneAcoustic signal transform coding method and decoding method having a high efficiency envelope flattening method thereinUS5686964Dec 4, 1995Nov 11, 1997Tabatabai; AliBit rate control mechanism for digital image and video data compressionUS5701346Feb 2, 1995Dec 23, 1997Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V.Method of coding a plurality of audio signalsUS5745275Oct 15, 1996Apr 28, 1998Lucent Technologies Inc.Multi-channel stabilization of a multi-channel transmitter through correlation feedbackUS5787390Dec 11, 1996Jul 28, 1998France TelecomMethod for linear predictive analysis of an audiofrequency signal, and method for coding and decoding an audiofrequency signal including application thereofUS5812971Mar 22, 1996Sep 22, 1998Lucent Technologies Inc.Enhanced joint stereo coding method using temporal envelope shapingUS5822370Apr 16, 1996Oct 13, 1998Aura Systems, Inc.Compression/decompression for preservation of high fidelity speech quality at low bandwidthUS5826221Oct 29, 1996Oct 20, 1998Oki Electric Industry Co., Ltd.Vocal tract prediction coefficient coding and decoding circuitry capable of adaptively selecting quantized values and interpolation valuesUS5835030Apr 3, 1995Nov 10, 1998Sony CorporationSignal encoding method and apparatus using selected predetermined code tablesUS5845243Feb 3, 1997Dec 1, 1998U.S. Robotics Mobile Communications Corp.Method and apparatus for wavelet based data compression having adaptive bit rate control for compression of audio informationUS5890108Oct 3, 1996Mar 30, 1999Voxware, Inc.Low bit-rate speech coding system and method using voicing probability determinationUS5956674May 2, 1996Sep 21, 1999Digital Theater Systems, Inc.Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channelsUS5969750 *Sep 4, 1996Oct 19, 1999Winbcnd Electronics CorporationMoving picture camera with universal serial bus interfaceUS5973629 *Nov 25, 1997Oct 26, 1999Yamaha CorporationDifferential PCM system with frame word length responsive to magnitudeUS5974380Dec 16, 1997Oct 26, 1999Digital Theater Systems, Inc.Multi-channel audio decoderUS5995151Sep 18, 1997Nov 30, 1999Tektronix, Inc.Bit rate control mechanism for digital image and video data compressionUS6016111 *Nov 26, 1997Jan 18, 2000Samsung Electronics Co., Ltd.Digital data coding/decoding method and apparatusUS6029126Jun 30, 1998Feb 22, 2000Microsoft CorporationScalable audio coder and decoderUS6041295Apr 10, 1996Mar 21, 2000Corporate Computer SystemsComparing CODEC input/output to adjust psycho-acoustic parametersUS6058362Jun 30, 1998May 2, 2000Microsoft CorporationSystem and method for masking quantization noise of audio signalsUS6064954Mar 4, 1998May 16, 2000International Business Machines Corp.Digital audio signal codingUS6104321Jul 15, 1994Aug 15, 2000Sony CorporationEfficient encoding method, efficient code decoding method, efficient code encoding apparatus, efficient code decoding apparatus, efficient encoding/decoding system, and recording mediaUS6104996Sep 30, 1997Aug 15, 2000Nokia Mobile Phones LimitedAudio coding with low-order adaptive prediction of transientsUS6115688Aug 16, 1996Sep 5, 2000Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V.Process and device for the scalable coding of audio signalsUS6115689 *May 27, 1998Sep 5, 2000Microsoft CorporationScalable audio coder and decoderUS6134523 *Dec 10, 1997Oct 17, 2000Kokusai Denshin Denwa Kabushiki KaishaCoding bit rate converting method and apparatus for coded audio dataUS6167373Dec 30, 1999Dec 26, 2000Matsushita Electric Industrial Co., Ltd.Linear prediction coefficient analyzing apparatus for the auto-correlation function of a digital speech signalUS6182034Jun 30, 1998Jan 30, 2001Microsoft CorporationSystem and method for producing a fixed effort quantization step size with a binary searchUS6185253 *Oct 31, 1997Feb 6, 2001Lucent Technology, Inc.Perceptual compression and robust bit-rate control systemUS6205430Sep 26, 1997Mar 20, 2001Stmicroelectronics Asia Pacific Pte LimitedAudio decoder with an adaptive frequency domain downmixerUS6212495 *Oct 8, 1998Apr 3, 2001Oki Electric Industry Co., Ltd.Coding method, coder, and decoder processing sample values repeatedly with different predicted valuesUS6226616Jun 21, 1999May 1, 2001Digital Theater Systems, Inc.Sound quality of established low bit-rate audio coding systems without loss of decoder compatibilityUS6240380Jun 30, 1998May 29, 2001Microsoft CorporationSystem and method for partially whitening and quantizing weighting functions of audio signalsUS6249614Mar 6, 1998Jun 19, 2001Alaris, Inc.Video compression and decompression using dynamic quantization and/or encodingUS6253185 *Nov 12, 1998Jun 26, 2001Lucent Technologies Inc.Multiple description transform coding of audio using optimal transforms of arbitrary dimensionUS6256608Jun 30, 1998Jul 3, 2001Microsoa CorporationSystem and method for entropy encoding quantized transform coefficients of a signalUS6341165Jun 3, 1997Jan 22, 2002Fraunhofer-Gesellschaft zur Förderdung der Angewandten Forschung E.V.Coding and decoding of audio signals by using intensity stereo and prediction processesUS6353807May 4, 1999Mar 5, 2002Sony CorporationInformation coding method and apparatus, code transform method and apparatus, code transform control method and apparatus, information recording method and apparatus, and program providing mediumUS6366881 *Feb 18, 1998Apr 2, 2002Sanyo Electric Co., Ltd.Voice encoding methodUS6370128Jan 21, 1998Apr 9, 2002Nokia Telecommunications OyMethod for control channel range extension in a cellular radio system, and a cellular radio systemUS6370502May 27, 1999Apr 9, 2002America Online, Inc.Method and system for reduction of quantization-induced block-discontinuities and general purpose audio codecUS6393392Sep 28, 1999May 21, 2002Telefonaktiebolaget Lm Ericsson (Publ)Multi-channel signal encoding and decodingUS6404827May 19, 1999Jun 11, 2002Matsushita Electric Industrial Co., Ltd.Method and apparatus for linear predictingUS6418405Sep 30, 1999Jul 9, 2002Motorola, Inc.Method and apparatus for dynamic segmentation of a low bit rate digital voice messageUS6424939Mar 13, 1998Jul 23, 2002Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V.Method for coding an audio signalUS6434190May 19, 2000Aug 13, 2002Texas Instruments IncorporatedGeneralized precoder for the upstream voiceband modem channelUS6445739Feb 5, 1998Sep 3, 2002Matsushita Electric Industrial Co., Ltd.Quantization matrix for still and moving picture codingUS6473561Jul 18, 2000Oct 29, 2002Samsung Electronics Co., Ltd.DVD disc, device and method for reproducing the sameUS6496798Sep 30, 1999Dec 17, 2002Motorola, Inc.Method and apparatus for encoding and decoding frames of voice model parameters into a low bit rate digital voice messageUS6499010 *Jan 4, 2000Dec 24, 2002Agere Systems Inc.Perceptual audio coder bit allocation scheme providing improved perceptual quality consistencyUS6594626Jan 8, 2002Jul 15, 2003Fujitsu LimitedVoice encoding and voice decoding using an adaptive codebook and an algebraic codebookUS6658162Oct 13, 1999Dec 2, 2003Sharp Laboratories Of AmericaImage coding method using visual optimizationUS6680972Jun 9, 1998Jan 20, 2004Coding Technologies Sweden AbSource coding enhancement using spectral-band replicationUS6697491Sep 3, 1998Feb 24, 2004Harman International Industries, Incorporated5-2-5 matrix encoder and decoder systemUS6704711Jan 5, 2001Mar 9, 2004Telefonaktiebolaget Lm Ericsson (Publ)System and method for modifying speech signalsUS6738074Dec 8, 2000May 18, 2004Texas Instruments IncorporatedImage compression system and methodUS6757654May 11, 2000Jun 29, 2004Telefonaktiebolaget Lm EricssonForward error correction in speech codingUS6760698Feb 12, 2001Jul 6, 2004Mindspeed Technologies Inc.System for coding speech information using an adaptive codebook with enhanced variable resolution schemeUS6766293Mar 13, 1998Jul 20, 2004Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V.Method for signalling a noise substitution during audio signal codingUS6771777Jun 3, 1997Aug 3, 2004Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.Process for coding and decoding stereophonic spectral valuesUS6774820Aug 18, 2003Aug 10, 2004Dolby Laboratories Licensing CorporationMatrix improvements to lossless encoding and decodingUS6807524Oct 27, 1999Oct 19, 2004Voiceage CorporationPerceptual weighting device and method for efficient coding of wideband signalsUS6836761Oct 20, 2000Dec 28, 2004Yamaha CorporationVoice converter for assimilation by frame synthesis with temporal alignmentUS6865534Jun 15, 1999Mar 8, 2005Nec CorporationSpeech and music signal coder/decoderUS6934677Dec 14, 2001Aug 23, 2005Microsoft CorporationQuantization matrices based on critical band pattern information for digital audio wherein quantization bands differ from critical bandsUS6940840Feb 21, 2001Sep 6, 2005Interdigital Technology CorporationApparatus for adaptive reverse power control for spread-spectrum communicationsUS7027982Dec 14, 2001Apr 11, 2006Microsoft CorporationQuality and rate control strategy for digital audioUS7050972Nov 15, 2001May 23, 2006Coding Technologies AbEnhancing the performance of coding systems that use high frequency reconstruction methodsUS7058571Jul 30, 2003Jun 6, 2006Matsushita Electric Industrial Co., Ltd.Audio decoding apparatus and method for band expansion with aliasing suppressionUS7062445Jan 26, 2001Jun 13, 2006Microsoft CorporationQuantization loop with heuristic approachUS7069212Sep 11, 2003Jun 27, 2006Matsushita Elecric Industrial Co., Ltd.Audio decoding apparatus and method for band expansion with aliasing adjustmentUS7096240 *Oct 30, 1999Aug 22, 2006Stmicroelectronics Asia Pacific Pte Ltd.Channel coupling for an AC-3 encoderUS7249016Feb 17, 2005Jul 24, 2007Microsoft CorporationQuantization matrices using normalized-block pattern of digital audioUS7269559Jan 24, 2002Sep 11, 2007Sony CorporationSpeech decoding apparatus and method using prediction and class tapsUS7328162Oct 9, 2003Feb 5, 2008Coding Technologies AbSource coding enhancement using spectral-band replicationUS7386132Aug 7, 2006Jun 10, 2008Harman International Industries, Incorporated5-2-5 matrix encoder and decoder systemUS7394903Jan 20, 2004Jul 1, 2008Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V.Apparatus and method for constructing a multi-channel output signal or for generating a downmix signalUS7460993Dec 14, 2001Dec 2, 2008Microsoft CorporationAdaptive window-size selection in transform codingUS7519538Oct 28, 2004Apr 14, 2009Koninklijke Philips Electronics N.V.Audio signal encoding or decodingUS7602922Mar 25, 2005Oct 13, 2009Koninklijke Philips Electronics N.V.Multi-channel encoderUS20020143556Jan 26, 2001Oct 3, 2002Kadatch Andrew V.Quantization loop with heuristic approachUS20030009327Apr 10, 2002Jan 9, 2003Mattias NilssonBandwidth extension of acoustic signalsUS20030050786Aug 7, 2001Mar 13, 2003Peter JaxMethod and apparatus for synthetic widening of the bandwidth of voice signalsUS20030115042Dec 14, 2001Jun 19, 2003Microsoft CorporationTechniques for measurement of perceptual audio qualityUS20030115052Dec 14, 2001Jun 19, 2003Microsoft CorporationAdaptive window-size selection in transform codingUS20030236580Jun 19, 2002Dec 25, 2003Microsoft CorporationConverting M channels of digital audio data into N channels of digital audio dataUS20040001608Jan 2, 2003Jan 1, 2004Rhoads Geoffrey B.Image processor and image processing methodUS20040044527Aug 15, 2003Mar 4, 2004Microsoft CorporationQuantization and inverse quantization for audioUS20040078194Oct 9, 2003Apr 22, 2004Coding Technologies Sweden AbSource coding enhancement using spectral-band replicationUS20040093208Nov 7, 2003May 13, 2004Lin YinAudio coding method and apparatusUS20040267543Apr 28, 2004Dec 30, 2004Nokia CorporationSupport of a multichannel audio extensionUS20050065780Oct 29, 2004Mar 24, 2005Microsoft CorporationDigital audio signal filtering mechanism and methodUS20050165611Jun 29, 2004Jul 28, 2005Microsoft CorporationEfficient coding of digital media spectral data using wide-sense perceptual similarityUS20050246164Apr 15, 2005Nov 3, 2005Nokia CorporationCoding of audio signalsUS20060013405Jul 14, 2005Jan 19, 2006Samsung Electronics, Co., Ltd.Multichannel audio data encoding/decoding method and apparatusUS20060106619Sep 16, 2005May 18, 2006Bernd IserBandwidth extension of bandlimited audio signalsUS20060259303May 12, 2005Nov 16, 2006Raimo BakisSystems and methods for pitch smoothing for text-to-speech synthesisUS20070081536Oct 12, 2006Apr 12, 2007Samsung Electronics Co., Ltd.Bit-stream processing/transmitting and/or receiving/ processing method, medium, and apparatusUS20070112559Apr 14, 2004May 17, 2007Koninklijke Philips Electronics N.V.Audio signal synthesisUS20070140499Feb 28, 2005Jun 21, 2007Dolby Laboratories Licensing CorporationMultichannel audio codingUS20070172071Jan 20, 2006Jul 26, 2007Microsoft CorporationComplex transforms for multi-channel audioUSRE36721Nov 22, 1995May 30, 2000Kabushiki Kaisha ToshibaSpeech coding and decoding apparatusEP0597649B1Nov 8, 1993Jan 21, 2004Sony CorporationHigh efficiency coding method and apparatusEP0663740A2Dec 30, 1994Jul 19, 1995Daewoo Electronics Co., LtdApparatus for adaptively encoding input digital audio signals from a plurality of channelsEP0669724A4Jul 15, 1994Sep 16, 1998Sony CorpHigh-efficiency encoding method, high-efficiency decoding method, high-efficiency encoding device, high-efficiency decoding device, high-efficiency encoding/decoding system and recording media.EP0910927B1Jun 3, 1997Jan 12, 2000AT &amp; T Laboratories/ResearchProcess for coding and decoding stereophonic spectral valuesEP0924962B1Apr 10, 1998Dec 12, 2012Sony CorporationEncoding method and device, decoding method and device, and recording mediumEP0931386B1Mar 13, 1998Jul 5, 2000Fraunhofer-Gesellschaft Zur Förderung Der Angewandten Forschung E.V.Method for signalling a noise substitution during audio signal codingEP1093113A3Sep 27, 2000Jan 15, 2003Motorola, Inc.Method and apparatus for dynamic segmentation of a low bit rate digital voice messageEP1175030B1Jun 14, 2001Feb 20, 2008Nokia Siemens Networks OyMethod and system for multichannel perceptual audio coding using the cascaded discrete cosine transform or modified discrete cosine transformEP1408484B1Jan 26, 2000Nov 30, 2005Coding Technologies ABEnhancing perceptual quality of sbr (spectral band replication) and hfr (high frequency reconstruction) coding methods by adaptive noise-floor addition and noise substitution limitingEP1617418B1Jan 26, 2000May 14, 2008Coding Technologies ABSpectral band replication and high frequency reconstruction audio coding methods and apparatuses using adaptive noise-floor addition and noise substitution limitingGB2318029B Title not available* Cited by examinerNon-Patent CitationsReference1"ISO/IEC 11172-3, Information Technology-Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1.5 Mbit/s-Part 3: Audio," 154 pp. (1993).2"ISO/IEC 11172-3, Information Technology—Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1.5 Mbit/s—Part 3: Audio," 154 pp. (1993).3"ISO/IEC 13818-7, Information Technology-Generic Coding of Moving Pictures and Associated Audio Information-Part 7: Advanced Audio Coding (AAC), Technical Corrigendum 1" 22 pp. (1998).4"ISO/IEC 13818-7, Information Technology—Generic Coding of Moving Pictures and Associated Audio Information—Part 7: Advanced Audio Coding (AAC), Technical Corrigendum 1" 22 pp. (1998).5"ISO/IEC 13818-7, Information Technology-Generic Coding of Moving Pictures and Associated Audio Information-Part 7: Advanced Audio Coding (AAC)," 174 pp. (1997).6"ISO/IEC 13818-7, Information Technology—Generic Coding of Moving Pictures and Associated Audio Information—Part 7: Advanced Audio Coding (AAC)," 174 pp. (1997).7"Method for Objective Measurements of Perceived Audio Quality," Rec. ITU-R BS.1387 (Question ITU-R 210/10), 1998.8"MPEG2 Audio for DVD: the Compromise Choice," 5 pp. (Oct. 1996).9"Radio Engineering," authored by KPR i-Services, Inc., printed from internet on Dec. 13, 2005, 3 pages.10"Smart Project-Algebraic Theorgy of Signal Processing," http://www.ece.cmu.edu/~smart/papers/dttaglo.html, printed from internet on Jun. 30, 2006, 2 pages.11"Smart Project—Algebraic Theorgy of Signal Processing," http://www.ece.cmu.edu/˜smart/papers/dttaglo.html, printed from internet on Jun. 30, 2006, 2 pages.12A.M. Kondoz, Digital Speech: Coding for Low Bit Rate Communications Systems, "Chapter 3.3: Linear Predictive Modeling of Speech Signals" and "Chapter 4: LPC Parameter Quantisation Using LSFs," John Wiley & Sons, pp. 42-53 and 79-97 (1994).13Advanced Television Systems Committee, ATSC Standard: Digital Audio Compression (AC-3), Revision A, 140 pp. (1995).14Audio Codec Processing Functions; Extended AMR Wideband Codec; Transcoding Functions (Release 6), 3rd Generation Partnership Technical Specification, Sep. 2004, pp. 1-86.15Autti et al., "Mobile Audio-from MP3 to AAC and further," Helsinki University of Technology, Nov. 2004, pp. 1-20.16Autti et al., "Mobile Audio—from MP3 to AAC and further," Helsinki University of Technology, Nov. 2004, pp. 1-20.17Beerends, "Audio Quality Determination Based on Perceptual Measurement Techniques," Applications of Digital Signal Processing to Audio and Acoustics, Chapter 1, Ed. Mark Kahrs, Karlheinz Brandenburg, Kluwer Acad. Publ., pp. 1-38 (1998).18Bier, "Digital Audio Compression: Why, What, and How," © 2000-2002 Berkeley Design Technology, Inc., Dec. 2, 2002, 15 pages.19Bosi et al., "ISO/IEC MPEG-2 Advanced Audio Coding," Journal of the Audio Engineering Society, Audio Engineering Society, vol. 45, No. 10, pp. 789-812 (1997).20Brandenburg et al, "Second Generation Perceptual Audio Coding: the Hybrid Coder," AES Preprint, 13 pp. (Mar. 1990).21Brandenburg et al., "ASPEC: Adaptive Spectral Entropy Coding of High Quality Music Signals," Proc. AES, 12 pp. (Feb. 1991).22Brandenburg et al., "Low Bit Rate Codecs for Audio Signals: Implementations in Real Time," Proc. AES, 12 pp. (Nov. 1988).23Brandenburg et al., "Low Bit Rate Coding of High-quality Digital Audio: Algorithms and Evaluation of Quality," Proc. AES, pp. 201-209 (May 1989).24Brandenburg, "ASPEC Coding", AES 10th International Conference, pp. 81-90 (1991).25Brandenburg, "ASPEC Coding", AES 10th International Conference, pp. 81-90 (Sep. 1991).26Brandenburg, "High Quality Sound Coding at 2.5 Bits/Sample," Proc. AES, 15 pp. (Mar. 1988).27Brandenburg, "MP3 and AAC Explained," AES 17th International Conference on High Quality Audio Coding, 1999, 12 pages.28Brandenburg, "OCF: Coding High Quality Audio with Data Rates of 64 kbit/sec," Proc. AES, 17 pp. (Mar. 1988).29Brandenburg, "OCF-A New Coding Algorithm for High Quality Sound Signals," Proc. ICASSP, pp. 5.1.1-5.1.4 (May 1987).30Brandenburg, "OCF—A New Coding Algorithm for High Quality Sound Signals," Proc. ICASSP, pp. 5.1.1-5.1.4 (May 1987).31Breebaart et al., "MPEG Spatial Audio Coding/MPEG Surround: Overview and Current Status," in Proc. 119th AES Conv., New York, NY, Oct. 7-10, 2005, pp. 1-17.32Breebaart et al., "Parametric Coding of Stereo Audio," EURASIP Jour. Applied Signal Proc., Sep. 2005, pp. 1305-1322.33Caetano et al., "Rate Control Strategy for Embedded Wavelet Video Coders," Electronics Letters, pp. 1815-1817 (Oct. 14, 1999).34Chen et al., U.S. Appl. No. 10/017,694, entitled, "Quality and Rate Control Strategy for Digital Audio," filed Dec. 14, 2001.35Chen et al., U.S. Appl. No. 10/017,702, entitled, "Quantization Matrices for Digital Audio," filed Dec. 14, 2001.36Chen et al., U.S. Appl. No. 10/017,861, entitled, "Techniques for Measurement of Perceptual Audio Quality," filed Dec. 14, 2001.37Chen et al., U.S. Appl. No. 10/020,708, entitled, "Adaptive Window-Size Selection in Transform Coding," filed Dec. 14, 2001.38Davis, "The AC-3 Multichannel Coder," Dolby Laboratories, 9 pp. (Downloaded from the World Wide Web on Aug. 15, 2002).39De Luca, "AN1090 Application Note: STA013 MPEG 2.5 Layer III Source Decoder," STMicroelectronics, 17 pp. (1999).40de Queiroz et al., "Time-Varying Lapped Transforms and Wavelet Packets," IEEE Transactions on Signal Processing, vol. 41, pp. 3293-3305 (1993).41Dietz et al., "Spectral Band Replication, a novel approach in audio coding," Preprint 5553, 112th AES Convention, Munich, 8 pages, May 2002.42Dolby Laboratories, "AAC Technology," 4 pp. [Downloaded from the web site aac-audio.com on World Wide Web on Nov. 21, 2001.].43Duhamel et al., "A Fast Algorithm for the Implementation of Filter Banks Based on Time Domain Aliasing Cancellation," Proc. Int'l Conf. Acous., Speech, and Sig. Process, pp. 2209-2212 (May 1991).44Edler et al., "Perceptual Audio Coding Using a Time-Varying Linear Pre- and Post-Filter," in AES 109th Convention, Los Angeles, California, 12 pp. (Sep. 2000).45Ekstrand, "Bandwidth Extension of Audio Signals by Spectral Band Replication," Proc 1st EEE Benelux Workshop on Model based Processing and Coding of Audio, Leuven, Belgium, pp. 73-79, Nov. 2002.46Fraunhofer-Gesellschaft, "MPEG Audio Layer-3," 4 pp. [Downloaded from the World Wide Web on Oct. 24, 2001.].47Fraunhofer-Gesellschaft, "MPEG-2 AAC," 3 pp. [Downloaded from the World Wide Web on Oct. 24, 2001.].48Geiger et al., "Audio Coding Based on Integer Transforms," AES Convention Paper 5471, 111th AES Convention, New York, NY, Sep. 21-24, 2001.49Gibson et al., Digital Compression for Multimedia, Title Page, Contents, "Chapter 7: Frequency Domain Coding," Morgan Kaufman Publishers, Inc., pp. iii, v-xi, and 227-262 (1998).50Gibson et al., Digital Compression for Multimedia, Title Page, Contents, "Chapter 8: Frequency Domain Speech and Audio Coding Standards," Morgan Kaufman Publishers, Inc., pp. 263-290 (1998).51Gillespie et al., "Speech dereverberation via maximum-kurtosis subband adaptive filtering," Proc. IEEE ICASSP, 2001, pp. 3701-3704.52H.S. Malvar, "Lapped Transforms for Efficient Transform/Subband Coding," IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 38, No. 6, pp. 969-978 (1990).53H.S. Malvar, "Signal Processing with Lapped Transforms," Artech House, Norwood, MA, pp. iv, vii-xi, 175-218, 353-357 (1992).54Hasegawa-Johnson et al., "Speech coding: fundamentals and applications," Handbook of Telecommunications, John Wiley and Sons, Inc., pp. 1-33 (2003). [available at http://citeseer.ist.psu.edu/617093.html].55Herley et al., "Tilings of the Time-Frequency Plane: Construction of Arbitrary Orthogonal Bases and Fast Tiling Algorithms," IEEE Transactions on Signal Processing, vol. 41, No. 12, pp. 3341-3359 (1993).56Herre et al., "Intensity Stereo Coding," presented at AES 96th Convention, 1994, 11 pages.57Herre et al., "The Reference Model Architecture for MPEG Spatial Audio Coding," Proc. 118th AES Convention, Barcelona, Spain, May 28-31, 2005, pp. 1-13.58Herre, "From Joint Stereo to Spatial Audio Coding-Recent Progress and Standardization," Proc. Of the 7th Int. Conference on Digital Audio Effects (DAFx'04), 2004, pp. 157-162.59Herre, "From Joint Stereo to Spatial Audio Coding—Recent Progress and Standardization," Proc. Of the 7th Int. Conference on Digital Audio Effects (DAFx'04), 2004, pp. 157-162.60ISO/IEC 11172-3, Information Technology-Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1.5 Mbit/s-Part 3: Audio, 154 pp. (1993).61ISO/IEC 11172-3, Information Technology—Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1.5 Mbit/s—Part 3: Audio, 154 pp. (1993).62ISO/IEC 13818-7, Information technology-Generic coding of moving pictures and associated audio information-Part 7: Advanced Audio Coding (AAC), 150 pp. (1997).63ISO/IEC 13818-7, Information technology—Generic coding of moving pictures and associated audio information—Part 7: Advanced Audio Coding (AAC), 150 pp. (1997).64ITU, Recommendation ITU-R BS 1115, Low Bit-Rate Audio Coding, 9 pp. (1994).65ITU, Recommendation ITU-R BS 1387, Method for Objective Measurements of Perceived Audio Quality, 89 pp. (1998).66Iwadare et al., "A 128 kb/s Hi-Fi Audio CODEC Based on Adaptive Transform Coding with Adaptive Block Size MDCT," IEEE. J. Sel. Areas in Comm., pp. 138-144 (Jan. 1992).67Jesteadt et al., "Forward Masking as a Function of Frequency, Masker Level, and Signal Delay," Journal of Acoustical Society of America, 71:950-962 (1982).68Johnston, "Perceptual Transform Coding of Wideband Stereo Signals," Proc. ICASSP, pp. 1993-1996 (May 1989).69Johnston, "Transform Coding of Audio Signals Using Perceptual Noise Criteria," IEEE J. Sel. Areas in Comm., pp. 314-323 (Feb. 1988).70Kadatch, U.S. Appl. No. 09/771,371, entitled, "Quantization Loop with Heuristic Approach," filed Jan. 26, 2001.71Kornagel, "Techniques for artificial bandwidth extension of telephone speech," Signal Processing, vol. 86, No. 6, pp. 1296-1306, Oct. 2005.72Kuo et al., "A Study of Why Cross Channel Prediction is Not Applicable to Perceptual Audio Coding," IEEE Signal Processing Letters, vol. 8, No. 9, 3 pp. (Sep. 2001).73Laaksonen, "Bandwidth extension in high-quality audio coding," Master's Thesis, 69 pp., May 30, 2005.74Lopez et al., "Software Toolbox for Multichannel Sound Reproduction," Proceedings of Digital Audio Effects Conference (DAFX), Barcelona, Spain, Dec. 1998.75Lufti, "Additivity of Simultaneous Masking," Journal of Acoustic Society of America, 73:262-267 (1983).76Lutfi, "Additivity of Simultaneous Masking," Journal of Acoustic Society of America, 73:262-267 (1983).77Mahieux et al., "Transform Coding of Audio Signals at 64 kbits/sec," Proc. Globecom, pp. 405.2.1-405.2.5 (Nov. 1990).78Malvar, "A Modulated Complex Lapped Transform and its Applications to Audio Processing," In Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Phoenix, AZ, May 1999, pp. 1-9.79Malvar, "Biorthogonal and Nonuniform Lapped Transforms for Transform Coding with Reduced Blocking and Ringing Artifacts," appeared in IEEE Transactions on Signal Processing, Special Issue on Multirate Systems, Filter Banks, Wavelets, and Applications, vol. 46, 29 pp. (1998).80Meares, D.J., "Matrixed Surround Sound in an MPEG Digital World," Journal of the Audio Engineering Society, vol. 46, No. 4, 13 pp. (Apr. 1998).81Moriya et al., "Extension and Complexity Reduction of TWINVQ Audio Coder," IEEE ICASSP, pp. 1029-1032 (May 1996).82Najafzadeh-Azghandi et al., "Improving perceptual coding of narrowband audio signals at low rates," Acoustics, Speech, and Signal Processings, IEEE International Conference on Phoenix, AZ, vol. 2, pp. 913-916, Mar. 15, 1999.83Najafzadeh-Azghandi et al., "Perceptual coding of narrowband audio signals at 8 Kbit/s" (1997), available at http://citeseer.ist.psu.edu/najafzadeh-azghandi97perceptual.html.84OPTICOM GmbH, "Objective Perceptual Measurement," 14 pp. [Downloaded from the World Wide Web on Oct. 24, 2001.].85Painter et al., "Perceptual Coding Of Digital Audio," Proceedings Of The IEEE, vol. 88, Issue 4, pp. 451-515, Apr. 2000, available at http://www.eas.asu.edu/~spanias/papers/paper-audio-tedspanias-00.pdf.86Painter et al., "Perceptual Coding Of Digital Audio," Proceedings Of The IEEE, vol. 88, Issue 4, pp. 451-515, Apr. 2000, available at http://www.eas.asu.edu/˜spanias/papers/paper-audio-tedspanias-00.pdf.87Phamdo, "Speech Compression," 13 pp. [Downloaded from the World Wide Web on Nov. 25, 2001.].88Princen et al., "Analysis/Synthesis Filter Bank Design Based on Time Domain Aliasing Cancellation," IEEE Trans. ASSP, pp. 1153-1161 (Oct. 1986).89Purnhagen, "Low Complexity Parametric Stereo Coding in MPEG-4," Proc. Of the 7th Int. Conference on Digital Audio Effects, Oct. 2004, pp. 163-168.90Püschel et al., "The Algebraic Approach to the Discrete Cosine and Sine Transforms and their Fast Algorithms," SIAM Journal of Computing 2003, vol. 32, No. 5, pp. 1280-1316.91Ribas Corbera et al., "Rate Control in DCT Video Coding for Low-Delay Communications," IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, No. 1, pp. 172-185 (Feb. 1999).92Schroder et al., "High Quality Digital Audio Encoding with 3.0 Bits/Semple using Adaptive Transform Coding," Proc. 80th Conv. Aud. Eng. Soc., 8 pp. (Mar. 1986).93Schroeder et al., "Code-excited linear prediction (CELP): High-quality speech at very low bit rates," Proc. IEEE Int. Conf ASSP, pp. 937-940, 1985.94Schroeder, "‘Colorless’ Artificial Reverberation," presented at Audio Engineering Society 12th Annual Meeting, 1960, 18 pages.95Schroeder, "'Colorless' Artificial Reverberation," presented at Audio Engineering Society 12th Annual Meeting, 1960, 18 pages.96Schroeder, "Natural Sounding Artificial Reverberation," presented at the Audio Engineering Society 13th Annual Meeting, 1961, 18 pages.97Schuijers et al., "Low Complexity Parametric Stereo Coding," 116th convention of the AES, May 2004, pp. 1-11.98Schulz, "Improving audio codecs by noise substitution," Journal Of The AES, vol. 44, No. 7/8, pp. 593-598, Jul./Aug. 1996.99Search Report for European Patent Application No. 03 020 110.7.100Search Report for European Patent Application No. 03 020 111.5.101Search Report for PCT/US2004/024935.102Seymour Schlien, "The Modulated Lapped Transform, Its Time-Varying Forms, and Its Application to Audio Coding Standards," IEEE Transactions on Speech and Audio Processing, vol. 5, No. 4, pp. 359-366 (Jul. 1997).103Smith, "Physical Audio Signal Processing: for Virtual Musical Instruments and Digital Audio Effects," (Global Contents-13 pages, Allpass Filters-2 pages, Schroeder Allpass Sections-2 pages, and A Schroeder Reverberator called JCRev-2 pages) of online book at http://ccrma.stanford.edu/~jos/pasp/, Center for Computer Research in Music and Acoustics (CCRMA), Stanford University, printed from internet on Dec. 20, 2005, 19 pages.104Smith, "Physical Audio Signal Processing: for Virtual Musical Instruments and Digital Audio Effects," (Global Contents—13 pages, Allpass Filters—2 pages, Schroeder Allpass Sections—2 pages, and A Schroeder Reverberator called JCRev—2 pages) of online book at http://ccrma.stanford.edu/˜jos/pasp/, Center for Computer Research in Music and Acoustics (CCRMA), Stanford University, printed from internet on Dec. 20, 2005, 19 pages.105Solari, Digital Video and Audio Compression, Title Page, Contents, "Chapter 8: Sound and Audio," McGraw-Hill, Inc., pp. iii, v-vi, and 187-211 (1997).106Soon et al., "Bandwidth Extension of Narrowband Speech Using Soft-decision Vector Quantization," ICICS 2005, pp. 734-738, Bangkok, Thailand (Dec. 2005).107Sporer et al., "The Use of Multirate Filter Banks for Coding of High Quality Digital Audio," 6th European Signal Processing Conference (EUSIPCO), Amsterdam, vol. 1, pp. 211-214, Jun. 1992.108Srinivasan et al., "High-Quality Audio Compression Using an Adaptive Wavelet Packet Decomposition and Psychoacoustic Modeling," IEEE Transactions on Signal Processing, vol. 46, No. 4, pp. 1085-1093 (Apr. 1998).109Stuart et al., "Lossless Compression for DVD-Audio," in AES 9th Regional Convention Tokyo, 4 pp. (1999).110Terhardt, "Calculating Virtual Pitch," Hearing Research, 1:155-182 (1979).111Theile et al., "Low-Bit Rate Coding of High Quality Audio Signals," Proc. AES, 32 pp. (Mar. 1987).112Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall Signal Processing Series, Cover page, pp. 745-751 (1992).113Van Assche et al., "Lossless Compression of Pre-Press Image Using a Novel Color Decorrelation Technique," Proc. SPIE, Very High Resolution and Quality III. vol. 3308, 8 pp. (1998).114Wang et al., "A Multichannel Audio Coding Algorithm for Inter-Channel Redundancy Removal," in AES 110th Convention, Amsterdam, the Netherlands, 6pp. (May 2001).115Wang et al., "EE225a Lecture 13: Karhunen Loève Transform and Discrete Cosine Transform," Department of EECS, University of California at Berkley, 10 pp. (Mar. 2002).116Wragg et al., "An Optimised Software Solution for an ARM Powered(TM) MP3 Decoder," 9 pp. [Downloaded from the World Wide Web on Oct. 27, 2001.].117Wragg et al., "An Optimised Software Solution for an ARM Powered™ MP3 Decoder," 9 pp. [Downloaded from the World Wide Web on Oct. 27, 2001.].118Wright, "Notes on Ogg Vorbis and the MDCT," www.free-comp-shop.com, 7 pp. (May 2003).119Yang et al., "Adaptive Karhunen-Loeve Transform for Enhanced Multichannel Audio Coding," Proc. SPIE vol. 4475, 13 pp., Mathematics of Data/Image Coding, Compression, and Encryption IV San Diego, CA. (Jul. 29-Aug. 3, 2001).120Yang et al., "An Inter-Channel Redundancy Removal Approach for High-Quality Multichannel Audio Compression," in AES 109th Convention, Los Angeles, California, 8 pp. (Sep. 2000).121Zwicker et al., Das Ohr als Nachrichtenempfänger, Title Page, Table of Contents, "I: Schallschwingungen," Index, Hirzel-Verlag, Stuttgart, pp. III, IX-XI, 1-26, and 231-232 (1967).122Zwicker, Psychoakustik, Title Page, Table of Contents, "Teil I: Einfuhrung," Index, Springer-Verlag, Berlin Heidelberg, New York, pp. II, IX-XI, 1-30, and 157-162 (1982).Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8190440 *Feb 27, 2009May 29, 2012Broadcom CorporationSub-band codec with native voice activity detectionUS8374883 *Oct 31, 2008Feb 12, 2013Panasonic CorporationEncoder and decoder using inter channel prediction based on optimally determined signalsUS8532982 *Jul 14, 2009Sep 10, 2013Samsung Electronics Co., Ltd.Method and apparatus to encode and decode an audio/speech signalUS9355646Sep 6, 2013May 31, 2016Samsung Electronics Co., Ltd.Method and apparatus to encode and decode an audio/speech signalUS20090089479 *Feb 19, 2008Apr 2, 2009Samsung Electronics Co., Ltd.Method of managing memory, and method and apparatus for decoding multi-channel dataUS20090222264 *Feb 27, 2009Sep 3, 2009Broadcom CorporationSub-band codec with native voice activity detectionUS20100010807 *Jul 14, 2009Jan 14, 2010Eun Mi OhMethod and apparatus to encode and decode an audio/speech signalUS20100250244 *Oct 31, 2008Sep 30, 2010Panasonic CorporationEncoder and decoderUS20130064383 *Nov 9, 2012Mar 14, 2013Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V.Information signal representation using lapped transform* Cited by examinerClassifications U.S. Classification704/500, 704/504, 704/200.1, 704/E19.005, 704/230, 704/501, 704/503, 704/E19.015International ClassificationG10L19/02Cooperative ClassificationG10L19/02, G10L19/008, G10L19/002European ClassificationG10L19/02Legal EventsDateCodeEventDescriptionAug 25, 2014FPAYFee paymentYear of fee payment: 4Dec 9, 2014ASAssignmentOwner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTONFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034542/0001Effective date: 20141014RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services