Source: https://patents.google.com/patent/US20090225991A1/en
Timestamp: 2018-10-16 02:42:43
Document Index: 696904300

Matched Legal Cases: ['art 110', 'art 110', 'art 120', 'art 112', 'art 130', 'art 512']

US20090225991A1 - Method and Apparatus for Decoding an Audio Signal - Google Patents
US20090225991A1
US20090225991A1 US11915319 US91531906A US2009225991A1 US 20090225991 A1 US20090225991 A1 US 20090225991A1 US 11915319 US11915319 US 11915319 US 91531906 A US91531906 A US 91531906A US 2009225991 A1 US2009225991 A1 US 2009225991A1
US11915319
US8917874B2 (en )
Method and apparatus for processing audio signals are provided. The method for decoding an audio signal includes extracting a downmix signal and spatial information from a received audio signal, generating surround converting information using the spatial information and rendering the downmix signal to generate a pseudo-surround signal in a previously set rendering domain, using the surround converting information. The apparatus for decoding an audio signal includes a demultiplexing part extracting a downmix signal and spatial information from a received audio signal, an information converting part generating surround converting information using the spatial information and a pseudo-surround generating part rendering the downmix signal to generate a pseudo-surround signal in a previously set rendering domain, using the surround converting information.
The psycho-acoustic model is a method to efficiently reduce amount of data as signals, which are not necessary in an encoding process, are removed, using a principle of human being's sound recognition manner. For example, human ears cannot recognize quiet sound immediately after loud sound, and also can hear only sound whose frequency is between 20˜20,000 Hz.
“Spatial information” in the present invention is indicative of information required to generate multi-channels by upmixing downmixed signal. Although the present invention will be described assuming that the spatial information is spatial parameters, it will be easily appreciated that the spatial information is not limited by the spatial parameters. Here, the spatial parameters include a Channel Level Differences (CLDs), Inter-Channel Coherences (ICCs), and Channel Prediction Coefficients (CPCs), etc. The Channel Level Difference (CLD) is indicative of an energy difference between two channels. The Inter-Channel Coherence (ICC) is indicative of cross-correlation between two channels. The Channel Prediction Coefficient (CPC) is indicative of a prediction coefficient to predict three channels from two channels.
“Core codec” in the present invention is indicative of a codec for coding an audio signal. The Core codec does not code spatial information. The present invention will be described assuming that a downmix audio signal is an audio signal coded by the Core codec. Also, the core codec may include Moving Picture Experts Group (MPEG) Layer-II, MPEG Audio Layer-III (MP3), AC-3, Ogg Vorbis, DTS, Window Media Audio (WMA), Advanced Audio Coding (AAC) or High-Efficiency AAC (HE-AAC). However, the core codec may not be provided. In this case, an uncompressed PCM signals is used. The codec may be conventional codecs and future codecs, which will be developed in the future.
“Channel splitting part” is indicative of a splitting part which can divide a particular number of input channels into another particular number of output channels, in which the output channel numbers are different from those of the input channels. The channel splitting part includes a two to three (TTT) box, which converts the two input channels to three output channels. Also, the channel splitting part includes a one to two (OTT) box, which converts the one input channel to two output channels. The channel splitting part of the present invention is not limited by the TTT and OTT boxes, rather it will be easily appreciated that the channel splitting part may be used in systems whose input channel number and output channel number are arbitrary.
When the N multi-channel audio signals X1, X2, . . . , XN are inputted the downmixing part 110 generates audio signals, depending on a certain downmixing method or an arbitrary downmix method. Here, the number of the audio signals outputted from the downmixing part 110 to the core encoding part 120 is less than the number “N” of the input multi-channel audio signals. The spatial information estimating part 112 extracts spatial information from the input multi-channel audio signals, and then transmits the extracted spatial information to the multiplexing part 130. Here, the number of the downmix channel may one or two, or be a particular number according to downmix commands. The number of the downmix channels may be set. Also, an arbitrary downmix signal is optionally used as the downmix audio signal.
The following is a description for an audio signal structure 140 according to an embodiment of the present invention, as shown in FIG. 1. When the audio signal is transmitted on the basis of a payload, it may be received through each channel or a single channel. An audio payload of 1 frame is composed of a coded audio data field and an ancillary data field. Here, the ancillary data field may include coded spatial information. For example, if a data rate of an audio payload is at 48˜128 kbps, the data rate of spatial information may be at 5˜32 kbps. Such an example will not limit the scope of the present invention.
Referring to FIG. 4, it is defined that the inputted stereo downmix signal is denoted by “x”, channel mapping coefficient, which is obtained by mapping spatial information to channel, is denoted by “D”, a proto-type HRTF filter coefficient of an external input is denoted by “G”, a temporary multi-channel signal is denoted by “p”, and an output signal which has undergone rendering is denoted by “y”. The notations “x”, “D”, “G”, “p”, and “y” may be expressed by a matrix form as following Equation 1. Equation 1 is expressed on the basis of the proto-type HRTF filter coefficient. However, when a modified HRTF filter coefficient is used in the following Equations, G must be replaced with G′ in the following Equations.
x = [ Li Ri ] ,  p = [ L Ls R Rs C LFE ] ,  D = [ D_L   1 D_L   2 D_Ls   1 D_Ls   2 D_R   1 D_R   2 D_Rs   1 D_Rs   2 D_C   1 D_C   2 D_LFE   1 D_LFE   2 ] ,  G = [  GL_L GLs_L GR_L GRs_L GC_L GLFE_L GL_R GLs_R GR_R GRs_R GC_R GLFE_R  ]  y = [ Lo Ro ] [ Equation   1 ]
Here, when each coefficient is a value of a frequency domain, the temporary multi-channel signal “p” may be expressed by the product of a channel mapping coefficient “D” by a stereo downmix signal “x” as the following Equation 2.
p = D · x ,  [ L Ls R Rs C LFE ] = [ D_L   1 D_L   2 D_Ls   1 D_Ls   2 D_R   1 D_R   2 D_Rs   1 D_Rs   2 D_C   1 D_C   2 D_LFE   1 D_LFE   2 ]  [ Li Ri ] [ Equation   2 ]
After that, the output signal “y” may be expressed by Equation 3, when rendering the temporary multi-channel “p” using the proto-type HRTF filter coefficient “G”.
y=G·p
Then, “y” may be expressed by Equation 4 if p=D·X is inserted.
Here, if H=GD is defined, the output signal “y” and the stereo downmix signal “x” have a relationship as following Equation 5.
H = [ HL_L HR_L HL_R HR_R ] ,  y = Hx [ Equation   5 ]
Therefore, the product of the filter coefficients allows “H” to be obtained. After that, the output signal “y” may be acquired by multiplying the stereo downmix signal “x” and the “H”.
H =  GD =  [  GL_L GLs_L GR_L GRs_L GC_L GLFE_L GL_R GLs_R GR_R GRs_R GC_R GLFE_R  ]  [ D_L   1 D_L   2 D_Ls   1 D_Ls   2 D_R   1 D_R   2 D_Rs   1 D_Rs   2 D_C   1 D_C   2 D_LFE   1 D_LFE   2 ] [ Equation   6 ]
According to the first method, the third rendering part 512 (for example, a HRTF filter) does not use a filter coefficient for a pseudo-surround sound but uses a value used when processing stereo downmix. Here, the value used when processing the stereo downmix may be coefficients (left front=1, right front=0, . . . , etc.), where the coefficient “left front” is for left output, and the coefficient “right front” is for right output.
Referring to FIG. 5, it is defined that the input mono downmix signal is denoted by “x”, a channel mapping coefficient is denoted by “D”, a proto-type HRTF filter coefficient of an external input is denoted by “G”, a temporary multi-channel signal is denoted by “p”, and an output signal which has undergone rendering is denoted by “y”, the notations “x”, “D”, “G”, “p”, and “y” may be expressed by a matrix form as following Equation 7.
x = [ Mi ] ,  p = [ L Ls R Rs C LFE ] ,  D = [ D_L D_Ls D_R D_Rs D_C D_LFE ]   G = [  GL_L GLs_L GR_L GRs_L GC_L GLFE_L GL_R GLs_R GR_R GRs_R GC_R GLFE_R  ]  ,   y = [ Lo Ro ] [ Equation   7 ]
Referring to FIG. 6, multi-channel signals L, R, C, LFE, Ls, Rs may be generated from the downmix signal “m”, using the OTT boxes 601, 602, 603, 604, 605 and spatial information, for example, CLD0, CLD1, CLD2, CLD3, CLD4, ICC0, ICC1, ICC2, ICC3, etc. For example, when the tree structure has 5151 configuration as shown in FIG. 6, the channel mapping output values may be obtained, using CLD only, as shown in Equation 8.
[ L R C LFE Ls Rs ] = [ D L D R D C D LFE D Ls D Rs ]  m = [ c 1 , O   T   T   3  c 1 , O   T   T   1  c 1 , O   T   T   0 c 2 , O   T   T   3  c 1 , O   T   T   1  c 1 , O   T   T   0 c 1 , O   T   T   4  c 2 , O   T   T   1  c 1 , O   T   T   0 c 2 , O   T   T   4  c 2 , O   T   T   1  c 1 , O   T   T   0 c 1 , O   T   T   2  c 2 , O   T   T   0 c 2 , O   T   T   2  c 2 , O   T   T   0 ]  m   Where ,  c  ? = 10  ? 1 + 10 C   L   D x ? 10 ,  c  ? = 1 1 + 10  ?   ?  indicates text missing or illegible when filed [ Equation   8 ]
Referring to FIG. 7, multi-channel signals L, Ls, R, Rs, C, LFE may be generated from the downmix signal “m”, using the OTT boxes 701, 702, 703, 704, 705 and spatial information, for example, CLD0, CLD1, CLD2, CLD3, CLD4, ICC0, ICC1, ICC3, ICC4, etc.
[ L Ls R Rs C LFE ] = [ D L D Ls D R D Rs D C D LFE ]  m = [ c 1 , O   T   T   3  c 1 , O   T   T   1  c 1 , O   T   T   0 c 2 , O   T   T   3  c 1 , O   T   T   1  c 1 , O   T   T   0 c 1 , O   T   T   4  c 2 , O   T   T   1  c 1 , O   T   T   0 c 2 , O   T   T   4  c 2 , O   T   T   1  c 1 , O   T   T   0 c 1 , O   T   T   2  c 2 , O   T   T   0 c 2 , O   T   T   2  c 2 , O   T   T   0 ]  m [ Equation   9 ]
In order to perform pseudo-surround rendering, a signal from a left channel source “L” 810 is filtered by a filter having a filter coefficient GL_L, and then the filtering result L*GL_L is transmitted as the left output. Also, a signal from the left channel source “L” 810 is filtered by a filter having a filter coefficient GL_R, and then the filtering result L*GL_R is transmitted as the right output. For example, the left and right outputs may attain to left and right ears of user, respectively. Like this, all left and right outputs are obtained by channels. Then, the obtained left outputs are summed to generate a final left output (for example, Lo), and the obtained right outputs are summed to generate a final right output (for example, Ro). Therefore, the final left and right outputs which have undergone pseudo-surround rendering may be expressed by following Equation 10.
Here, HM_L(n+j) and HM_R(n+j) are indicative of coefficients obtained by interpolating filter coefficient for pseudo-surround rendering, when a mono downmix signal is input. Also, HL_L(n+j), HR_L(n+j), HL_R(n+j) and HR_R(n+j) are indicative of coefficients obtained by interpolating filter coefficient for pseudo-surround rendering, when a stereo downmix signal is input. Here, ‘j’ and ‘k’ are integers, 0<j<k. Also, ‘a’ is a real number (0<a<1) and expressed by following Equation 15.
a=j/k
Equation 16 describes blurring through a 1-pole IIR filter, in which the blurring results may be obtained, as follows. That is, the filter coefficients HM_L(n) and HM_R(n) in the present block (n) are multiplied by “b”, respectively. And then, the filter coefficients HM_L(n−1)′ and HM_R(n−1)′ in the previous block (n−1) are multiplied by (1−b), respectively. The multiplying results are added as shown in Equation 16. Here, “b” is a constant (0<b<1). The smaller the value of “b” the more the blurring effect is increased. On the contrary, the larger the value of “b”, the less the blurring effect is increased. Similar to the above methods, the blurring of remaining filter coefficients may be performed.
1. A method for decoding an audio signal, the method comprising—:
2. The method of claim 1, further comprising—converting the pseudo-surround signal of the rendering domain to a pseudo-surround signal of an output domain.
the rendering domain includes at least one of frequency domain and time domain;
the frequency domain includes at least one of subband domain and discrete frequency domain; and
the subband domain includes at least one of simple subband domain and hybrid subband domain.
converting the downmix signal of a
wherein a downmix domain to the downmix signal of the previously set rendering domain when the downmix domain is different from the previously set rendering domain.
5. The method of claim 4, wherein the converting the downmix signal of the downmix domain—comprises at least one of the operations:
converting the downmix signal of a time domain into the downmix signal of the previously set rendering domain when the downmix domain is the time domain;
converting the downmix signal of a discrete frequency domain into the downmix signal of the previously set rendering domain when the downmix domain is the discrete frequency domain; and
converting the downmix signal of the discrete frequency domain into the downmix signal of the time domain, and then the downmix signal of the converted time domain into the downmix signal of the previously set rendering domain, when the downmix domain is the discrete frequency domain.
6. The method of claim 1, wherein the previously set rendering domain is a subband domain and the downmix signal comprises a first signal and a second signal, and the rendering of the downmix signal comprises:
applying the surround converting information to the first signal;
applying the surround converting information to the second signal; and
adding the first signal to the second signal.
7. The method of claim 1, wherein the surround converting information is generated using the spatial information and filter information.
generating channel mapping information by mapping the spatial information by channels;
generating the surround converting information using the channel mapping information and a filter information.
9. The method of claim 1, wherein the generating of the surround converting information comprises:
generating channel coefficient information using the spatial information and filter information; and,
generating the surround converting information using the channel coefficient information.
10. The method of claim 1, wherein the generating of the surround converting information comprises:
generating channel coefficient information using the channel mapping information and filter information; and
receiving the audio signal including the downmix signal and the spatial information,
wherein the downmix signal and the spatial information are extracted from the audio signal.
12. The method of claim 1, wherein the spatial information includes at least one of a channel level difference and an inter channel coherence.
a downmix signal which is generated by downmixing the audio signal having a plurality of channels; and
spatial information which is generated while the downmix signal is generated,
wherein the spatial information is converted to surround converting information, and the downmix signal is rendered to be converted to a pseudo-surround signal with the surround converting information being used, in a previously set rendering domain.
18. An apparatus for decoding an audio signal, the apparatus comprising:
an information converting part generating surround converting information using the spatial information; and
a pseudo-surround generating part rendering the downmix signal to generate a pseudo-surround signal in a previously set rendering domain, using the surround converting information.
19. The apparatus of claim 18, wherein the pseudo-surround generating part comprises an output domain converting part converting the pseudo-surround signal of the previously set rendering domain to a pseudo-surround signal of an output domain.
the previously set rendering domain includes at least one of frequency domain and time domain;
21. The apparatus of claim 18, wherein the pseudo-surround generating part comprises:
a rendering domain converting part converting the downmix signal of a downmix domain to the downmix signal of the previously set rendering domain when the downmix domain is different from the previously set rendering domain.
22. The apparatus of claim 21 wherein the rendering domain converting part comprises at least one of:
a first domain converting part converting the downmix signal of a time domain into the downmix signal of the previously set rendering domain when the downmix domain is the time domain;
a second domain converting part converting the downmix signal of a discrete frequency domain into the downmix signal of the previously set rendering domain when the downmix domain is the discrete frequency domain; and
a third domain converting part converting the downmix signal of the discrete frequency domain into the downmix signal of the time domain and then the downmix signal of the converted time domain into the downmix signal of the previously set rendering domain, when the downmix domain is the discrete frequency domain.
23. The apparatus of claim 18, wherein the previously set rendering domain is a subband domain and the downmix signal comprises a first signal and a second signal, and
the pseudo-surround generating part applies the surround converting information to the first signal applies the surround converting information to the second signal; and, adding the first signal to the second signal.
24. The apparatus of claim 18, wherein the surround converting information is generated using the spatial information and filter information.
a channel mapping part generating channel mapping information by mapping the spatial information by channels;
a coefficient generating part generating channel coefficient information from the channel mapping information and filter information; and,
a integrating part generating the surround converting information from the channel coefficient information.
28. The apparatus of claim 18, wherein the demultiplexing part receives the audio signal including the downmix signal and the spatial information, wherein the downmix signal and the spatial information are extracted from the audio signal.
US11915319 2005-05-26 2006-05-25 Method and apparatus for decoding an audio signal Active 2031-03-04 US8917874B2 (en)
US68457905 true 2005-05-26 2005-05-26
US60684579 2005-05-26
US60759980 2006-01-19
US60776724 2006-02-27
US60779441 2006-03-07
US60779442 2006-03-07
US60779417 2006-03-07
US1020060030670 2006-04-04
KR20060030670A KR20060122695A (en) 2005-05-26 2006-04-04 Method and apparatus for decoding audio signal
KR10-2006-0030670 2006-04-04
US11915319 US8917874B2 (en) 2005-05-26 2006-05-25 Method and apparatus for decoding an audio signal
PCT/KR2006/001987 WO2006126844A8 (en) 2005-05-26 2006-05-25 Method and apparatus for decoding an audio signal
PCT/KR2006/001987 A-371-Of-International WO2006126844A8 (en) 2005-05-26 2006-05-25 Method and apparatus for decoding an audio signal
US14558649 Continuation US9595267B2 (en) 2005-05-26 2014-12-02 Method and apparatus for decoding an audio signal
US20090225991A1 true true US20090225991A1 (en) 2009-09-10
US8917874B2 US8917874B2 (en) 2014-12-23
ID=37452464
US11915327 Active 2027-11-08 US8577686B2 (en) 2005-05-26 2006-05-25 Method and apparatus for decoding an audio signal
US11915319 Active 2031-03-04 US8917874B2 (en) 2005-05-26 2006-05-25 Method and apparatus for decoding an audio signal
US11915329 Active 2027-08-13 US8543386B2 (en) 2005-05-26 2006-05-26 Method and apparatus for decoding an audio signal
US (3) US8577686B2 (en)
EP (3) EP1899958B1 (en)
WO (3) WO2006126843A3 (en)
US20130022205A1 (en) * 2006-03-07 2013-01-24 Samsung Electronics Co., Ltd Binaural decoder to output spatial stereo sound and a decoding method thereof
US10091603B2 (en) 2017-12-15 2018-10-02 Dolby International Ab Binaural multi-channel decoder in the context of non-energy-conserving upmix rules
US9213703B1 (en) * 2012-06-26 2015-12-15 Google Inc. Pitch shift and time stretch resistant audio matching
US9264809B2 (en) * 2014-05-22 2016-02-16 The United States Of America As Represented By The Secretary Of The Navy Multitask learning method for broadband source-location mapping of acoustic sources
US9071920B2 (en) * 2006-03-07 2015-06-30 Samsung Electronics Co., Ltd. Binaural decoder to output spatial stereo sound and a decoding method thereof
US10097941B2 (en) 2017-12-20 2018-10-09 Dolby International Ab Binaural multi-channel decoder in the context of non-energy-conserving upmix rules
US10097940B2 (en) 2017-12-20 2018-10-09 Dolby International Ab Binaural multi-channel decoder in the context of non-energy-conserving upmix rules
EP1905002A2 (en) 2008-04-02 application
US8917874B2 (en) 2014-12-23 grant
WO2006126843A2 (en) 2006-11-30 application
US20080294444A1 (en) 2008-11-27 application
US8543386B2 (en) 2013-09-24 grant
EP1899958A2 (en) 2008-03-19 application
WO2006126855A2 (en) 2006-11-30 application
WO2006126844A3 (en) 2007-02-01 application
EP1905003A2 (en) 2008-04-02 application
EP1905003B1 (en) 2013-05-22 grant
WO2006126843A3 (en) 2007-03-08 application
EP1899958A4 (en) 2011-03-09 application
EP1905002B1 (en) 2013-05-22 grant
US8577686B2 (en) 2013-11-05 grant
WO2006126844A2 (en) 2006-11-30 application
WO2006126844A8 (en) 2008-01-03 application
WO2006126855A3 (en) 2007-01-11 application
EP1899958B1 (en) 2013-08-07 grant
US20080275711A1 (en) 2008-11-06 application
EP1905002A4 (en) 2011-03-09 application
EP1905003A4 (en) 2011-03-30 application
US20070071247A1 (en) 2007-03-29 Slot position coding of syntax of spatial audio application
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:OH, HYEN-O;PANG, HEE SUK;KIM, DONG SOO;AND OTHERS;REEL/FRAME:021582/0132