Source: https://patents.google.com/patent/JP4988717B2/en
Timestamp: 2019-10-23 19:26:27
Document Index: 35404340

Matched Legal Cases: ['art 110', 'art 120', 'art 130', 'art 111', 'art 112', 'art 160', 'art 170', 'art 180', 'art 200', 'art 190', 'art 300', 'art 230', 'art 180', 'art 310', 'art 320', 'art 330', 'art 340', 'art 414', 'art 512', 'art 511', 'art 513', 'art: 900', 'art: 900', 'art: 900', 'art 900', 'art 910', 'art 920', 'art 930', 'art: 900', 'art) 900', 'art) 900', 'art: 1000', 'art: 1000', 'art: 1000', 'art 1000', 'art 1020', 'art 1010', 'art 1010', 'art 1030']

JP4988717B2 - Audio signal decoding method and apparatus - Google Patents
Audio signal decoding method and apparatus Download PDF
JP4988717B2
JP4988717B2 JP2008513375A JP2008513375A JP4988717B2 JP 4988717 B2 JP4988717 B2 JP 4988717B2 JP 2008513375 A JP2008513375 A JP 2008513375A JP 2008513375 A JP2008513375 A JP 2008513375A JP 4988717 B2 JP4988717 B2 JP 4988717B2
JP2008513375A
JP2009501457A (en
ジェヒョン イム
ヒョンオ オ
ドンス キム
ヤンウォン ジョン
ヒソク パン
2005-05-26 Priority to US60/684,579 priority
2006-04-04 Priority to KR10-2006-0030670 priority
2006-05-25 Priority to PCT/KR2006/001987 priority patent/WO2006126844A2/en
2006-05-25 Application filed by エルジー エレクトロニクス インコーポレイティド filed Critical エルジー エレクトロニクス インコーポレイティド
2009-01-15 Publication of JP2009501457A publication Critical patent/JP2009501457A/en
2012-08-01 Publication of JP4988717B2 publication Critical patent/JP4988717B2/en
The present invention relates to audio signal processing, and more particularly, to an audio signal processing method and apparatus for generating a virtual surround signal (Pseudo surround signal).
In recent years, various coding technologies and methods for digital audio signals have been developed, and related products have been produced. In addition, a coding method for a multi-channel audio signal has been developed using a psychoacoustic model, and standardization work for this method is in progress.
According to the psychoacoustic model, a method in which a human recognizes a voice, for example, a small voice following a loud voice cannot be heard, and only a voice corresponding to a frequency of 20 Hz to 20000 Hz can be heard. By removing the signal for the part, it is possible to effectively reduce the amount of data required.
However, a processing method for an audio signal for generating a virtual surround signal from an audio bitstream including spatial information has not been specifically presented, and there have been many problems in efficiently processing an audio signal.
The present invention is to solve the above-described problems, and an object of the present invention is to provide an audio signal processing method and apparatus for providing a virtual three-dimensional sound effect (Pseudo surround effect) in an audio system.
According to an embodiment of the present invention, a step of extracting a downmix signal and spatial information from a received audio signal, a step of generating surround conversion information using the spatial information, and a preset rendering domain And a method of rendering a virtual surround signal by rendering the downmix signal using the surround conversion information.
According to another embodiment of the present invention, a demultiplexing unit that extracts a downmix signal and spatial information from a received audio signal, an information conversion unit that generates surround conversion information using the spatial information, Provided with an audio signal decoding device, comprising: a virtual surround generation unit that generates a virtual surround signal by rendering the downmix signal using the surround conversion information on a set rendering domain Is done.
According to still another embodiment of the present invention, a downmix signal downmixed from an audio signal having a plurality of channels, and spatial information generated when the downmixing is performed, wherein the spatial information is included. Is a data structure of an audio signal that is converted into surround conversion information and uses the surround conversion information on a preset rendering domain, and the downmix signal is rendered and converted into a virtual surround signal. Is provided.
According to still another embodiment of the present invention, a medium including an audio signal includes a downmix signal downmixed from an audio signal having a plurality of channels, and spatial information generated in the downmix process. Here, the spatial information is converted into surround conversion information, and the surround conversion information is used on a preset rendering domain, and the downmix signal is rendered and converted into a virtual surround signal. There is provided a medium characterized by comprising.
According to the audio signal decoding method and apparatus of the present invention, an audio bitstream generated by downmixing multichannels to generate a downmix channel and extracting spatial information of the multichannels. When the decoding apparatus that receives the signal is not in an environment capable of generating a multi-channel, it is possible to perform decoding so as to have a virtual surround effect (Pseudo surround effect).
The terms used in the present invention are general terms that are widely used as much as possible. However, in certain cases, there are terms arbitrarily selected by the applicant. Since the meaning is described in detail in the explanation part, the present invention must be grasped not by a simple term name but by the meaning of the term.
In the present invention, “spatial information” means information for generating a multi-channel by performing an up-mix on a down-mixed signal. To do. Here, the spatial information is described as a spatial parameter, but the present invention is not limited to this. This spatial parameter includes CLD (channel level difference), which means the energy difference between the two channels, and ICC (inter channel coordinates), which means the correlation between the two channels, and three channels from the two channels. There are CPC (channel prediction coefficients) which are prediction coefficients used.
In the present invention, the “core codec” refers to a codec that codes an audio signal that is not spatial information. In the present invention, an audio signal that is not spatial information will be described as a downmix audio signal. Also, the core codec can include MPEG Layer-II, MP3, OggVorbis, AC-3, DTS, WMA, AAC or HE-AAC. On the other hand, an uncompressed PCM signal may be used instead of the core codec. If a codec function is performed on an audio signal, not only an already developed codec but also a codec to be developed in the future can be included.
In the present invention, the “channel splitting part” means a splitting part that splits a specific number of input channels into a number of specific output channels different from the number of input channels. When there are two input channels, the channel division unit converts a TTT (two to three: hereinafter referred to as “TTT”) box to convert the output channel into three, or an input channel. If there is one, it includes an OTT (one to two: hereinafter referred to as “OTT”) box for converting the output channel into two. However, it is obvious that the channel division unit of the present invention is not limited to the TTT box and the OTT box, and can be applied to any case where the number of input channels and output channels is arbitrary.
FIG. 1 is a diagram illustrating a signal processing system according to an embodiment of the present invention. Referring to FIG. 1, the signal processing system includes an encoding device 100 and a decoding device 150. However, although an audio signal will be described here, it is obvious that the present invention can be applied to any signal processing besides an audio signal.
The encoding apparatus 100 includes a downmixing part 110, a core encoding part 120, and a multiplexing part 130. The downmixing unit 110 includes a channel downmixing part 111 and a spatial information extracting part 112.
When the audio signal is input to the N multi-channels X 1 , X 2 ,..., X 3 , the downmixing unit 110 performs a predetermined downmix method or an arbitrarily set downmix method (artistic downmix method). Thus, an audio signal of a channel smaller than the number of input channels is output, and the output signal is input to the core encoding unit 120. On the other hand, the spatial information extraction unit 112 extracts spatial information from the multichannel, and transmits the extracted spatial information to the multiplexing unit 130. Here, the downmix channel may have one channel or two channels, or may have a specific number of channels according to a downmix command. In this case, the number of downmix channels can be set. In addition, it is apparent that an artistic downmix signal can be used as the downmix audio signal.
The core encoding unit 120 encodes the downmix audio signal transferred through the downmix channel. The encoded downmix audio signal is input to the multiplexing unit 130.
The multiplexing unit 130 multiplexes the downmix audio signal and the spatial information to generate a bit stream, and transmits the generated bit stream to the decoding apparatus 150. At this time, the bitstream may include a core codec bitstream and a spatial information bitstream.
The decoding apparatus 150 includes a demultiplexing part 160, a core decoding part 170, and a virtual surround decoding part 180. The virtual surround decoding unit 180 may include a virtual surround generation part 200 and an information conversion unit 300. The decoding apparatus 150 may further include a spatial information decoding part 190. The demultiplexer 160 receives the bitstream and demultiplexes the received bitstream into a core codec bitstream and a spatial information bitstream. Further, the demultiplexer 160 can extract a downmix signal and spatial information from the received bitstream.
The core decoding unit 170 receives the core codec bitstream from the demultiplexing unit 160 and outputs a decoded downmix signal. For example, when the multi-channel is downmixed by the encoding apparatus, if the downmix signal is downmixed to a mono channel or a stereo channel, the decoded downmix signal can be a mono channel or a stereo channel signal. However, although the embodiments of the present invention will be described based on a mono channel or a stereo channel used as a downmix channel, the number of downmix channels is not limited.
The spatial information decoding unit 190 can receive the spatial information bitstream from the demultiplexing unit 160 and decode the spatial information bitstream to generate spatial information.
The virtual surround decoding unit 180 generates a virtual surround signal from the downmix signal using spatial information. Hereinafter, the information conversion unit 300 and the virtual surround generation unit 200 provided in the virtual surround decoding unit 180 will be described.
An information converting part 300 receives spatial information and receives filter information. Further, surround conversion information in a form that can be applied to generation of a virtual surround signal is generated using the filter information and the spatial information. The surround conversion information means a filter coefficient when the virtual surround generation unit 200 is a specific filter. Therefore, although the present invention will be described using filter coefficients as surround transform information, the present invention is not limited to the filter coefficients. Here, as an example of the filter information, HRTF (head-related transfer functions) may be mentioned, but the present invention is not limited to this.
In the present invention, a filter coefficient means a coefficient of a specific filter. For example, the filter coefficients can be named as follows: The original HRTF filter coefficient (proto-type HRTF filter coefficient) means an original filter coefficient of a specific HRTF filter and can be expressed by GL_L or the like. The modified HRTF filter coefficient (converted HRTF filter coefficient) means a filter coefficient after the original HRTF filter coefficient is modified, and can be expressed by GL_L ′ or the like. A spatialized HRTF filter coefficient (spatialized HRTF filter coefficient) means a filter coefficient obtained by spatializing an original HRTF filter coefficient for generating a virtual surround signal, and can be expressed by FL_L1 or the like. The master rendering coefficient means a filter coefficient necessary for rendering, and can be expressed by HL_L or the like. The interpolated master rendering coefficient means a filter coefficient obtained by interpolating and / or blurring the master rendering coefficient, and can be expressed by HL_L ′ or the like. However, it is obvious that the present invention is not limited to the names of the filter coefficients.
The virtual surround generation unit 200 receives the downmix signal decoded from the core decoding unit 170, receives the surround conversion information from the information conversion unit 300, and uses the decoded downmix signal and the surround conversion information. To generate a virtual surround signal. For example, a virtual surround signal is a signal that provides a virtual stereophonic effect in an audio system having only stereo devices. At this time, it is obvious that the present invention is not limited to an audio system having only a device whose output signal is stereo, and can be applied to other devices. Rendering performed by the virtual surround generation unit 200 can be performed in various ways according to a set mode.
As described above, according to the present invention, the encoding apparatus 100 does not transfer the multi-channel audio signal as it is, but transfers it down-mixed to a stereo or mono audio signal and transfers the spatial information of the multi-channel audio signal together. In this case, since the decoding apparatus 150 includes the virtual surround decoding unit 180 according to the present invention, the user can experience a virtual multi-channel effect even when the output channel is not a multi-channel but a stereo channel. This is a very good method.
An example of the audio signal structure 140 according to the present invention will be described. When the audio signal is transferred based on one payload, the audio signal may be received through each channel or may be received through one channel. . An audio payload 1 frame includes a field including coded audio data and an additional data field. Here, the additional data field may include coded spatial information. For example, when the audio payload is 48 to 128 kbps, the spatial information can have a range of about 5 to 32 kbps, but is not limited thereto.
FIG. 2 is a schematic block diagram of the virtual surround generator 200 according to an embodiment of the present invention.
In the present invention, the domain is a downmix domain where the downmix signal is decoded, a spatial information domain where the spatial information is processed to generate surround conversion information, and the downmix signal is rendered using the spatial information. A rendering domain and an output domain that outputs a virtual surround signal in the time domain. Here, the output domain is a domain of an audio signal that can be heard by humans, and means a time domain. The virtual surround generation unit 200 includes a rendering unit 220 and an output domain converting part 230. Further, when the downmix domain and the rendering domain are different from each other, the rendering domain converting unit 210 may perform domain conversion so that the downmix domain matches the rendering domain.
For example, the rendering domain conversion unit 210 performs domain conversion to match the rendering domain and the downmix domain. The domain method performed by the rendering domain conversion unit 210 will be described. The following first, second, and third methods are possible. Here, the rendering domain is set to the subband domain, but the present invention is not limited to this. The first method is to convert the time domain to the rendering domain if the downmix domain is the time domain. The second method is to convert the discrete frequency domain into a rendering domain when the downmix domain is a discrete frequency domain. A third method, when the downmix domain is a discrete frequency domain, after converting the the discrete frequency domain to the time domain, is to convert the rendering domain.
The rendering unit 220 performs virtual surround rendering of the downmix signal using the surround conversion information, and generates a virtual surround signal. At this time, when the output unit is a stereo channel, the virtual surround signal becomes a virtual surround stereo output having virtual stereophonic sound. Further, since the virtual surround signal output from the rendering unit 220 is a signal on the rendering domain, domain conversion is required when the rendering domain is not the time domain. Here, the output part (output part) of the virtual surround decoding part 180 is a stereo channel. However, in the present invention, the output part can be applied regardless of the number of channels.
For example, the virtual surround rendering method includes HRTF filtering performed by an HRTF (head-related transfer functions: hereinafter referred to as 'HRTF') filter. In this case, the spatial information may be a value that can be applied in a hybrid filterbank domain defined by MPEG Surround. The virtual surround rendering method can be implemented in the following embodiments depending on the domain. For this reason, it is necessary to match the downmix domain and the spatial information domain to the rendering domain.
The first embodiment is a method of performing virtual surround rendering on the downmix signal in the subband domain (QMF). The subband domain includes a simple subband domain and a hybrid domain. For example, when the downmix signal is a PCM signal and the downmix domain is not a subband domain, the domain conversion is performed from the rendering domain converter 210 to the subband domain, and the downmix signal is a subband domain. There is no need to perform domain conversion. If necessary, it is necessary to delay one of them in order to match the applied frame between the downmix signal and the spatial information. At this time, if the spatial information domain is a subband domain, conversion to the spatial information domain is not necessary. In addition, in order to generate a virtual surround signal on the time domain, the output domain converter 230 needs to convert the rendering domain into the time domain.
The second embodiment is a method of performing virtual surround rendering in a discrete frequency domain on a downmix signal. Here, the discrete frequency domain means a frequency domain other than the subband domain. For example, when the downmix domain is not a discrete frequency domain, the rendering domain conversion unit 210 performs domain conversion to the discrete frequency domain. At this time, if the spatial information domain is a subband domain, the spatial information domain is also converted into a discrete frequency domain. This method replaces filtering in the time domain with computation in the discrete frequency domain, and enables high-speed computation. In addition, in order to generate a virtual surround signal on the time domain, the output domain converter 230 needs to convert the rendering domain into the time domain.
The third embodiment is a method of performing virtual surround rendering in the time domain on a downmix signal. For example, when the downmix domain is not the time domain, the rendering domain conversion unit 210 performs domain conversion to the time domain. At this time, if the spatial information domain is a subband domain, the spatial information domain is also converted to a time domain. In this case, it is not necessary to perform domain conversion in the output domain conversion unit 230 in order to generate a virtual surround signal on the time domain.
FIG. 3 is a diagram illustrating an information conversion unit 300 according to an embodiment of the present invention. Referring to FIG. 3, the information converting unit 300 includes a channel mapping part 310, a coefficient generating part 320, and a synthesizing part 330. The information converting unit 300 may further include an additional processing unit that performs additional processing on the filter coefficient and / or a rendering domain converting part 340.
The channel mapping unit 310 performs channel mapping so that the input spatial information is mapped to at least one multi-channel signal, and generates a channel mapping output value. The coefficient generation unit 320 generates coefficient information corresponding to a channel, and the coefficient information may include channel-specific coefficient information or inter-channel coefficient information. Here, the channel-specific coefficient information represents size information, energy information, and the like, and the inter-channel coefficient information represents inter-channel correlation information calculated using a filter coefficient and a channel mapping output value. The coefficient generation unit 320 can include a plurality of channel-specific coefficient generation units, and generates coefficient information using filter information and channel mapping output values. Here, the channel includes at least one of a multi-channel, a downmix channel, and an output channel. In the following description, the channel is assumed to be multi-channel, and the channel-specific coefficient information is described as size information, but the present invention is not limited to this. The coefficient generation unit 320 may correspond to the number of channels or set the number according to other characteristics.
The synthesizing unit 330 that has received the channel-specific coefficients performs a function of generating a synthesis coefficient by integrating or adding the channel-specific coefficients, and generating a filter coefficient using the synthesis coefficient. In the process of generating the synthesis coefficient, additional information other than the channel-specific coefficient may be further synthesized to generate the synthesis coefficient. The combining unit 330 combines at least one channel according to the characteristics of the coefficient information, and performs each downmix channel, each output channel, one channel that combines the output channels, and a combination of these channels according to the characteristics. Can do. Then, the synthesis unit 330 may perform additional processing on the synthesis coefficient to generate a filter coefficient. For example, filter coefficients may be generated by performing additional processing on the synthesis coefficient, such as applying a separate function to the synthesis coefficient or combining a plurality of synthesis coefficients.
When the spatial information domain is different from the rendering domain, the rendering domain conversion unit 340 plays a role of matching the spatial information domain with the rendering domain. This is converted into a rendering domain for virtual surround rendering, and filter coefficients for virtual surround rendering are output.
Here, the synthesizing unit 330 has a function of reducing the amount of computation for virtual surround rendering, and can be omitted. Also, when the downmix signal is stereo, a coefficient set to be applied to the left and right downmix signals is generated in each channel coefficient generation process. Here, the filter coefficient set may include a coefficient transmitted from each channel to its own channel and a coefficient transmitted to the partner channel.
FIG. 4 is a diagram illustrating a virtual surround rendering process and a spatial information conversion process according to an embodiment of the present invention. In particular, the case where the downmix signal input to the virtual surround generation unit 410 is stereo is shown.
The information conversion unit 400 can generate the coefficient transmitted to the own channel of the virtual surround generation unit 410 and the coefficient transmitted to the partner channel using the spatial information. The information converting unit 400 is input to a first rendering unit 413 and transmitted to a left output (left out) that is its own channel output, and a right output ( coefficient HL_R transmitted to right out). Further, the information conversion unit 400 is input to a second rendering part 414, a coefficient HR_R transmitted to the right output that is its own channel output, and a coefficient transmitted to the left output that is the partner channel. HR_L is generated.
The virtual surround generation unit 410 includes a first rendering unit 413, a second rendering unit 414, and adders (Adders) 415 and 416. For example, when the downmix domain is not a subband domain and the rendering domain is a subband domain, domain conversion units (domain converting parts) 411 and 412 for domain conversion may be further provided for domain matching. it can. Here, inverse domain converting parts (Inverse domain converting parts) 417 and 418 for converting the subband domain into the time domain may be further provided. In this case, the user can listen to sound having a multi-channel effect with an earphone having a stereo channel.
The first rendering unit 413 and the second rendering unit 414 receive the downmix signal through the stereo channel, and receive filter coefficient sets applied to the left and right downmix signals output from the synthesis unit 403.
For example, the first rendering unit 413 and the second rendering unit 414 perform rendering for generating a virtual surround signal from the downmix signal using four filter coefficient sets (for example, HL_L, HL_R, HR_L, HR_R). It can be carried out.
More specifically, the first rendering unit 413 includes a filter coefficient set HL_L transmitted to the own channel from the left set (left set) that is a filter coefficient set, and a filter coefficient set HL_R transmitted to the partner channel. Can be used to render. The first rendering unit 413 may include a 1-1 rendering unit and a 1-2 rendering unit. The 1-1 rendering unit performs rendering using the filter coefficient set HL_L that is transmitted to the left output that is its own channel output, and the 1-2 rendering unit transmits to the right output that is the partner channel. Rendering can be performed using the filter coefficient set HL_R. In addition, the second rendering unit 414 may perform rendering using the filter coefficient set HR_R transmitted to the own channel from the right set which is the filter coefficient set and the filter coefficient set HR_L transmitted to the partner channel. it can. The second rendering unit 414 may include a 2-1 rendering unit and a 2-2 rendering unit. The 2-1 rendering unit performs rendering using the filter coefficient set HR_R that is transmitted to the right output that is its own channel output, and the 2-2 rendering unit transmits to the left output that is the partner channel. Rendering is performed using the filter coefficient set HR_L. Here, HL_R and HR_L are added to the other channel by adders 415 and 416. At this time, HL_R and HR_L may be 0 in some cases, which means that the coefficient of the cross term may be 0. Here, when HL_R and HR_L become 0, it means that both paths have no influence on each other.
On the other hand, even when the downmix signal is mono, rendering with a structure similar to that in FIG. 4 can be performed. For this reason, if the original mono input is the first channel signal and the signal that has been decorrelated to the first channel signal is the first channel signal, the first channel signal and the second channel signal Each of the channel signals can be rendered as an input to the first rendering unit 413 and the second rendering unit 414.
Hereinafter, as shown in FIG. 4, when the input signal is a stereo downmix signal, the downmix signal is x, the spatial mapping channel mapping coefficient is D, and the original HRTF is an external input. The filter coefficient is defined as G, the temporary multi-channel signal is defined as p, the rendered output signal is defined as y, and these are expressed by a determinant (matrix) as shown in the following Expression 1. Equation 1 below is based on the original HRTF filter coefficient, but it is clear that G is replaced by G 'in the following equation when a modified HRTF filter coefficient is used.
Here, if each coefficient is a value in the frequency domain, it can be developed in the following form. First, a temporary multi-channel signal can be represented by a product of a coefficient (Channel mapping coefficient) obtained by channel mapping spatial information and a downmix signal, which is expressed by the following Equation 2.
When the temporary multi-channel p is rendered using the original HRTF filter coefficient G, the following Equation 3 is obtained.
y = G ・ p
Here, y can be obtained by substituting p = D · x.
y = GDx
Here, if H is defined as H = GD, the rendered output signal y and the downmix signal x have the relationship of Equation 5 below.
Therefore, after the product between the filter coefficients is first processed to generate H, this can be multiplied by the downmix signal x to obtain y.
Therefore, the F coefficient described below can be obtained by the relationship of the following formula 6 where H = GD.
FIG. 5 is a diagram illustrating a virtual surround rendering process and a spatial information conversion process according to another embodiment of the present invention. In particular, the case where the decoded downmix signal input to the virtual surround generation unit 510 is mono is illustrated.
Referring to FIG. 5, the information conversion unit 500 includes a channel mapping unit 501, a coefficient generation unit 502, and a synthesis unit 503. The components of the information conversion unit 500 perform the same functions as the components of the information conversion unit 400 shown in FIG. However, the information conversion unit 500 can generate a final filter coefficient having the same domain as the rendering domain that performs virtual surround rendering. When the decoded downmix signal is mono, the filter coefficients are a filter coefficient set HM_L used to render the mono signal and output it to the left channel, and render the mono signal and output it to the right channel. The filter coefficient set HM_R used for
The virtual surround generation unit 510 includes a third rendering part 512. Moreover, the domain conversion part 511 and the reverse domain conversion part 513,514 can be further provided. The difference between the components of the virtual surround generation unit 510 and the virtual surround generation unit 410 shown in FIG. 4 is that the decoded downmix signal is mono, so there is one third rendering unit 512 that performs virtual surround rendering. In other words, one domain conversion unit 511 can be included. The third rendering unit 512 can receive the filter coefficient from the synthesizing unit 503 and perform virtual surround rendering for generating a virtual surround signal using the received filter coefficient. At this time, the filter coefficient includes a filter coefficient set HM_L used for rendering the mono signal and outputting it to the left channel, and a filter coefficient set HM_R used for rendering the mono signal and outputting it to the right channel.
On the other hand, when the output after virtual surround rendering is intended to obtain an output in the form of downmix stereo with respect to a mono downmix signal input, the following two methods are possible. is there.
First, the third rendering unit 512 (for example, the HRTF filter) does not use a filter coefficient for the virtual surround effect, but uses a value used at the time of stereo downmix (stereo downmix). In this case, the value used at the time of stereo downmix can be a coefficient left front = 1, right front = 0,.
Second, in the decoding process of generating multi-channel using spatial information from the downmix channel, the final multi-channel is not generated and decoding is performed only up to a corresponding step in order to obtain a desired number of channels. Can proceed.
Hereinafter, when the input signal is a mono downmix signal as shown in FIG. 5, x is the downmix signal, D is the channel mapping coefficient of spatial information, G is the original HRTF filter coefficient of the external input, and p is the temporary multichannel signal. When the rendered output signal is defined as y and these are expressed by a determinant, the following expression 7 is obtained.
Here, the relationship of the determinant has been described with reference to FIG. However, FIG. 4 shows an example in which the input downmix signal is stereo, and FIG. 5 shows an example in which the input downmix signal is mono.
6 and 7 illustrate a channel mapping process according to the present invention.
The channel mapping process refers to a process of generating a value to be mapped for each channel on the multi-channel so that the received spatial information matches the virtual surround generation unit. The channel mapping process is performed by the channel mapping units 401 and 501. At this time, it is possible to map at least two of the plurality of channels in consideration of all the channels in the process of mapping information mapped to each channel, for example, energy. In this case, the Lfe channel can be considered together with the center (C) channel, which eliminates the need for the number of channel divisions and simplifies the calculation.
For example, when the downmix signal is mono, channel mapping output values are generated using coefficients such as CLD1 to CLD5 and ICC1 to ICC5. The channel mapping output value can be D L , D R , D C , D LFE , D Ls , D Rs, etc., and is obtained using spatial information, so that various values can be obtained by various formulas. it is obvious. Here, the process of generating the channel mapping output value varies according to the tree configuration corresponding to the spatial information received by the decoding apparatus, the range of the spatial information used by the decoding apparatus, and the like.
6 and 7 are schematic block diagrams for explaining a channel mapping process according to the present invention. Here, the channel conversion unit forming the channel configuration is an OTT box, and the channel configuration has a 5151 structure.
Referring to FIG. 6, OTT boxes 601, 602, 603, 604, 605 and spatial information (for example, CLD 0 , CLD 1 , CLD 2 , CLD 3 , CLD 4 , ICC 0 , ICC 1 , ICC 2 , ICC 3, etc.) ) Can be used to generate multichannels L, R, C, LFE, Ls, and Rs from the downmix channel m. For example, when the tree structure is 5151, a method of obtaining a channel mapping output value using only the CLD is as shown in Equation 8 below.
Referring to FIG. 7, OTT boxes 701, 702, 703, 704, 705 and spatial information (for example, CLD 0 , CLD 1 , CLD 2 , CLD 3 , CLD 4 , ICC 0 , ICC 1 , ICC 3 , ICC 4, etc.) ) Can be used to generate multi-channels L, Ls, R, Rs, C, and LFE from the downmix channel m.
For example, when the tree structure is 5152, a method for obtaining a channel mapping output value using only the CLD is as shown in Equation 9 below.
The channel mapping output value has a different value for each frequency band, for each parameter band, and / or for each transmitted time slot. Here, if a value shift is large between adjacent bands and between time slots serving as boundaries, distortion may occur during virtual surround rendering. In order to prevent the generated distortion, a process of blurring in the frequency and time domains is required. A method for preventing the distortion is as follows. First, the frequency blurring and the time domain blurring described above can be used, and other methods suitable for virtual surround rendering can be used. Each channel mapping output value can be multiplied by a specific gain.
FIG. 8 is a diagram illustrating channel-specific filter coefficients according to the present invention. For example, the filter coefficient may be an HRTF coefficient.
For virtual surround rendering, a signal that has passed the GL_L filter is sent to the left channel source as a left output, and a signal that has passed the GL_R filter is sent as a right output. Thereafter, all signals received from the respective channels are combined to generate a left final output (for example, Lo) and a right final output (for example, Ro).
Therefore, the left and right channel outputs that have undergone virtual surround rendering are as shown in Equation 10 below.
Lo = L * GL_L + C * GC_L + R * GR_L + Ls * GLs_L + Rs * GRs_L
Ro = L * GL_R + C * GC_R + R * GR_R + Ls * GLs_R + Rs * GRs_R
According to one embodiment of the present invention, a method for obtaining L (810), C (800), R (820), Ls (830), Rs (840) is as follows. First, L (810), C (800), R (820), Ls (830), and Rs (840) are obtained using a decoding method that generates a multi-channel using a downmix channel and spatial information. be able to. For example, as a method for generating the multi-channel, there is an MPEG surround decoding method. Second, L (810), C (800), R (820), Ls (830), Rs (840) can be expressed by a relational expression of only spatial information.
9 to 11 are schematic block diagrams for explaining a process of generating virtual surround information according to the present invention.
FIG. 9 is a diagram illustrating a first embodiment of a process for generating virtual surround information according to the present invention. Referring to FIG. 9, the information conversion unit excluding the channel mapping unit includes a coefficient generation unit (coef_1 generating part: 900_1, coef_2 generating part: 900_2, ..., coef_N generating part: 900_N). A coefficient generating part 900 and an integrating part 910 are provided. In addition, an interpolating part 920 and a domain converting part 930 for additional processing of filter coefficients may be further provided.
The coefficient generation process performed by the coefficient generation unit 900 means a process of generating coefficients using filter information for spatial information. In this case, the coefficient generation process in the specific coefficient generation unit (for example, the first coefficient generation unit is coef_1 generating part: 900_1) can be expressed by the following equation.
For example, when the downmix channel is mono, the first coefficient generation unit 900_1 generates coefficients FL_L and FL_R for the multi-channel left channel using the coefficient D_L generated from the spatial information. The generated coefficients FL_L and FL_R can be expressed by Equation 11 below.
FL_L = D_L * GL_L (coefficient used to generate left output from mono input)
FL_R = D_L * GL_R (coefficient used to generate right output from mono input)
Here, D_L is a value generated from the spatial information in the channel mapping process of the spatial information. However, the process for obtaining D_L differs depending on the channel tree configuration (tree configuration) transmitted from the encoding apparatus and received by the decoding apparatus. In the second coefficient generation unit (coef_2 generating part) 900_2 and the third coefficient generation part (coef_3 generating part) 900_3, the second coefficient generation unit 900_2 generates FR_L and FR_R by the same method as the coefficient generation method. The third coefficient generator 900_3 can generate FC_L, FC_R, and the like.
For example, when the downmix channel is stereo, the first coefficient generation unit 900_1 uses the coefficients D_L1 and D_L2 generated from the spatial information to generate coefficients FL_L1, FL_L2, FL_R1, and FL_R2 for the multi-channel left channel. These can be generated by the following Equation 12.
FL_L1 = D_L1 * GL_L (coefficient used to generate left output from left input)
FL_L2 = D_L2 * GL_L (coefficient used to generate left output from right input)
FL_R1 = D_L1 * GL_R (coefficient used to generate right output from left input)
FL_R2 = D_L2 * GL_R (coefficient used to generate right output from right input)
Here, when the downmix channel is stereo, a plurality of coefficients can be generated by at least one coefficient generator in the same manner as when the downmix channel is mono.
The combining unit 910 generates filter coefficients by combining the channel-specific coefficients generated for each channel. The synthesis process in the synthesis unit 910 will be described separately for the case of mono input and the case of stereo input.
<Example of mono input>
HM_R = FL_R + FR_R + FC_R + FLS_R + FRS_R + FLFE_R
<Example of stereo input>
HL_L = FL_L1 + FR_L1 + FC_L1 + FLS_L1 + FRS_L1 + FLFE_L1
HL_R = FL_R1 + FR_R1 + FC_R1 + FLS_R1 + FRS_R1 + FLFE_R1
Here, HM_L and HM_R represent coefficients synthesized as filter coefficients for virtual surround rendering when they are mono inputs, and HL_L, HR_L, HL_R, and HR_R are synthesized as filter coefficients for virtual surround rendering when they are stereo inputs. Represents the calculated coefficient.
The interpolating unit 920 can perform interpolation on the filter coefficient. Also, time domain blurring can be performed as post-processing of filter coefficients. The time domain blurring is performed in a time blurring part. The interpolation in the interpolating unit 910 is performed in order to obtain spatial information that does not exist between the transferred and generated spatial information when the transferred and generated spatial information has a wide interval on the time axis. For example, if there is spatial information in the nth paramSlot and the n + kth paramSlot (k> 1), the generated coefficients (eg, HL_L, HR_L, HL_R, HR_R) are used on the paramSlot that has not been transferred. When linear interpolation is performed, the following expression 14 is obtained. The following Expression 14 is only one embodiment, and various interpolating methods can be applied.
HM_L (n + j) = HM_L (n) * a + HM_L (n + k) * (1-a)
HM_R (n + j) = HM_R (n) * a + HM_R (n + k) * (1-a)
HL_L (n + j) = HL_L (n) * a + HL_L (n + k) * (1-a)
HR_L (n + j) = HR_L (n) * a + HR_L (n + k) * (1-a)
HL_R (n + j) = HL_R (n) * a + HL_R (n + k) * (1-a)
HR_R (n + j) = HR_R (n) * a + HR_R (n + k) * (1-a)
Here, HM_L (n + j) and HM_R (n + j) represent coefficients obtained by interpolating coefficients synthesized as virtual surround rendering filter coefficients inputted in the case of mono input. HL_L (n + j), HR_L (n + j), HL_R (n + j), and HR_R (n + j) represent coefficients obtained by interpolating the coefficients synthesized as the virtual surround rendering filter coefficients inputted in the case of stereo input. Here, j and k are integers, 0 <j <k, and a is a real number where 0 <a <1, and is expressed by the following Expression 15.
Therefore, a mathematical expression for performing linear interpolation on the paramSlot that has not been transferred uses the value of the nth parameter slot and the value of the n + kth parameter slot, and parameters existing between them. This is a method of searching for a slot value. The value corresponding to the corresponding position is obtained on the line obtained by connecting the values in the two slots with a straight line according to the above formula 15.
In time blurring in the time blurring part, when a coefficient value suddenly changes between adjacent blocks in the time domain, a discontinuous point is generated and distortion is generated. ) Can be done to prevent problems. The time domain blurring can be parallel to the interpolation or the method applied can vary depending on its location. When the downmix channel is mono, the time domain blurring of the filter coefficient can be expressed by Equation 16 below.
HM_L (n) '= HM_L (n) * b + HM_L (n-1)' * (1-b)
HM_R (n) '= HM_R (n) * b + HM_R (n-1)' * (1-b)
That is, the filter coefficient (HM_L (n−1) ′ or HM_R (n−1) ′) in the previous block (n−1) is multiplied by (1−b) to generate the filter coefficient (HM_L) generated in the current block n. (N) or HM_R (n)) can be multiplied by b to perform a 1-pole IIR filter-type bulling. Here, b is a constant value of 0 <b <1, and the smaller the b value, the greater the blurring effect, and the greater the b value, the smaller the blurring effect. The same method can be applied to other filters.
When the interpolation and the blurring are expressed by one equation using the above equation 16 for the time domain blurring, the following equation 17 is obtained.
HM_L (n + j) '= (HM_L (n) * a + HM_L (n + k) * (1-a)) * b + HM_L (n + j-1)' * (1-b)
HM_R (n + j) '= (HM_R (n) * a + HM_R (n + k) * (1-a)) * b + HM_R (n + j-1)' * (1-b)
On the other hand, when the interpolation and the time-blurring process are performed in the interpolating unit 910 and / or the time-blurring unit, a filter coefficient having an energy value different from that of the original filter coefficient is obtained. Energy normalization work can be added to prevent seed problems.
When the rendering domain and the spatial information domain are not the same, the domain conversion unit 930 performs domain conversion to match the spatial information domain with the rendering domain. However, when the spatial information domain and the rendering domain are the same, domain conversion is not necessary. At this time, when the spatial information domain is a subband domain and the rendering domain is a frequency domain, the domain conversion may be a process of expanding and contracting the coefficient to fit the frequency and time range of each subband.
FIG. 10 is a diagram showing a second embodiment of the process of generating virtual surround information according to the present invention. Referring to FIG. 10, the information conversion unit excluding the channel mapping unit includes a coefficient generation unit (coef_1 generating part: 1000_1, coef_2 generating part: 1000_2, ..., coef_N generating part: 1000_N). a co-efficient generating part 1000 and an integrating part 1020; In addition, an interpolating part 1010 including at least one interpolating part 1010_1, 1010_2,..., 1010_N and a domain converting part 1030 may be further provided for additional processing. . The second embodiment shown in FIG. 10 is different from the first embodiment shown in FIG. 9 in that coefficients generated by the coefficient generator 1000 for each channel (for example, FL_L, FL_R in the case of mono, and stereo). In this case, all interpolation is performed on FL_L1, FL_L2, FL_R1, FL_R2).
FIG. 11 is a diagram showing a third embodiment of the process of generating virtual surround information according to the present invention. The embodiment of FIG. 11 differs from the first and second embodiments of FIG. 9 and FIG. 10 described above after interpolating the channel-mapped spatial information by the interpolating unit 1100. A coefficient for each channel is generated using the interpolated value.
In the method of each embodiment described with reference to FIGS. 9 to 11, the output value obtained by channel mapping spatial information is a frequency domain value (for example, a parameter band unit has a single value). The filter coefficient generation process and the like have been described on the assumption that the process proceeds in the frequency domain. Further, when virtual surround rendering is also performed in the subband region, the domain conversion unit does not play any role, and outputs the filter coefficients in the subband region as they are, or matches the frequency resolution (frequency resolution). Only the conversion process can be performed and output.
The present invention is not limited to the above-described embodiments, and it is obvious to those skilled in the art that various modifications can be made within the scope of the appended claims. Any variation is within the scope of the present invention.
It is a figure which shows the signal processing system by one Example of this invention. It is a schematic block diagram which shows the virtual surround production | generation part by one Example of this invention. It is a schematic block diagram which shows the information conversion part by one Example of this invention. 6 is a schematic block diagram for explaining a virtual surround rendering process and a spatial information conversion process according to an embodiment of the present invention; FIG. FIG. 6 is a schematic block diagram for explaining a virtual surround rendering process and a spatial information conversion process according to another embodiment of the present invention. 2 is a schematic block diagram illustrating a channel mapping process according to an embodiment of the present invention. FIG. FIG. 5 is a schematic block diagram for explaining a channel mapping process according to another embodiment of the present invention. It is the schematic for demonstrating the filter coefficient according to channel by one Example of this invention. FIG. 6 is a schematic block diagram for explaining a process of generating surround conversion information according to the present invention. FIG. 6 is a schematic block diagram for explaining a process of generating surround conversion information according to the present invention. FIG. 6 is a schematic block diagram for explaining a process of generating surround conversion information according to the present invention.
Using the spatial information to generate surround transform information;
Rendering a downmix signal using the surround transform information on a rendering domain to generate a virtual surround signal;
The spatial information is determined when generating the downmix signal;
The audio signal decoding method , wherein the virtual surround signal includes a first output channel signal and a second output channel signal .
The method of claim 1, further comprising converting the virtual surround signal of the rendering domain into a virtual surround signal of an output domain.
The rendering domain includes at least one of a frequency domain and a time domain;
The frequency domain includes at least one of a subband domain and a discrete frequency domain;
The method of claim 1, wherein the subband domain includes at least one of a simple subband domain and a hybrid subband domain.
The audio signal of claim 1, further comprising: converting the downmix signal of the downmix domain into the downmix signal of the rendering domain when a downmix domain is different from the rendering domain. Decoding method.
Transforming the downmix signal of the downmix domain comprises :
If the downmix domain is in the time domain, converting the downmix signal in the time domain to the downmix signal in the rendering domain;
If the downmix domain is a discrete frequency domain, converting the downmix signal in the discrete frequency domain to the downmix signal in the rendering domain;
If the downmix domain is the discrete frequency domain, the discrete after the downmix signal in the frequency domain by converting the downmix signal of the time domain, the downmix signal the rendering domain in the converted time-domain 5. The audio signal decoding method according to claim 4, further comprising at least one of converting the downmix signal into a downmix signal.
The rendering domain is a subband domain, the downmix signal includes a first channel signal and a second channel signal, and rendering the downmix signal comprises:
Applying the surround conversion information to the first channel signal;
Applying the surround conversion information to the second channel signal;
Adding the first channel signal and the second channel signal;
The audio signal decoding method according to claim 1, further comprising:
The audio signal decoding method according to claim 1, wherein the surround conversion information is generated using the spatial information and filter information.
The step of generating the surround conversion information includes:
Generating channel mapping information in which the spatial information is mapped for each channel;
Generating the surround conversion information using the channel mapping information and filter information;
Generating channel coefficient information using the spatial information and filter information;
Generating the surround transform information using the channel coefficient information;
Generating channel coefficient information using the channel mapping information and filter information;
Further comprising the downmix signal and the step of receiving the audio signal including the spatial information,
The method of claim 1, wherein the downmix signal and the spatial information are extracted from the audio signal.
2. The audio signal decoding method according to claim 1, wherein the spatial information includes at least one of CLD and ICC.
A demultiplexer that receives the downmix signal and spatial information;
An information conversion unit that generates surround conversion information using the spatial information;
A virtual surround generation unit that generates a virtual surround signal by rendering the downmix signal using the surround conversion information on a rendering domain;
The audio signal decoding apparatus , wherein the virtual surround signal includes a first output channel signal and a second output channel signal .
The audio signal decoding apparatus of claim 13, wherein the virtual surround generation unit further includes an output domain conversion unit that converts the virtual surround signal of the rendering domain into a virtual surround signal of an output domain. .
The apparatus of claim 13, wherein the subband domain includes at least one of a simple subband domain and a hybrid subband domain.
The virtual surround generation unit may include a rendering domain conversion unit that converts the downmix signal of the downmix domain into the downmix signal of the rendering domain when a downmix domain is different from the rendering domain. The audio signal decoding device according to claim 13.
The rendering domain conversion unit
A first domain conversion unit configured to change the downmix signal in the time domain to the downmix signal in the rendering domain when the downmix domain is a time domain;
When the downmix domain is a discrete frequency domain, a second domain conversion unit that converts the downmix signal in the discrete frequency domain into the downmix signal in the rendering domain;
If the downmix domain is the discrete frequency domain, the discrete after the downmix signal in the frequency domain by converting the downmix signal of the time domain, the downmix signal the rendering domain in the converted time-domain wherein among the third domain converter for converting the down-mix signal, characterized in that it comprises at least one, the decoding apparatus of an audio signal according to claim 16.
The rendering domain is a sub-band domain, and the downmix signal includes a first channel signal and a second channel signal;
The virtual surround generation unit applies the surround conversion information to the first channel signal, applies the surround conversion information to the second channel signal, and the first channel signal and the second channel signal 14. The audio signal decoding apparatus according to claim 13, wherein:
14. The audio signal decoding apparatus according to claim 13, wherein the surround conversion information is generated using the spatial information and filter information.
The information conversion unit according to claim 13, wherein the information conversion unit generates channel mapping information obtained by mapping the spatial information for each channel, and generates the surround conversion information using the channel mapping information and filter information. Audio signal decoding device.
The audio signal according to claim 13, wherein the information conversion unit generates channel coefficient information using the spatial information and filter information, and generates the surround conversion information using the channel coefficient information. Decoding device.
The information converter is
A channel mapping unit for generating channel mapping information obtained by mapping the spatial information for each channel;
A coefficient generator for generating channel coefficient information using the channel mapping information and filter information;
A combining unit that generates the surround conversion information using the channel coefficient information;
14. The apparatus for decoding an audio signal according to claim 13, further comprising:
The demultiplexing unit receives the audio signal including the downmix signal and the spatial information, the downmix signal and the spatial information is characterized by being extracted from the audio signal, according to claim 13 An audio signal decoding device according to claim 1.
14. The audio signal decoding apparatus according to claim 13, wherein the spatial information includes at least one of CLD and ICC.
JP2008513375A 2005-05-26 2006-05-25 Audio signal decoding method and apparatus Active JP4988717B2 (en)
US60/684,579 2005-05-26
PCT/KR2006/001987 WO2006126844A2 (en) 2005-05-26 2006-05-25 Method and apparatus for decoding an audio signal
JP2009501457A JP2009501457A (en) 2009-01-15
JP4988717B2 true JP4988717B2 (en) 2012-08-01
JP2008513375A Active JP4988717B2 (en) 2005-05-26 2006-05-25 Audio signal decoding method and apparatus
JP2008513374A Active JP4988716B2 (en) 2005-05-26 2006-05-25 Audio signal decoding method and apparatus
JP2008513378A Active JP4988718B2 (en) 2005-05-26 2006-05-26 Audio signal decoding method and apparatus
JP (3) JP4988717B2 (en)
HK (3) HK1119821A1 (en)
EP2191462A4 (en) * 2007-09-06 2010-08-18 Lg Electronics Inc A method and an apparatus of decoding an audio signal
JP5521908B2 (en) 2010-08-30 2014-06-18 ヤマハ株式会社 Information processing apparatus, acoustic processing apparatus, acoustic processing system, and program
JP5518638B2 (en) 2010-08-30 2014-06-11 ヤマハ株式会社 Information processing apparatus, sound processing apparatus, sound processing system, program, and game program
DE69511246D1 (en) 1994-02-25 1999-09-09 Moller Binaural synthesis, head-related transfer functions and their uses
JPH09224300A (en) * 1996-02-16 1997-08-26 Sanyo Electric Co Ltd Method and device for correcting sound image position
JPH1132400A (en) * 1997-07-14 1999-02-02 Matsushita Electric Ind Co Ltd Digital signal reproducing device
CA2742649C (en) 1999-04-07 2014-11-04 Dolby Laboratories Licensing Corporation Matrix improvements to lossless encoding and decoding
AU2002310642A1 (en) 2001-06-21 2003-01-08 1... Limited Loudspeaker
DE602006014809D1 (en) 2005-03-30 2010-07-22 Koninkl Philips Electronics Nv Scalable multi-channel audio encoding
EP1906706B1 (en) 2005-07-15 2009-11-25 Panasonic Corporation Audio decoder
JP4938015B2 (en) 2005-09-13 2012-05-23 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Method and apparatus for generating three-dimensional speech
AT476732T (en) 2006-01-09 2010-08-15 Nokia Corp Controlling the decoding of binaural audio signals
EP1982327A4 (en) 2006-02-07 2010-05-05 Lg Electronics Inc Apparatus and method for encoding/decoding signal
RU2407226C2 (en) 2006-03-24 2010-12-20 Долби Свидн Аб Generation of spatial signals of step-down mixing from parametric representations of multichannel signals
CN101411214B (en) 2006-03-28 2011-08-10 艾利森电话股份有限公司 Method and arrangement for a decoder for multi-channel surround sound
AT505912T (en) 2006-03-28 2011-04-15 Fraunhofer Ges Forschung Improved signal processing method for multi-channel audiore construction
2006-05-25 JP JP2008513375A patent/JP4988717B2/en active Active
2006-05-25 JP JP2008513374A patent/JP4988716B2/en active Active
2006-05-26 JP JP2008513378A patent/JP4988718B2/en active Active
2008-10-16 HK HK08111477.1A patent/HK1119821A1/en not_active IP Right Cessation
2008-10-16 HK HK08111481.5A patent/HK1119822A1/en not_active IP Right Cessation
2008-10-16 HK HK08111482.4A patent/HK1119823A1/en not_active IP Right Cessation
2014-12-02 US US14/558,649 patent/US9595267B2/en active Active
US20150088530A1 (en) 2015-03-26
HK1119821A1 (en) 2011-11-18
JP2008542815A (en) 2008-11-27
JP4988716B2 (en) 2012-08-01
JP2009501346A (en) 2009-01-15
US9595267B2 (en) 2017-03-14
JP2009501457A (en) 2009-01-15
HK1119823A1 (en) 2013-01-25
HK1119822A1 (en) 2013-06-07
JP4988718B2 (en) 2012-08-01
JP2013251919A (en) 2013-12-12 Multi-channel audio signal processor, multi-channel audio signal processing method, compression efficiency improving method, and multi-channel audio signal processing system