Source: http://www.google.com/patents/US8019095?dq=inventor:%22Arthur+R.+Hair%22&ei=VAy0Tsa4NYTl0QGQiqWiBA
Timestamp: 2016-04-30 13:43:51
Document Index: 299808411

Matched Legal Cases: ['application No. 2005299410', 'Application No. 03791682', 'Application No. 05818505', 'Application No. 05818505', 'Application No. 182097', 'Application No. 03819918', 'Application No. 165', 'Application No. 03819918', 'art 2', 'Application No. 0702926']

Patent US8019095 - Loudness modification of multichannel audio signals - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsScaling, by a desired amount sm, the overall perceived loudness Lm of a multichannel audio signal, wherein perceived loudness is a nonlinear function of signal power P, by scaling the perceived loudness of each individual channel Lc by an amount substantially equal to the desired amount of scaling of...http://www.google.com/patents/US8019095?utm_source=gb-gplus-sharePatent US8019095 - Loudness modification of multichannel audio signalsAdvanced Patent SearchPublication numberUS8019095 B2Publication typeGrantApplication numberUS 12/225,988PCT numberPCT/US2007/006444Publication dateSep 13, 2011Filing dateMar 14, 2007Priority dateApr 4, 2006Fee statusPaidAlso published asCN101411060A, CN101411060B, DE602007010912D1, EP2002539A1, EP2002539B1, US8600074, US8731215, US20100202632, US20110311062, US20120106743, US20140211946, WO2007123608A1Publication number12225988, 225988, PCT/2007/6444, PCT/US/2007/006444, PCT/US/2007/06444, PCT/US/7/006444, PCT/US/7/06444, PCT/US2007/006444, PCT/US2007/06444, PCT/US2007006444, PCT/US200706444, PCT/US7/006444, PCT/US7/06444, PCT/US7006444, PCT/US706444, US 8019095 B2, US 8019095B2, US-B2-8019095, US8019095 B2, US8019095B2InventorsAlan Jeffrey Seefeldt, Michael John SmithersOriginal AssigneeDolby Laboratories Licensing CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (123), Non-Patent Citations (111), Referenced by (9), Classifications (16), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetLoudness modification of multichannel audio signals
US 8019095 B2Abstract
Scaling, by a desired amount sm, the overall perceived loudness Lm of a multichannel audio signal, wherein perceived loudness is a nonlinear function of signal power P, by scaling the perceived loudness of each individual channel Lc by an amount substantially equal to the desired amount of scaling of the overall perceived loudness of all channels sm, subject to accuracy in calculations and the desired accuracy of the overall perceived loudness scaling sm. The perceived loudness of each individual channel may be scaled by changing the gain of each individual channel, wherein gain is a scaling of a channel's power. Optionally, in addition, the loudness scaling applied to each channel may be modified so as to reduce the difference between the actual overall loudness scaling and the desired amount of overall loudness scaling.
1. A method for scaling, by a desired amount sm, the overall perceived loudness Lm of a multichannel audio signal, wherein the overall perceived loudness Lm is a nonlinear function of the sum of the per channel signal power Pc, comprising
scaling the perceived loudness Lc of each individual channel by an amount substantially equal to the desired amount sm of scaling of the overall perceived loudness, subject to the desired accuracy of the overall perceived loudness scaling by said amount sm, wherein the perceived loudness of each individual channel is a nonlinear function of the individual channel's power Pc.
2. A method according to claim 1 wherein the perceived loudness of each individual channel is scaled by changing the gain gc of each individual channel, wherein gain is a scaling of a channel's power, such that the gain gc for each channel is related to the power Pc of each channel according to the relationship:
in which F−1 is the inverse of the nonlinear function F relating perceived loudness L to power P:
L=F{P}.
3. A method according to claim 2 further comprising modifying the loudness scaling applied to each channel so as to reduce the difference between [1] the overall perceived loudness scaling resulting from the individual scaling by sm, of each channel's perceived loudness and [2] the desired amount sm of overall perceived loudness scaling.
4. A method according to claim 3 wherein said difference is reduced by applying a common multiplier to the gain of each channel's power.
5. A method according to claim 3 wherein said difference is reduced by adding a common scaling offset to the scaling of each channel's loudness.
6. A method according to claim 4 or claim 5 wherein said difference may be represented as:
where Δsm is the difference and C is the total number of channels.
7. A method according to claim 2 wherein the perceived loudness of each channel and the overall perceived loudness are both measured in each of a plurality of frequency bands and the gain of each channel is adjusted in such frequency bands.
8. A method according to claim 5 wherein said frequency bands are commensurate with the critical bands.
9. A method according to claim 1 wherein the perceived loudness of each channel and the overall perceived loudness are both measured in a single wideband frequency band.
10. Apparatus comprising means adapted to perform all the steps of the method of claim 1.
11. A computer program, stored on a computer-readable medium for causing a computer to perform the methods of claim 1. Description
The invention relates to audio signal processing. In particular, the invention relates to adjusting the overall perceived loudness of a multichannel audio signal while retaining approximately the relative perceived loudness between all the channels in order to preserve the perceived spatial balance. The invention includes not only methods but also corresponding computer programs and apparatus.
Certain techniques for measuring and adjusting perceived (psychoacoustic loudness) useful in better understanding aspects the present invention are described in published International patent application WO 2004/111994 A2, of Alan Jeffrey Seefeldt et al, published Dec. 23, 2004, entitled “Method, Apparatus and Computer Program for Calculating and Adjusting the Perceived Loudness of an Audio Signal” and in “A New Objective Measure of Perceived Loudness” by Alan Seefeldt et al, Audio Engineering Society Convention Paper 6236, San Francisco, Oct. 28, 2004. Said WO 2004/111994 A2 application and said paper are hereby incorporated by reference in their entirety.
Certain other techniques for measuring and adjusting perceived (psychoacoustic loudness) useful in better understanding aspects the present invention are described in published International patent application WO 2006/047600 A1 of Alan Jeffrey Seefeldt, published May 4, 2006, entitled “Calculating and Adjusting the Perceived Loudness and/or the Perceived Spectral Balance of an Audio Signal.” Said WO 2006/047600 A1 application is hereby incorporated by reference in its entirety.
Many methods exist for objectively measuring the perceived loudness of audio signals. Examples of methods include A, B and C weighted power measures as well as psychoacoustic models of loudness such as “Acoustics—Method for calculating loudness level,” ISO 532 (1975) and said PCT/US2005/038579 application. Weighted power measures operate by taking the input audio signal, applying a known filter that emphasizes more perceptibly sensitive frequencies while deemphasizing less perceptibly sensitive frequencies, and then averaging the power of the filtered signal over a predetermined length of time. Psychoacoustic methods are typically more complex and aim to better model the workings of the human ear. They divide the signal into frequency bands that mimic the frequency response and sensitivity of the ear, and then manipulate and integrate these bands while taking into account psychoacoustic phenomenon such as frequency and temporal masking, as well as the non-linear perception of loudness with varying signal intensity. The aim of all methods is to derive a numerical measurement that closely matches the subjective impression of the audio signal.
Accurate modeling of the non-linearity of the human auditory system forms the basis of perceptual models of loudness. In the 1930's, Fletcher and Munson found that the relative change in sensitivity decreased as the level of sound increased. In the 1950's, Zwicker and Stevens built on the work of Fletcher and Munson and developed more accurate and realistic models. FIG. 1, published by Zwicker, shows the growth of loudness of both a 1 kHz tone and uniform exciting noise (UEN, noise with equal power in all critical bands). For a signal level below what is often termed the “hearing threshold,” no loudness is perceived. Above this threshold, there is a quick rise in perceived loudness up to an asymptote where loudness grows linearly with signal level. Where FIG. 1 shows the non-linear behavior for a 1 kHz tone, the equal loudness contours of ISO 226 in FIG. 2 show the same behavior but as a function of frequency for sinusoidal tones. The contour lines, at increments of 10 phon, show the sound pressure levels across frequency that the human ear perceives as equally loud. The lowest line represents the “hearing threshold” as a function of frequency. At lower levels the lines of equal loudness compress closer together such that relatively smaller changes in sound pressure level cause more significant changes in perceived loudness than at higher levels.
The non-linear and frequency varying behavior of the human auditory system has a direct impact on the perceived timbre and imaging of audio signals. A complex, wideband audio signal, for example music, presented at a particular sound pressure level is perceived as having a particular spectral balance or timbre. If the same audio signal is presented at a different sound pressure level and, as shown in FIG. 2, the growth of perceived loudness is different for different frequencies, the perceived spectral balance or timbre of the audio signal will be different. A complex, wideband multichannel audio signal, presented over multiple loudspeakers, is also perceived as having a particular spatial balance. Spatial balance refers to the impression of the location of sound elements in the mix as well as the overall diffuseness of the mix due to the relative level of audio signals between two or more loudspeakers. If the same multichannel audio signal is presented at a different overall sound pressure level, the non-linear growth in perceived loudness and differing growth of loudness across frequency leads to a change in the perceived spatial balance of the multichannel audio signal. This is especially apparent when there is a significant difference in level between channels. Quieter channels will be affected differently to louder channels which, for example, can lead to quiet channels dropping below the hearing threshold and audibly disappearing when the overall level is reduced.
In many situations there is a desire to adjust or scale the perceived loudness of an audio signal. The most obvious examples are the traditional volume or level controls that appear on many devices including consumer music players, home theater receiver/amplifiers and professional mixing consoles. This simple volume or level control gain adjusts the audio signal without any consideration of the human auditory system and resulting change in perceived timbre and spatial balance.
More recently Seefeldt et. al (said WO 2004/111994 A2 application) and Seefeldt (said PCT/US2005/038579 application) have disclosed inventions, aspects of which enable accurate scaling of the perceived loudness of a monophonic audio signal and, depending on whether implementations thereof are wideband or multiband, maintain the perceived timbre. According to aspects of such inventions, a desired loudness scaling or target loudness may be achieved by, in essence, inverting the loudness measurement model and calculating either a wideband gain or multiband gains that can be applied to the audio signal.
While such approaches solve the problem of adjusting the loudness of a monophonic audio signal, the question still remains of how to adjust the loudness of a multichannel audio signal.
Multichannel loudness is typically calculated as a function of the sum of the power in each channel. For weighted power methods such as the A, B and C weighted measures mentioned above, the multichannel loudness is a simple sum of the weighted power in each channel. Commonly for psychoacoustic models of loudness, a critical band power spectrum or excitation spectrum is first calculated for each channel and the excitation spectrums are then summed across all the channels to create a single excitation spectrum. Each excitation band is passed through a non-linearity, such as FIG. 1, to create a measure of loudness per band, known as specific loudness, and the specific loudness is summed across frequency to calculate a single, wideband loudness value. For both weighted power and psychoacoustics methods, the function of the sum of the power in each channel may include additional per channel weightings to take into account head related transfer function (HRTF) effects.
Because the loudness of a multichannel signal can be calculated relatively simply, it is possible to calculate a single gain that, when applied to all channels, causes an overall desired change in loudness. However, this single gain may have undesirable effects on other attributes of the multichannel presentation. If differences exist in the relative signal levels between channels in the multichannel presentation and if all channels are scaled by the same gain, quieter channels will have a larger perceived change in their loudness than louder channels. This may cause a change in the perceived spatial balance that is worst when some channels fall below the threshold of hearing. For example, in many 5.1 audio mixes for film, the front channels contain signals of a significantly higher level than the surround channels. The center channel in particular is generally used to reproduce dialogue. The lower level surround channels, however, may contain signals that create a sense of diffuseness in the mix. For example, they may contain the reverberant portion of the dialogue in order to simulate the effect of someone speaking in a large room. As the loudness of such a signal is decreased by applying the same gain to all channels, the surround channels decrease in loudness more rapidly than the front channels, eventually falling below the threshold of hearing. The result is a significant collapse in the intended diffuse spatial balance.
According to aspects of the present invention, a desired scaling in the overall perceived loudness of a multichannel presentation may be achieved to a desired accuracy, while retaining, to a desired accuracy, the relative perceived loudness among channels in order to preserve a perceived spatial balance or timbre.
FIG. 1 shows the non-linear growth of loudness for both a 1 kHz tone and uniform exciting noise (UEN).
FIG. 2 shows the equal loudness contours of ISO 226. The horizontal scale is frequency in Hertz (logarithmic base 10 scale) and the vertical scale is sound pressure level in decibels.
FIG. 3 shows a set of critical band filter responses useful for computing an excitation signal for a psychoacoustic loudness model.
FIGS. 4 a-f depict the specific loudness spectra and gains resulting from the modification of the specific loudness of a multichannel audio signal.
The invention is directed to a method for scaling, by a desired amount sm, the overall perceived loudness Lm of a multichannel audio signal, wherein perceived loudness is a nonlinear function of signal power P, by scaling the perceived loudness of each individual channel Lc by an amount substantially equal to the desired amount of scaling of the overall perceived loudness of all channels sm, subject to accuracy in calculations and the desired accuracy of the overall perceived loudness scaling sm. The perceived loudness of each individual channel may be scaled by changing the gain of each individual channel, wherein gain is a scaling of a channel's power.
Optionally, in addition, the loudness scaling applied to each channel is modified so as to reduce the difference between the actual overall loudness scaling and the desired amount of overall loudness scaling.
The loudness scaling applied to each channel may be modified by applying a common multiplier to the gain of each channel or by adding a common scaling offset to the scaling of each channel.
The perceived loudness of each channel and the overall perceived loudness may both be measured in each of a plurality of frequency bands and the amplitude of each channel adjusted in such frequency bands. The frequency bands may be critical bands. Alternatively, the perceived loudness of each channel and the overall perceived loudness may both be measured in a single wideband frequency band.
In another aspect, the invention may be practiced by apparatus adapted to perform any of the above-mentioned methods.
In yet another aspect, the invention may be practiced by a computer program, stored on a computer-readable medium for causing a computer to perform any of the above-mentioned methods.
In general terms, the measure of loudness L may be described as a function F of signal power P. Signal power P is a power measure of the audio signal. This could be the A, B or C weighted power or a multiband excitation spectrum. See, for example, ANSI S1.42-2001 (R2006), American National Standard Design Response of Weighting Networks for Acoustical Measurements. The function F is a non-linearity designed to approximate variations in the growth of loudness. This function could be as simple as the single UEN function of FIG. 1 applied to a single, wideband power measure or as complex as a psychoacoustic model of loudness where the excitation spectrum is converted, through different per-band nonlinearities, to a specific loudness spectrum and then to a single loudness value (as in said PCT/US2005/038579 application, for example). It should be noted that while traditional weighted power loudness measures such as A weighted power attempt to take into account the frequency varying sensitivity of the human auditory system, they do not take into account the variation in level sensitivity. It may therefore be useful to pass a traditional weighted power measure through a non-linearity such as the one described above.
L=F{P} (1)
Assuming that the loudness function is invertible, a gain scaling g of the signal power P may be calculated such that the gain change results in a particular, desired scaling s of the perceived loudness.
s . L = F { g . P } ( 2 a ) g = F - 1 { s . L } P ( 2 b ) Thus, gain g is a scaling of the power P, whereas s is a scaling of the loudness L.
If the function F were linear, then Eqn. 2a would simplify to sL=gF{P}=gL that yields the trivial solution g=s, independent of the signal power P. However, with a nonlinear function F, the gain g is, in general, a function of the signal power P as shown in Eqn. 2b. In other words, different signal powers P require different gains g for the same loudness scaling s.
The overall (all channel) measure of loudness Lm of a multichannel audio signal may, in practice, be approximated as a function of the sum of the per channel power Pc of each of the channels in the multichannel audio signal. The total number of channels is C.
Note that the sum of the per-channel power may be weighted to take into account head related transfer function (HRTF) effects. That is, signals from different spatial directions may have slightly different, relative perceived loudness. If one knows or assumes where the listener is in relation to the loudspeakers reproducing the multiple channels, then one may build a model of the signals arriving at a listener's ears as a function of the individual channel signals (generally, filtered and summed versions of the channel signals). The loudness may then be computed from such ear signals. In practice, however, performing a power sum of the channel signals works well for most listening environments.
Now again assuming that the loudness function is invertible, a single gain gm applied to all channels may be calculated such that the result is a desired scaling sm of the overall perceived loudness.
However, applying the same gain scaling gm to all the channels may undesirably affect the spatial balance of the modified audio. In particular, the computation of the gain gm will be most influenced by the channels with the greatest amount of power. If other channels have significantly less power, then the gain gm may cause a significantly different perceived change in these lower level channels in comparison to the higher level channels due to the non-linearity of human loudness perception. If the scaling sm corresponds to an attenuation in loudness, too much attenuation may be applied to these lower level channels. As a result, the relative contribution of such low level channels to the spatial balance of the mix will be diminished, and at worst, the channels will become completely inaudible.
The present invention addresses the problem of maintaining the spatial balance of a multichannel audio signal while imparting a desired change to its overall loudness. Accurately measuring and characterizing the spatial balance of a multichannel audio signal is highly complex. Portions of the spectra of the various channels may fuse perceptually into virtual sources located between the speakers through which the channels are played, while other portions of the channels may combine to form the perception of a diffuse sound field surrounding the listener. Measuring the perceived loudness of these various components in relation to each other is not a well understood problem as it involves the complex phenomenon of certain audio signal components partially masking other components. The degree of masking is a function of the level of each source as well as the spatial location and diffuseness of each source. Even if one were able to accurately measure all these aspects of the spatial balance, attempting to preserve their relative measures as the overall loudness is scaled would likely involve a complex non-linear optimization process.
Consider, however, a simple example of a two-channel signal in which each channel contains a signal that does not overlap spectrally with the signal in the other channel. Each channel will then be perceived as a distinct source with neither source masking the other. In this simple case, it becomes clear that maintaining the relative loudness of the two components may be achieved by scaling the loudness of each individual channel (rather than the gain of each channel) by the same amount. The inventors have found that applying this solution generally to a multichannel signal helps preserve the spatial balance without the introduction of any objectionable side-effects.
In a basic implementation of aspects of the invention, the perceived loudness of each individual channel Lc (taken in isolation) may be scaled by an amount of scaling sc substantially equal to a desired amount of scaling, sm, of the overall perceived loudness of all channels, subject to accuracy in calculations and the desired accuracy of the overall perceived loudness scaling. This solution mitigates the problem, mentioned above, of low level channels falling below the threshold of hearing due to the influence of higher level channels. Such a scaling in the perceived loudness of each individual channel Lc may be accomplished by controlling the individual gain gc of each channel (where such gain gc is a scaling of the channel's power Pc). Note, as discussed further below, that such individual channel gains gc generally are not the same as the gain gm mentioned above in connection Eqns. 4a and 4b. This may be may be better understood, for example, by reference to Eqns. 5a and 5b:
s m . L c = F { g c . P c } for each of C channels or ( 5 a ) g c = F - 1 { s m . L c } P c for each of C channels ( 5 b ) where sc=sm.
Although such a basic implementation of the invention substantially maintains the spatial balance and is usable in many applications, such implementations may not assure that the desired overall scaling sm of the multichannel perceived loudness Lm is achieved due to the non-linearity of the function F. Because F is non-linear, the gain gm given by Eqn. 4b is, in general, not equal to the gains gc given by Eqn. 5b. Therefore, the loudness of all channels after the application of gm to all channels is not equal, in general, to the loudness of all channels after applying the gains gc to each respective channel:
F { ∑ c = 1 C g m P c } ≠ F { ∑ c = 1 C g c P c } ( 6 a ) Substituting the left hand side of Eqn. 6a with Eqn. 4a and gc with Eqn. 5b yields the equivalent expression:
Thus, there may be a difference or error between (1) the perceived loudness of all channels resulting from scaling the perceived loudness of each of the individual channels Lc by the desired overall perceived loudness scaling factor sm (expressed, for example, by the right hand portion of Eqn. 6b) and (2) the perceived loudness of all channels resulting from scaling directly by the overall loudness scaling factor sm (expressed, for example, by the left hand portion of Eqn. 6b). One may express this error as a scaling delta Δsm which when summed with the desired overall loudness scaling sm turns Eqn. 6b into an equality:
( s m + Δ s m ) L m = F { ∑ c = 1 C F - 1 { s m L c } } ( 6 c ) or, rearranging,
For any function F that realistically models the non-linear level behavior of human perception, such errors are generally small because the growth of loudness is close to linear over a large range. However, to minimize such errors, it may be desirable to add an optional correction to the basic implementation of the invention. Without loss of generality, one may represent such a correction as scaling deltas Δsc introduced to the loudness scaling of each individual channel so that that the overall loudness scaling error Δsm in Eqn. 6d is reduced. Generally, the scaling deltas Δsc are different from channel to channel. Incorporation of these channel scaling deltas Δsc into Eqn. 6d yields the modified expression:
Δ s m = F { ∑ c = 1 C F - 1 { ( s m + Δ s c ) L c } } L m - s m ( 6 e ) The individual channel gains with the application of such a correction are then given by:
One may employ any suitable technique to arrive at channel scaling deltas Δsc, within some tolerable range, so that the absolute value of the overall loudness scaling error Δsm in Eqn. 6e is made smaller than that in Eqn. 6d. Thus, the absolute value of Δsm is made smaller. In the two implementation examples given below, it is, ideally, reduced to zero. However, the degree of the reduction in the absolute value of Δsm may be traded off against the size of each channel scaling delta Δsc so as to minimize audible channel loudness variation artifacts, in which case the ideal value of Δsm is not zero. The two examples of correction implementations are next described below.
An example of one way to implement such a correction is to compute first the individual channel gains gc according to a basic implementation of the invention as in Eqn. 5b and to compute next a single correction gain G for all channels that is multiplied by each channel gain gc to yield corrected channel gains, gc Δ=Ggc. The gain G is computed so that the overall loudness after the application of the gains gc Δ to each channel is equal to the original overall loudness scaled by the desired amount:
s m L m = F { ∑ c = 1 C g c Δ P c } = F { G ∑ c = 1 C g c P c } ( 7 a ) Solving for G yields:
This correction reduces the absolute value of the overall loudness scaling error Δsm. Ideally, as is evident from inspection of Eqn. 7a (there is no Δsm factor—the scaling error is set to zero), it is reduced to zero. In practical arrangements, the scaling error may not be zero as a result of calculation accuracy, signal processing time lags, etc. Also, as mentioned above, the size of each channel scaling delta Δsc may be taken into account in limiting the degree of reduction of the Δsm error factor.
The corresponding channel scaling deltas Δsc are not specified directly but rather implicitly through the calculation of G. Given G, one may rearrange Eqn. 6f to solve for each channel's scaling delta Δsc as the ratio of the loudness of the particular channel after the application of the corrected channel gain gc Δ to the loudness of the original channel minus the desired overall loudness scaling:
Note that it is not necessary to solve for Δsc (the desired correction to the overall (multichannel) loudness is effected by adjusting each channel's gain by applying the common G factor). Eqn. 7c is shown for the purpose of exposition in explaining the first correction example.
Because in practice the overall loudness scaling achieved by way of the individual channel gains is close to the desired overall loudness scaling sm, the resulting correction gain G typically is close to unity and the corresponding channel scaling deltas are close to zero. As a result, the correction is not likely to cause any objectionable spatial changes.
An example of another way to apply a correction is to find a channel scaling delta Δs common to all channels, such that Δsc=Δs for all channels, which results in reducing the absolute value of the overall loudness scaling error Δsm. Ideally, as is evident from inspection of Eqn. 8 (there is no Δsm factor—the scaling error is set to zero), it is reduced to zero. In practical arrangements, the scaling error may not be zero as a result of calculation accuracy, signal processing time lags, etc. Plugging these constraints into Eqn. 6e yields the condition:
s m L m = F { ∑ c = 1 C F - 1 { ( s m + Δ s ) L c } } ( 8 ) One may solve Eqn. 8 for Δs and then compute the corresponding corrected channel gains gc Δ using Eqn. 6f in which Δsc=Δs for all channels. In practice, solving Eqn. 8 for Δs requires an iterative numerical technique and is therefore less desirable than the first correction implementation described.
Aspects of the two above-described correction examples may be summarized in the following table:
Summary of Correction Examples
Loudness Scaling (per
Total gain of Ggc applied to
Different scaling sm +
each channel. G is the same
Δsc for each channel
for each channel, but gc is
different for each channel.
Solve for each channel's gc The loudness scaling
using Eqn. 5b and for
delta Δsc is implicitly
common G using Eqn. 7b
determined when
A different gain gc Δ is
Same scaling sm + Δs
applied to each channel.
Solve for each channel's
Solve for the loudness
gc Δ using Eqn. 8 and Eqn. 6f
scaling delta Δs using
in which Δsc = Δs for all
Other techniques may exist for applying approximately the same loudness scaling to each individual channel of a multichannel signal while at the same time applying approximately a desired change to the overall loudness, and this invention is meant to cover all such techniques.
In said WO 2004/111994 A2 application and said PCT/US2005/038579 application, Seefeldt et al and Seefeldt disclose, among other things, an objective measure of perceived loudness based on a psychoacoustic model. From a monophonic audio signal, x[n], the method first computes an excitation signal E[b,t] approximating the distribution of energy along the basilar membrane of the inner ear at critical band b during time block t. This excitation may be computed from the Short-time Discrete Fourier Transform (STDFT) of the audio signal as follows:
E [ b , t ] = λ b E [ b , t - 1 ] + ( 1 - λ b ) ∑ k  T [ k ]  2  C b [ k ]  2  X [ k , t ]  2 ( 9 ) where X[k,t] represents the STDFT of x[n] at time block t and bin k. T[k] represents the frequency response of a filter simulating the transmission of audio through the outer and middle ear, and Cb[k] represents the frequency response of the basilar membrane at a location corresponding to critical band b. FIG. 3 depicts a suitable set of critical band filter responses in which forty bands are spaced uniformly along the Equivalent Rectangular Bandwidth (ERB) scale, as defined by Moore and Glasberg (B. C. J. Moore, B. Glasberg, T. Baer, “A Model for the Prediction of Thresholds, Loudness, and Partial Loudness,” Journal of the Audio Engineering Society, Vol. 45, No. 4, April 1997, pp. 224-240). Each filter shape is described by a rounded exponential function and the bands are distributed using a spacing of 1 ERB. Lastly, the smoothing time constant λb in (9) may be advantageously chosen proportionate to the integration time of human loudness perception within band b.
Using equal loudness contours, such as those depicted in FIG. 2, the excitation at each band is transformed into an excitation level that would generate the same loudness at 1 kHz. Specific loudness, a measure of perceptual loudness distributed across frequency and time, is then computed from the transformed excitation, E1 kHz[b,t], through a compressive non-linearity. One such suitable function to compute the specific loudness N[b,t] is given by:
N [ b , t ] = β ( ( E 1 kHz [ b , t ] TQ 1 kHz ) α - 1 ) ( 10 ) where TQ1 kHz the threshold in quiet at 1 kHz and the constants β and α are chosen to match growth of loudness data as shown in FIG. 1. Finally, the total loudness, L[t], represented in units of sone, is computed by summing the specific loudness across bands:
For the purposes of adjusting the audio signal, one may wish to compute a wideband gain g[t], which when multiplied by the audio signal makes the loudness of the adjusted audio equal to some desired target loudness, {circumflex over (L)}[t], as measured by the described psychoacoustic technique. The target loudness {circumflex over (L)}[t] may be computed in a variety of ways. For example, in the case of a volume control it may be computed as a fixed scaling of the original loudness L[t]. Alternatively, more sophisticated functions of the loudness L[t] may be used, such as an Automatic Gain Control (AGC) or Dynamic Range Control (DRC). Regardless of how {circumflex over (L)}[t] is computed, the corresponding gain g[t] is computed in the same way. Letting the function FL represent the transformation from excitation to loudness such that
L[t]=FL{E[b,t]} (12a)
the gain g[t] is computed such that
{circumflex over (L)}[t]=FL{g[t]E[b,t]} (12b)
Rearranging (12a-b), one arrives at the solution
g [ t ] = F L - 1 { s [ t ] L [ t ] } E [ b , t ] for any b ( 12 c ) where s[t] is the loudness scaling associated with {circumflex over (L)}[t] such that
s [ t ] = L ^ [ t ] L [ t ] ( 12 d ) and the inverse function FL −1 is constrained to generate an excitation that is a wideband scaling of the original excitation E[b,t]. Due to the nature of the function FL (a non-linearity applied to each band followed by a summation across bands), a closed form solution for the inverse function FL −1 does not exist. Instead, an iterative technique described in said WO 2004/111994 A2 application may be used to solve for the gain g[t].
Rather than compute a wideband gain g[t] to modify the audio, one may instead compute a multiband gain g[b,t] which when applied to the original audio results in a modified audio signal whose specific loudness is substantially equal to some desired target specific loudness {circumflex over (N)}[b,t]. By computing a multiband gain instead of a wideband gain, control of the perceived spectral balance, or timbre, of the audio may be achieved. For example, with a volume control, the target specific loudness may be computed as a band-independent scaling of the original specific loudness N[b,t], thereby preserving the original timbre of the audio as the volume is changed. In said PCT/US2005/038579 application, a variety of other techniques for computing {circumflex over (N)}[b,t] as a function of N[b,t] are described, including AGC, multiband DRC, and Dynamic EQ (DEQ). Letting the function FN represent the transformation from excitation to specific loudness such that
N[b,t]=FN{E[b,t]} (13a)
the gain g[b,t] is computed such that
{circumflex over (N)}[b,t]=FN{g[b,t]} (13b)
Rearranging (13a-b), one arrives at the solution
g [ b , t ] = F N - 1 { s [ b , t ] N [ b , t ] } E [ b , t ] ( 13 c ) where s[b,t] is the specific loudness scaling associated with {circumflex over (N)}[b,t] such that
In said PCT/US2005/038579 application, several techniques for computing FN −1 in (12c) are described, including a closed form expression, a lookup table, and iterative search.
Consider now a multichannel audio signal xc[n], c=1 . . . C, from which an excitation Ec[b,t] may be computed for each channel c. A total excitation Em[b,t] for the multichannel signal may be computed by summing all the channel excitations:
E m [ b , t ] = ∑ c = 1 C E c [ b , t ] ( 14 a ) and a corresponding total loudness and specific loudness may be computed from the total excitation according to:
Lm[t]=FL{Em[b,t]} (14b)
Nm[b,t]=FN{Em[b,t]} (14c)
Likewise the loudness and specific loudness of each individual channel may be computed from each channel excitation:
Lc[t]=FL{Ec[b,t]} (15a)
Nc[b,t]=FN{Ec[b,t]} (15b)
Now suppose that one wishes to modify the multichannel audio signal so that either the total loudness Lm[t] is scaled by sm[t] or the total specific loudness Nm[b,t] is scaled by sm[b,t]. In the first case, one may solve for a wideband gain gm[t] such that
sm[t]Lm[t]=FL{gm[t]Em[b,t]} (16a)
and in the second case solve for a multiband gain gm[b,t] such that
sm[b,t]Nm[b,t]=FN{gm[b,t]Em[b,t]} (16b)
In both cases the same gain is then applied to all channels c, but as discussed earlier, this may result is a distortion of the perceived spatial balance of the multichannel signal. In order to preserve the spatial balance, one may instead compute gains gc[t] or gc[b,t] for each channel such that each individual channel loudness or specific loudness is scaled by the desired amount:
sm[t]Lc[t]=FL{gc[t]Ec[b,t]} (17a)
sm[b,t]Nc[b,t]=FN{gc[b,t]Ec[b,t]} (17b)
This way, the relative loudness or specific loudness between all channels is preserved. However, when these gains gc[t] or gc[b,t] are applied to the corresponding channels of the original multichannel audio, the total loudness of the resulting modified multichannel audio signal may not exactly equal the total loudness of the original multichannel audio signal scaled by the desired amount. More specifically,
In many cases, the two sides of Eqns. 18a and 18b will be nearly equal and therefore for some applications the resulting error may be ignored. For the best results, however, one may compute a correction gain G[t] or G[b,t] applied to all channels such that
This way the desired total loudness scaling may be achieved. In most cases, the correction gain G[t] or G[b,t] is small, and therefore the spatial balance of the multichannel signal is largely preserved.
In FIGS. 4 a-4 f are depicted plots of the specific loudness and multiband gains for the modification of a multichannel audio signal consisting of five channels: left, center, right, left-surround, and right-surround. This particular audio signal is dominated by dialogue in the center channel, with the remaining four channels containing ambience signals of a much lower level used to convey to the impression of being in a large hall. For this particular case, the multiband gains gm[b,t] and gc[b,t] (c=1 . . . 5) are computed in order to achieve a specific loudness scaling of sm[b,t]=0.16 for all bands b. Examining the center channel (c=2) in FIG. 4 b, one notes that the two specific loudness spectra resulting from the application of the same gain for all channels gm[b,t] and the channel-specific gain g2[b,t] are nearly identical. This is because the center channel contains the vast majority of the signal energy, and therefore computation of gm[b,t] from the combined excitation Em[b,t] is influenced mainly by this channel. Examining the remaining channels, however, one notes a large discrepancy between the two specific loudness spectra resulting from the application of gm[b,t] and gc[b,t]. In these cases, because the signals are so small in comparison to the center channel, application of gm[b,t] results in a modified specific loudness that is far smaller than the desired scaling of 0.16. For many bands, the modified specific loudness falls below the threshold of hearing. This is most evident in the left and right surround channels (c=4 and 5). Application of gc[b,t], on the other hand, results in the desired specific loudness scaling. In FIG. 4 f is depicted the specific loudness of all channels combined after the application of gm[b,t] to all channels and of gc[b,t] to each respective channel. One notes that, in the first case, the modified specific loudness is equal to the original combined specific loudness scaled by the desired amount, as expected. Application of gc[b,t] to each respective channel results in a modified specific loudness that is close to this result, but a small error exists at the lower and higher bands. This error is eliminated through the further application of the correction gain G[b,t], which is close to zero dB for most bands b. The average absolute value of G[b,t] across bands is 0.6 dB, and the maximum absolute value of G[b,t] is only 3.7 dB. Returning to FIGS. 4 a-4 e, one notes that the application of the correction gain has only a minor effect on the modified specific loudness of each individual channel.
The invention may be implemented in hardware or software, or a combination of both (e.g., programmable logic arrays). Unless otherwise specified, algorithms and processes included as part of the invention are not inherently related to any particular computer or other apparatus. In particular, various general-purpose machines may be used with programs written in accordance with the teachings herein, or it may be more convenient to construct more specialized apparatus (e.g., integrated circuits) to perform the required method steps. Thus, the invention may be implemented in one or more computer programs executing on one or more programmable computer systems each comprising at least one processor, at least one data storage system (including volatile and non-volatile memory and/or storage elements), at least one input device or port, and at least one output device or port. Program code is applied to input data to perform the functions described herein and generate output information. The output information is applied to one or more output devices, in known fashion.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS2808475Oct 5, 1954Oct 1, 1957Bell Telephone Labor IncLoudness indicatorUS4281218Oct 26, 1979Jul 28, 1981Bell Telephone Laboratories, IncorporatedSpeech-nonspeech detector-classifierUS4543537Apr 23, 1984Sep 24, 1985U.S. Philips CorporationMethod of and arrangement for controlling the gain of an amplifierUS4739514Dec 22, 1986Apr 19, 1988Bose CorporationAutomatic dynamic equalizingUS4887299Nov 12, 1987Dec 12, 1989Nicolet Instrument CorporationAdaptive, programmable signal processing hearing aidUS5027410Nov 10, 1988Jun 25, 1991Wisconsin Alumni Research FoundationAdaptive, programmable signal processing and filtering for hearing aidsUS5097510Nov 7, 1989Mar 17, 1992Gs Systems, Inc.Artificial intelligence pattern-recognition-based noise reduction system for speech processingUS5172358Mar 2, 1990Dec 15, 1992Yamaha CorporationLoudness control circuit for an audio deviceUS5278912Jun 28, 1991Jan 11, 1994Resound CorporationMultiband programmable compression systemUS5363147Jun 1, 1992Nov 8, 1994North American Philips CorporationAutomatic volume levelerUS5369711Aug 31, 1990Nov 29, 1994Bellsouth CorporationAutomatic gain control for a headsetUS5377277Nov 17, 1993Dec 27, 1994Bisping; RudolfProcess for controlling the signal-to-noise ratio in noisy sound recordingsUS5457769Dec 8, 1994Oct 10, 1995Earmark, Inc.Method and apparatus for detecting the presence of human voice signals in audio signalsUS5500902Jul 8, 1994Mar 19, 1996Stockham, Jr.; Thomas G.Hearing aid device incorporating signal processing techniquesUS5530760Apr 29, 1994Jun 25, 1996Audio Products International Corp.Apparatus and method for adjusting levels between channels of a sound systemUS5548638Aug 10, 1995Aug 20, 1996Iwatsu Electric Co., Ltd.Audio teleconferencing apparatusUS5583962Jan 8, 1992Dec 10, 1996Dolby Laboratories Licensing CorporationEncoder/decoder for multidimensional sound fieldsUS5615270Jun 6, 1995Mar 25, 1997International Jensen IncorporatedMethod and apparatus for dynamic sound optimizationUS5632005Jun 7, 1995May 20, 1997Ray Milton DolbyEncoder/decoder for multidimensional sound fieldsUS5633981Jun 7, 1995May 27, 1997Dolby Laboratories Licensing CorporationMethod and apparatus for adjusting dynamic range and gain in an encoder/decoder for multidimensional sound fieldsUS5649060Oct 23, 1995Jul 15, 1997International Business Machines CorporationAutomatic indexing and aligning of audio and text using speech recognitionUS5663727Jun 23, 1995Sep 2, 1997Hearing Innovations IncorporatedFrequency response analyzer and shaping apparatus and digital hearing enhancement apparatus and method utilizing the sameUS5682463Feb 6, 1995Oct 28, 1997Lucent Technologies Inc.Perceptual audio compression based on loudness uncertaintyUS5712954Aug 23, 1995Jan 27, 1998Rockwell International Corp.System and method for monitoring audio power level of agent speech in a telephonic switchUS5724433Jun 7, 1995Mar 3, 1998K/S HimppAdaptive gain and filtering circuit for a sound reproduction systemUS5727119Mar 27, 1995Mar 10, 1998Dolby Laboratories Licensing CorporationMethod and apparatus for efficient implementation of single-sideband filter banks providing accurate measures of spectral magnitude and phaseUS5819247Jul 29, 1997Oct 6, 1998Lucent Technologies, Inc.Apparatus and methods for machine learning hypothesesUS5848171Jan 12, 1996Dec 8, 1998Sonix Technologies, Inc.Hearing aid device incorporating signal processing techniquesUS5862228Feb 21, 1997Jan 19, 1999Dolby Laboratories Licensing CorporationAudio matrix encodingUS5878391Jul 3, 1997Mar 2, 1999U.S. Philips CorporationDevice for indicating a probability that a received signal is a speech signalUS5907622Sep 21, 1995May 25, 1999Dougherty; A. MichaelAutomatic noise compensation system for audio reproduction equipmentUS5909664May 23, 1997Jun 1, 1999Ray Milton DolbyMethod and apparatus for encoding and decoding audio information representing three-dimensional sound fieldsUS6002776Sep 18, 1995Dec 14, 1999Interval Research CorporationDirectional acoustic signal processor and method thereforUS6002966Apr 23, 1996Dec 14, 1999Advanced Bionics CorporationMultichannel cochlear prosthesis with flexible control of stimulus waveformsUS6021386Mar 9, 1999Feb 1, 2000Dolby Laboratories Licensing CorporationCoding method and apparatus for multiple channels of audio information representing three-dimensional sound fieldsUS6041295Apr 10, 1996Mar 21, 2000Corporate Computer SystemsComparing CODEC input/output to adjust psycho-acoustic parametersUS6061647Apr 30, 1998May 9, 2000British Telecommunications Public Limited CompanyVoice activity detectorUS6088461Sep 26, 1997Jul 11, 2000Crystal Semiconductor CorporationDynamic volume control systemUS6094489Sep 15, 1997Jul 25, 2000Nec CorporationDigital hearing aid and its hearing sense compensation processing methodUS6108431Oct 1, 1996Aug 22, 2000Phonak AgLoudness limiterUS6125343May 29, 1997Sep 26, 20003Com CorporationSystem and method for selecting a loudest speaker by comparing average frame gainsUS6148085Aug 28, 1998Nov 14, 2000Samsung Electronics Co., Ltd.Audio signal output apparatus for simultaneously outputting a plurality of different audio signals contained in multiplexed audio signal via loudspeaker and headphoneUS6182033Jul 22, 1998Jan 30, 2001At&T Corp.Modular approach to speech enhancement with an application to speech codingUS6185309Jul 11, 1997Feb 6, 2001The Regents Of The University Of CaliforniaMethod and apparatus for blind separation of mixed and convolved sourcesUS6233554Dec 12, 1997May 15, 2001Qualcomm IncorporatedAudio CODEC with AGC controlled by a VOCODERUS6240388Jul 8, 1997May 29, 2001Hiroyuki FukuchiAudio data decoding device and audio data coding/decoding systemUS6263371Jun 10, 1999Jul 17, 2001Cacheflow, Inc.Method and apparatus for seaming of streaming contentUS6272360Jul 3, 1997Aug 7, 2001Pan Communications, Inc.Remotely installed transmitter and a hands-free two-way voice terminal device using sameUS6275795Jan 8, 1999Aug 14, 2001Canon Kabushiki KaishaApparatus and method for normalizing an input speech signalUS6298139Dec 31, 1997Oct 2, 2001Transcrypt International, Inc.Apparatus and method for maintaining a constant speech envelope using variable coefficient automatic gain controlUS6301555Mar 25, 1998Oct 9, 2001Corporate Computer SystemsAdjustable psycho-acoustic parametersUS6311155May 26, 2000Oct 30, 2001Hearing Enhancement Company LlcUse of voice-to-remaining audio (VRA) in consumer applicationsUS6314396Nov 6, 1998Nov 6, 2001International Business Machines CorporationAutomatic gain control in a speech recognition systemUS6327366May 1, 1996Dec 4, 2001Phonak AgMethod for the adjustment of a hearing device, apparatus to do it and a hearing deviceUS6332119Mar 20, 2000Dec 18, 2001Corporate Computer SystemsAdjustable CODEC with adjustable parametersUS6351731Aug 10, 1999Feb 26, 2002Polycom, Inc.Adaptive filter featuring spectral gain smoothing and variable noise multiplier for noise reduction, and method thereforUS6351733May 26, 2000Feb 26, 2002Hearing Enhancement Company, LlcMethod and apparatus for accommodating primary content audio and secondary content remaining audio capability in the digital audio production processUS6353671Feb 5, 1998Mar 5, 2002Bioinstco Corp.Signal processing circuit and method for increasing speech intelligibilityUS6370255Jul 17, 1997Apr 9, 2002Bernafon AgLoudness-controlled processing of acoustic signalsUS6411927Sep 4, 1998Jun 25, 2002Matsushita Electric Corporation Of AmericaRobust preprocessing signal equalization system and method for normalizing to a target environmentUS6430533Apr 17, 1998Aug 6, 2002Lsi Logic CorporationAudio decoder core MPEG-1/MPEG-2/AC-3 functional algorithm partitioning and implementationUS6442278May 26, 2000Aug 27, 2002Hearing Enhancement Company, LlcVoice-to-remaining audio (VRA) interactive center channel downmixUS6442281May 21, 1997Aug 27, 2002Pioneer Electronic CorporationLoudness volume control systemUS6473731Nov 30, 2000Oct 29, 2002Corporate Computer SystemsAudio CODEC with programmable psycho-acoustic parametersUS6498855Apr 17, 1998Dec 24, 2002International Business Machines CorporationMethod and system for selectively and variably attenuating audio dataUS6529605Jun 29, 2000Mar 4, 2003Harman International Industries, IncorporatedMethod and apparatus for dynamic sound optimizationUS6570991Dec 18, 1996May 27, 2003Interval Research CorporationMulti-feature speech/music discrimination systemUS6625433Sep 29, 2000Sep 23, 2003Agere Systems Inc.Constant compression automatic gain control circuitUS6639989Sep 22, 1999Oct 28, 2003Nokia Display Products OyMethod for loudness calibration of a multichannel sound systems and a multichannel sound systemUS6650755Jun 25, 2002Nov 18, 2003Hearing Enhancement Company, LlcVoice-to-remaining audio (VRA) interactive center channel downmixUS6651041Jun 21, 1999Nov 18, 2003Ascom AgMethod for executing automatic evaluation of transmission quality of audio signals using source/received-signal spectral covarianceUS6700982Jun 7, 1999Mar 2, 2004Cochlear LimitedHearing instrument with onset emphasisUS6807525Oct 31, 2000Oct 19, 2004Telogy Networks, Inc.SID frame detection with human auditory perception compensationUS6823303Sep 18, 1998Nov 23, 2004Conexant Systems, Inc.Speech encoder using voice activity detection in coding noiseUS6889186Jun 1, 2000May 3, 2005Avaya Technology Corp.Method and apparatus for improving the intelligibility of digitally compressed speechUS6985594Jun 14, 2000Jan 10, 2006Hearing Enhancement Co., Llc.Voice-to-remaining audio (VRA) interactive hearing aid and auxiliary equipmentUS7065498Apr 7, 2000Jun 20, 2006Texas Instruments IncorporatedSupply of digital audio and video productsUS7068723Feb 28, 2002Jun 27, 2006Fuji Xerox Co., Ltd.Method for automatically producing optimal summaries of linear mediaUS7155385May 16, 2002Dec 26, 2006Comerica Bank, As Administrative AgentAutomatic gain control for adjusting gain during non-speech portionsUS7171272Aug 20, 2001Jan 30, 2007University Of MelbourneSound-processing strategy for cochlear implantsUS7212640Nov 29, 2000May 1, 2007Bizjak Karl MVariable attack and release system and methodUS7454331Aug 30, 2002Nov 18, 2008Dolby Laboratories Licensing CorporationControlling loudness of speech in signals that contain speech and other types of audio materialUS20010027393Dec 8, 2000Oct 4, 2001Touimi Abdellatif BenjellounMethod of and apparatus for processing at least one coded binary audio flux organized into framesUS20010038643Jan 29, 2001Nov 8, 2001British Broadcasting CorporationMethod for inserting auxiliary data in an audio data streamUS20020013698Aug 23, 2001Jan 31, 2002Vaudrey Michael A.Use of voice-to-remaining audio (VRA) in consumer applicationsUS20020040295Dec 10, 2001Apr 4, 2002Saunders William R.Method and apparatus for accommodating primary content audio and secondary content remaining audio capability in the digital audio production processUS20020076072Nov 7, 2001Jun 20, 2002Cornelisse Leonard E.Software implemented loudness normalization for a digital hearing aidUS20020097882Nov 29, 2001Jul 25, 2002Greenberg Jeffry AllenMethod and implementation for detecting and characterizing audible transients in noiseUS20020146137Apr 10, 2001Oct 10, 2002Phonak AgMethod for individualizing a hearing aidUS20020147595Feb 22, 2001Oct 10, 2002Frank BaumgarteCochlear filter bank structure for determining masked thresholds for use in perceptual audio codingUS20030002683Jun 25, 2002Jan 2, 2003Vaudrey Michael A.Voice-to-remaining audio (VRA) interactive center channel downmixUS20030035549Nov 29, 2000Feb 20, 2003Bizjak Karl M.Signal processing system and methodUS20040024591Oct 22, 2002Feb 5, 2004Boillot Marc A.Method and apparatus for enhancing loudness of an audio signalUS20040037421Dec 17, 2001Feb 26, 2004Truman Michael MeadParital encryption of assembled bitstreamsUS20040042617Oct 11, 2001Mar 4, 2004Beerends John GerardMeasuring a talking quality of a telephone link in a telecommunications neworkUS20040044525Aug 30, 2002Mar 4, 2004Vinton Mark StuartControlling loudness of speech in signals that contain speech and other types of audio materialUS20040076302Feb 14, 2002Apr 22, 2004Markus ChristophDevice for the noise-dependent adjustment of sound volumesUS20040122662Feb 12, 2002Jun 24, 2004Crockett Brett GrehamHigh quality time-scaling and pitch-scaling of audio signalsUS20040148159Feb 25, 2002Jul 29, 2004Crockett Brett GMethod for time aligning audio signals using characterizations based on auditory eventsUS20040165730Feb 26, 2002Aug 26, 2004Crockett Brett GSegmenting audio signals into auditory eventsUS20040172240Feb 22, 2002Sep 2, 2004Crockett Brett G.Comparing audio using characterizations based on auditory eventsUS20040184537Aug 7, 2003Sep 23, 2004Ralf GeigerMethod and apparatus for scalable encoding and method and apparatus for scalable decodingUS20040190740Feb 26, 2004Sep 30, 2004Josef ChalupperMethod for automatic amplification adjustment in a hearing aid device, as well as a hearing aid deviceUS20040213420Apr 24, 2003Oct 28, 2004Gundry Kenneth JamesVolume and compression control in movie theatersUS20060002572Jul 1, 2004Jan 5, 2006Smithers Michael JMethod for correcting metadata affecting the playback loudness and dynamic range of audio informationUS20060215852Feb 8, 2006Sep 28, 2006Dana TroxelMethod and apparatus for identifying feedback in a circuitUS20070291959Oct 25, 2005Dec 20, 2007Dolby Laboratories Licensing CorporationCalculating and Adjusting the Perceived Loudness and/or the Perceived Spectral Balance of an Audio SignalUSRE34961May 26, 1992Jun 6, 1995The Minnesota Mining And Manufacturing CompanyMethod and apparatus for determining acoustic parameters of an auditory prosthesis using software modelDE4335739A1Oct 20, 1993May 19, 1994Rudolf Prof Dr BispingAutomatically controlling signal=to=noise ratio of noisy recordingsDE19848491A1Oct 21, 1998Apr 27, 2000Bosch Gmbh RobertRadio receiver with audio data system has control unit to allocate sound characteristic according to transferred program type identification adjusted in receiving sectionEP0517233B1Jun 5, 1992Oct 30, 1996Matsushita Electric Industrial Co., Ltd.Music/voice discriminating apparatusEP0637011B1Jul 21, 1994Oct 14, 1998Philips Electronics N.V.Speech signal discrimination arrangement and audio device including such an arrangementEP0661905B1Mar 13, 1995Dec 11, 2002Phonak AgMethod for the fitting of hearing aids, device therefor and hearing aidEP0746116B1May 30, 1996Jul 9, 2003Mitsubishi Denki Kabushiki KaishaMPEG audio decoderEP1239269A4Mar 29, 2001Dec 19, 2007Nat Inst Of Advanced Ind ScienSound measuring method and device allowing for auditory sense characteristicsEP1251715B1Apr 18, 2002Feb 15, 2006Gennum CorporationMulti-channel hearing instrument with inter-channel communicationEP1387487A3Jul 17, 2003Oct 27, 2004Pioneer CorporationMethod and apparatus for adjusting frequency characteristic of signalEP1736966B1May 30, 2003Jul 7, 2010Dolby Laboratories Licensing CorporationMethod for generating audio informationFR2820573B1 Title not availableWO2007120452A1Mar 30, 2007Oct 25, 2007Dolby Laboratories Licensing CorporationAudio signal loudness measurement and modification in the mdct domainWO2007120453A1Mar 30, 2007Oct 25, 2007Dolby Laboratories Licensing CorporationCalculating and adjusting the perceived loudness and/or the perceived spectral balance of an audio signalWO2007127023A1Mar 30, 2007Nov 8, 2007Dolby Laboratories Licensing CorporationAudio gain control using specific-loudness-based auditory event detectionWO2008085330A1Dec 17, 2007Jul 17, 2008Dolby Laboratories Licensing CorporationHybrid digital/analog loudness-compensating volume controlNon-Patent CitationsReference1Atkinson, I. A., et al., "Time Envelope LP Vocoder: A New Coding Technology at Very Low Bit Rates," 4th ed., 1995, ISSN 1018-4074, pp. 241-244.2ATSC Standard A52/A: Digital Audio Compression Standard (AC-3), Revision A, Advanced Television Systems Committee, Aug. 20, 2001. The A/52A document is available on the World Wide Web at http://www./atsc.org. standards.html.3Australian Broadcasting Authority (ABA), "Investigation into Loudness of Advertisements," Jul. 2002.4Australian Government IP Australia, Examiner's first report on patent application No. 2005299410, mailed Jun. 25, 2009, Australian Patent Appln. No. 2005299410.5Belger, "The Loudness Balance of Audio Broadcast Programs," J. Audio Eng. Soc., vol. 17, No. 3, Jun. 1969, pp. 282-285.6Bertsekas, Dimitri P., "Nonlinear Programming," 1995, Chapter 1.2 "Gradient Methods-Convergence," pp. 18-46.7Bertsekas, Dimitri P., "Nonlinear Programming," 1995, Chapter 1.2 "Gradient Methods—Convergence," pp. 18-46.8Bertsekas, Dimitri P., "Nonlinear Programming," 1995, Chapter 1.8 "Nonderivative Methods,", pp. 142-148.9Blesser, Barry, "An Ultraminiature console Compression System with Maximum User Flexibility," Journal of Audio Engineering Society, vol. 20, No. 4, May 1972 (1972-05), pp. 297-302.10Bosi, et al., "High Quality, Low-Rate Audio Transform Coding for Transmission and Multimedia Applications," Audio Engineering Society Preprint 3365, 93rd AES Convention, Oct. 1992.11Bosi, et al., "ISO/IEC MPEG-2 Advanced Audio coding," J. Audio Eng. Soc., vol. 45, No. 10, Oct. 1997, pp. 789-814.12Brandenburg, et al., "Overview of MPEG Audio: Current and Future Standards for Low-Bit-Rate Audio Coding," J. Audio eng. Soc., vol. 45, No. 1/2, Jan./Feb. 1997.13Bray, et al.; "An "Optimized" Platform for DSP Hearing Aids," Sonic Innovations, vol. 1 No. 3 1998, pp. 1-4, presented at the Conference on Advanced Signal Processing Hearing Aids, Cleveland, OH, Aug. 1, 1998.14Bray, et al.; "Digital Signal Processing (DSP) Derived from a Nonlinear Auditory Model," Sonic Innovations, vol. 1 No. 1 1998, pp. 1-3, presented at American Academy of Audiology, Los Angeles, CA, Apr. 4, 1998.15Bray, et al.; "Optimized Target Matching: Demonstration of an Adaptive Nonlinear DSP System," Sonic Innovations vol. 1 No. 2 1998, pp. 1-4, presented at the American Academy of Audiology, Los Angeles, CA, Apr. 4, 1998.16Carroll, Tim, "Audio Metadata: You can get there from here", Oct. 11, 2004, pp. 1-4, XP002392570. http://tvtechnology.com/features/audio-notes/f-TC-metadata-08.21.02.shtml.17Carroll, Tim, "Audio Metadata: You can get there from here", Oct. 11, 2004, pp. 1-4, XP002392570. http://tvtechnology.com/features/audio—notes/f-TC-metadata-08.21.02.shtml.18CEI/IEC Standard 60804 published Oct. 2000.19Chalupper, Josef; "Aural Exciter and Loudness Maximizer: What's Psychoacoustic about " Psychoacoustic Processors?, Audio Engineering Society (AES) 108th Convention, Sep. 22-25, 2000, Los Angeles, CA, pp. 1-20.20Cheng-Chieh Lee, "Diversity Control Among Multiple Coders: A Simple Approach to Multiple Descriptions," IEE, September.21Claro Digital Perception Processing; "Sound Processing with a Human Perspective," pp. 1-8.22Communication Under Rule 51(4) EPC, European Patent Office, EP Application No. 03791682.2-2218, dated Dec. 5, 2005.23Crockett, Brett, "High Quality Multichannel Time Scaling and Pitch-Shifting using Auditory Scene Analysis," Audio Engineering Society Convention Paper 5948, New York, Oct. 2003.24Crockett, et al., "A Method for Characterizing and Identifying Audio Based on Auditory Scene Analysis," Audio Engineering Society Convention Paper 6416, 118th Convention, Barcelona, May 28-31, 2005.25Davis, Mark, "The AC-3 Multichannel Coder," Audio engineering Society, Preprint 3774, 95th AES Convention, Oct. 1993.26Dept of Justice & Human Rights of Republic of Indonesia, Directorate General Intellectual Property Rights, First Office Action received Apr. 22, 2010, Indonesian Patent Appln. No. WO0200701285.27European Patent Office Searching Authority, Int'l Search Report and Written Opinion, Int'l Appln. No. PCT/US2004/016964, mailed Jun. 20, 2005.28European Patent Office, Office Action dated Apr. 2, 2008, EP Application No. 05818505.9.29European Patent Office, Response to Office Action dated Apr. 2, 2008, EP Application No. 05818505.9.30Fielder, et al., "Introduction to Dolby Digital Plus, an Enhancement to the Dolby Digital Coding System," AES Convention Paper 6196, 117th AES Convention, Oct. 28, 2004.31Fielder, et al., "Professional Audio Coder Optimized fro Use with Video," AES Preprint 5033, 107th AES Conference, Aug. 1999.32Ghent, Jr., et al.; "Expansion as a Sound Processing Tool in Hearing Aids," American Academy of Audiology National Convention, Apr. 29-May 2, 1999, Miami Beach, FL.33Ghent, Jr., et al.; "Uses of Expansion to Promote Listening Comfort with Hearing Aids," American Academy of Audiology 12th Annual Convention, Mar. 16-19, 2000, Chicago, IL.34Ghent, Jr., et al.; "Uses of Expansion to Promote Listening Comfort with Hearing Aids," Sonic Innovations, vol. 3 No. 2, 2000, pp. 1-4, presented at American Academy of Audiology 12th Annual Convention, Chicago, IL, Mar. 16-19, 2000.35Glasberg, et al., "A Model of Loudness Applicable to Time-Varying Sounds," Journal of the Audio Engineering Society, Audio Engineering Society, New York, vol. 50, No. 5, May 2002, pp. 331-342.36Guide to the Use of the ATSC Digital Television Standard, Dec. 4, 2003.37H. H. Scott, "The Amplifier and its Place in the High Fidelity System," J. Audio Eng. Soc., vol. 1, No. 3, Jul. 1953.38Hauenstein M., "A Computationally Efficient Algorithm for Calculating Loudness Patterns of Narrowband Speech," Acoustics, Speech and Signal Processing 1997. 1997 IEEE International Conference, Munich Germany, Apr. 21-24, 1997, Los Alamitos, Ca, USA, IEEE Comput. Soc., US, Apr. 21, 1997, pp. 1311-1314.39Hermesand, et al., "Sound Design-Creating the Sound for Complex Systems and Virtual Objects," Chapter II, "Anatomy and Psychoacoustics," 2003-2004.40Hermesand, et al., "Sound Design—Creating the Sound for Complex Systems and Virtual Objects," Chapter II, "Anatomy and Psychoacoustics," 2003-2004.41Hoeg, W., et al., "Dynamic Range Control (DRC) and Music/Speech Control (MSC) Programme-Associated Data Services for DAB", EBU Review-Technical, European Broadcasting Union, Brussels, BE, No. 261, Sep. 21, 1994.42Intellectual Property Corporation of Malaysia, Substantive/Modified Substantive Examination Adverse Report (Section 30(1)/30(2)) and Search Report, dated Dec. 5, 2008, Malaysian Patent Appln. No. PI 20055232.43International Search Report, PCT/US2004/016964 dated Dec. 1, 2005.44International Search Report, PCT/US2005/038579 dated Feb. 21, 2006.45International Search Report, PCT/US2006/010823 dated Jul. 25, 2006.46International Search Report, PCT/US2007/006444 dated Aug. 28, 2007.47International Search Report, PCT/US2007/020747, dated May 21, 2008.48International Search Report, PCT/US2007/022132 dated Apr. 18, 2008.49ISO Standard 532:1975, published 1975.50ISO226 : 1987 (E), "Acoustics-Normal Equal Loudness Level Contours."51ISO226 : 1987 (E), "Acoustics—Normal Equal Loudness Level Contours."52Israel Patent Office, Examiner's Report on Israel Application No. 182097 mailed Apr. 11, 2010, Israel Patent Appln. No. 182097.53Johns, et al.; "An Advanced Graphic Equalizer Hearing Aid: Going Beyond Your Home Audio System," Sonic Innovations Corporation, Mar. 5, 2001, Http://www.audiologyonline.com/articles/pf-arc-disp.asp?id=279.54Johns, et al.; "An Advanced Graphic Equalizer Hearing Aid: Going Beyond Your Home Audio System," Sonic Innovations Corporation, Mar. 5, 2001, Http://www.audiologyonline.com/articles/pf—arc—disp.asp?id=279.55Lin, L., et al., "Auditory Filter Bank Design Using Masking Curves," 7th European Conference on Speech Communications and Technology, Sep. 2001.56Mapes, Riordan, et al., "Towards a model of Loudness Recalibration." 1997 IEEE ASSP workshop on New Paltz, NY USA, Oct. 19-22, 1997.57Martinez G., Isaac; " Automatic Gain Control (AGC) Circuits-Theory and Design," University of Toronto ECE1352 Analog Integrated Circuits I, Term Paper, Fall 2001, pp. 1-25.58Martinez G., Isaac; " Automatic Gain Control (AGC) Circuits—Theory and Design," University of Toronto ECE1352 Analog Integrated Circuits I, Term Paper, Fall 2001, pp. 1-25.59Masciale, John M.; "The Difficulties in Evaluating A-Weighted Sound Level Measurements," S&V Observer, pp. 2-3.60Mexican Patent Application No. PA/a/2005/002290-Response to Office Action dated Oct. 5, 2007.61Mexican Patent Application No. PA/a/2005/002290—Response to Office Action dated Oct. 5, 2007.62Moore, BCJ, "Use of a loudness model for hearing aid fitting, IV. Fitting hearing aids with multi-channel compression so as to restore "normal" loudness for speech at different levels." British Journal of Audiology, vol. 34, No. 3, pp. 165-177, Jun. 2000, Whurr Publishers, UK.63Moore, et al., "A Model for the Prediction of Thresholds, Loudness and Partial Loudness," Journal of the Audio Engineering Society, Audio Engineering Society, New York, vol. 45, No. 4, Apr. 1997, pp. 224-240.64Moulton, Dave, "Loud, Louder, Loudest!," Electronic Musician, Aug. 1, 2003.65Newcomb, et al., "Practical Loudness: an Active Circuit Design Approach," J. Audio eng. Soc., vol. 24, No. 1, Jan./Feb. 1976.66Nigo, et al., "Concert-Hall Realism through the Use of Dynamic Level Control," J. Audio Eng. Soc., vol. 1, No. 1, Jan. 1953.67Nilsson, et al.; "The Evolution of Multi-channel Compression Hearing Aids," Sonic Innovations, Presented at American Academy of Audiology 13th Convention, San Diego, CA, Apr. 19-22, 2001.68Notification of the First Office Action, Chinese Application No. 03819918.1, dated Mar. 30, 2007.69Notification of Transmittal of the International Search Report, PCT/US2006/011202, dated Aug. 9, 2006.70Notification of Transmittal of the International Search Report, PCT/US2007/0025747, dated Apr. 14, 2008.71Notification of Transmittal of the International Search Report, PCT/US2007/007945, dated Aug. 17, 2007.72Notification of Transmittal of the International Search Report, PCT/US2007/007946, dated Aug. 21, 2007.73Notification of Transmittal of the International Search Report, PCT/US2007/08313), dated Sep. 21, 2007.74Notification of Transmittal of the International Search Report, PCT/US2008/007570, dated Sep. 10, 2008.75Official Letter from the Intellectual Property Bureau, Ministry of Economic Affairs, Taiwan, dated Mar. 21, 2008.76Park, et al.; "High Performance Digital Hearing Aid Processor with Psychoacoustic Loudness Correction," IEEE FAM P3.1 0-7803-3734-4/97, pp. 312-313.77Response Office Action from the Israel Patent Office, Israel Patent Application No. 165,398, dated Dec. 29, 2008.78Response to Notification of the First Office Action, Chinese Application No. 03819918.1, dated Aug. 14, 2007.79Response to Official Letter from the Intellectual Property Bureau, Ministry of Economic Affairs, Taiwan, dated Jun. 25, 2008.80Riedmiller, Jeff, "Working Toward Consistency in Program Loudness," Broadcast Engineering, Jan. 1, 2004.81Robinson, et a., Dynamic Range Control via Metadata, 107th Convention of the AES, Sep. 14-27, 1999, New York.82Robinson, et al., "Time-Domain Auditory Model for the Assessment of High-Quality Coded Audio," 107th AES Convention, Sep. 1999.83Saunders, "Real-Time Discrimination of Broadcast Speech/Music," Proc. of Int. Conf on Acoust. Speech and Sig. Proce., 1996, pp. 993-996.84Schapire, "A Brief Introduction to Boosting," Proc. of the 16th Int. Joint Conference on Artificial Intelligence, 1999.85Scheirer and Slaney, "Construction and Evaluation of a robust Multifeature Speech/Music Discriminator," Proc. of Int. Conf. on Acoust. Speech and Sig. Proc., 1997, pp. 1331-1334.86Seefeldt, et al.; "A New Objective Measure of Perceived Loudness," Audio Engineering Society (AES) 117th Convention, Paper 6236, Oct. 28-31, 2004, San Francisco, CA, pp. 1-8.87Seo, et al., "Auditory Model Design for Objective Audio Quality Measurement," Department of Electronic Engineering, Dongguk University, Seoul Korea.88Smith, et al., "Tandem-Free VolP Conferencing: A Bridge to Next-Generation Networks," IEEE Communications Magazine, IEEE Service Center, New York, NY, vol. 41, No. 5, May 2003, pp. 136-145.89Soulodre, GA, "Evaluation of Objective Loudness Meters" Preprints of Papers Presented at the 116th AES Convention, Berlin, Germany, May 8, 2004.90State Intellectual Property Office of the People'S Republic of China, Notification of the Third Office Action, mailed Apr. 21, 2010, China Patent Appln. No. 200580036760.7.91Stevens, "Calculations of the Loudness of Complex Noise," Journal of the Acoustical Society of America, 1956.92The Written Opinion of the International Searching Authority, PCT/US2007/0025747, dated Apr. 14, 2008.93The Written Opinion of the International Searching Authority, PCT/US2007/007945, dated Aug. 17, 2007.94The Written Opinion of the International Searching Authority, PCT/US2007/007946, dated Aug. 21, 2007.95The Written Opinion of the International Searching Authority, PCT/US2007/08313), dated Sep. 21, 2007.96The Written Opinion of the International Searching Authority, PCT/US2008/007570, dated Sep. 10, 2008.97Todd, et al., "Flexible Perceptual Coding for Audio Transmission and Storage," 96th Convention of the Audio Engineering Society, Feb. 26, 1994, Preprint, 3796.98Trapee, W., et al., "Key distribution for secure multimedia multicasts via data embedding," 2001 IEEE International Conferenced on Acoustics, Speech, and Signal Processing. May 7-11, 2001.99Truman, et al., "Efficient Bit Allocation, Quantization, and Coding in an Audio Distribution System," AES Preprint 5068, 107th AES Conference, Aug. 1999.100Vernon, Steve, "Design and Implementation of AC-3 Coders," IEEE Trans. Consumer Electronics, vol. 41, No. 3, Aug. 1995.101Watson, et al., "Signal Duration and Signal Frequency in Relation to Auditory Sensitivity," Journal of the Acoustical Society of America, vol. 46, No. 4 (Part 2) 1969, pp. 989-997.102Written Opinion of the Intellectual Property Office of Singapore, Singapore Application No. 0702926-7, dated May 12, 2008.103Written Opinion of the International Search Authority, PCT/US2006/011202, dated Aug. 9, 2006.104Written Opinion of the International Searching Authority, PCT/US2004/016964 dated Dec. 1, 2005.105Written Opinion of the International Searching Authority, PCT/US2005/038579 dated Feb. 21, 2006.106Written Opinion of the International Searching Authority, PCT/US2006/010823 dated Jul. 25, 2006.107Written Opinion of the International Searching Authority, PCT/US2007/006444 dated Aug. 28, 2007.108Written Opinion of the International Searching Authority, PCT/US2007/022132 dated Apr. 18, 2008.109Zwicker, "Psychological and Methodical Basis of Loudness," Acoustica, 1958.110Zwicker, et al., "Psychoacoustics-Facts and Models," Springer-Verlag, Chapter 8, "Loudness," pp. 203-238, Berlin Heidelberg, 1990, 1999.111Zwicker, et al., "Psychoacoustics—Facts and Models," Springer-Verlag, Chapter 8, "Loudness," pp. 203-238, Berlin Heidelberg, 1990, 1999.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8315398Nov 20, 2012Dts LlcSystem for adjusting perceived loudness of audio signalsUS8315862 *Jun 5, 2009Nov 20, 2012Samsung Electronics Co., Ltd.Audio signal quality enhancement apparatus and methodUS8761415Apr 28, 2010Jun 24, 2014Dolby Laboratories CorporationControlling the loudness of an audio signal in response to spectral localizationUS8891789Apr 29, 2010Nov 18, 2014Dolby Laboratories Licensing CorporationAdjusting the loudness of an audio signal with perceived spectral balance preservationUS8938313Apr 12, 2010Jan 20, 2015Dolby Laboratories Licensing CorporationLow complexity auditory event boundary detectionUS9312829Apr 12, 2012Apr 12, 2016Dts LlcSystem for adjusting loudness of audio signals in real timeUS20090161883 *Dec 19, 2008Jun 25, 2009Srs Labs, Inc.System for adjusting perceived loudness of audio signalsUS20090306971 *Dec 10, 2009Samsung Electronics Co., Ltd & Kwangwoon University IndustryAudio signal quality enhancement apparatus and methodUS20130253923 *Mar 21, 2012Sep 26, 2013Her Majesty The Queen In Right Of Canada, As Represented By The Minister Of IndustryMultichannel enhancement system for preserving spatial cues* Cited by examinerClassifications U.S. Classification381/107, 704/225, 381/105, 381/106, 704/201, 381/104, 700/94International ClassificationH03G3/00Cooperative ClassificationH03G9/005, H03G3/20, H03G3/10, H04R5/04, H03G9/025European ClassificationH03G9/02B, H03G3/10, H03G9/00NLegal EventsDateCodeEventDescriptionJan 2, 2009ASAssignmentOwner name: DOLBY LABORATORIES LICENSING CORPORATION, CALIFORNFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SEEFELDT, ALAN;SMITHERS, MICHAEL;SIGNING DATES FROM 20081209 TO 20081215;REEL/FRAME:022049/0336Mar 13, 2015FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services