Patent Description:
Traditional DSP sound personalization methods often rely on administration of an audiogram to parameterize a frequency gain compensation function. Typically, a pure tone threshold (PTT) hearing test is employed to identify frequencies in which a user exhibits raised hearing thresholds and the frequency output is modulated accordingly. These gain parameters are stored locally on the user's device for subsequent audio processing.

The use of frequency compensation is inadequate to the extent that solely applying a gain function to the audio signal does not sufficiently restore audibility. The gain may enable the user to recapture previously unheard frequencies, but the user may subsequently experience loudness discomfort. Listeners with sensorineural hearing loss typically have similar, or even reduced, discomfort thresholds when compared to normal hearing listeners, despite their hearing thresholds being raised. To this extent, their dynamic aperture is narrower and simply adding gain would be detrimental to their hearing health in the long run.

Although hearing loss typically begins at higher frequencies, listeners who are aware that they have hearing loss do not typically complain about the absence of high frequency sounds. Instead, they report difficulties listening in a noisy environment and in hearing out the details in a complex mixture of sounds, such as in an audio stream of a radio interview conducted in a busy street. In essence, off frequency sounds more readily mask information with energy in other frequencies for hearing-impaired (HI) individuals - music that was once clear and rich in detail becomes muddled. This is because music itself is highly self-masking, i.e. numerous sound sources have energy that overlaps in the frequency space, which can reduce outright detectability, or impede the users' ability to extract information from some of the sources.

<CIT> discloses a method of generating a Personalized Audio Content (PAC) comprising selecting Audio Content (AC) to personalize, selecting an Earprint and generating a PAC using the Earprint to modify the AC.

<CIT> discloses systems and methods for modifying an audio signal using custom psychoacoustic models. A user's hearing profile is first obtained. Subsequently, an audio processing function is parameterized so as to optimize the user's perceptually relevant information. The method for calculating the user's perceptually relevant information comprises first processing audio signal samples using the parameterized processing function and then transforming samples of the processed audio signals into the frequency domain. Next, masking and hearing thresholds are obtained from the user's hearing profile and applied to the transformed audio sample, wherein the user's perceived data is calculated. Once perceptually relevant information is optimized, the resulting parameters are transferred to the audio processing function and an output audio signal is processed.

Other relevant prior art disclosures are known from <CIT>, <CIT>, <CIT>, and <CIT>.

As hearing deteriorates, the signal-conditioning capabilities of the ear begin to break down, and thus HI listeners need to expend more mental effort to make sense of sounds of interest in complex acoustic scenes (or miss the information entirely). A raised threshold in an audiogram is not merely a reduction in aural sensitivity, but a result of the malfunction of some deeper processes within the auditory system that have implications beyond the detection of faint sounds. To this extent, the addition of simple frequency gain provides an inadequate solution and the use of a multiband dynamic compression system would be more ideally suited as it more readily addresses the deficiencies of an impaired user.

Moreover, it is further inadequate to apply the same parameterized DSP algorithm to all types of audio content. Different forms of audio content require different DSP parameter settings as these systems aren't "one size fits all". For example, the requirements for voice processing are different than for more complex audio streams, such as for movies or music. For example, users are more willing to accept more aggressive forms of compression for voice calls to improve speech clarity than they are for music. Likewise, in a movie, a user may want to fit a specific DSP somewhere in between that of pure speech and music - so that a balance is achieved between voice clarity and greater detail in background sound and music.

Accordingly, it is an aspect of the present disclosure to provide systems and methods for providing content-specific, personalized audio replay on consumer devices.

According to aspect of the present disclosure, provided are systems and methods for providing content-specific, personalized audio replay on consumer devices. According to an aspect of the present disclosure, provided are methods and systems for processing an audio signal, the method comprising: storing a user hearing profile; calculating at least one set of audio content-specific DSP (digital signal processing) parameters for each of one or more sound personalization algorithms, the calculation of the content-specific DSP parameters based on at least the user hearing profile; associating one or more of the calculated sets of content-specific DSP parameters with a content-identifier for the specific content; in response to an audio stream on an audio output device, analyzing the audio stream to determine at least one content type of the audio stream; based on the at least one determined content type of the audio stream, outputting corresponding content-specific DSP parameters to the audio output device, wherein the corresponding content-specific DSP parameters are outputted based at least in part on their content-identifier. The method may further comprise processing, on the audio output device, an audio signal by using a given sound personalization algorithm parameterized by the corresponding content-specific DSP parameters.

In an aspect of the disclosure, the content-identifier further indicates the given sound personalization algorithm for which a given set of content-specific DSP parameters was calculated.

In a further aspect of the disclosure, the calculation of the content-specific DSP parameters further comprises applying a scaled processing level for the different types of specific content, wherein each scaled processing level is calculated based on one or more target age hearing curves that are different from a hearing curve of the user.

According to the claimed invention, the calculation of the content-specific DSP parameters further comprises calculating one or more wet mixing parameters and dry mixing parameters to optimize the content-specific DSP parameters for the different types of specific content.

In a further aspect of the disclosure, the calculation of the content-specific DSP parameters further comprises analyzing perceptually relevant information (PRI) to optimize a PRI value provided by the content-specific DSP parameters for the different types of specific content.

In a further aspect of the disclosure, the method may comeprise generating the user hearing profile. For example, the user hearing profile is generated by conducted at least one hearing test on the audio output device of a user.

In a further aspect of the disclosure, the hearing test is one or more of a masked threshold test (MT test), a pure tone threshold test (PTT test), a psychophysical tuning curve test (PTC test), or a cross frequency simultaneous masking test (xF-SM test).

In a further aspect of the disclosure, the user hearing profile is generated at least in part by analyzing a user input of demographic information to thereby interpolate a representative hearing profile.

In a further aspect of the disclosure, the user input of demographic information includes an age of the user.

In a further aspect of the disclosure, the sound personalization algorithm is a multiband dynamic processor; and the content-specific DSP parameters include one or more ratio values and gain values.

In a further aspect of the disclosure, the sound personalization algorithm is an equalization DSP; and the content-specific DSP parameters include one or more gain values and limiter values.

In a further aspect of the disclosure, the content type of the audio stream is determined by analyzing one or more metadata portions associated with the audio stream.

In a further aspect of the disclosure, the one or more metadata portions are extracted from the audio stream.

In a further aspect of the disclosure, the one or more metadata portions are calculated locally by an operating system of the audio output device.

In a further aspect of the disclosure, the content types include voice, video, music, and specific music genres.

In a further aspect of the disclosure, the audio output device is one of a mobile phone, a smart speaker, a television, headphones, or hearables.

In a further aspect of the disclosure, the at least one set of content-specific DSP parameters is stored on a remote server.

In a further aspect of the disclosure, the at least one set of content-specific DSP parameters is stored locally on the audio output device.

In a further aspect of the disclosure, the content type of the audio stream is determined by performing a Music Information Retrieval (MIR) calculation.

In a further aspect of the disclosure, the content type of the audio stream is determined by performing a spectral analysis calculation on the audio stream or providing the audio stream as input to a speech detection algorithm.

In a further aspect of the disclosure, an apparatus for processing an audio signal is provided. The apparatus comprises a processing unit and a memory for storing a user hearing profile. The apparatus may have an interface to an audio output device. The processing unit is configured for: calculating at least one set of audio content-specific DSP (digital signal processing) parameters for each of one or more sound personalization algorithms, the calculation of the content-specific DSP parameters based on at least the user hearing profile; associating one or more of the calculated sets of content-specific DSP parameters with a content-identifier for the specific content; in response to an audio stream on the audio output device, analyzing the audio stream to determine at least one content type of the audio stream; and based on the at least one determined content type of the audio stream, outputting corresponding content-specific DSP parameters to the audio output device for processing an audio signal on the audio output device by using a given sound personalization algorithm parameterized by the corresponding content-specific DSP parameters, wherein the corresponding content-specific DSP parameters are outputted based at least in part on their content-identifier.

In embodiments, the apparatus and the audio output device may be implemented in a same device, e.g. mobile phones, computers, televisions, hearing aids, headphones, smart speakers, hearables, and/or speaker systems. In this case, the content-specific DSP parameters are output from the apparatus itself to the DSP algorithm in the audio output device, e.g. via a software interface or API. Alternatively, apparatus and the audio output device may be different devices. For example, the apparatus may be implemented in a server and the audio output device may be a mobile phone, computer, television, hearing aid, headphone, smart speaker, hearable, and/or speaker system. In this case, the corresponding content-specific DSP parameters may be output to the audio output device via an interface. The interface between the apparatus and the audio output device may be implemented in various ways. The interface may be a software or hardware interface, communications interface, a direct or indirect interface via a cable, a wireless connection, a network connection, etc..

The audio stream may be received from the audio output device via the software or hardware interface. Alternatively, an indicator of the audio stream may be received from the audio output device via the interface and the audio stream accessed from an audio server based on the identifier. In examples, the processed audio signal may relate to the same audio content or content type as the analyzed audio stream. In general, it is envisioned that x-type of audio content has associated DSP parameters that can be reused whenever audio content of the same type x occurs. In variations, these DSP parameters can be used for different audio content types as well, e.g. for audio of similar content type.

The processing unit may comprise a processor and instructions to control execution of a computer program on the processor may be stored in the memory.

Another aspect of the present disclosure may relate to computer software, a computer program product or any media or data embodying computer software instructions for execution on a programmable computer or dedicated hardware comprising at least one processor, which causes the at least one processor to perform any of the method steps disclosed in the present disclosure.

Notably, it is understood that methods according to the disclosure relate to methods of operating the apparatuses according to the above aspect and variations thereof, and that respective statements made with regard to the apparatuses likewise apply to the corresponding methods, and vice versa, such that similar description may be omitted for the sake of conciseness.

Implementations of the disclosed apparatuses may include using, but not limited to, one or more processor, one or more application specific integrated circuit (ASIC) and/or one or more field programmable gate array (FPGA). Implementations of the apparatus may also include using other conventional and/or customized hardware such as software programmable processors.

The term "sound personalization algorithm", as used herein, is defined as any digital signal processing (DSP) algorithm that processes an audio signal to enhance the clarity of the signal to a listener. The DSP algorithm may be, for example: an equalizer, an audio processing function that works on the subband level of an audio signal, a multiband compressive system, or a non-linear audio processing algorithm.

The term "audio content type", as used herein, is defined as any specific type of audio content in an audio stream, such as voice, video, music, or specific genres of music, such as rock, jazz, classical, pop, etc..

The term "audio output device", as used herein, is defined as any device that outputs audio, including, but not limited to: mobile phones, computers, televisions, hearing aids, headphones, smart speakers, hearables, and/or speaker systems.

The term "headphone", as used herein, is any earpiece bearing a transducer that outputs soundwaves into the ear. The earphone may be a wireless hearable, a corded or wireless headphone, a hearable device, or any pair of earbuds.

The term "hearing test", as used herein, is any test that evaluates a user's hearing health, more specifically a hearing test administered using any transducer that outputs a sound wave. The test may be a threshold test or a suprathreshold test, including, but not limited to, a psychophysical tuning curve (PTC) test, a masked threshold (MT) test, a temporal fine structure test (TFS), temporal masking curve test and a speech in noise test.

The term "server", as used herein, generally refers to a computer program or device that provides functionalities for other programs or devices.

In order to describe the manner in which the above-recited and other advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific embodiments thereof, which are illustrated in the appended drawings. Understand that these drawings depict only exemplary embodiments of the disclosure and are not therefore to be considered to be limiting of its scope, the principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which:.

Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. Thus, the following description and drawings are illustrative and are not to be construed as limiting the scope of the embodiments described herein. Numerous specific details are described to provide a thorough understanding of the disclosure. However, in certain instances, well-known or conventional details are not described in order to avoid obscuring the description. References to one or an embodiment in the present disclosure can be references to the same embodiment or any embodiment; and, such references mean at least one of the embodiments.

Various example embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only.

It is an aspect of the present disclosure to provide systems and methods for providing audio content-specific, personalized audio replay on consumer devices. <FIG>underscore the importance of sound personalization, illustrating the deterioration of a listener's hearing ability over time. Past the age of <NUM> years old, humans begin to lose their ability to hear higher frequencies, as illustrated by <FIG> (albeit above the spectrum of human voice). This steadily becomes worse with age as noticeable declines within the speech frequency spectrum are apparent around the age of <NUM> or <NUM>. However, these pure tone audiometry findings mask a more complex problem as the human ability to understand speech may decline much earlier. Although hearing loss typically begins at higher frequencies, listeners who are aware that they have hearing loss do not typically complain about the absence of high frequency sounds. Instead, they report difficulties listening in a noisy environment and in hearing out the details in a complex mixture of sounds, such as in a telephone call. In essence, off-frequency sounds more readily mask a frequency of interest for hearing impaired individuals - conversation that was once clear and rich in detail becomes muddled. As hearing deteriorates, the signal-conditioning capabilities of the ear begin to break down, and thus hearing-impaired listeners need to expend more mental effort to make sense of sounds of interest in complex acoustic scenes (or miss the information entirely). A raised threshold in an audiogram is not merely a reduction in aural sensitivity, but a result of the malfunction of some deeper processes within the auditory system that have implications beyond the detection of faint sounds.

To this extent, <FIG> illustrates key, discernable age trends in suprathreshold hearing. Through the collection of large datasets, key age trends can be ascertained, allowing for the accurate parameterization of personalization DSP algorithms. In a multiband compressive system, for example, the threshold and ratio values of each sub-band signal dynamic range compressor (DRC) can be modified to reduce problematic areas of frequency masking, while post-compression sub-band signal gain can be further applied in the relevant areas. Masked threshold curves depicted in <FIG> represent a similar paradigm for measuring masked threshold. A narrow band of noise, in this instance around <NUM>, is fixed while a probe tone sweeps from <NUM>% of the noise band center frequency to <NUM>% of the noise band center frequency. Again, key age trends can be ascertained from the collection of large MT datasets.

<FIG> illustrate a method in which a PTC test <NUM> or MT test <NUM> may be conducted to assess a user's hearing. A psychophysical tuning curve (PTC), consisting of a frequency selectivity contour <NUM> extracted via behavioral testing, provides useful data to determine an individual's masking contours. In one embodiment of the test, a masking band of noise <NUM> is gradually swept across frequency, from below the probe frequency <NUM> to above the probe frequency <NUM>. The user then responds when they can hear the probe and stops responding when they no longer hear the probe. This gives a jagged trace that can then be interpolated to estimate the underlying characteristics of the auditory filter. Other methodologies known in the prior art may be employed to attain user masking contour curves. For instance, an inverse paradigm may be used in which a probe tone <NUM> is swept across frequency while a masking band of noise <NUM> is fixed at a center frequency (known as a "masked threshold test" or "MT test").

Other suprathreshold testing may be used. A cross frequency masked threshold test is illustrated in <FIG>. The y-axis represents the amplitude of the depicted signals, which include a noise masking probe M <NUM> and a tone signal probe <NUM>. The x-axis is logarithmic in frequency F. As illustrated, noise masking probe M <NUM> has a center frequency Fc and is kept at a fixed amplitude while being swept in frequency (i.e. the left to right progression seen in the graphs of <FIG>). In some embodiments, the absolute width of the masking probe M <NUM> is dynamic, e.g. <NUM> octaves on either side of the center frequency Fc. Tone signal probe <NUM> has a frequency Fs and a variable amplitude, i.e. an amplitude that is varied or adjusted while tone signal probe <NUM> is being swept in frequency, with an example variability or range of variability illustrated via arrow <NUM>. In some embodiments, the rate of variation of amplitude of tone signal probe <NUM> is independent of the rate at which the masking probe <NUM> and tone signal probe <NUM> are frequency swept, although in other embodiments a relationship is contemplated, as will be explained in greater depth below. While performing frequency sweeping of the tone signal probe <NUM> and the masking probe <NUM>, a fixed frequency ratio r is maintained, indicated in <FIG>at <NUM> and simply as 'r' elsewhere. In some embodiments, the fixed frequency ratio r is given by [Fs /Fc ] where <NUM> ≤ r ≤ <NUM>, although other ratio values may be utilized without departing from the scope of the present disclosure. As illustrated, masking probe <NUM> and signal probe <NUM> are then swept <NUM>, <NUM> simultaneously to higher frequencies while Bekesy-style user responses <NUM>, <NUM> are recorded and then interpolated to generate curve <NUM>.

<FIG> illustrates an exemplary embodiment of the present disclosure in which content-specific, personalized audio replay is carried out on an audio output devices. First, a hearing test is conducted <NUM> on one of a plurality of audio output devices. Alternatively, a user may just input their age, which would then input a representative hearing profile based on their age. The hearing test may be provided by any one of a plurality of hearing test options, including but not limited to: a masked threshold test (MT test) <NUM>, a pure tone threshold test (PTT test) <NUM>, a psychophysical tuning curve test (PTC test) <NUM>, a cross frequency simultaneous masking test (xF-SM) <NUM>, a speech in noise test <NUM>, or other suprathreshold test(s) <NUM>.

Next, hearing test results are used to calculate <NUM> at least one set of audio content-specific DSP parameters (also referred to herein as "content-specific" DSP parameters) for at least one sound personalization algorithm. The calculated DSP parameters for a given sound personalization algorithm may include, but are not limited to: ratio, threshold and gain values within a multiband dynamic processor, gain and limiter values for equalization DSPs, and/or parameter values common to other sound personalization DSPs (see, e.g., commonly owned <CIT> and <CIT>). One or more of the DSP parameter calculations may be performed directly or indirectly, as is explained below. The content specific parameters are then stored <NUM> on the audio output device and/or on a server database alongside a content identifier.

In some embodiments, when an audio stream is playing <NUM>, the audio content type may be identified through metadata associated with the audio stream. For example, the metadata may be contained within the audio file itself, or may be ascertained from the operating system of the audio output device. Various types of audio content may include: voice, video, music, or specific genres of music such as classical, rock, pop, jazz, etc. Alternatively or additionally, in some embodiments, other forms of audio signal analysis may be performed to identify audio content type, such as Music Information Retrieval (MIR), speech detection algorithms, or other forms of spectral analysis. After the audio content type has been identified or otherwise determined for the audio stream, content-specific DSP parameters are subsequently retrieved from the database using the audio content identifier as reference <NUM> and the parameters are then outputted to the device's sound personalization algorithm <NUM>. The audio stream is then processed by the sound personalization algorithm <NUM>.

<FIG> illustrates an exemplary multiband dynamic range compressor, which may be used to personalize sound for an audio stream. Here, each subband (n=<NUM>,. , x) contains at least variables threshold (tn) and ratio (rn) for the subband's dynamic range compressor and gain (gn). Other circuitries may be used (see for example, commonly owned <CIT>).

<FIG> further illustrates an exemplary method in which different parameter values are associated with various types of audio content. When the audio content-type is detected, these content-specific parameters are then outputted to the device's sound personalization algorithm and the audio stream is then processed accordingly. Content-specific parameters may be calculated according to a variety of methods, for example as illustrated in <FIG>. For example, parameter calculation may be based on a scale of processing level via target hearing age <NUM> in which higher levels of compression are used for voice - which is then progressively scaled lower for movies and music. For telephone calls, users are often willing to tolerate higher levels of compression, and thus distortion - in order to improve the speech clarity of the call. This compression tolerance is reduced when consuming more complex audio content - that is content across a broader spectral range, and content that contains multiple audio layers of voice, background noise and music. The level of processing may be calculated using a target age approach (see e.g., <FIG>,<FIG>). For instance, when a user is <NUM> years old - parameters for voice may be calculated according to a target age curve of a <NUM> year old (i.e., this represents higher levels of compression needed for call clarity), whereas for music or video, a target age curve of a <NUM> year old may be used (i.e., a less harsh form of processing).

A wet/ dry mixing approach may also be used, as seen in <FIG> (<NUM>), <FIG>, in which different ratios of wet and dry audio signals are selected for specific types of audio content, with higher levels of wet mix for audio content needing greater levels of processing. Specifically, a wide band audio signal is provided at processing input <NUM> and then divided into a first pathway (first signal pathway) <NUM> and a second pathway (second signal pathway) <NUM>. In this example, the second pathway <NUM> is only subject to a delay <NUM> and a protective limiter <NUM>. In contrast, in the first pathway <NUM>, the audio signal from the control input <NUM> is spectrally decomposed and processed according to the configuration of <FIG>. Each pathway <NUM>, <NUM> may include a weighting operator <NUM> and <NUM>, respectively. For example, these weighting operators <NUM> and <NUM> may be correlated by a common function that may be adjustable by a user by one single control variable <NUM>. Then these pathways <NUM> and <NUM> are recombined according to their weighting factors in operator <NUM> and provided to the processing output <NUM>.

Parallel compression provides the benefit of allowing the user to mix 'dry' unprocessed or slightly processed sound with 'wet' processed sound, enabling customization of processing based on subjective preference. For example, this enables hearing impaired users to use a high ratio of heavily processed sound relative to users with moderate to low hearing loss. Furthermore, by reducing the dynamic range of an audio signal by bringing up the softest sounds, rather than reducing the highest peaks, it provides audible detail to sound. The human ear is sensitive to loud sounds being suddenly reduced in volume, but less sensitive to soft sounds being increased in volume, and this mixing method takes advantage of this observation, resulting in a more natural sounding reduction in dynamic range compared with using a dynamic range compressor in isolation. Additionally, parallel compression is in particular useful for speech-comprehension and/or for listening to music with full, original timbre. To mix two different signal pathways requires that the signals in the pathways conform to phase linearity, or into the pathway's identical phase using phase distortion, or the pathway mixing modulator involves a phase correction network in order to prevent any phase cancellations upon summing the correlated signals to provide an audio signal to the control output.

A PRI optimization approach may also be employed, see <FIG> (<NUM>), <FIG>. DSP parameters in a multiband dynamic processor may be calculated by optimizing perceptually relevant information (e.g. perceptual entropy) through parameterization using user threshold and suprathreshold hearing data (see commonly owned US Patent No. <CIT> and <CIT>). Briefly, in order to optimally parameterize a multiband dynamic processor through perceptually relevant information, an audio sample <NUM>, or body of audio samples representing a specific content type, is first processed by a parameterized multiband dynamics processor <NUM> (see also <FIG>) and the perceptual entropy of the file is calculated <NUM> according to user threshold and suprathreshold hearing data <NUM>. After calculation, the multiband dynamic processor is re-parameterized <NUM> according to a given set of parameter heuristics, derived from optimization, and from this - the audio sample(s) is reprocessed <NUM> and the PRI calculated <NUM>. In other words, the multiband dynamics processor is configured to process the audio sample so that it has a higher PRI value for the particular listener, taking into account the individual listener's threshold and suprathreshold information <NUM>. To this end, parameterization of the multiband dynamics processor is adapted to increase the PRI of the processed audio sample over the unprocessed audio sample. The parameters of the multiband dynamics processor are determined by an optimization process that uses PRI as its optimization criteria. Optionally, the PRI optimization process may be subject to constraints <NUM> to make the optimization process more efficient and worthwhile. This is performed by evaluating parameters within a given set of criteria to direct the end result to a level of signal manipulation that the end user deems tolerable (e.g. using EQ coloration criteria or against harmonic distortion and noise criteria to limit the optimization space, see jointly owned <CIT> ).

PRI can be calculated according to a variety of methods found. One such method, also called perceptual entropy, was developed by James D. Johnston at Bell Labs, generally comprising: transforming a sampled window of audio signal into the frequency domain, obtaining masking thresholds using psychoacoustic rules by performing critical band analysis, determining noise-like or tone-like regions of the audio signal, applying thresholding rules for the signal and then accounting for absolute hearing thresholds. Following this, the number of bits required to quantize the spectrum without introducing perceptible quantization error is determined. For instance, Painter & Spanias disclose a formulation for perceptual entropy in units of bits/s, which is closely related to ISO/IEC MPEG-<NUM> psychoacoustic model <NUM> [<NPL>); see also generally Moving Picture Expert Group standards https://mpeg. chiariglione. org/standards].

Various optimization methods are possible to maximize the PRI of audio samples, depending on the type of the applied audio processing function such as the above-mentioned multiband dynamics processor. For example, a subband dynamic compressor may be parameterized by compression threshold, attack time, gain and compression ratio for each subband, and these parameters may be determined by the optimization process. In some cases, the effect of the multiband dynamics processor on the audio signal is nonlinear and an appropriate optimization technique such as gradient descend is required. The number of parameters that need to be determined may become large, e.g. if the audio signal is processed in many subbands and a plurality of parameters needs to be determined for each subband. In such cases, it may not be practicable to optimize all parameters simultaneously and a sequential approach for parameter optimization may be applied. Although sequential optimization procedures do not necessarily result in the optimum parameters, the obtained parameter values result in increased PRI over the unprocessed audio sample, thereby improving the listener's listening experience.

Other parameterization processes commonly known in the art may be used to calculate parameters based off user-generated threshold and suprathreshold information. For instance, common prescription techniques for linear and non-linear DSP may be employed. Well known procedures for linear hearing aid algorithms include POGO, NAL, and DSL. See, e.g.,<NPL>.

Fine tuning of any of the above-mentioned techniques may be estimated from manual fitting data. For instance, it is common in the art to fit a multiband dynamic processor according to series of subjective tests <NUM> given to a patient in which parameters are adjusted according to a patient's responses, e.g. a series of A/B tests, decision tree paradigms, 2D exploratory interface, in which the patient is asked which set of parameters subjectively sounds better. This testing ultimately guides the optimal parameterization of the DSP.

<FIG> and <FIG> demonstrate one way of configuring the ratio and threshold parameters for a frequency band in a multi-band compression system (see, e.g., commonly owned applications <CIT> and <CIT>) based upon a target curve / target age (see also <FIG>). Briefly, a user's masking contour curve is received <NUM>, a target masking curve is determined <NUM>, and is subsequently compared with the user masking contour curve <NUM> in order to determine and output user-calculated DSP parameter sets <NUM>.

<FIG> combines the visualization of a user masking contour curve <NUM> for a listener (listener) and a target masking contour curve <NUM> of a probe tone <NUM> (with the x-axis <NUM> being frequency, and the y-axis <NUM> being the sound level in dB SPL or HL) with an input/output graph of a compressor showing the input level <NUM> versus the output level <NUM> of a sound signal, in decibels relative to full scale (dB FS). The bisecting line in the input/output graph represents a <NUM>:<NUM> (unprocessed) output of the input signal with gain <NUM>.

The parameters of the multi-band compression system in a frequency band are threshold <NUM> and gain <NUM>. These two parameters are determined from the user masking contour curve <NUM> for the listener and target masking contour curve <NUM>. The threshold <NUM> and ratio <NUM> must satisfy the condition that the signal-to-noise ratio <NUM> (SNR) of the user masking contour curve <NUM> at a given frequency <NUM> is greater than the SNR <NUM> of the target masking contour curve <NUM> at the same given frequency <NUM>. Note that the SNR is herein defined as the level of the signal tone compared to the level of the masker noise. The broader the curve will be, the greater the SNR. The given frequency <NUM> at which the SNRs <NUM> and <NUM> are calculated may be arbitrarily chosen, for example, to be beyond a minimum distance from the probe tone frequency <NUM>.

The sound level <NUM> (in dB) of the target masking contour curve <NUM> at a given frequency corresponds (see bent arrow <NUM> in <FIG>) to an input sound level <NUM> entering the compression system. The objective is that the sound level <NUM> outputted by the compression system will match the user masking contour curve <NUM>, i.e., that this sound level <NUM> is substantially equal to the sound level (in dB) of the user masking contour curve <NUM> at the given frequency <NUM>. This condition allows the derivation of the threshold <NUM> (which has to be below the input sound level <NUM>) and the ratio <NUM>. In other words, input sound level <NUM> and output sound level <NUM> determine a reference point of the compression curve. As noted above, threshold <NUM> must be selected to be lower than input sound level <NUM> - if it is not, there will be no change, as below the threshold of the compressor, the system is linear). Once the threshold <NUM> is selected, the ratio <NUM> can be determined from the threshold and the reference point of the compression curve.

In the context of the present disclosure, a masking contour curve is obtained from a user hearing test. A target masking contour curve <NUM> is interpolated from at least the user masking contour curve <NUM> and a reference masking contour curve, representing the curve of a normal hearing individual. The target masking contour curve <NUM> is preferred over a reference curve because fitting an audio signal to a reference curve is not necessarily optimal. Depending on the initial hearing ability of the listener, fitting the processing according to a reference curve may cause an excess of processing to spoil the quality of the signal. The objective is to process the signal in order to obtain a good balance between an objective benefit and a good sound quality.

The given frequency <NUM> is then chosen. It may be chosen arbitrarily, e.g., at a certain distance from the tone frequency <NUM>. The corresponding sound levels of the listener and target masking contour curves are determined at this given frequency <NUM>. The value of these sound levels may be determined graphically on the y-axis <NUM>.

The right panel in <FIG> (see the contiguous graph) illustrates a hard knee DRC, with a threshold <NUM> and a ratio <NUM> as parameters that need to be determined. An input sound signal having a sound level <NUM>/<NUM> at a given frequency <NUM> enters the compression system (see bent arrow <NUM> indicating correspondence between <NUM>/<NUM>). The sound signal should be processed by the DRC in such a way that the outputted sound level is the sound level of the user masking contour curve <NUM> at the given frequency <NUM>. The threshold <NUM> should not exceed the input sound level <NUM>, otherwise compression will not occur. Multiple sets of threshold and ratio parameters are possible. Preferred sets can be selected depending on a fitting algorithm and/or objective fitting data that have proven to show the most benefit in terms of sound quality. For example, either one of the threshold <NUM> and ratio <NUM> may be chosen to have a default value, and the respective other one of the parameters can then be determined by imposing the above-described condition.

In some embodiments, content-specific DSP parameter sets may be calculated indirectly from a user hearing test based on preexisting entries or anchor points in a server database. An anchor point comprises a typical hearing profile constructed based at least in part on demographic information, such as age and sex, in which DSP parameter sets are calculated and stored on the server to serve as reference markers. Indirect calculation of DSP parameter sets bypasses direct parameter sets calculation by finding the closest matching hearing profile(s) and importing (or interpolating) those values for the user.

<FIG> illustrate three conceptual user masked threshold (MT) curves for users x, y, and z, respectively. The MT curves are centered at frequencies a-d, each with curve width d, which may be used to as a metric to measure the similarity between user hearing data. For instance, a root mean square difference calculation may be used to determine if user y's hearing data is more similar to user x's or user z's, e.g. by calculating: <MAT>.

<FIG> illustrates three conceptual audiograms of users x, y and z, each with pure tone threshold values <NUM>-<NUM>. Similar to above, a root mean square difference measurement may also be used to determine, for example, if user y's hearing data is more similar to user x's than user z's, e.g., by calculating: <MAT> As would be appreciated by one of ordinary skill in the art, other methods may be used to quantify similarity amongst user hearing profile graphs, where the other methods can include, but are not limited to, methods such as a Euclidean distance measurements, e.g. ((y1 - x<NUM>) + (y<NUM> - x<NUM>). > (y1 - x<NUM>) + (y<NUM> - x<NUM>)). or other statistical methods known in the art. For indirect DSP parameter set calculation, then, the closest matching hearing profile(s) between a user and other preexisting database entries or anchor points can then be used.

<FIG> illustrates an exemplary embodiment for calculating sound personalization parameter sets for a given algorithm based on preexisting entries and/or anchor points. Here, server database entries <NUM> are surveyed to find the best fit(s) with user hearing data input <NUM>, represented as MT<NUM> and PTT<NUM> for (u_id)<NUM>. This may be performed by the statistical techniques illustrated in <FIG> and <FIG>. In the example of <FIG>, (u_id)<NUM> hearing data best matches MT<NUM> and PTT<NUM> data <NUM>. To this extent, (u_id)<NUM> associated parameter sets, [DSPq-param <NUM>], are then used for the (u_id)<NUM> parameter set entry, illustrated here as [(u_id)<NUM>, t<NUM>, MT<NUM>, PTT<NUM>, DSPq-param <NUM>].

<FIG> illustrates an exemplary embodiment for calculating sound personalization parameter sets for a given algorithm based on preexisting entries or anchor points, according to aspects of the present disclosure. Here, server database entries <NUM> are employed to interpolate <NUM> between two nearest fits <NUM> with user hearing data input <NUM> MT<NUM> and PT<NUM> for (u_id)<NUM>. In this example, the (u_id)<NUM> hearing data fits nearest between: MT<NUM> <IMG> MT<NUM> <IMG> MT<NUM> and PTT<NUM> <IMG> PTT<NUM><IMG> PTT<NUM> <NUM>. To this extent, (u_id)<NUM> and (u_id)<NUM> parameter sets are interpolated to generate a new set of parameters for the (u_id)<NUM> parameter set entry, represented here as [(u_id)<NUM>, t<NUM>, MT<NUM>, PTT<NUM>, DSPq-param3/<NUM>] <NUM>. In a further embodiment, interpolation may be performed across multiple data entries to calculate sound personalization parameters.

DSP parameter sets may be interpolated linearly, e.g., a DRC ratio value of <NUM> for user <NUM> (u_id)<NUM> and <NUM> for user <NUM> (u_id)<NUM> would be interpolated as <NUM> for user <NUM> (u_id)<NUM> in the example of <FIG>, assuming user <NUM>'s hearing data was halfway in-between that of users <NUM> and <NUM>. In some embodiments, DSP parameter sets may also be interpolated non-linearly, for instance using a squared function, e.g. a DRC ratio value of <NUM> for user <NUM> and <NUM> for user <NUM> would be non-linearly interpolated as <NUM> for user <NUM> in the example of <FIG>.

<FIG> shows an example of computing system <NUM>, which can be for example any computing device making up (e.g., mobile device <NUM>, server, etc.) or any component thereof in which the components of the system are in communication with each other using connection <NUM>. Connection <NUM> can be a physical connection via a bus, or a direct connection into processor <NUM>, such as in a chipset architecture. Connection <NUM> can also be a virtual connection, networked connection, or logical connection.

In some embodiments computing system <NUM> is a distributed system in which the functions described in this disclosure can be distributed within a datacenter, multiple datacenters, a peer network, etc. In some embodiments, one or more of the described system components represents many such components each performing some or all of the function for which the component is described. In some embodiments, the components can be physical or virtual devices.

Example system <NUM> includes at least one processing unit (CPU or processor) <NUM> and connection <NUM> that couples various system components including system memory <NUM>, such as read only memory (ROM) <NUM> and random access memory (RAM) <NUM> to processor <NUM>. Computing system <NUM> can include a cache of high-speed memory <NUM> connected directly with, in close proximity to, or integrated as part of processor <NUM>.

Processor <NUM> can include any general-purpose processor and a hardware service or software service, such as services <NUM>, <NUM>, and <NUM> stored in storage device <NUM>, configured to control processor <NUM> as well as a special-purpose processor where software instructions are incorporated into the actual processor design. Processor <NUM> may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.

To enable user interaction, computing system <NUM> includes an input device <NUM>, which can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech, etc. Computing system <NUM> can also include output device <NUM>, which can be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems can enable a user to provide multiple types of input/output to communicate with computing system <NUM>. Computing system <NUM> can include communications interface <NUM>, which can generally govern and manage the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

Storage device <NUM> can be a non-volatile memory device and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs), read only memory (ROM), and/or some combination of these devices.

The storage device <NUM> can include software services, servers, services, etc., that when the code that defines such software is executed by the processor <NUM>, it causes the system to perform a function. In some embodiments, a hardware service that performs a particular function can include the software component stored in a computer-readable medium in connection with the necessary hardware components, such as processor <NUM>, connection <NUM>, output device <NUM>, etc., to carry out the function.

The presented technology offers an efficient and accurate way to personalize audio replay automatically for a variety of audio content types. It is to be understood that the present disclosure contemplates numerous variations, options, and alternatives.

Such instructions can comprise, for example, instructions and data which cause or otherwise configure a general-purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions.

Claim 1:
A method (<NUM>) for processing an audio stream, the method comprising:
storing a user hearing profile;
calculating (<NUM>, <NUM>) at least one set of audio content-specific DSP, digital signal processing, parameters for each of one or more sound personalization algorithms, the calculation of the content-specific DSP parameters based on at least the user hearing profile, wherein the calculation (<NUM>, <NUM>) of the content-specific DSP parameters further comprises
calculating (<NUM>) one or more wet mixing parameters and dry mixing parameters to optimize the content-specific DSP parameters for the different types of specific content;
associating one or more of the calculated sets of content-specific DSP parameters with a content-identifier for the specific content;
in response to the audio stream on an audio output device (<NUM>), analyzing (<NUM>) the audio stream to determine at least one content type of the audio stream; and
based on the at least one determined content type of the audio stream, outputting (<NUM>) corresponding content-specific DSP parameters to the audio output device (<NUM>) for processing the audio stream on the audio output device (<NUM>) by using a given sound personalization algorithm parameterized by the corresponding content-specific DSP parameters, wherein the corresponding content-specific DSP parameters are outputted based at least in part on their content-identifier.