Patent Description:
Audio processing systems use one or more speakers to produce sound in a given space. The one or more speakers generate a sound field, where a user in the environment receives the sound included in the sound field. When the user hears the sound, the user determines a spatial point from which the sound appears to originate. Various audio processing systems perform audio processing and reproduction techniques to reproduce two-dimensional or three-dimensional audio, where the user hears the reproduced audio as appearing to come from one or more specific originating points in the environment. When generating the sound field, an audio processing system uses one or more finite impulse response (FIR) filters to generate the sounds that create the sound field. For example, the audio processing system uses a sparse set of FIR filters to estimate the impulse response at various locations within the sound field. Using such methods, the audio processing system determines the impulse response of the sound field at a given point in space and adjusts the audio output based on the impulse response.

At least one drawback with conventional audio processing systems is that such audio processing systems do not provide an audio output based on an accurate sound field for all locations within the sound field. For example, audio processing systems use a sparse set of FIR filters to generate portions of the sound field for a limited number of locations in the environment and use linear interpolation to estimate impulse responses for other locations in the environment. However, such audio processing systems do not account for many characteristics of the sound field and cannot accurately estimate impulse responses for all the locations in the sound field. For example, sound fields that are produced from highly-directive sources and sound fields having complex structures vary greatly over different locations in the environment. In such instances, the audio processing systems require higher spatial sampling of impulse responses. As a result, the audio processing systems require a larger number of FIR filters for additional locations in the environment, or otherwise do not accurately estimate the impulse response at specific locations in the environment. The error in estimation causes errors in audio reproduction and degrades the auditory experience for the user.

As the foregoing illustrates, what is needed in the art are more effective techniques for generating sound fields in an environment.

Various embodiments disclose a computer-implemented method comprising determining a target location in an environment, determining a set of sub-band impulse responses for a first frequency sub-band, each sub-band impulse response in the set of sub-band impulse responses being associated with a corresponding location that is proximate to the target location, selecting a first pair of sub-band impulse responses for the first frequency sub-band from among pairs of sub-band impulse responses in the set of sub-band impulse responses, computing a first coherence value indicating a level of coherence between sub-band impulse responses in the first pair, determining that the first coherence value is below a coherence threshold, in response to determining that the first coherence value is below the coherence threshold, combining the sub-band impulse responses in the first pair using a non-linear interpolation technique to generate an estimated impulse response for the first frequency sub-band for the target location, generating, based at least on the estimated impulse response, a filter for a speaker, filtering, by the filter, an audio signal to generate a filtered audio signal, and causing the speaker to output the filtered audio signal.

Further embodiments provide, among other things, one or more non-transitory computer-readable media and systems configured to implement the method set forth above.

At least one technical advantage of the disclose techniques relative to the prior art is that, with the disclosed techniques, an audio processing system can more accurately generate a sound field for a particular location in an environment, which increases the auditory experience of a user at the particular location. Further, the disclosed techniques are able to generate impulse response filters more accurately for the particular location from a smaller set of impulse response filters than prior art techniques. The disclosed techniques therefore reduce the memory used by the audio processing system when estimating impulse responses at particular locations. Further, the disclosed techniques reduce the time spent collecting measurements of impulse responses at locations within a listening environment that are needed to generate an accurate sound field. These technical advantages provide one or more technological advancements over prior art approaches.

So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description of the inventive concepts, briefly summarized above, may be had by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of the inventive concepts and are therefore not to be considered limiting of scope in any way, and that there are other equally effective embodiments.

In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of skilled in the art that the inventive concepts may be practiced without one or more of these specific details.

<FIG> is a schematic diagram illustrating an audio processing system <NUM> according to various embodiments. As shown, the audio processing system <NUM> includes, without limitation, a computing device <NUM>, one or more sensors <NUM>, and one or more speakers <NUM>. The computing device <NUM> includes, without limitation, a processing unit <NUM> and memory <NUM>. The memory <NUM> stores, without limitation, an audio processing application <NUM>, location data <NUM>, impulse response data <NUM>, and one or more filters <NUM>. The audio processing application <NUM> includes, without limitation, an impulse response coherence calculator <NUM>, an interpolator, <NUM>, and a filter calculator <NUM>.

In operation, the audio processing system <NUM> processes sensor data from the one or more sensors <NUM> to track the location of one or more listeners within the listening environment to identify one or more target locations within the listening environment. The audio processing application <NUM> included in the audio processing system <NUM> retrieves measured impulse responses for various locations within the listening environment and selects a subset of the measured impulse responses surrounding each target location. The impulse response coherence calculator <NUM> processes the selected measured impulse responses to determine a set of impulse responses to use and whether to use linear or non-linear interpolation to estimate the impulse response over a given frequency range for the target location. The interpolator <NUM> uses the determined interpolation technique to generate an estimated impulse response for the target location. The filter calculator sets the parameters for the filters <NUM> based at least on the estimated impulse response at the target location. The audio processing application <NUM> uses the filters <NUM> that are generated according to the parameters to filter an audio signal and reproduce a sound field within the listening environment.

The computing device <NUM> is a device that drives speakers <NUM> to generate, in part, a sound field. In various embodiments, the computing device <NUM> is a central unit in a home theater system, a soundbar, a vehicle system, and so forth. In some embodiments, the computing device <NUM> is included in one or more devices, such as consumer products (e.g., portable speakers, gaming, gambling, etc. products), vehicles (e.g., the head unit of a car, truck, van, etc.), smart home devices (e.g., smart lighting systems, security systems, digital assistants, etc.), communications systems (e.g., conference call systems, video conferencing systems, speaker amplification systems, etc.), and so forth. In various embodiments, the computing device <NUM> is located in various environments including, without limitation, indoor environments (e.g., living room, conference room, conference hall, home office, etc.), and/or outdoor environments, (e.g., patio, rooftop, garden, etc.).

The processing unit <NUM> can be any suitable processor, such as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), and/or any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, the processing unit <NUM> can be any technically feasible hardware unit capable of processing data and/or executing software applications.

Memory <NUM> can include a random-access memory (RAM) module, a flash memory unit, or any other type of memory unit or combination thereof. The processing unit <NUM> is configured to read data from and write data to the memory <NUM>. In various embodiments, the memory <NUM> includes non-volatile memory, such as optical drives, magnetic drives, flash drives, or other storage. In some embodiments, separate data stores, such as an external data stores included in a network ("cloud storage") can supplement the memory <NUM>. The audio processing application <NUM> within the memory <NUM> can be executed by the processing unit <NUM> to implement the overall functionality of the computing device <NUM> and, thus, to coordinate the operation of the audio processing system <NUM> as a whole. In various embodiments, an interconnect bus (not shown) connects the processing unit <NUM>, the memory <NUM>, the speakers <NUM>, the sensors <NUM>, and any other components of the computing device <NUM>.

The audio processing application <NUM> executes various techniques to determine the location of a listener within a listening environment and sets the parameters for one or more filters <NUM> to generate a sound field for the location of the listener. In various embodiments, the audio processing application <NUM> receives location data <NUM> to identify the location of the listener and receives and impulse response data <NUM> for various locations where an impulse response within the listening environment has been determined. The audio processing application <NUM> uses the location data <NUM> to set the location of the listener as the target location. The target location is then used to select measured impulse responses near the target location from the impulse response data <NUM>. For example, the audio processing application <NUM> acquires the location data <NUM> from the sensors <NUM> (e.g., received optical data and/or other tracking data) to determine the position of the listener. The audio processing application <NUM> also acquires the impulse response data <NUM> to determine the locations within the listening environment where impulse responses were measured. Based on the locations of the measured impulse responses, the audio processing application <NUM> estimates the impulse response at the target location and updates the impulse response data <NUM> that is used to set the parameters for the filters <NUM>. In some embodiments, the audio processing application <NUM> sets the parameters for multiple filters <NUM> corresponding to multiple speakers <NUM>. Additionally or alternatively, the audio processing application <NUM> tracks the positions of multiple listeners. In such instances, the audio processing application <NUM> determines multiple target locations and estimates impulse responses at each of the target locations. The audio processing application <NUM> can then update the impulse response data <NUM> to include each of the estimated impulse responses.

The filters <NUM> include one or more filters that modify an input audio signal. In various embodiments, a given filter <NUM> modifies the input audio signal by modifying the energy within a specific frequency range, adding directivity information, and so forth. For example, the filter <NUM> can include filter parameters, such as a set of values that modify the operating characteristics (e.g., center frequency, gain, Q factor, cutoff frequencies, etc.) of the filter <NUM>. In some embodiments, the filter parameters include one or more digital signal processing (DSP) coefficients that steer the generated soundwave in a specific direction. In such instances, the generated filtered audio signal is used to generate a soundwave in the direction specified in the filtered audio signal. For example, the one or more speakers <NUM> reproduce using one or more filtered audio signals to generate a sound field. In some embodiments, the audio processing application <NUM> sets separate filter parameters for separate filters <NUM>. In such instances, one or more speakers <NUM> generate the sound field using the separate filters <NUM>. For example, each filter <NUM> can generate a filtered audio signal for a single speaker <NUM> within the listening environment.

The impulse response data <NUM> includes measured impulse responses within the listening environment. The impulse response data <NUM> includes a set of measured impulse responses at locations within the listening environment. In some embodiments, the impulse response data <NUM> also includes previously estimated impulse responses. In such instances, the audio processing application <NUM> checks the impulse response data <NUM> for a previously estimated impulse response for the target location before generating an estimated impulse response for the target location.

In some embodiments, the impulse response data <NUM> includes filter parameters for one or more filters <NUM>, such as one or more finite impulse response (FIR) filters. In various embodiments, the audio processing application <NUM> initially sets filter parameters for filters <NUM> corresponding to each speaker <NUM> and updates the filter parameters for a specific speaker (e.g., a first filter <NUM>(<NUM>) for a first speaker <NUM>(<NUM>)) when the listener moves. For example, the audio processing application <NUM> can initially generate filter parameters for a set of filters <NUM>. Upon determining that the listener has moved to a new location, the audio processing application <NUM> then determines whether any of the speakers <NUM> require updates to the corresponding filters <NUM>. The audio processing application <NUM> updates the filter parameters for any filter <NUM> that requires updating. In some embodiments, audio processing application <NUM> generates each of the filters <NUM> independently. For example, upon determining that a listener has moved, the audio processing application <NUM> can update the filter parameters for a single filter <NUM> (e.g., <NUM>(<NUM>) for a specific speaker <NUM> (e.g., <NUM>(<NUM>)). Alternatively, the audio processing application <NUM> updates multiple filters <NUM>. In some embodiments, the audio processing application <NUM> uses multiple filters <NUM> to modify the audio signal. For example, the audio processing application <NUM> can use a first filter <NUM>(<NUM>) to add directivity information to an audio signal and can use separate filters <NUM>, such as equalization filters, spatialization filters, etc., to further modify the audio signal.

The location data <NUM> is a dataset that includes positional information for one or more locations within the listening environment. In some embodiments, the location data <NUM> includes specific coordinates relative to a reference point. For example, the location data <NUM> can store the current positions and/or orientations of each respective speaker <NUM> as a distance and angle from a specific reference point. In some embodiments, the location data <NUM> can include additional orientation information, such as a set of angles (e.g., {µ, φ, ψ}) relative to a normal orientation. In such instances, the position and orientation of a given speaker <NUM> is stored in the location data <NUM> as a set of distances and angles relative to a reference point. In various embodiments, the location data <NUM> also includes computed directions between points. For example, the audio processing application <NUM> can compute the direction of the target location and/or a specific listener relative to the position and orientation of the speaker <NUM> and can store the direction as a vector in the location data <NUM>. In such instances, the audio processing application <NUM> retrieves the stored direction when setting the filter parameters of the one or more filters <NUM>.

The sensors <NUM> include various types of sensors that acquire data about the listening environment. For example, the computing device <NUM> can include auditory sensors to receive several types of sound (e.g., subsonic pulses, ultrasonic sounds, speech commands, etc.). In some embodiments, the sensors <NUM> includes other types of sensors. Other types of sensors include optical sensors, such as RGB cameras, time-of-flight cameras, infrared cameras, depth cameras, a quick response (QR) code tracking system, motion sensors, such as an accelerometer or an inertial measurement unit (IMU) (e.g., a three-axis accelerometer, gyroscopic sensor, and/or magnetometer), pressure sensors, and so forth. In addition, in some embodiments, sensor(s) <NUM> can include wireless sensors, including radio frequency (RF) sensors (e.g., sonar and radar), and/or wireless communications protocols, including Bluetooth, Bluetooth low energy (BLE), cellular protocols, and/or near-field communications (NFC). In various embodiments, the audio processing application <NUM> uses the sensor data acquired by the sensors <NUM> to generate the location data <NUM>. For example, the computing device <NUM> includes one or more emitters that emit positioning signals, where the computing device <NUM> includes detectors that generate auditory data that includes the positioning signals. In some embodiments, the audio processing application <NUM> combines multiple types of sensor data. For example, the audio processing application <NUM> can combine auditory data and optical data (e.g., camera images or infrared data) in order to determine the position and orientation of the listener at a given time.

<FIG> illustrates an example speaker arrangement of the audio processing system <NUM> of <FIG> within a listening environment <NUM>, according to various embodiments. As shown, and without limitation, the listening environment <NUM> includes a listener <NUM>, a set of speakers <NUM>(<NUM>)-<NUM>(<NUM>), stored impulse response locations <NUM>, a target location <NUM>, and an impulse response subset <NUM>.

Each speaker <NUM> is physically located at a different position within the listening environment. At various times, the impulse response at a given location is determined. For example, each speaker <NUM>(<NUM>)-<NUM>(<NUM>) can emit an audio impulse and a microphone positioned at the given location (e.g., location <NUM>(<NUM>)) can record the impulse response. In such instances, the impulse response can be separately measured for an audio impulse emitted by each of the speakers <NUM>(<NUM>)-<NUM>(<NUM>). Upon recording the impulse response, the measured impulse response and the corresponding impulse response location <NUM> can be stored in the impulse response data <NUM>. In some embodiments, the audio processing application <NUM> generates an estimated impulse response for a target location <NUM>. In such instances, the audio processing application <NUM> stores the estimated impulse response and the corresponding location (e.g., setting the target location <NUM> as a stored impulse response location <NUM>) in the impulse response data <NUM>. In various embodiments, the group of stored impulse responses and stored impulse response locations <NUM> acts as a sparse set of known impulse responses for which the audio processing application <NUM> can determine the impulse responses at other locations.

A listener <NUM> is positioned in proximity to one or more of the speakers <NUM>. As shown in the embodiments of <FIG>, the listener <NUM> is oriented such that the front of listener <NUM> is facing speaker <NUM>(<NUM>). Speakers <NUM>(<NUM>) and <NUM>(<NUM>) are positioned to the front left and front right, respectively, of the listener <NUM>. Speakers <NUM>(<NUM>) and <NUM>(<NUM>) are positioned behind the listener <NUM>. In some embodiments, speakers <NUM>(<NUM>) and <NUM>(<NUM>) form a dipole group.

Listener <NUM> listens to sounds emitted by the audio processing system <NUM> via the speakers <NUM>. As shown in <FIG>, the listener <NUM> is associated with a target location <NUM> (e.g., a specific ear or ears of the listener, a center point between the ears of the listener, and/or the like) within the listening environment <NUM>. In various embodiments, the audio processing system <NUM> outputs a sound field that is heard by listener <NUM>. In order to generate the filters <NUM> for speakers <NUM>, the audio processing application <NUM> first determines whether an impulse response was measured at the target location <NUM>. When audio processing application <NUM> determines that an impulse response was measured at the target location <NUM> or determines that an impulse response for the target location <NUM> has already been estimated (e.g., the impulse response data <NUM> includes stored impulse response for the target location <NUM>), the audio processing application <NUM> sets filters <NUM> based on the impulse response for the target location <NUM>. Otherwise, the audio processing application <NUM> determines that an impulse response for the target location <NUM> cannot be retrieved and generates an estimated impulse response for the target location <NUM>.

In some embodiments, the measured impulse responses for the listening environment <NUM> include measured impulse responses at various locations <NUM> (e.g., <NUM>(<NUM>)-<NUM>(<NUM>)) within the listening environment <NUM>. The audio processing application <NUM> identifies two or more locations within the listening environment <NUM> that are near to target location and for which an impulse response has been measured. For example, the subset <NUM> could include impulse responses measured at locations <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>). In some embodiments, the audio processing application <NUM> use a set of criteria to determine the subset <NUM>. For example, the audio processing application <NUM> can select the three nearest locations <NUM>, measured by Euclidean distance in space and/or a perceived spatial auditory distance to the target location <NUM>, that combine to surround the target location <NUM>.

In some embodiments, the audio processing application <NUM> determines the specific stored locations <NUM> to include in the subset <NUM> using a set of one or more heuristics and/or rules in addition to or in lieu of distance to the target location <NUM>. The set of one or more heuristics and/or rules could consider, the number of listeners <NUM> (e.g., <NUM>(<NUM>), <NUM>(<NUM>), etc.) within the listening environment <NUM>, the position of the listener(s) <NUM>, the orientation of the listener(s) <NUM>, the number of speakers <NUM> in the audio processing system <NUM>, the location of the speakers <NUM>, whether a pair of speakers <NUM> form a dipole group, the position of the speakers <NUM> relative to the position of the listener(s) <NUM>, the type of listening environment, and/or other characteristics of the listening environment <NUM> and/or the audio processing system <NUM>. The specific heuristics and/or rules may vary, for example, depending on the audio processing system <NUM>, the listening environment <NUM>, the type of audio being played, user-specified preferences (e.g., noise cancellation mode), and so forth.

<FIG> is a technique for generating an estimated impulse response <NUM> at a target location <NUM>, according to various embodiments. As shown, the audio processing system <NUM> includes the impulse response coherence calculator <NUM>, the interpolator <NUM>, the filter calculator <NUM>, and the filter <NUM>(<NUM>).

In various embodiments, the audio processing application <NUM> selects a subset <NUM> of stored impulse responses <NUM>(<NUM>)-<NUM>(N) for locations <NUM> that are near the target location <NUM>. In some embodiments, the audio processing application <NUM> selects for the subset <NUM> each stored impulse response <NUM> that was measured at a location within a threshold distance of the target location <NUM> (e.g., N locations within a threshold distance). For example, the audio processing application <NUM> can select <NUM> impulse responses that are located within a threshold distance of the target location <NUM>. Additionally or alternatively, the audio processing application <NUM> selects a specific number of measured impulse responses <NUM>, such as the three closest locations (measured by the distance) that form an area encompassing at least a portion of the target location <NUM>.

In various embodiments, the audio processing application <NUM> separates each measured impulse response <NUM> into sub-band impulse responses <NUM>, <NUM>. For example, the audio processing application <NUM> decomposes each of the N stored impulse responses <NUM>(<NUM>)-<NUM>(N) (where N ≥ <NUM>) included in the subset <NUM> into separate groups of signals corresponding to impulse responses for a specific sub-band (e.g., decomposing the impulse response <NUM>(<NUM>) into X sub-band impulse responses <NUM>(<NUM>)-<NUM>(X) corresponding to X separate sub-bands and similarly for stored impulse responses <NUM>(<NUM>)-<NUM>(N)). Alternatively, in some embodiments, the audio processing application <NUM> retrieves sub-band impulse responses that were previously decomposed and stored. The audio processing application <NUM> groups the sub-band impulse responses into X separate sub-band groupings <NUM>(<NUM>)-<NUM>(X). For example, upon decomposing each of the N impulse responses in the subset <NUM> (e.g., decomposing the first impulse response <NUM>(<NUM>) through the Nth impulse response <NUM>(N)) into separate sub-band impulse responses <NUM>(<NUM>)-<NUM>(X),. <NUM>(<NUM>)-<NUM>(X), the audio processing application <NUM> generates a sub-band grouping <NUM>(<NUM>) for the first sub-band that includes each of the impulse responses for the first sub-band. In various embodiments, the audio processing application <NUM> also generates separate sub-band groupings <NUM>(<NUM>)-<NUM>(X) (not shown) that correspond to the other sub-bands.

In various embodiments, the impulse response coherence calculator <NUM> included in the audio processing application <NUM> iteratively calculates a separate coherence value <NUM> (e.g., <NUM>(<NUM>)-<NUM>(Z)) for each paired combination of the sub-band impulse responses <NUM>, <NUM> included in the sub-band grouping <NUM>. The impulse response coherence calculator <NUM> generates the coherence value set <NUM> for the sub-band grouping <NUM>, where the coherence value set <NUM> includes coherence values <NUM>(<NUM>)-<NUM>(Z) for each paired combination of sub-band, where Z is equivalent to <MAT> combinations of pairs of sub-band impulse responses within the sub-band grouping <NUM>. The impulse response coherence calculator <NUM> selects two sub-band impulse responses (e.g., <NUM>(<NUM>) and <NUM>(<NUM>)) from the sub-band grouping <NUM> and computes the coherence value (e.g., <NUM>(<NUM>)) for the paired combination.

In various embodiments, the impulse response coherence calculator <NUM> initially computes the coherence signal between two sub-band impulse responses. In some embodiments, the coherence value <NUM> can be a magnitude-squared coherence signal that is a function of a first sub-band impulse response (e.g., x(ω)) and a second sub-band impulse response (e.g., y(ω)): <MAT>.

Where Sxx and Syy are the power-spectral densities (PSDs) of the first and second sub-band impulse responses, respectively, and Syx is the cross-spectral density between the first and second sub-band impulse responses. In such instances, the impulse response coherence calculator <NUM> can store the coherence signal as the coherence value <NUM>.

In some embodiments, the impulse response coherence calculator <NUM> can determine a single coherence value from the coherence signal (e.g., averaging the coherence signal). Alternatively, in some embodiments, the impulse response coherence calculator <NUM> generates a single coherence value directly from the two sub-band impulse responses included in the paired combination. Upon calculating the coherence value <NUM>, the impulse response coherence calculator <NUM> adds the coherence value <NUM> for the paired combination into the coherence value set <NUM>. In some embodiments, the impulse response coherence calculator <NUM> includes an index that maps coherence value <NUM> to the associated pair of sub-band impulse responses.

In various embodiments, the impulse response coherence calculator <NUM>, upon determining that the coherence value set <NUM> for the sub-band grouping <NUM> is complete, selects an impulse response pair <NUM> and a corresponding coherence value <NUM> based on the coherence values <NUM> included in the coherence value set <NUM>. In some embodiments, the impulse response coherence calculator <NUM> selects the impulse response pair <NUM> from the impulse response pairs that has a highest corresponding coherence value <NUM>. For example, when each coherence value <NUM> is a single value, the impulse response coherence calculator <NUM> determines the maximum coherence value from the coherence value set <NUM>. The impulse response coherence calculator <NUM> selects the impulse response pair <NUM> corresponding to the maximum coherence value and sets the selected coherence value <NUM> equal to the maximum coherence value. In some embodiments, the coherence values <NUM> varies over the frequency range of the sub-band. In such instances, the impulse response coherence calculator <NUM> the maximum average value and selects the impulse response pair <NUM>. Alternatively, the impulse response coherence calculator <NUM> selects an impulse response pair <NUM> corresponding to a specific coherence value <NUM> from the coherence value set <NUM> using different criteria. For example, the impulse response coherence calculator <NUM> can select the impulse response pair <NUM> having a coherence value <NUM> corresponding to the median, mean, or minimum value in the coherence value set <NUM>.

In various embodiments, the interpolator <NUM> compares the selected coherence value <NUM> to a coherence threshold. In some embodiments, two or more sub-bands share a common coherence threshold. Alternatively, the interpolator <NUM> maintains separate coherence thresholds for each sub-band. When the interpolator <NUM> determines that the selected coherence value <NUM> is equal to or above the coherence threshold, the interpolator <NUM> determines to use linear interpolation to generate the portion of the estimated impulse response <NUM>. For example, the interpolator <NUM> can use a specific linear interpolation technique such as weighted interpolation, where the interpolator <NUM> estimates the impulse response as inversely proportional to the distance between the target location <NUM> and the respective locations <NUM> of each of the sub-band impulse responses included in the selected impulse response pair <NUM>. Otherwise, the interpolator <NUM> determines that the selected coherence value <NUM> is below the coherence threshold and selects a non-linear interpolation technique to generate the portion of the estimated impulse response <NUM>.

In some embodiments, the interpolator <NUM> selects a non-linear interpolation technique from a group of available non-linear interpolation techniques. For example, the interpolator <NUM> can select a non-linear interpolation technique that uses at least the impulse responses from the selected impulse response pair <NUM>. In various embodiments, the interpolator <NUM> can select one of a Lagrange interpolation, a least-squares interpolation, a bicubic spline interpolation, a cosine interpolation, or a parabolic interpolation. Alternatively, the interpolator <NUM> can set one of the impulse responses included in the selected impulse response pair <NUM> as the estimated impulse response <NUM>.

In various embodiments, the filter calculator <NUM> sets one or more filter parameters <NUM> based at least one the estimated impulse response <NUM>. In various embodiments, the filter calculator <NUM> determines filter parameters <NUM>, which include a set of values that modify the operating characteristics (e.g., center frequency, gain, Q factor, cutoff frequencies, etc.) of the filter <NUM>. In such instance, the filter calculator <NUM> modifies the filter parameters <NUM> such that the filter <NUM> enables the corresponding speaker to generate a specific sound filed. For example, the filter parameters <NUM> can include one or more DSP coefficients that steer the generated soundwave in a specific direction. In various embodiments, the filter calculator <NUM> uses the estimated impulse response <NUM> to set filter parameters <NUM> to ensure that the sound field accurately reproduces the audio signal at the target location <NUM>.

<FIG> sets forth a flow chart of method steps for generating a filter for a speaker based on an estimated impulse response for a target location, according to one or more embodiments. Although the method steps are described with reference to the systems and embodiments of <FIG>, persons skilled in the art will understand that any system configured to implement the method steps, in any order, falls within the scope of the present disclosure.

As shown, the method <NUM> begins at step <NUM>, where the audio processing application <NUM> identifies a location requiring an estimated impulse response. In various embodiments, the audio processing application <NUM> determines a target location <NUM> within a listening environment <NUM> for which an accurate sound field is to be produced. For example, the audio processing application <NUM> acquires tracking data from one or more sensors <NUM> that indicate the location of a listener within the listening environment <NUM>.

At step <NUM>, the audio processing application <NUM> acquires a set of stored impulse responses <NUM> near the target location <NUM>. In various embodiments, the audio processing application <NUM> identifies two or more measured impulse responses for locations <NUM> within the listening environment <NUM> that are proximate to the target location <NUM>. For example, the audio processing application <NUM> retrieves impulse response data <NUM> that includes a dataset mapping various measured impulse responses in the environment to corresponding locations <NUM> in the listening environment <NUM>. In some embodiments, the dataset includes a sparse set of stored impulse responses <NUM>. For example, the dataset can include a small group of stored impulse responses <NUM> that were measured at predetermined positions within the listening environment <NUM> (e.g., various positions within a room).

In various embodiments, the audio processing application <NUM> selects from the dataset a subset <NUM> of stored impulse responses <NUM> corresponding to locations <NUM> near the target location <NUM>. In some embodiments, the audio processing application <NUM> selects each stored impulse response for a location within a threshold Euclidean or perceived audio distance of the target location <NUM>. Additionally or alternatively, the audio processing application <NUM> selects from the dataset a specific number of stored impulse responses <NUM>, such as stored impulse responses for three locations <NUM> closest to the target location <NUM> by Euclidean or perceived audio distance that also form an area encompassing at least a portion of the target location <NUM>. In some embodiments, the audio processing application <NUM> can use other criteria and/or employ other heuristics to add impulse responses from the dataset in the impulse response data <NUM> into the subset <NUM>.

At step <NUM>, the audio processing application <NUM> separates each stored impulse response into responses for multiple frequency sub-bands. In various embodiments, the audio processing application <NUM> decomposes each of the stored impulse responses <NUM> included in the subset <NUM> of stored impulse responses to generate a plurality of signals corresponding to frequency impulse responses for a specific sub-band frequency range. In some embodiments, the audio processing application <NUM> uses a filter bank, separate from the filters <NUM>, to generate the respective sub-band impulse responses. For example, the audio processing application <NUM> can use a filter bank of separate bandpass filters, DSP-based bandpass filters, and/or the like. In various embodiments, the audio processing application <NUM> groups sub-band impulse responses by frequency sub-band. For example, upon decomposing each of the N impulse responses in the subset <NUM> (e.g., each of the first impulse response <NUM>(<NUM>) through the Nth impulse response <NUM>(N)) into X separate sub-band impulse responses <NUM>(<NUM>)-<NUM>(X),. <NUM>(<NUM>)-<NUM>(X), the audio processing application <NUM> generates a sub-band grouping <NUM>(<NUM>) for the first frequency sub-band that includes each of the N impulse responses for the first frequency sub-band. In such instances, the audio processing application <NUM> repeats steps <NUM>-<NUM> for each of the X-<NUM> remaining sub-band groupings <NUM>(<NUM>)-<NUM>(X).

At step <NUM>, the audio processing application <NUM> determines whether each sub-band grouping <NUM> has been processed. In various embodiments, the audio processing application <NUM> determines whether each of X sub-band groupings <NUM> has been processed by determining whether the audio processing application <NUM> has generated estimated impulse responses <NUM>(<NUM>)-<NUM>(X) for each sub-band grouping <NUM>(<NUM>)-<NUM>(X). When the audio processing application <NUM> determines that each sub-band grouping <NUM>(<NUM>)-<NUM>(X) has a corresponding estimated impulse response <NUM>(<NUM>)-<NUM>(X), the audio processing application <NUM> proceeds to step <NUM>. Otherwise, the audio processing application <NUM> determines that the impulse responses for each sub-band grouping <NUM> has not been processed and responds by selecting an unprocessed sub-band grouping <NUM> and proceeds to step <NUM>.

At step <NUM>, the audio processing application <NUM> selects impulse response pair a based on coherence values for a group of impulse response pairs. In various embodiments and as will be discussed further in relation to <FIG>, the impulse response coherence calculator <NUM> included in the audio processing application <NUM> iteratively calculates separate coherence values <NUM> based on the spectral densities of the respective impulse responses. In some embodiments, the coherence value is based on a cross-spectral density between two impulse responses and indicates the coherence between pairs of sub-band impulse responses included in the sub-band grouping <NUM>. Upon computing each coherence value <NUM>, the audio processing application <NUM> selects a sub-band impulse response pair <NUM> and corresponding coherence value <NUM>.

For example, when computing a coherence value <NUM> for a pair of sub-band impulse responses, the audio processing application <NUM> initially computes a coherence signal between two sub-band impulse responses, then determines a single coherence value by averaging the coherence signal. Upon calculating the coherence value <NUM>, the audio processing application <NUM> adds the coherence value for the paired combination of sub-band impulse responses to a coherence value set <NUM>. When the coherence value set <NUM> is complete, the audio processing application <NUM> identifies a coherence value from the coherence value set <NUM> that meets specific criteria. The audio processing application <NUM> selects an impulse response pair <NUM> that produced the identified coherence value. In various embodiments, the impulse response coherence calculator <NUM> compares the coherence values <NUM> included in the coherence value set <NUM> and determines a coherence value from the coherence value set <NUM> that meets one or more criteria. In some embodiments, the impulse response coherence calculator <NUM> selects the highest value coherence values <NUM> and selects the pair of sub-band impulse responses that have the corresponding coherence value.

At step <NUM>, the audio processing application <NUM> determines whether the selected coherence value corresponding to the selected impulse response pair <NUM> is below a coherence threshold. In various embodiments, the interpolator <NUM> compares the selected coherence value <NUM>, which is equal to the coherence value corresponding to the selected impulse response pair <NUM>, to a coherence threshold. In some embodiments, the audio processing application <NUM> uses a same coherence threshold for multiple sub-bands. Alternatively, each sub-band has a distinct coherence threshold. When the interpolator <NUM> determines that the selected coherence value <NUM> is equal to or above the coherence threshold, the interpolator <NUM> proceeds to step <NUM>. Otherwise, the interpolator <NUM> determines that the selected coherence value <NUM> is below the coherence threshold and proceeds to step <NUM>.

At step <NUM>, the audio processing application <NUM> estimates the impulse response for the sub-band using a linear interpolation technique. In various embodiments, the interpolator <NUM> uses a linear interpolation technique to generate a portion of the estimated impulse response <NUM> for the target location <NUM>. In some embodiments, the interpolator uses one or more additional sub-band impulse responses <NUM>(<NUM>)-<NUM>(<NUM>) included in the sub-band grouping <NUM> during the linear interpolation. Upon the interpolator <NUM> generating the portion of the estimated impulse response <NUM>, the audio processing application <NUM> returns to step <NUM> to process any of the remaining sub-band groupings <NUM>.

At step <NUM>, the audio processing application <NUM> selects a non-linear interpolation technique. In various embodiments, upon the impulse response coherence calculator <NUM> determining that the selected coherence value <NUM> is below the coherence threshold, the interpolator <NUM> selects a non-linear interpolation technique. For example, the interpolator <NUM> selects a non-linear interpolation technique for combining the selected impulse response pair <NUM>. For example, the interpolator <NUM> can select one of a Lagrange interpolation, a least-squares interpolation, a bicubic spline interpolation, a cosine interpolation, or a parabolic interpolation. Alternatively, the interpolator can select the closest impulse response from the selected impulse response pair (e.g., nearest-neighbor interpolation).

At step <NUM>, the audio processing application <NUM> estimates the impulse response for the sub-band using the selected non-linear interpolation technique. In various embodiments, the interpolator <NUM> generates a portion of the estimated impulse response <NUM> for the frequency sub-band using the selected non-linear interpolation technique. The number of sub-band impulse responses <NUM>(<NUM>)-<NUM>(<NUM>) that the interpolator <NUM> uses when generating the portion of the estimated impulse response <NUM> varies based on the selected non-linear technique. For example, when the selected non-linear interpolation technique uses data from three or more impulse responses, the interpolator <NUM> selects additional sub-band impulse responses from the sub-band grouping <NUM> when generating the portion of the estimated impulse response. In another example, when the selected non-linear interpolation technique uses data from one or two impulse responses, the interpolator <NUM> uses one or both of the selected impulse response pair <NUM> to generate the portion of the estimated impulse response <NUM>. Upon the interpolator <NUM> generating the portion of the estimated impulse response <NUM> for the frequency sub-band, the audio processing application <NUM> returns to step <NUM> to process any of the remaining sub-band groupings <NUM>.

At step <NUM>, the audio processing application <NUM> generates a filter based on a complete estimated impulse response. In various embodiments, upon the interpolator <NUM> completing the estimated impulse response <NUM>, the filter calculator <NUM> sets one or more filter parameters <NUM> based at least one the estimated impulse response <NUM>. In various embodiments, the filter calculator <NUM> determines a set of filter parameters <NUM> that modify the operating characteristics (e.g., center frequency, gain, Q factor, cutoff frequencies, etc.) of the filter <NUM>.

At step <NUM>, the audio processing application <NUM> drives the speaker <NUM> to generate a sound field using the filter <NUM> generated in step <NUM>. In various embodiments, upon setting the filter <NUM>, the audio processing application <NUM> drives the speaker <NUM> to generate a portion of a sound field. The audio processing application <NUM> drives the speaker <NUM> to process an audio signal using the filter <NUM> to generate a filtered audio signal. In some embodiments, the filtered audio signal includes directivity information corresponding to the direction towards the target location <NUM> relative to the specific position of the speaker <NUM> (e.g., the position and/or orientation of the speaker <NUM>). The speaker <NUM> reproduces the filtered audio signal, generating an audio output corresponding to the filtered audio signals created by the filter <NUM>. For example, the audio processing application <NUM> drives the speaker <NUM> to generate a set of soundwaves in the direction toward the target location <NUM>. In various embodiments, the set of soundwaves that the speaker <NUM> generates combines with other soundwaves produced by other speakers <NUM> to generate a sound field that accurately reflects the estimated impulse response <NUM>. In some embodiments, the audio processing application <NUM> returns to step <NUM> to determine whether the target location <NUM> has changed.

In some embodiments, the audio processing application <NUM> repeats steps <NUM>-<NUM> for each speaker <NUM> that is to produce the sound field. For example, the audio processing application <NUM> uses the estimated impulse response <NUM> for each speaker <NUM> to generate distinct filter parameters <NUM> for the respective filters <NUM> of each speaker <NUM> in the listening environment <NUM>. Alternatively, in some embodiments, the audio processing application <NUM> repeats steps <NUM>-<NUM> for subsets of speakers <NUM> that generate different sound fields. For example, the audio processing device can determine distinct filter parameters <NUM> for the respective filters <NUM> of separate subsets of speakers <NUM> in the listening environment <NUM> that are to generate different sound fields for different target locations (e.g., separate sound fields for passengers in a vehicle).

In some embodiments, the audio processing application <NUM> tracks multiple listeners. In such instances, the audio processing application <NUM> can separately set the location of the respective listeners as one of multiple target locations <NUM> requiring an estimated impulse response. The audio processing application <NUM> repeats steps <NUM>-<NUM> to generate sound fields for each of the respective target locations <NUM>. For example, the audio processing application <NUM> initially determines a first estimated impulse response for a first location corresponding to a first listener, generates a sound field for the first listener, and then determines a second estimated impulse response for a second location corresponding to a second listener. In some embodiments, the audio processing application <NUM> can set different filter parameters <NUM> for different subsets of speakers <NUM> to generate separate sound fields for each of the respective target locations <NUM>.

<FIG> sets forth a flow chart of method steps for selecting a pair of sub-band impulse responses for use in an interpolation from a sub-band impulse response grouping <NUM>, according to one or more embodiments. Although the method steps are described with reference to the embodiments of <FIG>, persons skilled in the art will understand that any system configured to implement the method steps, in any order, falls within the scope of the present disclosure.

Method <NUM> begins at step <NUM>, the audio processing application <NUM> determines whether each pair of sub-band impulse responses in a sub-band grouping <NUM> has been processed. In various embodiments, the impulse response coherence calculator <NUM> included in the audio processing application <NUM> determines whether the coherence value set <NUM> for the sub-band grouping <NUM> currently stores Z coherence values <NUM>(<NUM>)-<NUM>(Z), where Z is equivalent to <MAT> combinations of N sub-band impulse responses included in the sub-band grouping <NUM>. When the impulse response coherence calculator <NUM> determines that each sub-band pair has been processed, the impulse response coherence calculator <NUM> proceeds to step <NUM>. Otherwise, the impulse response coherence calculator <NUM> determines that at least one sub-band pair requires processing and proceeds to step <NUM>.

At step <NUM>, the audio processing application <NUM> selects a first sub-band impulse response (IR) from the sub-band grouping <NUM>. In various embodiments, the impulse response coherence calculator <NUM> iteratively generates each coherence value <NUM> by selecting a first sub-band impulse response (j) of a sub-band impulse response pair from the sub-band grouping <NUM>.

At step <NUM>, the audio processing application <NUM> determines whether the coherence value set <NUM> for the first sub-band impulse response is complete. In various embodiments, the impulse response coherence calculator <NUM> determines whether the coherence value set <NUM> includes a coherence value <NUM> for each paired combination that includes the first sub-band impulse response. When the impulse response coherence calculator <NUM> determines that the coherence value set <NUM> includes each of the requisite coherence values <NUM> for the first sub-band impulse response, the impulse response coherence calculator <NUM> returns to step <NUM> to determine whether the select a different first sub-band impulse response. Otherwise, the impulse response coherence calculator <NUM> determines that the coherence value set <NUM> requires at least one coherence value <NUM> including the first sub-band impulse response and proceeds to step <NUM>.

At step <NUM>, the audio processing application <NUM> selects a second sub-band impulse response from the sub-band grouping. In various embodiments, the impulse response coherence calculator <NUM> selects at second sub-band impulse response (k) to form a paired combination with the first sub-band impulse response. When selecting the second sub-band impulse response, the impulse response coherence calculator <NUM> identifies, from the sub-band grouping <NUM>, a subgroup of sub-band impulse responses for which the impulse response coherence calculator <NUM> has not yet computed a coherence value <NUM> as part of a paired combination with the first sub-band impulse response. The impulse response coherence calculator <NUM> then selects one sub-band impulse response from the subgroup as the second sub-band impulse response.

At step <NUM>, the audio processing application <NUM> computes a coherence value for the paired combination that includes the first sub-band impulse response and the second sub-band impulse response. In various embodiments, the impulse response coherence calculator <NUM> computes a coherence value <NUM> for the paired combination (j, k) of sub-band impulse responses based on the spectral density of the impulse responses. In some embodiments, the impulse response coherence calculator <NUM> performs actions similar to step <NUM> of method <NUM> by calculating a coherence value <NUM>, such as a coherence signal, for the paired combination of the first and second sub-band impulse responses. In some embodiments, the impulse response coherence calculator <NUM> can determine a single coherence value from the coherence signal (e.g., averaging the coherence signal). Alternatively, in some embodiments, the impulse response coherence calculator <NUM> generates a single coherence value for the paired combination.

At step <NUM>, the audio processing application <NUM> adds the computed coherence value for the paired combination to the coherence value set <NUM>. In various embodiments, upon calculating the coherence value <NUM>, the impulse response coherence calculator <NUM> adds the coherence value <NUM> for the paired combination (j, k) into the coherence value set <NUM>. Upon adding the coherence value <NUM> to the coherence value set <NUM>, the impulse response coherence calculator <NUM> returns to the step <NUM> to determine whether the coherence value set <NUM> for the first sub-band impulse response is complete.

At step <NUM>, the audio processing application <NUM> selects an impulse response pair based on the coherence values in the coherence value set. In various embodiments, upon determining that the coherence value set <NUM> for the sub-band grouping <NUM> is complete, the interpolator <NUM>, based on the coherence values <NUM> included in the coherence value set <NUM>, selects an impulse response pair <NUM>. In some embodiments, the impulse response coherence calculator <NUM> performs actions similar to step <NUM> of method <NUM> by comparing the coherence values <NUM>(<NUM>)-<NUM>(Z) included in the coherence value set <NUM> and determining a coherence value that meets a set of one or more criteria and identifies the impulse response pair corresponding to the coherence value. For example, when each coherence value is a single value, the interpolator <NUM> determines the maximum coherence value, identifies the impulse response pair corresponding to the maximum coherence value, and sets the selected coherence value <NUM> equal to the maximum coherence value. In some embodiments, the coherence values <NUM> varies over the frequency range of the sub-band. In such instances, the interpolator <NUM> selects the impulse response pair <NUM> associated with the coherence signal possessing the maximum average value. Alternatively, the interpolator <NUM> selects the impulse response pair <NUM> using different criteria. For example, the interpolator <NUM> can select the impulse response pair <NUM> associated with a coherence value that corresponds to the median, mean, or minimum value in the coherence value set <NUM>.

In sum, an audio processing application sets the parameters for one or more filters that are used by a speaker to generate a sound field when reproducing an audio signal. The audio processing application generates the parameters for the one or more filters based on estimated impulse responses at a target location in a listening environment, such as where a listener is located. The audio processing application estimates the impulse response for a target location by acquiring stored impulse response data, such measured impulse responses at multiple locations within the listening environment. The audio processing system selects a subset of the stored impulse responses surrounding the target location based on one or more characteristics, such as Euclidean or perceived acoustic distance of the locations corresponding to the impulse responses relative to the target location. For each of the selected impulse responses, the audio processing application filters the impulse response into separate sub-band frequency impulse responses, each representing a separate frequency range. The audio processing application groups the selected impulse responses by sub-band, where a given sub-band grouping contains multiple impulse responses of a common sub-band.

For each sub-band grouping, the audio processing system selects a pair of impulse responses that are most similar (e.g., have a highest coherence value) to each other. If the impulse responses in the selected pair are sufficiently similar, a linear interpolation technique is used to combine the impulse responses in the selected pair. If the impulse responses in the selected pair are not sufficiently similar, a non-linear interpolation technique is used to combine the impulse responses in the selected pair. The combined impulse responses are then used to set the parameters of the one or more filters. The one or more filters are then used to process audio to be emitted by the speaker in order to generate a desired sound filed at the target location.

At least one technical advantage of the disclose techniques relative to the prior art is that, with the disclosed techniques, an audio processing system can more accurately generate a sound field for a particular location in an environment, which increases the auditory experience of a user at the particular location. Further, the disclosed techniques are able to generate impulse response filters more accurately for the particular location from a smaller set of impulse response filters than prior art techniques. The disclosed techniques therefore reduce the memory used by the audio processing system when estimating impulse responses at particular locations. Further, the disclosed techniques reduce the time needed to collect measurements of impulse responses at locations within a listening environment needed to generate an accurate sound field. These technical advantages provide one or more technological advancements over prior art approaches.

Aspects of the present embodiments may be embodied as a system, method, or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "module," a "system," or a "computer. " In addition, any hardware and/or software technique, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or set of circuits. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Claim 1:
A computer-implemented method comprising:
determining a target location in an environment;
determining a set of sub-band impulse responses for a first frequency sub-band, each sub-band impulse response in the set of sub-band impulse responses being associated with a corresponding location that is proximate to the target location;
selecting a first pair of sub-band impulse responses for the first frequency sub-band from among pairs of sub-band impulse responses in the set of sub-band impulse responses;
computing a first coherence value indicating a level of coherence between sub-band impulse responses in the first pair;
determining that the first coherence value is below a coherence threshold;
in response to determining that the first coherence value is below the coherence threshold, combining the sub-band impulse responses in the first pair using a non-linear interpolation technique to generate an estimated impulse response for the first frequency sub-band for the target location;
generating, based at least on the estimated impulse response, a filter for a speaker;
filtering, by the filter, an audio signal to generate a filtered audio signal; and
causing the speaker to output the filtered audio signal.