Patent Description:
Conventionally, head-related transfer functions (HRTF)s are determined in a sound dampening chamber for many different source locations (e.g., typically more than <NUM>) relative to a person. The determined HRTFs may then be used to provide spatialized audio content to the person. Moreover, to reduce error, it is common to determine multiple HRTFs for each source location (i.e., each speaker is generating a plurality of discrete sounds). Accordingly, for high quality spatialization of audio content it takes a relatively long time (e.g., more than an hour) to determine the HRTFs as there are multiple HRTFs determined for many different speaker locations. Additionally, the infrastructure for measuring HRTFs sufficient for quality surround sound is rather complex (e.g., sound dampening chamber, one or more speaker arrays, etc.). Accordingly, conventional approaches for obtaining HRTFs are inefficient in terms of hardware resources and/or time needed.

<CIT> discloses a sound reproduction system comprising an audio processor capable of applying an HRTF to an input audio signal, and arranged to apply a first head related transfer function over a first frequency range and a second head related transfer function over a second frequency range.

<CIT> discloses systems and methods to calibrate listening devices including a method to calibrate earphones including determining HRTFs corresponding to different parts of a user's anatomy (e.g., one or both of a listener's pinnae). The resulting HRTFs are combined to form a composite HRTF.

<CIT> discloses a system which calibrates one or more HRTFs for a user. An indicator is presented on a head-mounted display, where the indicator prompts the user to turn the user's head to view the indicator and effectively change the user's head orientation. Responsive to the indicator being viewed by the user, a sound source at a fixed position transmits a test sound and the test sound is received at microphones coupled to the user's ears. By analyzing the test sound received at the microphones, a unique HRTF associated with a relative position between the sound source and each ear can be obtained.

According to a first aspect of the present invention, there is provided a method comprising: capturing audio data of test sounds through a microphone of a headset worn by a user, the said test sounds played by an external speaker, and the said audio data including audio data captured for different orientations of the said headset with respect to the said external speaker; calculating a set of head-related transfer function (HRTFs) based at least in part on the said audio data of the test sounds at the different orientations of the said headset, the said set of HRTFs individualized to a user while wearing the said headset; discarding a portion of the said set of HRTFs to create an intermediate set of HRTFs, the said discarded portion including at least some HRTFs corresponding to orientations of the said headset where sound from the said external speaker was incident on the said headset prior to reaching an ear canal of the said user; and characterised by generating one or more HRTFs that correspond to the said discarded portion using at least some of the said intermediate set of HRTFs to create an individualized set of HRTFs for the said user; wherein said discarded portion is determined using a distortion mapping that identifies one or more distortion regions where sound is distorted by the presence of a headset, and wherein the said distortion mapping is based in part on comparisons between a set of HRTFs measured with at least one test user wearing a test headset and a set of HRTFs measured with the said at least one test user not wearing the said test headset.

Preferably, said generating the said one or more HRTFs that correspond to the said discarded portion using at least some of the said intermediate set of HRTFs comprises: interpolating at least some of the said intermediate set of HRTFs to generate the said one or more HRTFs that correspond to the said discarded portion.

Preferably, capturing the said audio data for different orientations of the said headset with respect to the said external speaker further comprises:.

Preferably, the method further comprises: uploading the said individualized set of HRTFs to a HRTF system, wherein the said HRTF system uses at least some of the said individualized set of HRTFs to update a the distortion mapping generated from comparisons between a set of HRTFs measured with at least one test user wearing a test headset and a set of HRTFs measured with the said at least one test user not wearing the said test headset.

According to a further aspect of the present invention there is provided a non-transitory computer-readable storage medium storing executable computer program instructions, the said instructions executable to perform steps comprising: receiving, from a headset, data representative of audio data of test sounds captured through a microphone of a the headset worn by a user, the said test sounds played by an external speaker, and the said audio data including audio data captured for different orientations of the said headset with respect to the said external speaker; calculating a set of head-related transfer function (HRTFs) based at least in part on the data representative of said audio data of the said test sounds at the said different orientations of the said headset, the set of HRTFs individualized to a user while wearing the said headset; and discarding a portion of the set of HRTFs to create an intermediate set of HRTFs, the said discarded portion including at least some HRTFs corresponding to data representative of orientations of the said headset where sound from the said external speaker was incident on the said headset prior to reaching an ear canal of the said user; characterised by the steps further comprising generating one or more HRTFs that correspond to the said discarded portion using at least some of the said intermediate set of HRTFs to create an individualized set of HRTFs for the said user and determining the said discarded portion using data representative of a distortion mapping that identifies one or more distortion regions where sound is distorted by the presence of a headset, and wherein the said distortion mapping is based in part on comparisons between a set of HRTFs measured with at least one test user wearing a test headset and a set of HRTFs measured with the said at least one test user not wearing the said test headset.

Preferably, receiving data representative of the said audio data for different orientations of the said headset with respect to the said external speaker further comprises: instructions to cause the headset to execute the steps of:
generating an indicator at a coordinate of a virtual space, the said indicator corresponding to a specific orientation of headset worn by the said user relative to an external speaker; and presenting, on a display of the said headset, the said indicator of the said coordinate in the said virtual space; determining that a first orientation of the said headset relative to the said external speaker is the specific orientation; instructing the said external speaker to play a test sound while the said headset is at the first orientation; obtaining the said audio data from the said microphone.

Preferably, the non-transitory computer-readable storage medium further comprises:
instructions, the instructions being executable to upload the said individualized set of HRTFs to a HRTF system, wherein the said HRTF system uses at least some of the said individualized set of HRTFs to update a distortion mapping generated from comparisons between a set of HRTFs measured with at least one test user wearing a test headset and a set of HRTFs measured with the said at least one test user not wearing the said test headset.

According to a further aspect of the present invention there is provided a system comprising: an external speaker configured to play one or more test sounds; a microphone assembly configured to capture audio data of the one or more test sounds; and a headset configured to be worn by a user, comprising an audio controller configured to: calculate a set of head-related transfer function (HRTFs) based at least in part on the said audio data of the said test sounds and at a plurality of different orientations of the said headset, the set of HRTFs individualized to a user while wearing the said headset; and discard a portion of the set of HRTFs to create an intermediate set of HRTFs, the portion including at least some HRTFs corresponding to orientations of the said headset where sound from the said external speaker was incident on the said headset prior to reaching an ear canal of the said user; characterised in that the controller is configured to generate one or more HRTFs that correspond to the said discarded portion using at least some of the said intermediate set of HRTFs to create an individualized set of HRTFs for the said user; and determinethe said discarded portion using a distortion mapping that identifies one or more distortion regions where sound is distorted by the presence of a headset, and wherein the said distortion mapping is based in part on comparisons between a set of HRTFs measured with at least one test user wearing a test headset and a set of HRTFs measured with the said at least one test user not wearing the said test headset.

The figures depict embodiments of the present disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles, or benefits touted, of the disclosure described herein.

Embodiments of the present disclosure may include or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some manner before presentation to a user, which may include, e.g., a virtual reality (VR), an augmented reality (AR), a mixed reality (MR), a hybrid reality, or some combination and/or derivatives thereof. Artificial reality content may include completely generated content or generated content combined with captured (e.g., real-world) content. The artificial reality content may include video, audio, haptic feedback, or some combination thereof, and any of which may be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional effect to the viewer). Additionally, in some embodiments, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof, that are used to, e.g., create content in an artificial reality and/or are otherwise used in (e.g., perform activities in) an artificial reality. The artificial reality system that provides the artificial reality content may be implemented on various platforms, including a headset, a headset connected to a host computer system, a standalone headset, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

An HRTF system herein is used to collect audio test data to determine common portions of HRTFs that are distorted by the presence of a headset. The HRTF system captures audio test data at a test user's ear canal in an acoustic chamber, both with the test user wearing a headset and without the headset. The audio test data is analyzed and compared to determine the effect of the presence of the headset on individualized HRTFs. The audio test data is collected for a population of test users and used to determine a set of distortion regions where the HRTFs are commonly distorted by the presence of the headset.

An audio system of a headset uses information from the HRTF system to calculate for a user a set of individualized HRTFs that compensate for the effects of the headset on the HRTFs. The user wears the headset and the audio system captures audio data of test sounds emitted from an external speaker. The external speaker may be, e.g., physically separate from the headset and audio system. The audio system calculates a set of initial HRTFs based at least in part on the audio data of the test sound at different orientations of the headset. The audio system discards a portion (based in part on at least some of the distortion regions determined by the HRTF server) of the set of initial HRTFs to create an intermediate set of HRTFs. The intermediate set of HRTFs is formed from the non-discarded HRTFs of the set of HRTFs. The discarded portion of the set of HRTFs corresponds to one or more distortion regions that are caused by the presence of the headset. The audio system generates one or more HRTFs (e.g., via interpolation) that correspond to the discarded portion of the set, which are combined with at least some of the intermediate set of HRTFs to create a set of individualized HRTFs for the user. The set of individualized HRTFs are customized to the user such that errors in the HRTFs caused by wearing the headset are mitigated, and thereby mimic actual HRTFs of the user without a headset. The audio system may use the set of individualized HRTFs to present spatialized audio content to the user. Spatialized audio content is audio that can be presented as if it is positioned at a specific point in three-dimensional space. For example, in a virtual environment, audio associated with a virtual object that is being displayed by the headset can appear to originate from the virtual object.

Note that in this manner, the audio system is effectively able to generate an individualized set of HRTFs for the user, even though the user is wearing the headset. This is much faster, easier, and cheaper than conventional methods of measuring a user's actual HRTFs in a customized sound dampening chamber.

<FIG> is a diagram of a sound measurement system (SMS) <NUM> for obtaining audio test data associated with a test user <NUM> wearing a headset <NUM>, in accordance with one or more embodiments. The sound measurement system <NUM> is part of an HRTF system (e.g., as described below with regard to <FIG>). The SMS <NUM> includes a speaker array <NUM>, and a binaural microphones 140a, 140b. In the illustrated embodiment, the test user <NUM> is wearing the headset <NUM> (e.g., as described in more detail in relation to <FIG> and <FIG>). The headset <NUM> may be called a test headset. The SMS <NUM> is used for measuring audio test data to determine a set of HRTFs for a test user <NUM>. The SMS <NUM> is housed in an acoustically treated chamber. In one particular embodiment, the SMS <NUM> is anechoic down to a frequency of approximately <NUM> hertz (Hz).

In some embodiments, the test user <NUM> is a human. In these embodiments, it is useful to collect audio test data for a large number of different people. The people can be of different ages, different sizes, different genders, have different hair lengths, etc. In this manner audio test data can be collected over a large population. In other embodiments, the test user <NUM> is a manikin. The manikin may, e.g., have physical features (e.g., ear shape, size, etc.) representative of an average person.

The speaker array <NUM> emits test sounds in accordance with instructions from a controller of the SMS <NUM>. A test sound is an audible signal transmitted by a speaker that may be used to determine a HRTF. A test sound may have one or more specified characteristics, such as frequency, volume, and length of the transmission. The test sounds may include, for example, a continuous sinusoidal wave at a constant frequency, a chirp, some other audio content (e.g., music), or some combination thereof. A chirp is a signal whose frequency is swept upward or downward for a period of time. The speaker array <NUM> comprises a plurality of speakers, including a speaker <NUM>, that are positioned to project sound to a target area. The target area is where the test user <NUM> is located during operation of the SMS <NUM>. Each speaker of the plurality of speakers is in a different location relative to the test user <NUM> in the target area. Note that, while the speaker array <NUM> is depicted in two-dimensions in <FIG> and <FIG>, it is noted that the speaker array <NUM> can also include speakers in other locations and/or dimensions (e.g., span three-dimensions). In some embodiments, the speakers in the speaker array <NUM> are positioned spanning in elevation from -<NUM>° to +<NUM>° with a spacing of <NUM>°-<NUM>° between each speaker <NUM> and spans every <NUM>° in azimuth around a full sphere. That is, <NUM> azimuths and <NUM> elevations, creating a total of <NUM> different angles of speakers <NUM> with respect to the test user <NUM>. In some embodiments, one or more speakers of the speaker array <NUM> may dynamically change their position (e.g., in azimuth and/or elevation) relative to the target area. Note in the above description the test user <NUM> is stationary (i.e., the position of the ears within the target area stays substantially constant).

The binaural microphones 140a, 140b (collectively referred to as "<NUM>") capture the test sounds emitted by the speaker array <NUM>. The captured test sounds are referred to as audio test data. The binaural microphones <NUM> are each placed in an ear canal of the test user. As illustrated, the binaural microphone 140a is placed in the ear canal of the right ear of the user, and the microphone 140b is placed in the ear canal of the left ear of the user. In some embodiments, the microphones <NUM> are embedded in foam earplugs that are worn by the test user <NUM>. As discussed in detail below with regard to <FIG>, the audio test data can be used to determine a set of HRTFS. For example, test sounds emitted by a speaker <NUM> of the speaker array <NUM> are captured by the binaural microphones <NUM> as audio test data. The speaker <NUM> has a specific location relative to the ears of the test user <NUM>, accordingly, there is a specific HRTF for each ear that can be determined using the associated audio test data.

<FIG> is a diagram of the SMS <NUM> of <FIG> configured to obtain audio test data associated with the test user <NUM> not wearing a headset, in accordance with one or more embodiments. In the illustrated embodiments, the SMS <NUM> collects audio test data in the same way described above with respect to <FIG>, except that the test user <NUM> in <FIG> is not wearing a headset. Accordingly, the audio test data collected can be used to determine actual HRTF's of the test user <NUM> that do not include distortion introduced by wearing the headset <NUM>.

<FIG> is a block diagram a HRTF system <NUM>, in accordance with one or more embodiments. The HRTF system <NUM> captures audio test data and determines portions of HRTFs commonly distorted by a headset. The HRTF system <NUM> includes a sound measurement system <NUM>, and a system controller <NUM>. In some embodiments some or all of the functions of the system controller <NUM> may be shared and/or performed by the SMS <NUM>.

The SMS <NUM> captures audio test data to be used by the HRTF system <NUM> to determine a mapping of distortion regions. In particular, the SMS <NUM> is used to capture audio test data that is used to determine HRTFs of a test user. The SMS <NUM> includes a speaker array <NUM> and microphones <NUM>. In some embodiments, the SMS <NUM> is the SMS <NUM> described in relation to <FIG> and <FIG>. The captured audio data is stored in the HRTF data store <NUM>.

The speaker array <NUM> emits test sounds in accordance with instructions from the system controller <NUM>. The test sounds transmitted by the speaker array <NUM> may include, for example, a chirp (a signal whose frequency is swept upward or downward for a period of time), some other audio signal that may be used for HRTF determination, or some combination thereof. The speaker array <NUM> comprises one or more speakers that are positioned to project sound to a target area (i.e., location where a test user is located). In some embodiments, the speaker array <NUM> includes a plurality of speakers and each speaker of the plurality of speakers is in a different location relative to the test user in the target area. In some embodiments, one or more speakers of the plurality of speakers may dynamically change their position (e.g., in azimuth and/or elevation) relative to the target area. In some embodiments, one or more speakers of the plurality of speakers may change their position (e.g., in azimuth and/or elevation) relative to the test user by instructing the test user to rotate his/her head. The speaker array <NUM> is an embodiment of the speaker array <NUM>.

The microphones <NUM> capture the test sounds emitted by the speaker array <NUM>. The captured test sounds are referred to as audio test data. The microphones <NUM> include binaural microphones for each ear canal, and may include additional microphones. The additional microphones may be placed, e.g., in areas around the ears, along different portions of the headset, etc. The binaural microphones <NUM> are an embodiment of the microphones <NUM>.

The system controller <NUM> generates control components of the HRTF system <NUM>. The system controller <NUM> includes an HRTF data store <NUM>, a HRTF module <NUM>, and a distortion identification module <NUM>. Some embodiments of the system controller <NUM> may include other components than those described herein. Similarly, the functions of components may be distributed differently than described here. For example, in some embodiments, some or all of the functionality of the HRTF module <NUM> may be part of the SMS <NUM>.

The HRTF data store <NUM> stores data relating to the HRTF system <NUM>. The HRTF data store <NUM> may store, e.g., audio test data associated with test users, HRTFs for test users wearing a headset, HRTFs for test users that are not wearing the headset, distortion mappings including sets of distortion regions for one or more test users, distortion mappings including sets of distortion regions for one or more populations of test users, parameters associated with physical characteristics of the test users, other data relating to the HRTF system <NUM>, or some combination thereof. The parameters associated with physical characteristics of the test users may include gender, age, height, ear geometry, head geometry, and other physical characteristics that affect how audio is perceived by a user.

The HRTF module <NUM> generates instructions for the speaker array <NUM>. The instructions are such that the speaker arrays <NUM> emits test sounds that can be captured at the microphones <NUM>. In some embodiments, the instructions are such that each speaker of the speaker array <NUM> plays one or more a respective test sounds. And each test sound may have one or more of a specified length of time, a specified volume, a specified start time, a specified stop time, and a specified waveform (e.g., chirp, frequency tone, etc.). For example, the instructions may be such that one or more speakers of the speaker array <NUM> play, in sequence, a <NUM>-second logarithmic sine sweep, ranging infrequency from <NUM> to <NUM> at a sampling frequency of <NUM>, with a sounds level of <NUM> decibel of sounds pressure level (dB SPL). In some embodiments, each speaker of the speaker array <NUM> is associated with a different position relative to the target area, accordingly, each speaker is associated with a specific azimuth and elevation relative to the target area. In some embodiments, one or more speakers of the speaker array <NUM> may be associated with multiple positions. For example, the one or more speakers may change position relative to the target area. In these embodiments, the generated instructions may also control motion of some or all of speakers in the speaker array <NUM>. In some embodiments, one or more speakers of the speaker array <NUM> may be associated with multiple positions. For example, the one or more speakers may change position relative to the test user by instructing the target user to rotate his/her head. In these embodiments, the generated instructions may also be presented to the test user. The HRTF module <NUM> provides the generated instructions to the speaker array <NUM> and/or the SMS <NUM>.

The HRTF module <NUM> determines HRTFs for the test user using the audio test data captured via the microphones <NUM>. In some embodiments, for each test sound played by a speaker of the speaker array <NUM> at a known elevation and azimuth, the microphones <NUM> capture audio test data of the test sound at the right ear and audio test data at the left ear (e.g., using binaural microphones as the microphones <NUM>). The HRTF module <NUM> uses audio test data for the right ear and the audio test data for the left ear to determine a right-ear HRTF and a left-ear HRTF, respectfully. The right-ear and left-ear HRTFs are determined for a plurality of different directions (elevation and azimuth) that each correspond to a different location of a respective speaker in the speaker array <NUM>.

Each set of HRTFs is calculated from captured audio test data for a particular test user. In some embodiments, the audio test data is a head-related impulse response (HRIR), where the test sound is the impulse. A HRIR relates the location of the sound source (i.e., a particular speaker in the speaker array <NUM>) to the location of the test user's ear canal (i.e., the location of the microphones <NUM>). The HRTFs are determined by taking the Fourier transform of each corresponding HRIR. In some embodiments, error in the HRTFs is mitigated using free-field impulse response data. The free-field impulse response data may be deconvolved from the HRIRs to remove the individual frequency response of the speaker array <NUM> and the microphones <NUM>.

The HRTFs are determined at each direction both with the test user wearing a headset <NUM> (e.g., as shown in <FIG>) and the test user not wearing a headset (e.g., as shown in <FIG>). For example, the HRTFs are determined at each elevation and azimuth with the test user wearing the headset <NUM> (as shown in <FIG>), then the headset <NUM> is removed, and the HRTFs are measured at each elevation and azimuth with the user not wearing the headset <NUM> (as shown in <FIG>). Audio test data at each speaker direction, both with and without the headset <NUM>, may be captured for a population (e.g., hundreds, thousands, etc.) of test users. The population of test users may include individuals of differing ages, sizes, genders, hair lengths, head geometry, ear geometry, some other factor that can affect an HRTF, or some combination thereof. For each test user, there is a set of individualized HRTFs with the headset <NUM> and a set of individualized HRTFs without the headset <NUM>.

The distortion identification module <NUM> compares one or more of the sets of HRTFs of a test user wearing a headset to one or more of the sets of HRTFs of the test user not wearing a headset. In one embodiment, the comparison involves the evaluation of the two sets of HRTFs using spectral difference error (SDE) analysis and determining discrepancies in the interaural time difference (ITD).

The SDE between the set of HRTFs without the headset and the set of HRTFs with the headset, for a particular test user, is calculated based on the formula: <MAT>.

Where Ω is direction angle (azimuth and elevation), f is the frequency of the test sound, HRTFWO(Ω, f) is the HRTF without the headset for the direction Ω and frequency f, and HRTFHeadset(Ω, f) is the HRTF with the headset for the direction Ω and frequency f. The SDE is calculated for each pair of HRTFs with and without the headset at a particular frequency and direction. The SDE is calculated for both ears at each frequency and direction.

In one embodiment, ITD error is also estimated by determining the time when the result of the correlation between the right and the left HRIRs reaches a maximum. For each measured test user, the ITD error may be calculated as the absolute value of the difference between the ITD of the HRTF without the headset and with the headset for each direction.

In some embodiments, a comparison of the set of HRTFs of a test user wearing a headset to the set of HRTFs of the test user not wearing a headset includes an additional subjective analysis. In one embodiment, each test user who had their HRTFs measured with and without the headset participates in a Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) listening test to corroborate the results of the objective analysis. In particular, the MUSHRA test consists of a set of generalized HRTFs without the headset, a set of generalized HRTFs with the headset, the test user's individualized set of HRTFs without the headset, and the test user's individualized set of HRTFs with the headset, wherein the set of individualized HRTFs without the headset is the hidden reference and there is no anchor.

The distortion identification module <NUM> determines an average comparison across the population of test users. To determine an average comparison the SDEWO-Headset(Ω, f) for each test user is averaged across the population of test users at each frequency and direction, denoted by SDEWO-Headset(Ω, f): <MAT>.

Where N is the total number of test users in the population of users. In alternate embodiments, SDEWO-Headset(Ω, f) may be determined by alternate calculations.

In one embodiment, the determination further includes averaging across the span of frequencies measured (e.g., <NUM>-<NUM>), denoted by SDEWO-Headset(Ω). The SDE is found to generally be higher at higher frequencies. That is, the HRTF with the headset differs more dramatically from the HRTF without the headset at higher frequencies due to the fact that at high frequencies the wavelengths are large relative to the headset's form factor. Because of the general trend that the SDE is greater at higher frequencies, averaging across all frequencies allows for determination of particular azimuths and elevations at which the distortion due to the headset is more extreme.

The average ITD error across the population of test users, ITDWO-Headset(Ω), is calculated based on the following formula: <MAT>.

Where N is the total number of test users in the population of test users, ITDWOi(Ω) is the maximum ITD of the HRTF without the headset at direction Ω of user i, and ITDHeadseti(Ω) is the maximum ITD of the HRTF with the headset at direction Ω of user i.

The distortion identification module <NUM> determines a distortion mapping that identifies a set of one or more distortion regions based on portions of HRTFs commonly distorted across the population of test users. Using the SDEWO-Headset(Ω) and ITDWO-Headset(Ω), the directional dependence of the distortion of the HRTFs based on the presence of the headset can be determined. Both SDEWO-Headset(Ω) and ITDWO-Headset(Ω) can be plotted in two dimensions to determine particular azimuths and elevations where the errors are the greatest in magnitude. In one embodiment, the directions with the greatest error are determined by a particular threshold value of SDE and/or ITD. The determined directions of greatest error are the set of one or more distortion regions.

In one example, the threshold is high error in the contralateral direction greater than 4dB of SDE. In this example, based on the SDEWO-Headset(Ω) for the left-HRTFs, regions of azimuth [-<NUM>°, -<NUM>°] and elevation [-<NUM>°, <NUM>°] and regions of azimuth [-<NUM>°, -<NUM>°] and elevation [-<NUM>°, <NUM>°] are above the SDE threshold. These regions are thereby determined to be the distortion regions.

In another example, the threshold is ITDWO-Headset(Ω) > <NUM>. In this example, directions corresponding to the regions of azimuth [-<NUM>°, -<NUM>°] and elevation [-<NUM>°, <NUM>°], azimuth [-<NUM>°, -<NUM>°] and elevation [<NUM>°, <NUM>°], azimuth [<NUM>°, <NUM>°] and elevation [<NUM>°, <NUM>°], and azimuth [<NUM>°, -<NUM>°] and elevation [-<NUM>°, <NUM>°] are above the ITD threshold. These regions are thereby determined to be the distortion regions.

The SDE and ITD analysis and thresholds may determine different distortion regions. In particular, the ITD analysis may result in smaller distortion region than the SDE analysis. In different embodiments, the SDE and ITD analyses may be used independently from one another, or used together.

Note that the distortion mapping is based on the HRTFs determined for a population of test users. In some embodiments, the population may be a single manikin. But in other embodiments, the population may include a plurality of test users having a large cross section of different physical characteristics. Note that in some embodiments, distortion maps are determined for populations having one or more common physical characteristics (e.g., age, gender, size, etc.). In this manner, the distortion identification module <NUM> may determine multiple distortion mappings that are each indexed to one or more specific physical characteristics. For example, one distortion mapping could be specific to adults that identifies a first set of distortion regions, and a separate distortion map could be specific to children that may identify a second set of distortion regions that are different than the first set of distortion regions.

The HRTF system <NUM> may communicate with one or more headsets and/or consoles. In some embodiments, the HRTF system <NUM> is configured to receive a query for distortion regions from a headset and/or console. In some embodiments, the query may include parameters about a user of the headset, which is used by the distortion identification module <NUM> to determine a set of distortion regions. For example, the query may include specific parameters about the user, such as height, weight, age, gender, dimensions of ears, and/or type of headset being worn. The distortion identification module <NUM> can use one or more of the parameters to determine a set of distortion regions. That is, the distortion identification module <NUM> uses parameters provided by the headset and/or console to determine a set of distortion regions from audio test data captured from test users with similar characteristics. The HRTF server <NUM> provides the determined set of distortion regions to the requesting headset and/or console. In some embodiments, the HRTF server <NUM> receives information (e.g., parameter about a user, sets of individualized HRTFs, HRTFs measured while a user is wearing a headset from a headset and/or console, or some combination thereof) from a headset (e.g., via a network). The HRTF server <NUM> may use the information to update one or more distortion mappings.

In some embodiments, the HRTF system <NUM> may be remote and/or separate from the sound measurement system <NUM>. For example, the sound measurement system <NUM> may be communicatively coupled with the HRTF system <NUM> via a network (e.g., local area network, Internet, etc.). Similarly, the HRTF system <NUM> may connect to other components via a network, as discussed in greater detail below in reference to <FIG> and <FIG>.

<FIG> is a flowchart illustrating a process <NUM> of obtaining a set of distortion regions, in accordance with one or more embodiments. In one embodiment, the process <NUM> is performed by the HRTF system <NUM>. Other entities may perform some or all of the steps of the process <NUM> in other embodiments (e.g., a server, headset, other connected device). Likewise, embodiment may include different and/or additional steps or perform the steps in a different order.

The HRTF system <NUM> determines <NUM> a set of HRTFs for a test user wearing a headset and a set of HRTFs for the test user not wearing the headset. Audio test data is captured by one or more microphones that are at or near the ear canals of a test user. The audio test data is captured for test sounds played from a variety of orientations, both with the test user wearing a headset and the user not wearing the headset. The audio test data is collected at each orientation both with and without the headset such that the audio test data can be compared for the instances with the headset and the instances without the headset. In one embodiment, this is done by the processes discussed above in relation to <FIG> and <FIG>.

Note that audio test data can be captured over a population of test users that includes one or more test users from which audio test data was measured. In some embodiments, the population of test users can be one or more people. The one or more people can be further divided into subsets of the population based on different physical characteristics, such as gender, age, ear geometry, head dimensions, some other factor that may affect HRTFs for the test user, or some combination thereof. In other embodiments, a test user may be a manikin head. In some embodiments, a first manikin head may have average physical characteristics, whereas other manikins may have different physical characteristics and be similarly subdivided into subsets based on the physical characteristics.

The HRTF system <NUM> compares <NUM> the set of HRTFs for the test user wearing a headset and the set of HRTFs for the test user not wearing a headset. In one embodiment, the comparison <NUM> is performed using SDE analysis and/or ITD, as previously discussed in relation to the HRTF module <NUM> of <FIG> and equation (<NUM>). The comparison <NUM> may be repeated for a population of test users. The sets of HRTFs and corresponding audio test data can be grouped based on the physical characteristics of the population of test users.

The HRTF system <NUM> determines <NUM> a set of distortion regions based on portions of the HRTFs commonly distorted across a population of test users. In some embodiments, the population of test users is a subset of the previously discussed population of test users. In particular the distortion regions may be determined for a population of test users that is a subset of the total population of test users that meet one or more parameters based on physical characteristics. In one embodiment, the HRTF system <NUM> determines <NUM> using an average of the SDE and average of the ITD, as previously discussed in relation to the distortion identification module <NUM> of <FIG> and equations (<NUM>) and (<NUM>).

An audio system uses information from an HRTF system and HRTFs calculated while a user of a headset is wearing the headset to determine a set of individualized HRTFs that compensate for the effects of the headset. The audio system collects audio data for a user wearing a headset. The audio system may determine HRTFs for the user wearing the headset and/or provide the audio data to a separate system (e.g., HRTF system and/or console) for the HRTF determination. In some embodiments, the audio system requests a set of distortion regions based on the audio test data previously captured by the HRTF system, and uses the set of distortion regions to determine he individualized set of HRTFS for the user.

<FIG> is a diagram of an example artificial reality system <NUM> for obtaining audio data associated with a user <NUM> wearing a headset <NUM> using an external speaker <NUM> and a generated virtual space <NUM>, in accordance with one or more embodiments. The audio data obtained by the artificial reality system <NUM> is distorted by the presence of the headset <NUM>, which is used by an audio system to calculate an individualized set of HRTFs for the user <NUM> that compensates for the distortion. The artificial reality system <NUM> uses artificial reality to enable measurement of individualized HRTFs for the user <NUM> without the use of anechoic chamber, such as the SMSs <NUM>, <NUM> previously discussed in <FIG>.

The user <NUM> is an individual, distinct from the test user <NUM> of <FIG> and <FIG>. The user <NUM> is an end-user of the artificial reality system <NUM>. The user <NUM> may use the artificial reality system <NUM> to create a set of individualized HRTFs that compensate for distortion of the HRTFs caused by the headset <NUM>. The user <NUM> wears a headset <NUM> and a pair of microphones 450a, 450b (collectively referred to as "<NUM>"). The headset <NUM> can be the same type, model, or shape as the headset <NUM>, as described in more detail in relation to <FIG> and <FIG>. The microphones <NUM> can have the same properties as the binaural microphones <NUM>, as discussed in relation to <FIG>, or the or microphones <NUM>, as discussed in relation to <FIG>. In particular, the microphones <NUM> are located at or near the entrance to the ear canals of the user <NUM>.

The external speaker <NUM> is a device configured to transmit sound (e.g., test sounds) to the user <NUM>. For example, the external speaker <NUM> may be a smartphone, a tablet, a laptop, a speaker of a desktop computer, a smart speaker, or any other electronic device capable of playing sound. In some embodiments, the external speaker <NUM> is driven by the headset <NUM> via a wireless connection. In other embodiments, the external speaker <NUM> is driven by a console. In one aspect, the external speaker <NUM> is fixed at one position and transmits test sounds that the microphones <NUM> can receive for calibrating HRTFs. For example, the external speaker <NUM> may play test sounds that are the same as those played by the speaker array <NUM>, <NUM> of the SMS <NUM>, <NUM>. In another aspect, the external speaker <NUM> provides test sounds of frequencies that the user <NUM> can optimally hear based on audio characterization configuration, in accordance with the image presented on the headset <NUM>.

The virtual space <NUM> is generated by the artificial reality system <NUM> to direct the orientation of the head of the user <NUM> while measuring the individualized HRTFs. The user <NUM> views the virtual space <NUM> through a display of the headset <NUM>. The term "virtual space" <NUM> is not intended to be limiting. In some various embodiments the virtual reality space <NUM> may include virtual reality, augmented reality, mixed reality, or some other form of artificial reality.

In the embodiment illustrated, the virtual reality space <NUM> includes an indicator <NUM>. The indicator <NUM> is presented on the display of the headset <NUM> to direct the orientation of the head of the user <NUM>. The indicator <NUM> can be light, or a marking presented on the display of the headset <NUM>. The position of the headset <NUM> can be tracked through an imaging device and/or an IMU (show in in <FIG> and <FIG>) to confirm whether the indicator <NUM> is aligned with the desired head orientation.

In one example, the user <NUM> is prompted to view the indicator <NUM>. After confirming that the indicator <NUM> is aligned with the head orientation, for example based on the location of the indicator <NUM> displayed on the HMD <NUM> with respect to a crosshair, the external speaker <NUM> generates a test sound. For each ear a corresponding microphone 450a, 450b captures the received test sound as audio data.

After the microphones <NUM> successfully capture the audio data, the user <NUM> is prompted to direct their orientation towards a new indicator <NUM> at a different location in the virtual space <NUM>. The process of capturing the audio data at indicator <NUM> is repeated to capture audio data at indicator <NUM>. Indicators <NUM>, <NUM> are generated at different locations in the virtual space <NUM> to capture audio data to be used to determine HRTFs at different head orientations of the user <NUM>. Each indicator <NUM>, <NUM> at a different location in the virtual space <NUM> enables the measurement of an HRTF at a different direction (elevation and azimuth). New indicators are generated and the process of capturing audio data is repeated to sufficiently span elevations and azimuths within the virtual space <NUM>. The use of an external speaker <NUM> and a display of indicators <NUM>, <NUM> within the virtual space <NUM> displayed via a headset <NUM> enables relatively convenient measurement the measurement of individualized HRTFs for a user <NUM>. That is, the user <NUM> can perform these steps at their convenience in their own home with an artificial reality system <NUM>, without the need for an anechoic chamber.

<FIG> is a diagram of a display <NUM> in which an alignment prompt <NUM> and an indicator <NUM> are displayed by a headset and a user's head is not at a correct orientation, in accordance with one or more embodiments. As shown in <FIG>, a display <NUM> presents an alignment prompt <NUM> on a center of the display <NUM> or at one or more predetermined pixels of the display <NUM>. In this embodiment, the alignment prompt <NUM> is a crosshair. But more generally, the alignment prompt <NUM> is any text and/or graphical interface that shows the user whether the user's head is at the correct orientation relative to a displayed indicator <NUM>. In one aspect, the alignment prompt <NUM> reflects a current head orientation and the indicator <NUM> reflects a target head orientation. The correct orientation occurs when the indicator <NUM> is at the center of the alignment prompt <NUM>. In the example depicted in <FIG>, the indicator <NUM> is positioned on a top left corner of the display <NUM>, rather than on the alignment prompt <NUM>. Accordingly, the head orientation is not at the correct orientation. Moreover, because the indicator <NUM> and the alignment prompt <NUM> are not aligned it is apparent to the user that his/her head is not at the proper orientation.

<FIG> is a diagram of the display of <FIG> in which the user's head is at a correct orientation, in accordance with one or more embodiments. The display <NUM> on <FIG> is substantially similar to the display <NUM> of <FIG>, except the indicator <NUM> is now displayed on the crosshair <NUM>. Hence, it is determined the head orientation is properly aligned with the indicator <NUM> and the user's HRTF is measured for the head orientation. That is, a test sound is played by the external speaker <NUM> and captured as audio data at the microphones <NUM>. Based on the audio data, an HRTF is determined for each ear at the current orientation. The process described in relation to <FIG> is repeated for a plurality of different orientations of the head of the user <NUM> with respect to the external speaker <NUM>. A set of HRTFs for the user <NUM> comprises an HRTF at each measured head orientation.

<FIG> is a block diagram of a system environment <NUM> of a system for determining individualized HRTFs for a user, in accordance with one or more embodiments. The system environment <NUM> comprises an external speaker <NUM>, the HRTF system <NUM>, a network <NUM>, and a headset <NUM>. The external speaker <NUM>, the HRTF system <NUM>, and the headset <NUM> are all connected via the network <NUM>.

The external speaker <NUM> is a device configured to transmit sound to the user. In one embodiment, the external speaker <NUM> is operated according to commands from the headset <NUM>. In other embodiments, the external speaker <NUM> is operated by an external console. The external speaker <NUM> is fixed at one position and transmits test sounds. Test sounds transmitted by the external speaker <NUM> include, for example, a continuous sinusoidal wave at a constant frequency, or a chirp. In some embodiments, the external speaker <NUM> is the external speaker <NUM> of <FIG>.

The network <NUM> couples the headset <NUM> and/or the external speaker <NUM> to the HRTF system <NUM>. The network <NUM> may couple additional components to the HRTF system <NUM>. The network <NUM> may include any combination of local area and/or wide area networks using both wireless and/or wired communication systems. For example, the network <NUM> may include the Internet, as well as mobile telephone networks. In one embodiment, the network <NUM> uses standard communications technologies and/or protocols. Hence, the network <NUM> may include links using technologies such as Ethernet, <NUM>, worldwide interoperability for microwave access (WiMAX), <NUM>/<NUM>/<NUM> mobile communications protocols, digital subscriber line (DSL), asynchronous transfer mode (ATM), InfiniBand, PCI Express Advanced Switching, etc. Similarly, the networking protocols used on the network <NUM> can include multiprotocol label switching (MPLS), the transmission control protocol/Internet protocol (TCP/IP), the User Datagram Protocol (UDP), the hypertext transport protocol (HTTP), the simple mail transfer protocol (SMTP), the file transfer protocol (FTP), etc. The data exchanged over the network <NUM> can be represented using technologies and/or formats including image data in binary form (e.g. Portable Network Graphics (PNG)), hypertext markup language (HTML), extensible markup language (XML), etc. In addition, all or some of links can be encrypted using conventional encryption technologies such as secure sockets layer (SSL), transport layer security (TLS), virtual private networks (VPNs), Internet Protocol security (IPsec), etc..

The headset <NUM> presents media to a user. Examples of media presented by the headset <NUM> include one or more images, video, audio, or any combination thereof. The headset <NUM> comprises a display assembly <NUM>, and an audio system <NUM>. In some embodiments, the headset <NUM> is the headset <NUM> of <FIG>. Specific examples of embodiments of the headset <NUM> are described with regard to <FIG> and <FIG>.

The display assembly <NUM> displays visual content to the user wearing the headset <NUM>. In particular, the display assembly <NUM> displays 2D or 3D images or video to the user. The display assembly <NUM> displays the content using one or more display elements. A display element may be, e.g., an electronic display. In various embodiments, the display assembly <NUM> comprises a single display element or multiple display elements (e.g., a display for each eye of a user). Examples of display elements include: a liquid crystal display (LCD), a light emitting diode (LED), display, a micro-light-emitting diode (µLED) display, an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode display (AMOLED), a waveguide display, some other display, or some combination thereof. In some embodiments, the display assembly <NUM> is at least partially transparent. In some embodiments, the display assembly <NUM> is the display <NUM> of <FIG>.

The audio system <NUM> determines a set of individualized HRTFs for the user wearing the headset <NUM>. In one embodiment, the audio system <NUM> comprises hardware, including one or more microphones <NUM> and a speaker array <NUM>, as well as an audio controller <NUM>. Some embodiments of the audio system <NUM> have different components than those described in conjunction with <FIG>. Similarly, the functions further described below may be distributed among components of the audio system <NUM> in a different manner than is described here. In some embodiments, some of the functions described below may be performed by other entities (e.g., the HRTF system <NUM>).

The microphone assembly <NUM> captures audio data of the test sounds emitted by the external speaker <NUM>. In some embodiment, the microphone assembly <NUM> is one or more microphones <NUM> located at or near the ear canal of the user. In other embodiments, the microphone assembly <NUM> is external from the headset <NUM> and is controlled by the headset <NUM> via the network <NUM>. The microphone assembly <NUM> may be the pair of microphones <NUM> of <FIG>.

The speaker array <NUM> play audio for the user in accordance with instructions from the audio controller <NUM>. The audio played for the user by the speaker array <NUM> may include instructions to facilitating the capture of the test sound audio by the one or more microphones <NUM>. The speaker array <NUM> is distinct from the external speaker <NUM>.

The audio controller <NUM> controls components of the audio system <NUM>. In some embodiments, the audio controller <NUM> may also control the external speaker <NUM>. The audio controller <NUM> includes a plurality of modules including a measurement module <NUM>, a HRTF module <NUM>, a distortion module <NUM>, and an interpolation module <NUM>. Note that in alternate embodiments, some or all of the modules of the audio controller <NUM> may be performed (wholly or in-part) by other entities (e.g., the HRTF system <NUM>). The audio controller <NUM> is coupled to other components of the audio system <NUM>. In some embodiments, the audio controller <NUM> is also coupled to the external speaker <NUM> or other components of the system environment <NUM> via communication coupling (e.g., wired or wireless communication coupling). The audio controller <NUM> may perform initial processing of data obtained from the microphone assembly <NUM> or other received data. The audio controller <NUM> communicates received data to other components in the headset <NUM> and the system environment <NUM>.

The measurement module <NUM> configures the capture of audio data of test sounds played by the external speaker <NUM>. The measurement module <NUM> provides instructions to the user to orient their head in a particular direction via the headset <NUM>. The measurement module <NUM> sends signals via the network <NUM> to the external speaker <NUM> to play one or more test sounds. The measurement module <NUM> instructs the one or more microphones <NUM> to capture audio data of the test sounds. The measurement module <NUM> repeats this process for a predetermined span of head orientations. In some embodiments, the measurement module <NUM> uses the process described in relation to <FIG>.

In one embodiment, the measurement module <NUM> sends instructions to the user to orient their head in a particular direction using the speaker array <NUM>. The speaker array <NUM> may play audio with verbal instructions or other audio to indicate a particular head orientation. In other embodiments, the measurement module <NUM> uses the display assembly <NUM> to provide the user with visual cues to orient his/her head. The measurement module <NUM> may generate a virtual space with an indicator, such as the virtual space <NUM> and the indicator <NUM> of <FIG>. The visual cue provided via the display assembly <NUM> to the user may be similar to the prompt <NUM> on the display <NUM> of FIGS.

When the measurement module <NUM> has confirmed the user has the desired head orientation, the measurement module <NUM> instructs the external speaker <NUM> to play a test sound. The measurement module <NUM> specifies the characteristics of the test sound, such as frequency, length, type (e.g., sinusoidal, chirp, etc.). To capture the test sound, the measurement module <NUM> instructs the one or more microphones <NUM> to record audio data. Each microphone captures audio data, (e.g., HRIR) of the test sound at its respective location.

The measurement module <NUM> iterates through the above described steps for a predetermined set of head orientations that span a plurality of azimuths and elevations. In one embodiment, the predetermined set of orientations spans the <NUM> directions described in relation to <FIG>. In another embodiment, the predetermined set of orientations spans a subset of the set of directions measured by the sound measurement system <NUM>. The process performed by the measurement module <NUM> enables convenient and relatively easy measurement of audio data for the determination of an individualized set of HRTFs.

The HRTF module <NUM> calculates an initial set of HRTFs for the audio data captured by the measurement module <NUM> for the user wearing the headset <NUM>. The initial set of HRTFs determined by the HRTF module <NUM> includes one or more HRTFs that are distorted by the presence of the headset <NUM>. That is, the HRTFs of one or more particular directions (e.g., ranges of elevations and azimuths) are distorted by the presence of the headset, such that sound played with the HRTFs gives an impression that the user is wearing the headset (versus giving the impression to the user that they are not wearing the headset - e.g., as part of a VR experience). In an embodiment where the measurement module <NUM> captures audio data in the form of HRIRs, the HRTF module <NUM> determines the initial set of HRTFs by taking the Fourier transform of each corresponding HRIR. In some embodiments, each HRTF in the initial set of HRTFs is directionally-dependent, H(Ω), where Ω is direction. The direction further comprises an elevation angle, θ, and an azimuth angle, ϕ, represented as Ω = (θ,ϕ). That is, an HRTF is calculated corresponding to each measured direction (elevation and azimuth). In other embodiments, each HRTF is frequency- and directionally-dependent, H(Ω, f), where f is a frequency.

In some embodiments, the HRTF module <NUM> utilizes data of sets of individualized HRTFs or a generalized set of HRTFs to calculate the initial set of HRTFs. The data may be preloaded on the headset <NUM> in some embodiments. In other embodiments, the data may be accessed by the headset <NUM> via the network <NUM> from the HRTF system <NUM>. In some embodiments, the HRTF module <NUM> may use processes and computations substantially similar to the SMS <NUM> of <FIG>.

The distortion module <NUM> modifies the initial set of HRTFs calculated by the HRTF module <NUM> to remove portions distorted by the presence of the headset <NUM>, creating an intermediate set of HRTFs. The distortion module <NUM> generates a query for a distortion mapping. As discussed above with regard to <FIG>, the distortion mapping includes a set of one or more distortion regions. The query may include one or more parameters corresponding to physical features of the user, such as gender, age, height, ear geometry, head geometry, etc. In some embodiments, the distortion module <NUM> sends the query to a local storage of the headset. In other embodiments, the query is sent to the HRTF system <NUM> via the network. The distortion module <NUM> receives some or all of a distortion mapping that identifies a set of one or more distortion regions. In some embodiments, the distortion mapping may be specific to a population of test users having one or more physical characteristic in common with some or all of the parameters in the query. The set of one or more distortion regions include directions (e.g., azimuth and elevation relative to the headset) of HRTFs that are commonly distorted by the headset.

In some embodiments, the distortion module <NUM> discards portions of the initial set of HRTFs corresponding to the set of one or more distortion regions, resulting in an intermediate set of HRTFs. In some embodiments, the distortion module <NUM> discards the portions of the directionally-dependent HRTFs corresponding to the particular directions (i.e., azimuths and elevations) of the set of one or more distortion regions. In other embodiments, the distortion module <NUM> discards the portions of the frequency- and directionally-dependent HRTFs corresponding to the particular directions and frequencies of the set of distortion regions.

For example, the set of one or more distortion regions comprise the region of azimuth [-<NUM>°, -<NUM>°] and elevation [-<NUM>°, <NUM>°] and region of azimuth [-<NUM>°, -<NUM>°] and elevation [-<NUM>°, <NUM>°]. The HRTFs in the initial set of HRTFs corresponding to directions comprised in these regions are removed from the set of HRTFs, creating an intermediate set of HRTFs. For example, the HRTF H(Ω = (<NUM>°, -<NUM>°)) falls within one of the distortion regions and is removed from the set of HRTFs by the distortion module <NUM>. The HRTF H(Ω = (<NUM>°, <NUM>°)) falls outside the directions comprised in the set of distortion regions and is included in the intermediate set of HRTFs. A similar process is followed when the distortion regions further comprise particular frequencies.

The interpolation module <NUM> may use the intermediate set of HRTFs to generate an individualized set of HRTFs that compensates for the presence of the headset <NUM>. The interpolation module <NUM> interpolates some or all of the intermediate set to generate a set of interpolated HRTFs. For example, the interpolation module <NUM> may select HRTFs that are within some angular range from the discarded portions, and use interpolation and the selected HRTFs to generate a set of interpolated HRTFs. The set of interpolated HRTFs combined with the intermediate set of HRTFs produce a complete set of individualized HRTFs that mitigate headset distortion.

In some embodiments, the generated individualized set of HRTFs that compensate for the distortion caused by a headset is stored. In some embodiments, the generated individualized set of HRTFs is maintained on the local storage of the headset and can be used in the future by the user. In other embodiments, the generated individualized set of HRTFs is uploaded to the HRTF system <NUM>.

Producing a set of individualized HRTFs that compensate for distortion caused by a headset improves a virtual reality experience of a user. For example, a user is wearing the headset <NUM> and experiencing a video-based virtual reality environment. The video-based virtual reality environment is intended to make the user forget that the reality is virtual, both in terms of video and audio quality. The headset <NUM> does this by removing ques (visual and auditory) to the user that they are wearing the headset <NUM>. The headset <NUM> provides an easy and convenient way to measure HRTFs of the user. However, HRTFs measured with the headset <NUM> being worn by the user have inherent distortion caused by the presence of the headset <NUM>. Playing audio using the distorted HRTFs would maintain an auditory que to the user that the headset is being worn - and would not align with a VR experience that makes it as-if no headset is worn by the user. And as exampled above the audio system <NUM> generates an individualized set of HRTFs using the measured HRTFs and a distortion mapping. The audio system <NUM> can then present audio content to the user using the individualized HRTFs in a manner such that the audio experience is as-if the user is not wearing a headset and, thereby, would align with a VR experience that makes it as-if no headset is worn by the user.

<FIG> is a flowchart illustrating a process <NUM> of obtaining a set of individualized HRTFs for a user, in accordance with one or more embodiments. In one embodiment, the process <NUM> is performed by the headset <NUM>. Other entities may perform some or all of the steps of the process <NUM> in other embodiments (e.g., the external speaker <NUM>, or the HTRF server <NUM>). Likewise, embodiments may include different and/or additional steps, or perform the steps in different orders.

The headset <NUM> captures <NUM> audio data of test sounds at different orientations. The headset <NUM> prompts the user to orient his/her head in a particular direction while wearing the headset <NUM>. The headset <NUM> instructs a speaker (e.g., the external speaker <NUM>) to play a test sound and audio data of the test sound is captured <NUM> by one or more microphones (e.g., microphones <NUM>) at or near the user's ear canal. The capturing <NUM> is repeated for a plurality of different head orientations of the user. <FIG> illustrate one embodiment of the capture <NUM> of audio data. The measurement module <NUM> of <FIG> performs the capturing <NUM>, according to some embodiments.

The headset <NUM> determines <NUM> a set of HRTFs based on the audio data at the different orientations. In some embodiments, an HRTF module (e.g., the HRTF module <NUM>) calculates the set of HRTFs using the audio data. The headset <NUM> may use conventional methods for calculating an HRTF using audio data originating from a specific location relative to the headset. In other embodiments, the headset may provide the audio data to an external device (e.g., a console and/or a HRTF system) to calculate the set of HRTFs.

The headset <NUM> discards <NUM> portions of the HRTFs corresponding to a set of distortion regions to create an intermediate set of HRTFs. The headset <NUM> generates a query for a set of distortion regions. In some embodiments, the headset <NUM> sends the query to a local storage of the headset <NUM> (e.g., the distortion regions are pre-loaded). In other embodiments, the headset <NUM> sends the query to the HRTF system <NUM> via the network <NUM>, in which case the distortion regions are determined by an external system (e.g. the HRTF system <NUM>). The set of distortion regions may be determined based on HRTFs of a population of test users or based on a manikin. Responsive to the query, the headset <NUM> receives a set of distortion regions and discards the portions of the set of HRTFs corresponding to one or more directions comprised within the set of distortion regions. According to some embodiments, the distortion module <NUM> of <FIG> performs the discarding <NUM>.

The headset <NUM> generates <NUM> an individualized set of HRTFs using at least some of the intermediate set of HRTFs. The missing portions are interpolated based on the intermediate set of HRTFs and, in some embodiments, a distortion mapping of HRTFs associated with the distortion regions. In some embodiments, the interpolation module <NUM> of <FIG> performs the generating <NUM>. In other embodiments, the headset <NUM> and generates <NUM> the individualized set of HRTFs.

In some embodiments, the HRTF system <NUM> performs at least some of the steps of the process. That is, the HRTF system <NUM> provides instructions to the headset <NUM> and external speakers <NUM> to capture <NUM> audio data of test sounds at different orientations. The HRTF system <NUM> sends a query to the headset <NUM> for audio data and receives the audio data. The HRTF system <NUM> calculates <NUM> a set of HRTFs based on the audio data at the different orientations and discards <NUM> portions of the HRTFs corresponding to distortion regions to create an intermediate set of HRTFs. The HRTF system <NUM> generates <NUM> an individualized set of HRTFs using at least some of the intermediate set of HRTFs and provides the individualized set of HRTFs to the headset <NUM> for use.

<FIG> is a perspective view of a headset <NUM> implemented as an eyewear device, in accordance with one or more embodiments. In some embodiments, the eyewear device is a near eye display (NED). In general, the headset <NUM> may be worn on the face of a user such that content (e.g., media content) is presented using a display assembly, such as the display assembly <NUM> of <FIG>, and/or an audio system, such as the audio system <NUM> of <FIG>. However, the headset <NUM> may also be used such that media content is presented to a user in a different manner. Examples of media content presented by the headset <NUM> include one or more images, video, audio, or some combination thereof. The headset <NUM> includes a frame, and may include, among other components, a display assembly including one or more display elements <NUM>, a depth camera assembly (DCA), an audio system, and a position sensor <NUM>. While <FIG> illustrates the components of the headset <NUM> in example locations on the headset <NUM>, the components may be located elsewhere on the headset <NUM>, on a peripheral device paired with the headset <NUM>, or some combination thereof. Similarly, there may be more or fewer components on the headset <NUM> than what is shown in <FIG>.

The frame <NUM> holds the other components of the headset <NUM>. The frame <NUM> includes a front part that holds the one or more display elements <NUM> and end pieces (e.g., temples) to attach to a head of the user. The front part of the frame <NUM> bridges the top of a nose of the user. The length of the end pieces may be adjustable (e.g., adjustable temple length) to fit different users. The end pieces may also include a portion that curls behind the ear of the user (e.g., temple tip, ear piece).

The one or more display elements <NUM> provide light to a user wearing the headset <NUM>. The one or more display elements may be part of the display assembly <NUM> of <FIG>. As illustrated the headset includes a display element <NUM> for each eye of a user. In some embodiments, a display element <NUM> generates image light that is provided to an eyebox of the headset <NUM>. The eyebox is a location in space that an eye of user occupies while wearing the headset <NUM>. For example, a display element <NUM> may be a waveguide display. A waveguide display includes a light source (e.g., a two-dimensional source, one or more line sources, one or more point sources, etc.) and one or more waveguides. Light from the light source is in-coupled into the one or more waveguides which outputs the light in a manner such that there is pupil replication in an eyebox of the headset <NUM>. In-coupling and/or outcoupling of light from the one or more waveguides may be done using one or more diffraction gratings. In some embodiments, the waveguide display includes a scanning element (e.g., waveguide, mirror, etc.) that scans light from the light source as it is in-coupled into the one or more waveguides. Note that in some embodiments, one or both of the display elements <NUM> are opaque and do not transmit light from a local area around the headset <NUM>. The local area is the area surrounding the headset <NUM>. For example, the local area may be a room that a user wearing the headset <NUM> is inside, or the user wearing the headset <NUM> may be outside and the local area is an outside area. In this context, the headset <NUM> generates VR content. Alternatively, in some embodiments, one or both of the display elements <NUM> are at least partially transparent, such that light from the local area may be combined with light from the one or more display elements to produce AR and/or MR content.

In some embodiments, a display element <NUM> does not generate image light, and instead is a lens that transmits light from the local area to the eyebox. For example, one or both of the display elements <NUM> may be a lens without correction (non-prescription) or a prescription lens (e.g., single vision, bifocal and trifocal, or progressive) to help correct for defects in a user's eyesight. In some embodiments, the display element <NUM> may be polarized and/or tinted to protect the user's eyes from the sun.

Note that in some embodiments, the display element <NUM> may include an additional optics block (not shown). The optics block may include one or more optical elements (e.g., lens, Fresnel lens, etc.) that direct light from the display element <NUM> to the eyebox. The optics block may, e.g., correct for aberrations in some or all of the image content, magnify some or all of the image, or some combination thereof.

The DCA determines depth information for a portion of a local area surrounding the headset <NUM>. The DCA includes one or more imaging devices <NUM> and a DCA controller (not shown in <FIG>), and may also include an illuminator <NUM>. In some embodiments, the illuminator <NUM> illuminates a portion of the local area with light. The light may be, e.g., structured light (e.g., dot pattern, bars, etc.) in the infrared (IR), IR flash for time-of-flight, etc. In some embodiments, the one or more imaging devices <NUM> capture images of the portion of the local area that include the light from the illuminator <NUM>. As illustrated, <FIG> shows a single illuminator <NUM> and two imaging devices <NUM>. In alternate embodiments, there is no illuminator <NUM> and at least two imaging devices <NUM>.

The DCA controller computes depth information for the portion of the local area using the captured images and one or more depth determination techniques. The depth determination technique may be, e.g., direct time-of-flight (ToF) depth sensing, indirect ToF depth sensing, structured light, passive stereo analysis, active stereo analysis (uses texture added to the scene by light from the illuminator <NUM>), some other technique to determine depth of a scene, or some combination thereof.

The audio system provides audio content. The audio system may be an embodiment of the audio system <NUM> of <FIG>. In one embodiment, the audio system includes a transducer array, a sensor array, and an audio controller <NUM>. However, in other embodiments, the audio system may include different and/or additional components. Similarly, in some cases, functionality described with reference to the components of the audio system can be distributed among the components in a different manner than is described here. For example, some or all of the functions of the controller may be performed by a remote server, such as the HRTF system <NUM>.

The transducer array presents sound to user. The transducer array includes a plurality of transducers. A transducer may be a speaker <NUM> or a tissue transducer <NUM> (e.g., a bone conduction transducer or a cartilage conduction transducer). Although the speakers <NUM> are shown exterior to the frame <NUM>, the speakers <NUM> may be enclosed in the frame <NUM>. In some embodiments, instead of individual speakers for each ear, the headset <NUM> includes a speaker array, such as the speaker array <NUM> of <FIG>, comprising multiple speakers integrated into the frame <NUM> to improve directionality of presented audio content. The tissue transducer <NUM> couples to the head of the user and directly vibrates tissue (e.g., bone or cartilage) of the user to generate sound. The number and/or locations of transducers may be different from what is shown in <FIG>.

The sensor array detects sounds within the local area of the headset <NUM>. The sensor array includes a plurality of acoustic sensors <NUM>. An acoustic sensor <NUM> captures sounds emitted from one or more sound sources in the local area (e.g., a room). Each acoustic sensor is configured to detect sound and convert the detected sound into an electronic format (analog or digital). The acoustic sensors <NUM> may be acoustic wave sensors, microphones, sound transducers, or similar sensors that are suitable for detecting sounds.

In some embodiments, one or more acoustic sensors <NUM> may be placed in an ear canal of each ear (e.g., acting as binaural microphones, or the microphone assembly <NUM> of <FIG>). In some embodiments, the acoustic sensors <NUM> may be placed on an exterior surface of the headset <NUM>, placed on an interior surface of the headset <NUM>, separate from the headset <NUM> (e.g., part of some other device), or some combination thereof. The number and/or locations of acoustic sensors <NUM> may be different from what is shown in <FIG>. For example, the number of acoustic detection locations may be increased to increase the amount of audio information collected and the sensitivity and/or accuracy of the information. The acoustic detection locations may be oriented such that the microphone is able to detect sounds in a wide range of directions surrounding the user wearing the headset <NUM>.

The audio controller <NUM> processes information from the sensor array that describes sounds detected by the sensor array. The audio controller <NUM> may comprise a processor and a computer-readable storage medium. The audio controller <NUM> may be configured to generate direction of arrival (DOA) estimates, generate acoustic transfer functions (e.g., array transfer functions and/or head-related transfer functions), track the location of sound sources, form beams in the direction of sound sources, classify sound sources, generate sound filters for the speakers <NUM>, or some combination thereof. The audio controller <NUM> is an embodiment of the audio controller <NUM> of <FIG>.

The position sensor <NUM> generates one or more measurement signals in response to motion of the headset <NUM>. The position sensor <NUM> may be located on a portion of the frame <NUM> of the headset <NUM>. The position sensor <NUM> may include an inertial measurement unit (IMU). Examples of position sensor <NUM> include: one or more accelerometers, one or more gyroscopes, one or more magnetometers, another suitable type of sensor that detects motion, a type of sensor used for error correction of the IMU, or some combination thereof. The position sensor <NUM> may be located external to the IMU, internal to the IMU, or some combination thereof.

In some embodiments, the headset <NUM> may provide for simultaneous localization and mapping (SLAM) for a position of the headset <NUM> and updating of a model of the local area. For example, the headset <NUM> may include a passive camera assembly (PCA) that generates color image data. The PCA may include one or more RGB cameras that capture images of some or all of the local area. In some embodiments, some or all of the imaging devices <NUM> of the DCA may also function as the PCA. The images captured by the PCA and the depth information determined by the DCA may be used to determine parameters of the local area, generate a model of the local area, update a model of the local area, or some combination thereof. Furthermore, the position sensor <NUM> tracks the position (e.g., location and pose) of the headset <NUM> within the room. Additional details regarding the components of the headset <NUM> are discussed below in connection with <FIG>.

<FIG> is a perspective view of a headset <NUM> implemented as a HMD, in accordance with one or more embodiments. In embodiments that describe an AR system and/or a MR system, portions of a front side of the HMD are at least partially transparent in the visible band (~<NUM> to <NUM>), and portions of the HMD that are between the front side of the HMD and an eye of the user are at least partially transparent (e.g., a partially transparent electronic display). The HMD includes a front rigid body <NUM> and a band <NUM>. The headset <NUM> includes many of the same components described above with reference to <FIG>, but modified to integrate with the HMD form factor. For example, the HMD includes a display assembly, a DCA, an audio system (e.g., an embodiment of the audio system <NUM>), and a position sensor <NUM>. <FIG> shows the illuminator <NUM>, a plurality of the speakers <NUM>, a plurality of the imaging devices <NUM>, a plurality of acoustic sensors <NUM>, and the position sensor <NUM>.

<FIG> is a is a system <NUM> that includes a headset <NUM>, in accordance with one or more embodiments. In some embodiments, the headset <NUM> may be the headset <NUM> of <FIG> or the headset <NUM> of <FIG>. The system <NUM> may operate in an artificial reality environment (e.g., a virtual reality environment, an augmented reality environment, a mixed reality environment, or some combination thereof). The system <NUM> shown by <FIG> includes the headset <NUM>, an input/output (I/O) interface <NUM> that is coupled to a console <NUM>, the network <NUM>, and the HRTF system <NUM>. While <FIG> shows an example system <NUM> including one headset <NUM> and one I/O interface <NUM>, in other embodiments any number of these components may be included in the system <NUM>. For example, there may be multiple headsets each having an associated I/O interface <NUM>, with each headset and I/O interface <NUM> communicating with the console <NUM>. In alternative configurations, different and/or additional components may be included in the system <NUM>. Additionally, functionality described in conjunction with one or more of the components shown in <FIG> may be distributed among the components in a different manner than described in conjunction with <FIG> in some embodiments. For example, some or all of the functionality of the console <NUM> may be provided by the headset <NUM>.

The headset <NUM> includes the display assembly <NUM>, the audio system <NUM>, an optics block <NUM>, one or more position sensors <NUM>, and a Depth Camera Assembly (DCA) <NUM>. Some embodiments of headset <NUM> have different components than those described in conjunction with <FIG>. Additionally, the functionality provided by various components described in conjunction with <FIG> may be differently distributed among the components of the headset <NUM> in other embodiments, or be captured in separate assemblies remote from the headset <NUM>.

In one embodiment, the display assembly <NUM> displays content to the user in accordance with data received from the console <NUM>. The display assembly <NUM> displays the content using one or more display elements (e.g., the display elements <NUM>). A display element may be, e.g., an electronic display. In various embodiments, the display assembly <NUM> comprises a single display element or multiple display elements (e.g., a display for each eye of a user). Examples of an electronic display include: a liquid crystal display (LCD), an organic light emitting diode (OLED) display, an active-matrix organic light-emitting diode display (AMOLED), a waveguide display, some other display, or some combination thereof. Note in some embodiments, the display element <NUM> may also include some or all of the functionality of the optics block <NUM>.

The optics block <NUM> may magnify image light received from the electronic display, corrects optical errors associated with the image light, and presents the corrected image light to one or both eyeboxes of the headset <NUM>. In various embodiments, the optics block <NUM> includes one or more optical elements. Example optical elements included in the optics block <NUM> include: an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, a reflecting surface, or any other suitable optical element that affects image light. Moreover, the optics block <NUM> may include combinations of different optical elements. In some embodiments, one or more of the optical elements in the optics block <NUM> may have one or more coatings, such as partially reflective or anti-reflective coatings.

Magnification and focusing of the image light by the optics block <NUM> allows the electronic display to be physically smaller, weigh less, and consume less power than larger displays. Additionally, magnification may increase the field of view of the content presented by the electronic display. For example, the field of view of the displayed content is such that the displayed content is presented using almost all (e.g., approximately <NUM> degrees diagonal), and in some cases all, of the user's field of view. Additionally, in some embodiments, the amount of magnification may be adjusted by adding or removing optical elements.

In some embodiments, the optics block <NUM> may be designed to correct one or more types of optical error. Examples of optical error include barrel or pincushion distortion, longitudinal chromatic aberrations, or transverse chromatic aberrations. Other types of optical errors may further include spherical aberrations, chromatic aberrations, or errors due to the lens field curvature, astigmatisms, or any other type of optical error. In some embodiments, content provided to the electronic display for display is pre-distorted, and the optics block <NUM> corrects the distortion when it receives image light from the electronic display generated based on the content.

The position sensor <NUM> is an electronic device that generates data indicating a position of the headset <NUM>. The position sensor <NUM> generates one or more measurement signals in response to motion of the headset <NUM>. The position sensor <NUM> is an embodiment of the position sensor <NUM>. Examples of a position sensor <NUM> include: one or more IMUs, one or more accelerometers, one or more gyroscopes, one or more magnetometers, another suitable type of sensor that detects motion, or some combination thereof. The position sensor <NUM> may include multiple accelerometers to measure translational motion (forward/back, up/down, left/right) and multiple gyroscopes to measure rotational motion (e.g., pitch, yaw, roll). In some embodiments, an IMU rapidly samples the measurement signals and calculates the estimated position of the headset <NUM> from the sampled data. For example, the IMU integrates the measurement signals received from the accelerometers over time to estimate a velocity vector and integrates the velocity vector over time to determine an estimated position of a reference point on the headset <NUM>. The reference point is a point that may be used to describe the position of the headset <NUM>. While the reference point may generally be defined as a point in space, however, in practice the reference point is defined as a point within the headset <NUM>.

The DCA <NUM> generates depth information for a portion of the local area. The DCA includes one or more imaging devices and a DCA controller. The DCA <NUM> may also include an illuminator. Operation and structure of the DCA <NUM> is described above with regard to <FIG>.

The audio system <NUM> provides audio content to a user of the headset <NUM>. The audio system <NUM> may comprise one or acoustic sensors, one or more transducers, and the audio controller <NUM>. The audio system <NUM> may provide spatialized audio content to the user. In some embodiments, the audio system <NUM> may request a distortion mapping from the HRTF system <NUM> over the network <NUM>. As described above with regard to <FIG> and <FIG>, the audio system instructs the external speaker <NUM> to emit test sounds, and captures audio data of the test sounds using a microphone assembly. The audio system <NUM> calculates a set of initial HRTFs based at least in part on the audio data of the test sound at different orientations of the headset <NUM>. The audio system <NUM> discards a portion (based in part on at least some of the distortion regions determined by the HRTF server) of the set of initial HRTFs to create an intermediate set of HRTFs. The intermediate set of HRTFs is formed from the non-discarded HRTFs of the set of HRTFs. The audio system <NUM> generates one or more HRTFs (e.g., via interpolation) that correspond to the discarded portion of the set, which are combined with at least some of the intermediate set of HRTFs to create a set of individualized HRTFs for the user. The set of individualized HRTFs are customized to the user such that errors in the HRTFs caused by wearing the headset <NUM> are mitigated, and thereby mimic actual HRTFs of the user without a headset. The audio system <NUM> may generate one or more sound filters using the individualized HRTFs, and use the sound filters to provide spatialized audio content to the user.

The I/O interface <NUM> is a device that allows a user to send action requests and receive responses from the console <NUM>. An action request is a request to perform a particular action. For example, an action request may be an instruction to start or end capture of image or video data, or an instruction to perform a particular action within an application. The I/O interface <NUM> may include one or more input devices. Example input devices include: a keyboard, a mouse, a game controller, or any other suitable device for receiving action requests and communicating the action requests to the console <NUM>. An action request received by the I/O interface <NUM> is communicated to the console <NUM>, which performs an action corresponding to the action request. In some embodiments, the I/O interface <NUM> includes an IMU that captures calibration data indicating an estimated position of the I/O interface <NUM> relative to an initial position of the I/O interface <NUM>. In some embodiments, the I/O interface <NUM> may provide haptic feedback to the user in accordance with instructions received from the console <NUM>. For example, haptic feedback is provided when an action request is received, or the console <NUM> communicates instructions to the I/O interface <NUM> causing the I/O interface <NUM> to generate haptic feedback when the console <NUM> performs an action.

The console <NUM> provides content to the headset <NUM> for processing in accordance with information received from one or more of: the DCA <NUM>, the headset <NUM>, and the I/O interface <NUM>. In the example shown in <FIG>, the console <NUM> includes the external speaker <NUM>, an application store <NUM>, a tracking module <NUM>, and an engine <NUM>. Some embodiments of the console <NUM> have different modules or components than those described in conjunction with <FIG>. In particular, the external speaker <NUM> is independent of the console <NUM> in some embodiments. Similarly, the functions further described below may be distributed among components of the console <NUM> in a different manner than described in conjunction with <FIG>. In some embodiments, the functionality discussed herein with respect to the console <NUM> may be implemented in the headset <NUM>, or a remote system.

The external speaker <NUM> plays test sounds in response to instructions from the audio system <NUM>. In other embodiments, the external speaker <NUM> receives the instructions from the console <NUM>, in particular from the engine <NUM> as described in greater detail below.

The application store <NUM> stores one or more applications for execution by the console <NUM>. An application is a group of instructions, that when executed by a processor, generates content for presentation to the user. Content generated by an application may be in response to inputs received from the user via movement of the headset <NUM> or the I/O interface <NUM>. Examples of applications include: gaming applications, conferencing applications, video playback applications, or other suitable applications.

The tracking module <NUM> tracks movements of the headset <NUM> or of the I/O interface <NUM> using information from the DCA <NUM>, the one or more position sensors <NUM>, or some combination thereof. For example, the tracking module <NUM> determines a position of a reference point of the headset <NUM> in a mapping of a local area based on information from the headset <NUM>. The tracking module <NUM> may also determine positions of an object or virtual object. Additionally, in some embodiments, the tracking module <NUM> may use portions of data indicating a position of the headset <NUM> from the position sensor <NUM> as well as representations of the local area from the DCA <NUM> to predict a future location of the headset <NUM>. The tracking module <NUM> provides the estimated or predicted future position of the headset <NUM> or the I/O interface <NUM> to the engine <NUM>.

The engine <NUM> executes applications and receives position information, acceleration information, velocity information, predicted future positions, or some combination thereof, of the headset <NUM> from the tracking module <NUM>. Based on the received information, the engine <NUM> determines content to provide to the headset <NUM> for presentation to the user. For example, if the received information indicates that the user has looked to the left, the engine <NUM> generates content for the headset <NUM> that mirrors the user's movement in a virtual local area or in a local area augmenting the local area with additional content. Additionally, in some embodiments, responsive to received information that indicates the user has positioned their head in a particular orientation, the engine <NUM> provides instructions to the external speaker <NUM> to play a test sound. Additionally, the engine <NUM> performs an action within an application executing on the console <NUM> in response to an action request received from the I/O interface <NUM> and provides feedback to the user that the action was performed. The provided feedback may be visual or audible feedback via the headset <NUM> or haptic feedback via the I/O interface <NUM>.

The network <NUM> couples the headset <NUM> and/or the console <NUM> to the HRTF system <NUM>. The network <NUM> may couple additional or fewer components to the HRTF system <NUM>. The network <NUM> is described in further detail in relation to <FIG>.

The foregoing description of the embodiments of the disclosure has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure.

Some portions of this description describe the embodiments of the disclosure in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.

Embodiments of the disclosure may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.

Embodiments of the disclosure may also relate to a product that is produced by a computing process described herein. Such a product may comprise information resulting from a computing process, where the information is stored on a non-transitory, tangible computer readable storage medium and may include any embodiment of a computer program product or other data combination described herein.

Claim 1:
A method comprising:
capturing audio data of test sounds through a microphone (140a, 140b) of a headset worn by a user (<NUM>), the said test sounds played by an external speaker (<NUM>), and the said audio data including audio data captured for different orientations of the said headset (<NUM>) with respect to the said external speaker (<NUM>);
calculating a set of head-related transfer function (HRTFs) based at least in part on the said audio data of the test sounds at the different orientations of the said headset (<NUM>), the said set of HRTFs individualized to a user (<NUM>) while wearing the said headset (<NUM>);
discarding a portion of the said set of HRTFs to create an intermediate set of HRTFs, the said discarded portion including at least some HRTFs corresponding to orientations of the said headset (<NUM>) where sound from the said external speaker (<NUM>) was incident on the said headset (<NUM>) prior to reaching an ear canal of the said user (<NUM>); and
characterised by generating one or more HRTFs that correspond to the said discarded portion using at least some of the said intermediate set of HRTFs to create an individualized set of HRTFs for the said user (<NUM>);
wherein the said discarded portion is determined using a distortion mapping that identifies one or more distortion regions where sound is distorted by the presence of a headset, and wherein the said distortion mapping is based in part on comparisons between a set of HRTFs measured with at least one test user wearing a test headset and a set of HRTFs measured with the said at least one test user not wearing the said test headset.