Patent Description:
This disclosure relates to ear-wearable devices.

A sound processing system is a system of one or more devices designed to stimulate a user's auditory system. For example, a sound processing system may comprise one or more auditory prostheses, hearing aids, hearables, virtual-sound systems, headsets, earbuds, and other types of devices.

Typically, a sound processing system has at least one microphone and at least one speaker. The microphone detects sounds from the user's environment. The speaker (which may also be referred to as a"receiver") generates sounds directed into the user's ear canal. Because of signal distortions or information loss in a sound processing system, or of acoustic characteristics of a sound scene (e.g., low signal-to-noise ratio), or of limitations in auditory physiological or perceptual abilities (e.g., hearing impairment), the user may have difficulty with detecting, discriminating, identifying, locating, following over time, or other aspects of the perception of, a particular sound that to which the user wants to listen. For example, it may be difficult for the user to understand what a person that the user wants to listen to is saying, in the presence of other sound sources in the environment. A location of a sound that a user wants to hear, or listen to, is referred to as the user's auditory attention locus. Usually, but not necessarily, an auditory attention locus corresponds to the location of a sound source. A sound source can be real or virtual. For example, some sound reproduction or sound synthesis systems can recreate or simulate for the user, a sensation of sound sources at different positions.

To overcome listening difficulties in environments containing multiple sound sources, makers of sound processing systems have attempted to implement directional processing modes in ear-wearable devices. For example, almost all hearing-aids today have directional modes, in which sounds coming from some directions (e.g., behind the user) are attenuated relative to sounds coming from other directions (e.g., from in-front of the user). In this context, it is advantageous to have a system for automatically determining a user's auditory attention locus or, at least, the direction in which a user's auditory attention is directed. Previous attempts to design such a system have involved, for example, a smartphone application whereby the user can manually indicate their auditory attention locus. However, this type of solution involving manual input from the user is demanding (the user must provide manual input whenever their locus of attention changes), intrusive (the user must temporarily disengage from what they are doing to provide such input) and not discreet (the user must make gestures in order to interact manually with the input device, including gestures that may be visible to others). Other attempts to solve this problem have used electroencephalography (EEG) or electrooculography (EOG) to determine the user's auditory attention locus based on brain or eye signals.

<CIT> relates to communication devices. Such devices may comprise input for receiving sound signal to be processed and presented to a user, and output for outputting the processed signal to a user perceivable as sound.

<CIT> describes the use of at least one electrical sensor which can either be implanted beneath the skin or placed on the surface of the skin as a part of a headset on either one side or if more than one sensor is used on both sides of a person's head in electrical communication with a vestigial periauricular nerve or muscle to enable a person to control the real or virtual action or movement of an output device in from one to three dimensions.

<CIT> relates to a hearing device, e.g. a hearing aid, comprising a sensor part adapted for being located at or in an ear or for fully or partially for being implanted in the head of a user.

<CIT> is a novelty only citation under Article <NUM>(<NUM>) EPC which recites a method for operating a hearing device for a user.

This disclosure describes techniques for inferring a user's auditory attention locus. Information concerning listening intentions of a user of a sound processing system (including, but not limited to, auditory prostheses, hearables, and virtual-sound systems), and in particular, information concerning the direction (angle) or location (angle and distance) of a listener's current auditory attentional focus, is difficult to obtain non-intrusively (i.e., without asking the user). The sound processing system may use such information in various applications, including but not limited to, applications involving a hearing device, such as a hearing-aid or hearable. For example, the sound processing system may use information regarding the direction or location of the user's current auditory attention locus to control a directional sound-processing algorithm on a hearing aid. Techniques of this disclosure may also be used in the context of a video game or a sensory-reeducation application (e.g., bio-feedback). As described herein, techniques of this disclosure may be used in situations in which it may be advantageous to infer non-intrusively (i.e., without the need for the user to alter their usual behavior) and/or covertly (i.e., non-observably to a third-party), using an ear-wearable device (e.g., hearing-aid, hearable, insertable earphones, etc.), the direction or locus where a user of the ear-wearable device is currently directing, or wanting to direct, his or her auditory attention.

In accordance with some techniques of this disclosure, a sound processing system (a) comprises at least one electrode placed in at least one external ear of a user, for the purpose of measuring at least one electromyographic (EMG) signal generated by, at least, one of the user's periauricular muscles (PAMs), (b) uses the measurements to estimate an angle and/or a distance of the user's current focus of spatial auditory attention (i.e., auditory attention locus), and (c) uses the latter estimate(s) to control a directional or spatial sound-processing system. The directional or spatial sound-processing system may be implemented in hardware or software designed to process sound-related signals in a manner that depends on an angle of and/or a distance to a real or virtual sound source, relative to a spatial reference point.

In one example, this disclosure describes a method comprising: measuring one or more periauricular muscle signals from one or more periauricular muscles of a user; computing, based on the periauricular muscle signals, an estimate of an angle corresponding to a direction of a current auditory attention locus of the user with respect to a reference point or plane; and controlling, based on the estimate of the angle, an operating characteristic of a sound processing system.

In another example, this disclosure describes a sound processing system comprising: one or more electrodes configured to measure one or more periauricular muscle signals form one or more periauricular muscles of a user; and one or more processors configured to: compute, based on the periauricular muscle signals, an estimate of an angle corresponding to a direction of a current auditory attention locus of the user with respect to a reference point or plane; and control, based on the estimate of the angle, an operating characteristic of a sound processing system.

In another example, this disclosure describes a sound processing system comprising: means for measuring one or more periauricular muscle signals from one or more periauricular muscles of a user; means for computing, based on the periauricular muscle signals, an estimate of an angle corresponding to a direction of a current auditory attention locus of the user with respect to a reference point or plane; and means for controlling, based on the estimate of the angle, an operating characteristic of a sound processing system.

Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description, drawings, and claims.

Techniques of this disclosure may have one or more advantages over existing approaches for estimating a listener's auditory attention locus, such as approaches using electroencephalography (EEG) or electrooculography (EOG). For example, PAM signals are usually substantially larger than EEG and EOG signals when PAM signals are measured using electrodes placed in or close to the ear, which is a particularly convenient location, in the context of ear-wearable devices, such as hearing-aids or hearables. In another example, PAM activity is more directly related to the locus of auditory spatial attention. Therefore, the direction or locus of the user's auditory attention can be inferred more straightforwardly (e.g., potentially using fewer computational resources, less battery, and less delay) based on PAM signals than solely on EEG or EOG signals, which may be influenced by many more factors than auditory attention.

As noted above, this disclosure describes a sound processing system that measures PAM-related EMG signals and then uses such signals to control a directional or spatial sound processing system. In some examples, the sound processing system uses electrodes placed in the concha and optionally inside the ear canal to detect the PAM-related EMG signals. In some examples, the sound processing system additionally uses electrodes placed around the ear (e.g., on the skull) or on the pinna (e.g., as part of a circumaural or supra-aural headphone) to detect the PAM-related EMG signals.

In some examples, the sound processing system uses a difference between PAM-related EMG signals across the left and right sides and/or across different PAM muscles (e.g., anterior and posterior) on the same side of the head, to more reliably estimate a user's auditory attention locus. For instance, the sound processing system may compare differential PAM-related EMG signals (e.g., differential anterior-posterior PAM signals) on the right side of the head with differential PAM-related EMG signals on the left side of the head of a user, to estimate the angle and distance of a user's auditory-attention locus.

<FIG> illustrates an example sound processing system <NUM>, in accordance with one or more aspects of this disclosure. In the example of <FIG>, sound processing system <NUM> comprises one or more processors <NUM>, one or more audio input sources <NUM>, one or more electrodes <NUM>, and one or more speakers <NUM>. The components of sound processing system <NUM> (i.e., processors <NUM>, audio input sources <NUM>, electrodes <NUM>, and speakers <NUM>) may be incorporated into one or more physical devices. For example, each of the components of sound processing system <NUM> may be included in a single ear-wearable device. In another example, sound processing system <NUM> comprises two ear-wearable devices, each of which include one or more of processors <NUM>, one or more of audio input sources <NUM>, one or more of electrodes <NUM>, and one or more of speakers <NUM>. In another example, sound processing system <NUM> may comprise one or more ear-wearable devices, each of which may include one or more of processors <NUM>, audio input sources <NUM>, electrodes <NUM>, and speakers <NUM>, and sound processing system <NUM> may also comprise one or more separate devices, such as a smartphone or special purpose device, that comprises one or more of processors <NUM>.

An ear-wearable device may comprise various types of devices designed to be worn in and/or on an ear of a wearer. For example, an ear-wearable device may comprise a hearing assistance device (e.g., a hearing aid device, a Personal Sound Amplification Product (PSAP), etc.), a wireless headset, a headphone, a wireless earbud, or another type of device. In another example, an ear-wearable device comprises a hearable with amplification and/or cancelation features.

In examples where sound processing system <NUM> comprises multiple devices, the devices of sound processing system <NUM> may communicate using wired or wireless communication technologies. For example, devices of sound processing system <NUM> may communicate wirelessly using a BLUETOOTH ™ technology, a WIFI ™ technology, or another type of wireless communication technology. In examples where processing system <NUM> comprises multiple processors <NUM>, such processors may communicate with each other to accomplish the tasks described herein as being performed by processors <NUM>.

Processor <NUM> may comprise one or more processing units, such as microprocessors, digital signal processors, application-specific integrated circuits, or other types of circuits. Audio input sources <NUM> provide audio data to processors <NUM>. For example, audio input sources <NUM> may comprise one or more microphones. In some examples, audio input sources <NUM> may comprise software, such as video games or decoders of recorded media, for generating audio data. As described below, electrodes <NUM> include one or more electrodes that may be positioned to detect electromyographical (EMG) signals from one or more of a user's periauricular muscles (i.e., PAM signals). Speakers <NUM> output sound based on audio signals output by processors <NUM>.

Sound processing system <NUM> may implement a variety of features that help a user of sound processing system <NUM> hear sounds. For example, processors <NUM> may generate audio signals that amplify the intensity of incoming sounds in audio signals generated by audio input sources <NUM>, amplify the intensity of certain frequencies of the incoming sounds, translate or compress frequencies of the incoming sound, output recorded or dynamically-generated sound, or otherwise generate sound.

In accordance with techniques of this disclosure, processors <NUM> may implement a directional processing mode in which processors <NUM> selectively amplify sound originating from a particular direction (e.g., to the front of the wearer) and/or fully or partially cancel sound originating from other directions. In some examples, processors <NUM> may reduce noise by canceling out certain frequencies.

The directional processing mode may help wearers understand conversations occurring in crowds or other noisy environments. In other words, use of a directional processing mode may be useful in a situation in which a hearing-impaired user wears hearing-aids equipped with a directional-microphone system in an environment containing multiple sound sources, such as multiple talkers. In this situation, it may be advantageous for an algorithm controlling the directional-microphone system to have information regarding the direction and/or locus in space where the user is attending or wanting to attend. This disclosure may refer to the locus in space where the user is attending or wanting to attend as the user's current auditory attention locus.

Furthermore, in some examples, sound processing system <NUM> may help a wearer enjoy audio media, such as music or sound components of visual media, by outputting sound based on audio data wirelessly transmitted to sound processing system <NUM> (e.g., via audio input sources <NUM>). In other words, the directional processing mode may be used when a user is listening to sounds (e.g., music, a television program, sounds from a video game) for which it is potentially advantageous to apply different signal-processing depending on the user's listening intentions, such as whether the user wishes to attend to sounds on the user's right or the left side. A specific instance of this type of situation involves a virtual ('3D') sound system, where different sounds in the simulated acoustic environment can be differentially enhanced, or attenuated, depending on their spatial positions and on the user's listening goals.

There are several challenges in implementing directional mode processing effectively. For example, estimating a user's current auditory attention locus is challenging for directional sound-processing algorithms such as those found on modern hearing-aids, e.g., 'beamformer' algorithms, which can selectively attenuate or amplify sounds coming from various directions, relative to sounds coming from other directions. Indeed, such systems are only beneficial to the user insofar as the sounds which they amplify are those which the user actually wants to listen to; failing this, directional sound-processing systems can actually be detrimental to the user. Additionally, it is challenging to locate the user's current auditory attention locus non-intrusively.

The techniques described in this disclosure may estimate the user's current auditory locus attention non-intrusively, using an ear-wearable device (e.g., a device placed inside the ear canal, in the concha, or around/behind the pinna), such as a hearing aid, where, or in which direction, the user of ear-wearable device <NUM> is currently focusing, or trying to focus, his/her auditory attention, and when that attentional focus is shifting.

The techniques of this disclosure for estimating the user's auditory attention locus may have use in a variety of scenarios including those discussed above. Additionally, the techniques of this disclosure for estimating the user's auditory attention locus may also be used in a sensory training or rehabilitation program involving an auditory attention component. For example, such a program might involve individuals with auditory, visual, or other sensory or neurological pathologies, who must be taught to better orient or focus their auditory attention in space.

Additionally, the techniques of this disclosure for estimating the user's auditory attention locus may help a user avoid front-back confusions. Front-back confusions are a frequent, and potentially life-threatening, perceptual error in hearing-aid wearers. (See e.g., <NPL>); <NPL>). Specifically, the techniques of this disclosure for estimating the user's auditory attention locus may enable sound processing system <NUM> to determine whether the attention of a hearing-impaired user fitted bilaterally with hearing-aids is erroneously directed toward the user's back (e.g., due to the rear-facing orientation of microphones on the hearing-aids creating a misleading sound-location perception). This may be achieved by comparing the direction of the user's auditory attention (inferred using the disclosed techniques of this disclosure), to the estimated direction of a sound source of interest (e.g., speech) determined using a directional microphone system. If the listener's auditory attention is consistently directed toward the back, while speech is consistently coming from the front, this would indicate a consistent front-back confusion. This information could then be recorded and be accessible to the audiologist, the hearing-aid manufacturer, and/or the user. Alternatively, or in addition, the information could be used to trigger the activation of an algorithm on the hearing-aid (e.g., a directional-microphone mode or simulated pinna effect) designed to reduce or eliminate front-back confusions.

As described herein, sound processing system <NUM> may perform a three-step process. In a first step, electrodes <NUM> measure electromyographical signals from at least one of the periauricular muscles (PAMs) of a user. This disclosure may refer to an electromyographical signal from a PAM as a PAM signal. In a second step, processors <NUM> use the PAM signals to estimate a direction and/or distance of the user's current spatial auditory attention locus (i.e., where the listener's auditory attention is currently directed). In a third step, processors <NUM> use the resulting estimate to control a spatial or directional sound-processing system, for example, a beamformer. In other words, processors <NUM> may control, based on the estimate of an angle of a direction of the user's current auditory attention locus (and, in some instances, a distance to the user's current auditory additional locus), an operating characteristic of sound processing system <NUM>. For instance, the operating characteristic may be a volume and/or frequency of sounds originating from the direction of the user's current auditory attention locus relative to sounds not originating from the direction and/or distance of the user's current auditory attention locus. The three steps are detailed below. However, this disclosure first explains why an ear-wearable device may use PAM signals for tracking a user's locus of spatial auditory attention.

The PAMs form part of a neuromusculuar pinna-orienting system, which in some animal species (e.g., canines and felines) is used to swivel the pinna(s) toward spatial locations corresponding to sound sources of interest. See e.g., Gobrecht, W. , & Wilson, E. , "A system of human anatomy, general and special," Philadelphia: Blanchard and Lea (<NUM>). In humans, the pinnas are strongly anchored to the skull, making it difficult for most individuals to appreciably move them. However, a few individuals can wiggle their pinnas, albeit feebly compared to felines. Nonetheless, the PAMs still contract depending on the position and volume of external sounds (reflex) as well as under volitional orientation of auditory attention, and these contractions, or micro-contractions, can be recorded using electrodes placed on the muscles. See e.g., <NPL>). In addition to being triggered reflexively in response to sound stimulation or eye rotations, PAM contractions can be elicited voluntarily by the individual; such volitional control of PAM contractions can be trained using bio-feedback methods. Therefore, PAM signals are prime candidates for non-intrusively tracking the focus of a listener's spatial auditory attention over time.

<FIG> is a conceptual diagram illustrating a human ear and PAMs, along with an example location for wear of an ear-wearable device, in accordance with a technique of this disclosure. As shown in <FIG>, PAMs include an auricularis superior muscle <NUM>, an auricularis anterior muscle <NUM>, and an auricularis posterior muscle <NUM>. The ear itself has a concha <NUM>. Arrow <NUM> indicates a location in concha <NUM> where an ear-wearable device configured in accordance with techniques of this disclosure may be worn.

PAM activity is typically measured using electrodes <NUM> (<FIG>). Some or all of electrodes <NUM> are located in, on, and/or around the user's ear. In contrast, some prior technologies have involved electrodes located behind the ear (i.e., on or near the mastoid bone), or above the ear (i.e., on the temporal bone). In accordance with the techniques of this disclosure, electrodes <NUM> in, and optionally on or around the user's ear may be used with a behind-the-ear (BTE) device (e.g., a hearing aid or a hearable), in an in-the-ear (ITE) device, or in an in-the-canal (ITC) device. Sound processing system <NUM> uses electrodes <NUM> placed at least partially in the concha and optionally on, atop, or anterior to, and outside the ear canal or pinna for recording the activity of anterior PAM <NUM>. Anterior to the ear canal means closer to the front of the body than the ear canal.

<FIG> is a conceptual diagram illustrating an ear-facing view of an example ear-wearable device <NUM> designed for use inside an external ear canal of a user, in accordance with a technique of this disclosure. Sound processing system <NUM> (<FIG>) may include ear-wearable device <NUM>. <FIG> is a schematic coronal plane view of ear-wearable device <NUM>, in accordance with a technique of this disclosure. Ear-wearable device <NUM> measures PAM signals involving a receiver in the canal (RIC), which can be placed entirely, or in part, inside a user's external ear canal (i.e., auditory meatus).

In the example of <FIG>, ear-wearable device <NUM> comprises a shell <NUM>, a RIC <NUM>, a plug <NUM>, electrical wires <NUM>, and one or more electrodes. In the example of <FIG>, ear-wearable device <NUM> has four electrodes 310A, 310B, 310C, and 310D (collectively, "electrodes <NUM>"). Electrodes <NUM> (<FIG>) of sound processing system <NUM> may include electrodes <NUM>. Three of electrodes <NUM> are for measuring PAM signals, and one of electrodes <NUM> serves as a reference. To optimize signal-to-noise ratio, the suggested locations for three of the four electrodes <NUM> are close to the expected insertion points of the muscle tendons around the ear canal; the fourth electrode is used for the ground (or reference) signal. Electrodes 310A, 310B, and 310C are positioned near the anatomical insertion points of the anterior, superior, and posterior PAMs in the ear canal. Electrode 310D is a ground electrode. The electrode locations shown in <FIG> are suggestive rather than prescriptive. Depending on the application, other or additional electrode locations may be used. The example illustrated in <FIG> is compatible with a BTE (with RIC) device, an in-the-ear (ITE) device, an in-the-canal (ITC) device, a completely-in-the-canal (CIC), or an invisible-in-the-canal form (IIC) hearing-aid or a hearable device.

Shell <NUM> may comprise a flexible body that is shaped to contain receiver <NUM> and hold electrodes <NUM> in the correct positions. Plug <NUM> may cover an open end of a cavity defined by shell <NUM>. RIC <NUM> may provide sound delivery. Speakers <NUM> (<FIG>) may include RIC <NUM>. As shown in the example of FIG. 3C, electrodes <NUM> are supported by wires <NUM>, which may act as springs pushing electrodes <NUM> against the skin of the ear canal to ensure good electrical contact between electrodes <NUM> and the skin.

<FIG> show examples of custom earmolds 400A, 400B, and 400C (collectively, "earmolds <NUM>") which may be used in the context of custom intra-auricular ear-wearable devices. <FIG> show example electrode locations <NUM> for measuring PAM signals from within the ear-canal and concha. In the example of <FIG>, filled ellipses correspond to example electrode locations directly visible with the current viewing angle; dashed ellipses correspond to example electrode locations on the opposite side, not directly visible with the current viewing angle.

In the example of <FIG>, electrode 404A measures signals from the superior auricular muscle. Electrode 404B measures signals from the anterior auricular muscle. Electrode 404C measures signals from the posterior auricular muscle. In the examples of <FIG>, at least one of electrodes <NUM> is located outside the ear canal, in the concha. Electrodes <NUM> (<FIG>) may include electrodes 404A, 404B, and 404C. The designs illustrated in <FIG>-3C are provided as examples. Other examples in accordance with techniques of this disclosure may use different designs, such as with fewer or more electrodes, and different electrode locations than those shown in <FIG> and <FIG>.

<FIG> shows an example circuit <NUM> comprising a power supply and bridge voltage divider for a preamplifier, which may be used in conjunction with electrodes <NUM> to measure PAM signals. <FIG> shows an example electrode-signal preamplifier circuit <NUM>. Together, <FIG> represent a schema of electronic circuits, which may be used to collect and pre-amplify PAM signals. Processors <NUM> (<FIG>) or other components of sound processing system <NUM> (<FIG>) may include circuit <NUM> and circuit <NUM>.

Electronic circuit <NUM> may measure EMG signals from the anterior, superior, and/or posterior PAMs. The operational-amplifier (op-amp) featured in circuit <NUM> may be one of the four op-amps featured in the TLC <NUM> integrated circuit (IC). The TLC <NUM> is a type of precision amplifier provide by Texas Instruments Inc. of Dallas, Texas. Advantageously, the TLC <NUM> IC exists in ultra-CMS format, compatible with miniaturization constraints for an in-ear, or on-the-ear, device. Advantageously, the op-amps in the TLC <NUM> IC use MOSFET technology, yielding a high input impedance compatible with the use of dry electrodes, which may be more convenient than wet electrodes in the context of consumer-device applications of the current disclosure. Additionally, this power supply/stabilizer design operates on <NUM>. 4V (-<NUM>. 7V to +<NUM>. 7V), compatible with battery voltage specifications used in current hearing aids (<NUM>. 55V, <NUM>-size battery), and its design features an arrangement of selected resistors and capacitors which, in combination with the op-amp, limit current leaks through the midpoint, thus limiting contamination of the EMG recordings by undesirable electronic noise.

In the example of <FIG>, the label `x <NUM>' is used to indicate that the plate (i.e., everything within the dashed line) can be replicated, up to <NUM> times, so as to obtain one preamplifier for each of the three periauricular muscles (posterior, anterior, superior), as needed for the application considered. The labels 'E1', 'E2', and 'E3' are used indicate that each replication of the plate should be connected to a different electrode. In most applications, these three electrodes are each connected to a different periauricular muscle. The label, 'E4', is used to denote a fourth electrode (e.g., electrode 310D of <FIG>), connected to the ground. The labels 'S1', 'S2', and 'S3' denote three amplified output signals, corresponding to the signals from the three electrodes labeled E1 through E3, respectively. In the example of <FIG>, resistors are shown by rectangles and capacitors are shown as parallel lines. The resistance values of the resistors are expressed in ohms and the capacitance values of the capacitors are expressed in farads.

As mentioned above, a second step comprises estimation of a direction or locus of the user's current auditory attentional focus. Particularly, processors <NUM> may estimate an angle of a direction between the user's current auditory attention locus and at least one reference point or plane. To estimate the angle between the user's current spatial auditory attention locus and at least one reference point or plane, processors <NUM> (<FIG>) combine the PAM signals from at least one channel (where a channel is defined by a pair of electrodes including the ground electrode). In some examples, processors <NUM> estimate the angle based on a difference between the channels corresponding to the anterior and posterior PAMs on the same side of the user's head. Alternatively, or additionally, processors <NUM> may use the difference between one channel on the left and one channel on the right (for instance, the difference between the left and right posterior-PAM channels) to estimate the angle.

<FIG> is a schematic illustration of an example setup showing four different sound source locations around a user. In the example of <FIG>, LE indicates left ear and RE indicates right ear. <FIG> illustrates an example of different signals corresponding to activities of anterior and posterior PAMs in the left and right side of the user's head. Together, <FIG> represent a schematic illustration of a principle for estimating the angle of a sound source in an azimuthal plane relative to a user's head, based on EMG signals from the posterior and anterior PAMs.

For instance, in the example of <FIG>, processors <NUM> may determine that the user's auditory attention locus is at an angle of <NUM>° (i.e., directly behind the user) when the muscle activity for both the left and right posterior PAMs (expressed in <FIG> as scaled muscle signal amplitude) is at approximately <NUM> units, and the muscle activity for both the left and right anterior PAMs is <NUM> units. Similarly, processors <NUM> may determine that the user's auditory attention locus is at an angle of <NUM>° (i.e., directly left of the user) when the muscle activity of the left posterior PAM and the muscle activity of the right anterior PAM are each at approximately <NUM> units.

In some examples, processors <NUM> determine the distance between a reference point and the user's auditory attention locus in addition to determining an angle between the direction of a user's auditory attention locus and a reference point or plane. The reference point may be one of the user's ears, a location midway between the user's ears, or another location. To determine the distance between the reference point and the user's auditory attention locus, processors <NUM> may combine signals from at least one channel on the left side of the user's head and at least one channel on the right side of the user's head. In one example, processors <NUM> compare the difference between the anterior and posterior channel signals on the right side to the difference between the anterior and posterior channel signals on the left side, according to a 'triangulation' formula.

In different examples, processors <NUM> may estimate the angle and/or distance between the user's current spatial auditory attention locus and at least one reference point or plane in different ways. These examples are described in this disclosure by way of illustration only.

In one example, processors <NUM> use PAM signals from one periauricular muscle, such as the posterior periauricular muscle corresponding to the user's right ear. Such PAM signals may be sufficient to obtain a first, coarse estimate of the angle of the direction of the user's current attentional locus since, for instance, the PAM signal varies as a function of this angle. In this situation, processors <NUM> may compute an estimate, â, of the angle, a, at time t, as: <MAT> where s(t) is the PAM signal from the user's periauricular muscle at time t, and f denotes a nonlinear function, which maps the PAM signal to an angle. Typically, but not necessarily, the function, f, is the inverse of a periodic function; for example, <MAT> where ζ-<NUM>(. ) denotes the inverse of ζ(. ), and, <MAT> where θ is an angle in degrees, while α, β, γ, κ, and µ are parameters, the values of which may be estimated based on data obtained using a particular realization of the invention, for an individual user, or averaged across a group of users. In this context, µ is the angle for which the function reaches its maximum, κ and γ control the width and shape of the peak, and α and β serve to shift and scale the function up or down. <FIG> shows an example of functions generated using Eq. <NUM>, with γ set to <NUM>, and α and β adjusted in such a way that the resulting function values span the interval (<NUM>; <NUM>). Note that the signal, s(t), need not reflect the activity of the periauricular muscle at time t only, and may reflect also contributions from the periauricular muscle at preceding time points. For instance, this would be the case in applications where s(t) is computed, through analog or digital circuitry, as a time-integral or time-weighted average of muscle activity signals unfolding over time.

In most applications, although not necessarily all applications, the function, f, is monotonic with s(t). Thus, the estimated angle increases or decreases with increasing or decreasing magnitude of activity of a periauricular muscle. In applications in which s(t) increases with the magnitude of activation of the posterior auricular muscle, f is monotonically increasing with s(t) such that the estimated angle of the direction of the user's auditory attention locus with regard to a reference point situated in front of the user (defined as an angle of <NUM>) increases with s(t).

In another example, to obtain an estimate of the angle of the direction of the user's attentional locus, processors <NUM> combine PAM signals across the left and right sides of the user's head. In this example, processors <NUM> compute the estimate, â(t), by subtracting at least one PAM signal on one side of the user's head from at least one PAM signal on the opposite side of the user's head. Denoting as sl(t) and sr(t) the PAM signals corresponding to the left side and the right side of the user's head, respectively, processors <NUM> may compute an estimate â, of the angle, a, at time t, as: <MAT> where fl and fr are linear or nonlinear functions applied to the left and right signals prior to subtraction, and the operator g serves to map the result of the subtraction into an angle. Note that this allows for the case where no transformation is applied to the muscle signals prior to their subtraction, since the functions fl and fr may be each identified with a linear function having a slope of <NUM> and an intercept of <NUM>. In some applications involving this estimation technique, the signals, sl(t) and sr(t), reflect the activity of the left and right posterior auricular muscles. In this situation, and if the functions, fl and fr are set to linear functions, then the mapping, g, may take the form of the inverse of a function, g-<NUM>, defined as, <MAT>.

In a variant of the two examples above (illustrated using Equations <NUM> and <NUM>), the signal, s(t), sl(t), or sr(t) corresponds to a linear or nonlinear combination of at least two PAM signals measured on the same side of the user's head. In some examples, although not necessarily all examples, processors <NUM> combine signals from the posterior and anterior auricular muscles in this way. Moreover, in some examples, although not necessarily all examples, the combination involves a subtraction. For example, denoting as sp(t) the signal corresponding to the posterior auricular muscle located on one side of the user's head, and sa(t) the signal corresponding to the posterior auricular muscle located on the same side of the user's head, processors <NUM> may compute the angle estimate, â(t) as: <MAT> where fp, fa, and h are nonlinear functions used to map the muscle-activation signals and their combination into an angle estimate.

In another example, processors <NUM> mathematically or electronically combine signals from at least two periauricular muscles located on the right side of the user's head with signals from at least two periauricular muscles located on the left side of the user's head. In this example, processors <NUM> may compute the angle estimate as: <MAT> where spr, sar, spl, and sal are signals corresponding to the posterior-right, anterior-right, posterior-left and anterior-left periauricular muscles, respectively, fpr, far, fpl, and fal are linear or nonlinear functions applied to these signals (for example, these functions could be defined as illustrated in Eqs. <NUM> and <NUM>), and hl, hr, and v, are nonlinear functions used to map intermediate variables into an angle estimate.

By combining signals from at least two periauricular muscles on one side of the user's head and signals from at least two periauricular muscles on the opposite side of the user's head, processors <NUM> may compute, in addition to an estimate of the angle between the direction of the user's attentional focus, an estimate of the distance between a reference point on the user (typically, the user's head) and the user's current auditory attention locus. The user's current auditory attention locus may correspond to an actual sound source in the user's physical environment, to a virtual sound source, or to an imaginary (or erroneously perceived) sound-source location. Processors <NUM> may obtain an estimate of the distance by combining said estimated angles (corresponding to the direction of the user's auditory attention) on the right side of the user's head, and the estimated angle on the left side of the user's head, using triangulation. Specifically, denoting as âr(t) and âl(t) estimates of the angles between (a) a line passing through the user's current auditory attention locus and a reference point on the user's right (for âr(t)) or left (for âr(t)) ear, and (b) a coronal plane passing through the user's left and right ear canals, and denoting as dlr the distance between the reference points on the user's left and right ears, processors <NUM> may compute the distance, denoted dr, between the right-ear reference point and the current locus of the user's auditory attention focus as: <MAT> Similarly, processors <NUM> may compute the distance, denoted dl, between the left-ear reference point and the current locus of the user's auditory attention focus as: <MAT> Angles in the above equations are in degrees.

<FIG> is a schematic illustration of the 'triangulation' principle, which may be used to estimate the angle and distance of a sound source relative to a reference point. For example, in the example of <FIG>, processors <NUM> (<FIG>) determine âl(t) is approximately <NUM>° and that âr(t) is approximately <NUM>°. Moreover, in the example of <FIG>, dlr is assumed to be <NUM>. Accordingly, processors <NUM> may determine that the dl is equal to approximately <NUM> and that dr is equal to approximately <NUM>. In the example of <FIG>, the line between the user's ears may correspond to a reference plane; a point between the user's ears may correspond to a reference point.

<FIG> is an example graph showing a technique for estimating an angle âl(t) of a direction to a user's current auditory attention locus and the user's left ear and an angle âr(t) of a direction to the user's current auditory attention locus and the user's right ear. As shown in the example of <FIG>, processors <NUM> (<FIG>) may determine that âl(t) is equal to <NUM>° relative to a vertical plane extending forward from the user's left ear (i.e., <NUM>° relative to a horizontal line extending through the user's left and right ears) when the muscle activity of the left anterior auricular muscle is approximately <NUM> units (on a normalized measurement scale from <NUM> to <NUM>, where <NUM> corresponds to no detectable muscle signal and <NUM> corresponds to the maximum muscle signal strength measurable by the device), and the muscle activity of the left posterior muscle is approximately <NUM> units (on the same normalized measurement scale). Similarly, processors <NUM> may determine that âr(t) is equal to <NUM>° relative to a vertical plane extending forward from the user's right ear (i.e., <NUM>° relative to the horizontal line extending through the user's left and right ears) when the muscle activity of the right anterior muscle is at <NUM> units and the muscle activity of the right posterior muscle is at <NUM> units.

Because the characteristics (e.g., absolute and relative amplitudes, latencies, signal-to-noise ratio, etc.) of the different PAM signals, and difference signals, can vary across individuals (e.g., depending on anatomical variability, skin conductance, and a myriad of other factors), as well as over time within a given user, processors <NUM> may use machine-learning algorithms that adapt, not only to a particular user, but also to changing conditions (i.e., changes in skin conductance depending on humidity, etc.) within a given user. Specifically, processors <NUM> may perform a 'tuning' procedure for the initial setting of the parameters of the algorithms that estimate the direction and distance based on PAM activity signals, as described above. In some examples, an initial setting is performed under well-controlled sound-stimulation conditions, with the device(s) placed on the user, and at least one sound source, positioned at (a) predefined distance(s) from a reference point placed, for instance, at a pre-defined position on the user's head. In some examples, processors <NUM> perform subsequent tunings each time the user puts ear-wearable device(s) (e.g., ear-wearable device <NUM> of <FIG>) in his/her ear(s). For these subsequent tunings, a single, readily-available sound source (e.g., smartphone) may be placed at an approximate position in front of the user, in a combination with a smartphone app to calibrate the algorithms. In some examples, the techniques described above may be advantageously supplemented with an apparatus that uses microphones to automatically estimate the angle and/or distance of a sound source in the user's environment (see, e.g., <CIT>, issued <NUM>; <NPL>); and <NPL>).

In addition, processors <NUM> may track drifts in the measured PAM signals, so that such drifts can be compensated for in the estimation of the direction and/or distance of the sound source. For example, suppose that a muscle-activity signal, s(t), follows a decreasing trend over the course of several minutes to a few hours; the ear-worn device, or an external device to which the ear-worn device sends information, can detect, compensate for, and extrapolate, such a drift by recording the muscle-activity signal at different moments, extracting relevant statistics (e.g., maximum value or upper decile) over successive time periods, and then comparing these values across the successive time periods. More advanced signal-processing or machine-learning algorithms (e.g., Kalman or particle filters) can also be used, to more accurately detect, track, extrapolate, and compensate for, any drift in muscle-activity signals over time.

A third step of the process uses the inferred direction of the user's current auditory attention locus to control an operating characteristic of a spatial or directional sound-processing system. A directional sound processing system is any processing device or algorithm that measures or computes the direction of arrival of a sound, or modifies a signal related to this sound in a direction-dependent manner. A spatial sound processing system is any device or algorithm that measures or computes spatial characteristics of a sound, such as its direction, distance, or spread (relative to one or several points or planes of reference), or that modifies a signal related to spatial characteristics of this sound. In this third step of processing, processors <NUM> may use the direction or position estimate derived in the previous step to control a directional or spatial sound-processing system.

In its simplest form, the target direction of a directional microphone system, such as a beamformer, is set to correspond to the currently-estimated direction of the user's auditory attention. More sophisticated versions of such processing involve a form of temporal smoothing, to avoid abrupt changes in the setting of the target direction for the directional sound system.

Other, more sophisticated processing performed during this third step could involve, but is not necessarily limited to, processors <NUM> using estimates of the location (i.e., direction and distance) of the user's auditory attentional locus. Processors <NUM> may use this advantageously in conjunction with a multi-microphone sound-processing system which can selectively attenuate, amplify, or otherwise process incoming sounds depending, not just on their angles of incidence, but also, depending on their distance, relative to the microphones.

It should be noted that this third step is optional, and could be replaced with other advantageous uses of the auditory-attention direction and or distance information estimated in the second step.

The techniques of this disclosure may contribute to the development of 'intelligent' hearing-aids or hearables which can infer, on a moment-by-moment basis, the user's listening intentions. In this context, it may be important to be able to determine the user's current auditory attention locus, or at least the direction of the user's current auditory attention locus. For instance, it may be important to determine whether the user currently attending (or trying to attend) to a sound source located in front of the user, to the user's right, to the user's left, or in the user's back. It may also be important to detect when the user's current auditory attention locus or the direction thereof changes, as happens when the user shifts the user's auditory attention from one talker to another in a multi-talker conversation.

Up to now, researchers in academia and in industry have focused on EEG or EOG signals. However, these have important shortcomings (poor signal-to-noise ratio, contamination by various artifacts, and/or need for computationally-heavy post processing), which currently limit their integration into products such as hearing-aids. In this context, PAM signals may be advantageous because firstly, they are usually markedly larger, and thus easier to measure reliably, from within or around the ear than EEG or EOG signals. Secondly, PAM signals may more straightforwardly relate to the user's current auditory attention locus or the direction thereof than EEG and EOG signals. This is because, in several mammals, the PAMs form part of a neuromuscular pinna-control system that evolved to improve sound detection and localization. Although in primates this system is largely vestigial, PAMs continue to be under the control of exogenous (reflex) and endogenous (deliberate) auditory attention. Given these advantages, the techniques of this disclosure may provide both a safer (i.e., more likely to succeed) and more direct route toward developing a user-brain-controlled hearing-aid/hearable than approaches based on EEG or EOG.

It is possible that one limitation of the techniques of this disclosure, over techniques using EEG signals for guiding the processing in hearing-aids or other ear-level devices, is that the information obtained from PAM signals is mostly limited to where in space a user's attention is directed, whereas EEG signals (if they can be reliably measured from within the ear in everyday life situations) may ultimately provide richer information about the user's listening goals. Hence, the techniques of this disclosure may be used in conjunction with EEG signals. For example, information derived from EMG signals from the periauricular muscles may be combined with information derived from EEG signals in order to improve the accuracy of an estimate of which sound source a listener is currently attending to in an auditory scene, or of where that sound source is located. To take an even more specific example of how this could work, there is a possibility of using brain activity measured using EEG measured using an ear-worn device, to infer which of two concurrent speech streams, corresponding to the voices of two talkers located in different positions in real or virtual soundspace, a listener is currently attending to (see, e.g., <NPL>)). However, inferences obtained in this way are not always sufficiently accurate to support certain real-world applications of such technology (e.g., Mirkovic, Bleichner, De Vos, & Debener, <NUM>). The accuracy of the estimation of which voice a user wants to attend to can be improved by combining such EEG-based technology with the EMG-based aspect of the inventions described here, as the latter provides additional information concerning the direction or location to which the user is currently attending, or trying to attend.

One currently available approach for measuring a user's intentions is to have the user interact, manually or though voice commands, with the hearing device (e.g., using knobs on the hearing device), or with an external device connected to the hearing device (e.g., smartphone). For example, there exist smartphone apps with interfaces that let a user indicate the direction from which the sounds to which the user wishes to listen are coming; this information can then be used to inform the activation, or the setting of parameters, of algorithms on the hearing device, for instance, where to place the 'nulls' (maximum attenuation) of a directional-microphone system.

Another approach, which has been used previously in an attempt to gain information as to the direction in which a listener's auditory attention is directed, relies on head position or head movements. Directional sound systems on most hearing-aids today are based on this idea, and on the assumption that users tend to turn their head so as to face the sound source which they are wanting to listen to. Accordingly, most 'fixed' directional sound systems attenuate sounds coming from directions other than the front. Adaptive directional systems also exist, which attenuate sounds coming from directions other than one or several 'target' directions identified automatically by the device based on, for instance, the presence of salient speech-sound characteristics in this (or those) target direction(s).

Although not yet used in commercially-available hearing aids, another approach, which may be used in an attempt to obtain information regarding the listening intentions of a user of hearing device, relies on 'covert' measures, in particular, biophysical measures. These include electrophysiological measures of brain activity (using, e.g., EEG), or of movements of the eyes (using, e.g., EOG) or other body parts (e.g., finger, toe).

There are several disadvantages to the existing approaches. For example, the overt approach requires explicit, potentially frequent user motor input (manual or otherwise), which may be tedious for the user and may be unlikely to be used consistently over time by users of hearing-aids or hearables. Particularly, the difficulty comes from the fact that, in many real-life listening situations, the direction of auditory attention changes rapidly and often, as when, for instance, listening to a conversation with more than one interlocutor. Asking users to press a button to indicate every time that their auditory attentional focus changes may be unacceptable.

There are also disadvantages to approaches based solely on head movements. For example, one limitation of this approach is that sound sources of interest to the user are not always in front of the user. For instance, in a multi-talker conversation at a restaurant, head turns are not fast enough to ensure that the listener always faces the current talker. Information derived from PAM signals can usefully replace, or supplement, head-movement or head-position information.

There are also disadvantages to approaches based solely on eye movements (EOG). For example, eye movements are not straightforwardly related to auditory attention; they are also influenced by visual attention, and a myriad of other factors. Secondly, EOG signals are typically of smaller magnitude than PAM signals, especially when recorded using electrodes in or close to the ear.

Furthermore, there are disadvantages to approaches based solely on EEG. For example, EEG signals are of much smaller magnitude (i.e., have lower signal-to-noise ratios (SNR)) than PAM signals when recorded at or close to the ear. In another example, estimating the current direction or locus of a listener's auditory attention using EEG is much less straightforward computationally than estimating the same information based on PAM signals. The EEG-based approach also poses major basic research challenges (e.g., how to process the EEG?), computational challenges (e.g., can this processing be implemented on a miniature device?), and power-consumption (e.g., will it consume batteries?) challenges, which the PAM-based approach does not.

In contrast to overt measures, which require explicit and deliberate user interactions, the techniques of various examples of this disclosure use covert, implicit biophysical measures, which may be essentially seamless ('transparent') to the user and therefore, more likely to be accepted or tolerated by the user than overt (manual or other) deliberate interactions with the device. Furthermore, in contrast to previously-proposed covert measures for inferring the direction or locus of a listener's auditory attention, namely EEG and EOG, EMG signals from the PAMs are primarily, and almost exclusively, related to the direction or locus of the listener's auditory attention. Therefore, with the PAM signals, relatively little processing is performed in comparison to EEG or EOG signals to infer the direction or locus of the listener's auditory attention. In addition, because the PAMs are located very close to the ear, EMG signals generated by the activation of the PAMs are usually considerably larger, and thus easier to detect than EEG or EOG signals, when measured using electrodes placed in or near the ear.

Designers of video games, especially, those involving a form of virtual reality (VR), in particular, virtual audio (or `3D sound'), could find it advantageous to use techniques of this disclosure to verify that the spatial rendering of sound in a game is effective, or further, to estimate the accuracy of such rendering. To this aim, designers could play a sound to a user in a predetermined position of the virtual audio space under headphones, then use the techniques of this disclosure to infer the direction and/or locus of the listener's auditory attention, and confirm that the locus or direction thus inferred corresponds to the direction or locus of the virtual sound being played. If there is a consistent discrepancy, the test could be repeated using a sound at another location in virtual space, and adjustments could be made to parameters of the virtual-sound positioning algorithm so as to resolve the discrepancy between the attended spatial location and the actual sound location, thus allowing a more accurate positioning of sounds in virtual space than would otherwise be possible. Several other possible advantageous uses of the techniques of this disclosure in the context of the video-game industry are conceivable, but are not described here.

As briefly mentioned above, the techniques of this disclosure for determining a direction and/or locus of a user's auditory attention may be used in a reeducation, or 'training', context, where an individual with an auditory impairment, for instance, an auditory attention deficit, must be taught to improve his/her ability to focus or maintain his/her auditory attention. For example, a sound scene comprising two or more sound sources at virtual (simulated) locations could be played to the user under headphones or through insert earphones, and the user would be instructed to focus and maintain his attention on one of the sound sources. Using the techniques of this disclosure, the sound that the user is currently attending would be modified in some way (e.g., its volume could be slightly enhanced), as long as the user is attending to it (as determined using the techniques of this disclosure), thus providing a 'biofeedback' signal to the user; such biofeedback loop could form the basis of the training program. As an added benefit, using the techniques of this disclosure for estimating a direction and/or locus of the user's auditory attention, an operator or sound processing system <NUM> (automated mode) may verify that the user is indeed able to perform this spatially-selective-attention task.

In the context of this document, muscle signals are defined to mean any signal related to the voluntary or reflex stimulation of a muscle, whether or not this signal is accompanied or followed by an observable muscle contraction. The phrases `directional sound processing system' or 'spatial sound processing system' mean any system that processes electrical signals related to sounds in a manner that depends on the location of the sound-generating sources, or the direction which these sounds form relative to a reference point, which may be on the device user.

<FIG> is a block diagram illustrating example components of ear-wearable device <NUM>, in accordance with one or more aspects of this disclosure. Sound processing system <NUM> (<FIG>) may include ear-wearable device <NUM>. In the example of <FIG>, ear-wearable device <NUM> comprises one or more storage device(s) <NUM>, a radio <NUM>, a receiver <NUM>, one or more processor(s) <NUM>, a microphone <NUM>, one or more electrodes <NUM>, a battery <NUM>, and one or more communication channels <NUM>. Communication channels <NUM> provide communication between storage device(s) <NUM>, radio <NUM>, receiver <NUM>, processor(s) <NUM>, a microphone <NUM>, and electrodes <NUM>. Components <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> may draw electrical power from battery <NUM>, e.g., via appropriate power transmission circuitry. Processors <NUM> (<FIG>) may include processor(s) <NUM>. Electrodes <NUM> (<FIG>) may include electrodes <NUM>. In other examples, ear-wearable device <NUM> may include more, fewer, or different components.

Storage device(s) <NUM> may store data. Storage device(s) <NUM> may comprise volatile memory and may therefore not retain stored contents if powered off. Examples of volatile memories may include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art. Storage device(s) <NUM> may further be configured for long-term storage of information as non-volatile memory space and retain information after power on/off cycles. Examples of non-volatile memory configurations may include flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories.

Radio <NUM> may enable ear-wearable device <NUM> to send data to and receive data from one or more other computing devices. For example, radio <NUM> may enable ear-wearable device <NUM> to send data to and receive data from other devices of sound processing system <NUM> (<FIG>). Radio <NUM> may use various types of wireless technology to communicate. For instance, radio <NUM> may use Bluetooth, <NUM>, <NUM>, <NUM> LTE, ZigBee, WiFi, Near-Field Magnetic Induction (NFMI), or another communication technology.

Receiver <NUM> comprises one or more speakers for generating audible sound. Microphone <NUM> detects incoming sound and generates an electrical signal (e.g., an analog or digital electrical signal) representing the incoming sound. Processor(s) <NUM> may process the signal generated by microphone <NUM> to enhance, amplify, or cancel-out particular channels within the incoming sound. Processor(s) <NUM> may then cause receiver <NUM> to generate sound based on the processed signal. Furthermore, processor(s) <NUM> may implement the techniques of this disclosure for estimating a direction and/or locus of a user's auditory attention. In some examples, processor(s) <NUM> include one or more digital signal processors (DSPs).

In some examples, ear-wearable device <NUM> comprises a custom earmold or a standard receiver module at the end of a RIC cable. The additional volume in a custom earmold may allow room for components such as sensors (accelerometers, heartrate monitors, temp sensors), a woofer-tweeter, (providing richer sound for music aficionados), and an acoustic valve that provides occlusion when desired. In some examples, a six-conductor RIC cable is used for in ear-wearable devices with sensors, woofer-tweeters, and/or acoustic valves.

<FIG> is a flowchart illustrating an example operation of sound processing system <NUM> in accordance with a technique of this disclosure. The flowcharts of this disclosure are provided as examples. Other examples may include more, fewer, or different actions.

In the example of <FIG>, electrodes <NUM> (<FIG>) measure one or more periauricular muscle signals from one or more periauricular muscles of a user (<NUM>). Electrodes <NUM> may include one or more electrodes located at least partly inside an ear canal of the user. According to the invention, electrodes <NUM> include one or more electrodes located at least partly inside a concha of the user. In some examples, electrodes <NUM> include one or more electrodes located at least partly atop or behind a pinna of the user. In some examples, electrodes <NUM> include one or more electrodes located at least partly anterior to an ear canal and pinna of the user and the one or more signals from the one or more periauricular muscles. In some examples, electrodes <NUM> measure a periauricular muscle signal from at least one anterior auricular muscle and at least one posterior auricular muscle on the same side of a head of the user.

In another example, electrodes <NUM> measure one or more signals from one or more periauricular muscles on a right side of the head of the user. In this example, electrodes <NUM> also measure one or more signals from one or more periauricular muscles on a left side of the head of the user. In some instances of this example, electrodes <NUM> measure signals from an anterior auricular muscle and a posterior auricular muscle on the right side of the head of the user and measure signals from an anterior auricular muscle and a posterior auricular muscle on the left side of the head of the user.

Furthermore, processors <NUM> may compute, based on the periauricular muscle signals, an estimate of an angle corresponding to a direction of a current auditory attention locus of the user with respect to a reference point or plane (<NUM>). In some examples where electrodes <NUM> measure one or more signals from one or more periauricular muscles on a right side of the head of the user and measure one or more signals from one or more periauricular muscles on a left side of the head of the user, processors <NUM> compute, based on the one or more signals from the one or more periauricular muscles on the right side of the head of the user and the one or more signals from the one or more periauricular muscles on the right side of the head of the user, an estimate of the distance corresponding to the current auditory attention locus of the user. For instance, processors <NUM> may use the triangulation techniques described elsewhere in this disclosure to determine the distance.

In some examples, processors <NUM> use the one or more periauricular muscle signals to compute an estimate of an elevation angle corresponding to the current auditory attention locus of the user, relative to a reference point or plane. The elevation angle may be a component (e.g., a z component) of the angle used in controlling an operating characteristic of sound processing system <NUM>. For instance, in this example, as part of measuring the set of one or more periauricular muscle signals, one or more electrodes may measure a signal from a superior auricular muscle of the user. In this example, processors <NUM> may use a signal from the superior auricular muscle to compute the estimate of the elevation angle corresponding to the current auditory attention locus of the user, relative to the reference point or plane. For example, the processor could take as input a measure of the amplitude, or intensity, of the superior auricular muscle activity signal, and compute a monotonically increasing transformation of this input, such that higher muscle-activity signal amplitudes, or higher intensities, correspond to a higher estimated elevation angle relative to a transverse plane through the user's ear canals or other reference plane through the user's body. The transformation used for this purpose could be implemented using a look-up table, wherein each superior auricular muscle signal amplitude comprised within the range of amplitudes that can be measured by the device has a corresponding elevation-angle estimate. The values in such a table could be determined based on measurements of posterior auricular muscle activity signals for different elevations of a real, virtual, or imagined sound source. Such measurements could be performed, specifically, on the user of the system to obtain individual data, or on a group of individuals to obtain average data. Alternatively, the transformation of said muscle-activity signal amplitude into an estimate of the elevation angle could be implemented as a linear or nonlinear function. The parameters of such a function could be determined by computing the best fit (according to a maximum-likelihood, minimum squared error, or some other criterion) of said function to measurements of at least one posterior muscle-activity signal amplitude.

In some examples, sound processing system <NUM> comprises one or more electrodes for measuring eye signals. The eye signals comprise at least one of eye-movement signals of the user or eye-position signals of the user. For instance, the eye signals may comprise EOG signals. The electrodes for generating the EOG signals may be in the ear canal of the user, or they may be placed elsewhere, for example, closer to at least one of the two eyes of the user. Processors <NUM> may combine such eye signals with the periauricular muscle signals mathematically, in order to compute a combined signal. The combined signal may provide a better estimate of the angle corresponding to the direction of the user's current auditory attention locus, relative to the reference point or plane. For example, if the user's eyes move in the direction corresponding to the user's auditory attention target, the processor can compute a weighted mean of the target-angle estimate (in complex-number form) computed based on the eye-movement signal, and of the target-angle estimate (in complex-number form) computed based on the peri-auricular muscle signals, as follows, <MAT> where θa is the angle of the user's auditory-attention target computed using periauricular-muscle signals only, θe is the angle of the user's auditory-attention target computed using eye signals only, wa is the relative weight of the angle information derived from peri-auricular muscle signals, and θ̂ is the resulting estimate of the user's auditory attention target, obtained by combining mathematically the periauricular-muscle and eye signals. If the relative weight, wa, is chosen appropriately, a more accurate and precise estimate of the direction of the user's auditory attention can be obtained in this way, than might be possible based on the eye signals alone, or on the peri-auricular muscle signals alone. For example, the weight may be inversely related to the measurement error of the periauricular muscle signals, relative to the measurement error of the eye-movement signal.

In some examples, sound processing system <NUM> comprises electrodes for measuring brain signals of the user. For instance, electrodes <NUM> may include electrodes for generating an EEG signals of the user. The electrodes for generating the EEG may be in the ear canal of the user. Furthermore, in this example, processors <NUM> may generate, based on the brain signals and the periauricular muscle signals, a combined signal. Processors <NUM> may use the combined signal to generate a more accurate and precise estimate of the angle corresponding to the direction in which the user's auditory attention is oriented, than might be achievable using either type of signal alone. For example, the mathematical combination of the EEG and periauricular-muscle signals may take the same form as described in Eq. <NUM>, with the angle estimate computed using eye signals replaced by an angle estimate computed using EEG signals. The latter estimate may be computed using existing techniques for processing EEG signals such as, for example, as described in <NPL>). As described in Wong, subjects kept their eyes fixated straight ahead while <NUM>-channel EEG was recorded. A <NUM>-level wavelet decomposition was then performed to split the data into <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM> and <NUM>-<NUM> bands. For each band, the <NUM> independent components that had the largest power were taken. Over data segments of <NUM> seconds, the power within each component was then computed for each band. A deep neural network was trained on these features, using stochastic backpropagation. The output of the network was the angle of the incoming sound source.

In some examples, sound processing system <NUM> comprises sensors for measuring head signals. The head signals of the user are indicative of head movements and/or head positions of the user. In some examples, sound processing system <NUM> comprises accelerometers, gyroscopes, or other types of sensors to determine the head signals. Processors <NUM> may generate, based on the head signals and the periauricular muscle signals, a combined signal that provides a more accurate and precise estimate of the angle corresponding to the direction in which the user's auditory attention is oriented. For example, an estimate of said angle computed using head-motion signals alone can be combined mathematically with an estimate of the angle computed using periauricular muscle signals alone following an equation similar to Eq. <NUM>, but with the angle estimate based on eye signals replaced by an angle estimate based on head movements. Alternatively, or in addition, head-motion signals can be used in a similar fashion.

In the example of <FIG>, processors <NUM> control, based on the estimate of the angle, an operating characteristic of sound processing system <NUM> (<NUM>). In some examples, the operating characteristic is a relative volume of sounds at a target angle. The target angle corresponds to the estimate of the angle corresponding to the direction of the current auditory attention locus of the user with respect to the reference point or plane. In such examples, as part of controlling the operating characteristic of sound processing system <NUM>, processors <NUM> selectively amplify sounds at a target angle relative to one or more non-target angles. In some examples, processors <NUM> selectively attenuate sounds at a non-target angle relative to the target angle.

In some examples, as part of controlling the operating characteristics, processors <NUM> selectively amplify sounds in a target spatial region relative to one or more non-target spatial regions. The target spatial region is defined by the direction of the current auditory attention locus of the user and a distance of the current auditory attention locus of the user relative to a reference points or plane. In some examples, processors <NUM> selectively attenuate sounds in a non-target spatial region relative to one or more target spatial regions.

In some examples, as part of controlling the operating characteristics of sound processing system <NUM>, processors <NUM> may reproduce sounds recorded on at least a first track of recorded audio data and a second track of the recorded audio data. In this example, the first track and the second track correspond to sound sources at different positions. In this example, processors <NUM> selectively amplify or selectively attenuate the first track or the second track depending on the current auditory attention locus of the user.

Furthermore, in some examples, prior to performing the actions shown in <FIG>, sound processing system <NUM> may perform a calibration process. Sound processing system <NUM> may perform the calibration process in various ways. For example, processors <NUM> may estimate the angle of a sound relative to the reference point or plane. Processors <NUM> may estimate the angle of sound in the manner described elsewhere in this disclosure. Additionally, processors <NUM> may compare the angle of the sound with the angle corresponding to the direction of the user's auditory attention, which has been estimated using the periauricular muscle signals, in order to compute an error signal. For example, the error signal may be computed as the arithmetic difference between the measured angle of the sound, and the estimated angle of the user's auditory attention. Furthermore, in this example, processors <NUM> may use the error signal to initialize or calibrate a process used to compute, based on the periauricular muscle signals, the estimate of the angle corresponding to the direction of the user's auditory attention. For example, the error signal defined above may be subtracted from subsequent user's auditory-attention angle estimates computed using periauricular muscle signals, so that the auditory-attention angle estimates are properly "corrected" or "calibrated".

In some examples of the calibration process, processors <NUM> estimate a distance of a sound source relative to the reference point or plane. Processors <NUM> may estimate the distance in the manner described elsewhere in this disclosure. Additionally, processors <NUM> may compare the estimated distance of the sound source with the signals from the periauricular muscles in order to compute an error signal. For example, the error signal may be computed as the arithmetic difference between the estimate of the sound-source distance computed based on other signals than the periauricular muscles, and the distance of the user's auditory attention target computed based on the periauricular muscle signals. Processors <NUM> may then use the error signal to initialize or calibrate a process used to compute, based on the periauricular muscle signals, the estimate of the distance. For example, the error signal defined above in this paragraph may be subtracted from subsequent distance estimates computed based on periauricular muscle signals, so that the latter estimates are properly "corrected" or "calibrated".

<FIG> is a cross section view of an example ear-wearable device <NUM>, in accordance with a technique of this disclosure. <FIG> is a top view of an example ear-wearable device <NUM>, in accordance with a technique of this disclosure. <FIG> is a top-cross section view of example ear-wearable device <NUM>, in accordance with a technique of this disclosure. <FIG> is an outside section view of example ear-wearable device <NUM>, in accordance with a technique of this disclosure.

In the example of <FIG>, ear-wearable device <NUM> comprises a soft shell <NUM>, a receiver <NUM>, and a tab <NUM>. Receiver <NUM> may be a speaker that outputs sound. Additionally, tab <NUM> is placed at the lowest part of the ear, between the anti-tragus and tragus. This way, electrodes <NUM> line-up with the tendon insertions of the peri-auricular muscles. In the example of <FIG>, the walls of shell <NUM> may be <NUM>-mm thick, with a <NUM>-mm pocket for electrodes <NUM>. Electrodes <NUM> may comprise <NUM>-mm in diameter gold balls that seat in shell <NUM> and stick out into the ear minimally.

If implemented in software, the functions may be stored on or transmitted over, as one or more instructions or code, a computer-readable medium and executed by a hardware-based processing unit. Data storage media may be any available media that can be accessed by one or more computers or one or more processing circuits to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure.

By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, cache memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.

Functionality described in this disclosure may be performed by fixed function and/or programmable processing circuitry. For instance, instructions may be executed by fixed function and/or programmable processing circuitry. Such processing circuitry may include one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Processing circuits may be coupled to other components in various ways. For example, a processing circuit may be coupled to other components via an internal device interconnect, a wired or wireless network connection, or another communication medium.

In this disclosure, ordinal terms such as "first," "second," "third," and so on, are not necessarily indicators of positions within an order, but rather may simply be used to distinguish different instances of the same thing. Examples provided in this disclosure may be used together, separately, or in various combinations.

Claim 1:
A method comprising:
measuring one or more periauricular muscle signals from one or more periauricular muscles of a user with a set of one or more electrodes (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) placed at least partially inside a concha of the user;
computing, based on the periauricular muscle signals, an estimate of an angle corresponding to a direction of a current auditory attention locus of the user with respect to a reference point or plane; and
controlling, based on the estimate of the angle, an operating characteristic of a sound processing system (<NUM>, <NUM>).