Patent Description:
Hearing loss is a type of sensory impairment that is generally of two types, namely conductive and/or sensorineural. Conductive hearing loss occurs when the normal mechanical pathways of the outer and/or middle ear are impeded, for example, by damage to the ossicular chain or ear canal. Sensorineural hearing loss occurs when there is damage to the inner ear, or to the nerve pathways from the inner ear to the brain.

Individuals who suffer from conductive hearing loss typically have some form of residual hearing because the hair cells in the cochlea are undamaged. As such, individuals suffering from conductive hearing loss typically receive an auditory prosthesis that generates motion of the cochlea fluid. Such auditory prostheses include, for example, acoustic hearing aids, bone conduction devices, and direct acoustic stimulators.

In many people who are profoundly deaf, however, the reason for their deafness is sensorineural hearing loss. Those suffering from some forms of sensorineural hearing loss are unable to derive suitable benefit from auditory prostheses that generate mechanical motion of the cochlea fluid. Such individuals can benefit from implantable auditory prostheses that stimulate nerve cells of the recipient's auditory system in other ways (e.g., electrical, optical and the like). Cochlear implants are often proposed when the sensorineural hearing loss is due to the absence or destruction of the cochlea hair cells, which transduce acoustic signals into nerve impulses. An auditory brainstem stimulator is another type of stimulating auditory prosthesis that might also be proposed when a recipient experiences sensorineural hearing loss due to damage to the auditory nerve.

<CIT> discloses an eyewear device comprising four microphones including left and right reference microphones. The signals from the four microphones are combined in a monophonic beamformer output. Left and right output signals are generated by combining the amplitude of the beamformer output to the respective phase signals of the left and right reference microphones.

In one aspect, a method according to claim <NUM> is provided.

In another aspect, an auditory prosthesis according to claim <NUM> is provided.

Embodiments of the present invention are described herein in conjunction with the accompanying drawings, in which:.

Presented herein are techniques for generating a combinatory microphone signal from sounds captured at a microphone array. More specifically, sound signals captured by a microphone array are used to generate first and second directional signals. A cross-power signal is computed from the first and second directional signals. The cross-power signal is converted into an amplitude domain output signal, and a phase of the amplitude domain output signal is reconstructed in order to generate a combinatory microphone signal, which that has at least one area of broad-side sensitivity and is useable for subsequent sound processing operations.

Merely for ease of description, the combinatory microphone techniques presented herein are primarily described herein with reference to one illustrative implantable auditory/hearing prosthesis, namely a cochlear implant. However, it is to be appreciated that the combinatory microphone techniques presented herein may also be used with a variety of other types of devices, including other auditory prostheses. For example, the techniques presented herein may be implemented in, for example, acoustic hearing aids, auditory brainstem stimulators, bone conduction devices, middle ear auditory prostheses, direct acoustic stimulators, bimodal auditory prosthesis, bilateral auditory prosthesis, etc. The combinatory microphone techniques presented herein may also be executed in any other device that includes a plurality of microphones (e.g., laptops, mobile phones, headsets, etc.). As such, description of the invention with reference to a cochlear implant should not be interpreted as a limitation of the scope of the techniques presented herein.

<FIG> is a schematic diagram of an exemplary cochlear implant <NUM> configured to implement aspects of the combinatory microphone techniques presented herein, while <FIG> is a block diagram of the cochlear implant <NUM>. For ease of illustration, <FIG> and <FIG> will be described together.

The cochlear implant <NUM> comprises an external component <NUM> and an internal/implantable component <NUM>. The external component <NUM> is directly or indirectly attached to the body of the recipient and typically comprises an external coil <NUM> and, generally, a magnet (not shown in <FIG>) fixed relative to the external coil <NUM>. The external component <NUM> also comprises one or more input elements/devices <NUM> for receiving input signals at a sound processing unit <NUM>. In this example, the one or more input devices <NUM> include a plurality of microphones <NUM> (e.g., microphones positioned by auricle <NUM> of the recipient, telecoils, etc.) configured to capture/receive input acoustic/sound signals (sounds), one or more auxiliary input devices <NUM> (e.g., a telecoil, one or more audio ports, such as a Direct Audio Input (DAI), a data port, such as a Universal Serial Bus (USB) port, cable port, etc.), and a wireless transmitter/receiver (transceiver) <NUM>, each located in, on, or near the sound processing unit <NUM>.

In certain examples, the microphones <NUM> are referred to as "closely-spaced" microphones, meaning that the microphones are generally separated by less than <NUM> centimeters (cm). In further examples, the microphones <NUM> are referred to as "very closely-spaced" microphones, meaning that the microphones are generally separated by less than <NUM>. Auditory prostheses, in particular, have very closely-spaced microphones due to, for example, manufacturing constraints, the need to make the prostheses as small and unobtrusive as possible, need to be positioned on the head of a recipient, etc..

The sound processing unit <NUM> also includes, for example, at least one battery <NUM>, a radio-frequency (RF) transceiver <NUM>, and a processing block <NUM>. The processing block <NUM> comprises a number of elements, including a directional pre-processing module <NUM>, a combinatory processing module <NUM>, and a sound processing module <NUM>. Each of the directional pre-processing module <NUM>, the combinatory processing module <NUM>, and the sound processing module <NUM> may be formed by one or more processors (e.g., one or more Digital Signal Processors (DSPs), one or more uC cores, etc.), firmware, software, etc. arranged to perform operations described herein. That is, the directional pre-processing module <NUM>, the combinatory processing module <NUM>, and the sound processing module <NUM> may each be implemented as firmware elements, partially or fully implemented with digital logic gates in one or more application-specific integrated circuits (ASICs), partially or fully in software, etc..

As described further below, the combinatory processing module <NUM> is configured to generate a combinatory microphone signal based on the sounds captured by the plurality of microphones <NUM>. More particularly, as described further below, the directional pre-processing module <NUM> generates two directional microphone signals from the captured sounds. These two directional signals are then processed by the combinatory processing module <NUM>, as described further below, to generate the combinatory microphone signal.

Returning to the example embodiment of <FIG> and <FIG>, the implantable component <NUM> comprises an implant body (main module) <NUM>, a lead region <NUM>, and an intra-cochlear stimulating assembly <NUM>, all configured to be implanted under the skin/tissue (tissue) <NUM> of the recipient. The implant body <NUM> generally comprises a hermetically-sealed housing <NUM> in which RF interface circuitry <NUM> and a stimulator unit <NUM> are disposed. The implant body <NUM> also includes an internal/implantable coil <NUM> that is generally external to the housing <NUM>, but which is connected to the RF interface circuitry <NUM> via a hermetic feedthrough (not shown in <FIG>).

As noted, stimulating assembly <NUM> is configured to be at least partially implanted in the recipient's cochlea <NUM>. Stimulating assembly <NUM> includes a plurality of longitudinally spaced intra-cochlear electrical stimulating contacts (electrodes) <NUM> that collectively form a contact or electrode array <NUM> for delivery of electrical stimulation (current) to the recipient's cochlea. Stimulating assembly <NUM> extends through an opening in the recipient's cochlea (e.g., cochleostomy, the round window, etc.) and has a proximal end connected to stimulator unit <NUM> via lead region <NUM> and a hermetic feedthrough (not shown in <FIG>). Lead region <NUM> includes a plurality of conductors (wires) that electrically couple the electrodes <NUM> to the stimulator unit <NUM>.

As noted, the cochlear implant <NUM> includes the external coil <NUM> and the implantable coil <NUM>. The coils <NUM> and <NUM> are typically wire antenna coils each comprised of multiple turns of electrically insulated single-strand or multi-strand platinum or gold wire. Generally, a magnet is fixed relative to each of the external coil <NUM> and the implantable coil <NUM>. The magnets fixed relative to the external coil <NUM> and the implantable coil <NUM> facilitate the operational alignment of the external coil with the implantable coil. This operational alignment of the coils <NUM> and <NUM> enables the external component <NUM> to transmit data, as well as possibly power, to the implantable component <NUM> via a closely-coupled wireless link formed between the external coil <NUM> with the implantable coil <NUM>. In certain examples, the closely-coupled wireless link is a radio frequency (RF) link. However, various other types of energy transfer, such as infrared (IR), electromagnetic, capacitive and inductive transfer, may be used to transfer the power and/or data from an external component to an implantable component and, as such, <FIG> illustrates only one example arrangement.

As noted above, the processing block <NUM> includes sound processing module <NUM>. The sound processing module <NUM> is configured to, in general, convert input audio signals into stimulation control signals <NUM> for use in stimulating a first ear of a recipient (i.e., the sound processing module <NUM> is configured to perform sound processing on input audio signals received at the one or more input devices <NUM>). Stated differently, the sound processing module <NUM> (e.g., one or more processing elements implementing firmware, software, etc.) is configured to convert the captured input audio signals into stimulation control signals <NUM> that represent electrical stimulation for delivery to the recipient. The input audio signals that are processed and converted into stimulation control signals may be audio signals received via the sound input devices <NUM> and, as described further below, pre-processed by the directional pre-processing module <NUM> and the combinatory processing module <NUM>.

In the embodiment of <FIG>, the stimulation control signals <NUM> are provided to the RF transceiver <NUM>, which transcutaneously transfers the stimulation control signals <NUM> (e.g., in an encoded manner) to the implantable component <NUM> via external coil <NUM> and implantable coil <NUM>. That is, the stimulation control signals <NUM> are received at the RF interface circuitry <NUM> via implantable coil <NUM> and provided to the stimulator unit <NUM>. The stimulator unit <NUM> is configured to utilize the stimulation control signals <NUM> to generate electrical stimulation signals (e.g., current signals) for delivery to the recipient's cochlea via one or more stimulating contacts <NUM>. In this way, cochlear implant <NUM> electrically stimulates the recipient's auditory nerve cells, bypassing absent or defective hair cells that normally transduce acoustic vibrations into neural activity, in a manner that causes the recipient to perceive one or more components of the input audio signals.

<FIG> and <FIG> illustrate an arrangement in which the cochlear implant <NUM> includes an external component. However, it is to be appreciated that embodiments of the present invention may be implemented in cochlear implants having alternative arrangements. For example, elements of the sound processing unit <NUM> (e.g., such as the processing block <NUM>, power source <NUM>, etc.), may be implanted in the recipient.

Directional microphone systems/arrays are formed by a plurality of individual microphones (e.g., omni-directional microphones) where the sounds detected by the each of the individual microphones are combined through digital signal processing (or historically through analogue or physical combination). In general, there are two conventional classes of directional microphone systems, namely additive microphone systems and differential microphone systems. Proposed herein is a new class of directional microphone systems, referred to as a "combinatory" microphone system in which two directional signals are used to generate a combinatory microphone signal that has features of both input directional signals. One implementation of a combinatory directional microphone is able to produce primary or supplemental "off-axis" or "broad-side" directionality/sensitivity. As used herein, off-axis or broad-side sensitivity refers to a pickup pattern that captures sound signals received at one or more angles relative to the microphone axis (i.e., the line along which the plurality of microphones are positioned). Typically, delay-and-sum structure directional microphones are sensitive to the end-fire direction, not the broad-side direction.

Additive microphone systems synchronize and add the microphone array sensor outputs. It is broadly understood for acoustic signals that additive microphone systems are a collective for all the directional microphone arrays with large inter-element spacing and optimal gain in broadside direction (orthogonal to the microphone array axis, in the case of linear arrays). In differential microphone systems, one signal received at a first microphone is subtracted from the signal received at a second microphone to exploit time differences between the signals. It is broadly understood that differential directional microphone systems are a collective for all the directional microphone arrays which small inter-element spacing and have optimal gain in the end-fire direction (in the direction of the microphone array axis, in the case of linear arrays).

The distinction between additive (broadside) and differential (end-fire) directional microphone systems is determined by whether the acoustic wavelength, λ, is smaller than the distance between microphones, δ (i.e., whether λ <δ). As noted, many devices are small and require microphones to be located close to each other (i.e., closely-spaced or very closely-spaced microphones). As noted, auditory prostheses (e.g., hearing aids, bone conduction devices, cochlear implants, etc.) in particular, generally use very closely-spaced microphones, while a range of other devices such as, mobile phones, wireless streaming devices, recording devices, etc., may also use closely-spaced or very closely-spaced microphones.

Acoustic signals have a wide range of useful frequencies for human listening. The widest limits of these are assumed to be between <NUM> Hertz (Hz) and <NUM> kilohertz (kHz). The range of acoustic frequencies particularly useful in small devices is usually more limited than this, in the range of <NUM> to <NUM> and particularly frequencies around <NUM>. Frequency must be considered to understand the differential microphone distance. Equation <NUM>, below, describes the general understanding of close spaced microphones when considering frequency. <MAT> where f is the frequency of the signal (inverse of the wavelength λ), δ is the distance between the microphones in meters, and c is the speed of sound.

The simplest directional microphone systems have two (<NUM>) omnidirectional microphones, where a noisy signal is received at both microphones. For a speech signal (x) and a noise signal (n), the noisy speech signal under additive assumptions is given as shown below in Equation <NUM>. <MAT> where t is the time and i is the microphone index.

For close spaced microphones (less than approximately <NUM>), a range of first order directional microphone shapes are possible. In the time domain, standard first-order differential polar patterns can be calculated through real-time windowed delay and subtract methods. For instance, forward-facing cardoid, rear facing cardioid, super cardioid, hyper cardioid, and figure-<NUM> patterns can be created. The general first order (FO) differential delay and subtract is described as shown below in Equation <NUM>. <MAT> where d<NUM> and d<NUM> are electrical delays of a signal for each of the two microphones. For a signal coming from the direction of the microphone axis, the time delay between a signal from one microphone to the second microphone can be determined from Equation <NUM>, below.

<FIG> is a general block of a first-order directional microphone system (with two microphones). By changing the delay of the two delays (d<NUM> and d<NUM>), the full array of first-order directional microphone patterns can be formed, as shown below in Table <NUM>. Common directional microphone polar responses are also depicted in <FIG>.

The omni-directional microphone pattern can be achieved with one single microphone. The first order directional microphone patterns of cardioid, super cardioid, hyper cardioid and figure-<NUM> (bidirectional) are achieved with the differential arrangement.

A problem with conventional directional microphone systems is that they have the direction of greatest sensitivity in the direction of the microphone axis (e.g., either forwards or backwards on close spaced unilateral systems, in the direction of the axis on which the Stated differently, these conventional directional microphone systems are unable to have the direction of greatest sensitivity in a different direction, such as orthogonal to the microphone axis, without the aid of a separate/remote microphone placed some distance from the directional microphone system (e.g., on the other ear). Off-axis sensitivity for a directional microphone system, without the requirement for a remote microphone, may be advantageous in a number of different devices.

As such, presented herein are microphone processing techniques, referred to as combinatory microphone techniques or a combinatory microphone system, in which two directional signals are used to generate a combinatory microphone signal that has off-axis sensitivity for sound signals detected by microphone array. More specifically, in accordance with the combinatory microphone techniques presented herein, a plurality of microphones forming a microphone array each capture sound signals. The plurality of microphones each output a corresponding microphone signal and these microphone signals are combined through a spectral cross correlation process. After application of a Fourier transform to each microphone signal, the microphone signals can each be expressed in the frequency domain as shown below in Equation <NUM>. <MAT> where k is the frame index and ω = 2πl/L and where <NUM> = <NUM>,<NUM>,<NUM>. L-<NUM> and L is the frame length.

To create a combinatory directional microphone signal in accordance with the techniques presented herein, the element wise cross power spectrum (power spectrum density) of the directional microphone signals, in the frequency domain, is computed as shown below in Equation <NUM>.

To recreate the time domain signal, the cross power signal is converted back to an amplitude/magnitude signal (e.g., via application of the square root which under some circumstances has the property of being real, or with the use of the absolute function, which results in a phase symmetric for both the left and right directional microphone signals). This results in an amplitude combinatory directional microphone signal. Then, in certain embodiments, an inverse Fourier transform may be applied to the combinatory directional microphone signal ϕ into an amplitude signal in the time domain. The combination of these processes is shown below in Equation <NUM>.

<FIG> generally illustrates a portion of a device <NUM>(A) configured to generate a combinatory directional microphone signal from any two first order microphone signals, FO1 and FO2. Device <NUM>(A) may be, for example, an auditory prosthesis (e.g., cochlear implant, hearing aid, bone conduction device, etc.), a mobile phone, a laptop, a headset, headphones etc..

<FIG> illustrates a microphone array <NUM> that comprises a first microphone <NUM>(<NUM>) and second microphone <NUM>(<NUM>). The microphones <NUM>(<NUM>) and <NUM>(<NUM>) are disposed along a microphone array axis <NUM>, and separated by a distance, δ. In an auditory prosthesis worn on the head of a recipient, the microphone array axis <NUM> is generally parallel to the side of the recipient's head and the first microphone <NUM>(<NUM>) is located relatively closer to the front of the head of the recipient, while the second microphone <NUM>(<NUM>) is located closer to the back of the head of the recipient.

The first microphone <NUM>(<NUM>) generates a first microphone signal <NUM>(<NUM>), y1, while the second microphone <NUM>(<NUM>) generates a second microphone signal <NUM>(<NUM>), y2. As shown, the device <NUM>(A) includes a directional pre-processing module <NUM> that is configured to implement windowed delay and subtract methods (e.g., in accordance with Equations <NUM> above) to create two first order directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>), referred to as yF01 and yF02, respectively, from the first and second microphone signals <NUM>(<NUM>) an <NUM>(<NUM>). Directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) are in the time domain, similar to the sound captured by the microphone <NUM> and the microphone signals <NUM>(<NUM>) and <NUM>(<NUM>).

The device <NUM>(A) also comprises a combinatory processing module <NUM>(A) configured to implement aspects of the combinatory microphone techniques presented herein. As shown in <FIG>, the two first order directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) (i.e., yF01 and yF02) are provided to the combinatory processing module <NUM>(A) and, as described further below, used to generate a combinatory directional microphone signal <NUM>.

The two first order directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) are amplitude signals in the time domain. At blocks <NUM>(<NUM>) and <NUM>(<NUM>), respectively, a Fourier transform (e.g., a fast Fourier transform (FFT), short-time Fourier transform (STFT), discrete Fourier transform (DFT), other frequency domain type transforms etc.) is applied to each of the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>), where directional microphone signals can be expressed in the frequency domain as shown above Equation <NUM>. In general, the Fourier transform blocks <NUM>(<NUM>) and <NUM>(<NUM>) are understood as the buffering, windowing, and a Fourier transform process that separates the signals into a plurality of frequency components.

The frequency domain versions of the directional microphone signal <NUM>(<NUM>) and the directional microphone signal <NUM>(<NUM>) are referred to as frequency domain directional microphone signal <NUM>(<NUM>) (YF01) and frequency domain directional microphone signal <NUM>(<NUM>) (YF02), respectively. Frequency domain directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) are representations of an amplitude signal, but in the frequency domain and include imaginary parts. In other words, at processing block <NUM>(<NUM>), a plurality of frequency components associated with a first directional signal (i.e., directional microphone signal <NUM>(<NUM>)) are determined and, at processing block <NUM>(<NUM>), a plurality of frequency components associated with a second directional signal (i.e., directional microphone signal <NUM>(<NUM>)) are determined.

At processing block <NUM>, an element wise multiplication is performed to determine the cross-power spectrum <NUM> (i.e., a cross-power signal) of the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) (e.g., as described above with reference to in Equation <NUM>). That is, at <NUM>, a plurality of frequency components associated with a first directional signal (i.e., directional microphone signal <NUM>(<NUM>)) are multiplied with a plurality of frequency components associated with a second directional signal (i.e., directional microphone signal <NUM>(<NUM>)).

In the example of <FIG>, the cross-power signal <NUM> is converted to an amplitude or magnitude signal via processing blocks <NUM> and <NUM>. More specifically, at processing block <NUM>, a square root of the cross-power signal <NUM> is calculated to generate an intermediate signal <NUM> (which in some cases is used as the combinatory directional microphone signal <NUM> without the need of magnitude and phase block <NUM> as it contains amplitude information, as shown in <FIG>). At <NUM>, the absolute operator is used to determine the magnitude of the signal. <FIG> illustrates an example in which any imaginary parts of the intermediate signal <NUM> are removed by calculating an absolute value of the intermediate signal <NUM>. However, in alternative embodiments, any imaginary parts of the intermediate signal <NUM> may be removed by computing the real part of the intermediate signal <NUM>. Regardless of the specific procedure, the result is conversion of the intermediate signal <NUM> into a magnitude or amplitude domain signal <NUM>.

The computation of the cross-power signal <NUM> (element wise cross-power spectrum) results in a loss of meaningful phase information from the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>). Therefore, at processing block <NUM>, the phase of the amplitude domain signal <NUM> is reconstructed from (based on) a phase of, for example, one or more of the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>). The resulting signal, Φ, is the combinatory directional microphone signal <NUM> (i.e., amplitude domain output signal with the reconstructed phase), sometimes referred to herein as combinatory signal <NUM>.

To obtain the phase information, the combinatory processing module <NUM>(A) includes phase extraction blocks <NUM>(<NUM>) and <NUM>(<NUM>). The phase extraction blocks <NUM>(<NUM>) and <NUM>(<NUM>) receive the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>), respectively, and extract phase information therefrom. This phase information, sometimes referred to herein as a phase signal, extracted from directional microphone signal <NUM>(<NUM>) is represented in <FIG> by arrow <NUM>(<NUM>), while the phase information extracted from directional microphone signal <NUM>(<NUM>) is represented in <FIG> by arrow <NUM>(<NUM>). Although <FIG> illustrates the extraction of the phase information from both of the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>), it is to be appreciated that other embodiments may only extract and use the phase from one of the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>).

In certain examples, the processing block <NUM>, or another element, is configured to generate a longer term amplitude estimate of the sound signals received at the microphone array <NUM>. The processing block <NUM> is configured to adjust a shorter term power signal of combinatory directional microphone signal <NUM> so as to approximate the longer term amplitude estimate of the sound signals. More specifically, the cross-power spectrum (power signal) <NUM> does not have natural growth, in that, for example, it gets <NUM> dB softer for every <NUM> dB decrease in the actual sound environment level. If the sound signals indicate that the environment level is, say, around <NUM> dB, then it may be desirable to match the power CDM to this level so it is about the same level. If the environmental signal changes to <NUM> dB (<NUM> dB louder) then the power CDM will be at <NUM> dB (<NUM> dB louder). Although it may not be desirable to change the short term amplitude and undo the pattern (e.g., cardioid), it may be desirable to change the longer term amplitude to match the listening level to the environment. In the second case, the system may turn the signal down by <NUM> dB, slowly (maybe over seconds), to match the longer term environmental loudness.

In certain examples, the phase information from directional microphone signal <NUM>(<NUM>) may be used to reconstruct the phase of the of the amplitude domain signal <NUM>. In other embodiments, the phase information from directional microphone signal <NUM>(<NUM>) may be used to reconstruct the phase of the of the amplitude domain signal <NUM>. In still other embodiments, block <NUM> may be configured to use the phase information from both of the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>). For example, in one embodiment, block <NUM> may be configured to compute a mean of the phase information extracted from the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>). In another example, block <NUM> may be configured to compute the weighted mean (by vector magnitude for example) of the phase information extracted from the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>).

Returning to the specific example of <FIG>, the device <NUM>(A) also includes an inverse Fourier transform block <NUM>. That is, at block <NUM>, an inverse Fourier transform (e.g., inverse fast Fourier transform (IFFT), inverse discrete Fourier transform (IDFT), inverse short time Fourier transform (ISTFT), or other conversion from the frequency to the time domain transform) is applied to the combinatory signal <NUM> in order to convert the combinatory signal <NUM> into the time domain. At the output of the inverse Fourier transform block <NUM>, the combinatory signal <NUM> is referred to as time-domain amplitude combinatory signal <NUM>.

<FIG> also illustrates that device <NUM>(A) includes a time domain frequency filter (HL) <NUM>. In this embodiment, the frequency filter <NUM> provides frequency specific gain to the signal to make the time-domain combinatory signal <NUM> flat across frequency, in the case of the operations in the combinatory processing module <NUM>(A) amplifies the low frequencies for instance. Stated differently, the frequency filter <NUM> is provided to compensate for unintended frequency shaping introduced in the directional microphone signals (e.g., aim to make it the output flat, or to have a specific frequency shape). The frequency filter <NUM> also has a second purpose in that it would also apply a high pass function at a certain frequency to remove any aliasing from insufficient FFT length, as described further below. At the output of the frequency filter <NUM>, the time-domain combinatory signal is referred to as frequency-adjusted combinatory signal <NUM>.

As noted above, in order to ensure that combinatory directional microphone signal includes minimal audible distortion, the phase information can be reconstructed. That is, the signal amplitude signal is computed from the cross-power spectrum, but this computation introduces phase distortions that need to be addressed. In the example of <FIG>, the phase information is reconstructed from one or both of the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>).

According to an example not falling within the scope of the claims, <FIG> illustrates a device <NUM>(B) (e.g., auditory prosthesis, mobile phone, laptop, headset, headphones etc.) that, similar to device <NUM>(A) of <FIG>, comprises microphone array <NUM>, directional pre-processing module <NUM>, inverse Fourier transform block <NUM>, and frequency filter <NUM>. Device <NUM>(B) also comprises a combinatory processing module <NUM>(B) which is similar to combinatory processing module <NUM>(A), except that, in <FIG>, the phase information is reconstructed from the first microphone signal <NUM>(<NUM>) (i.e., y1, the front omnidirectional microphone signal). To this end, the combinatory processing module <NUM>(B) comprises, in addition to the elements described with reference to <FIG>, an additional Fourier transform block <NUM>. The additional Fourier transform block <NUM> receives the first microphone signal <NUM>(<NUM>) and applies a Fourier transform (e.g., a fast Fourier transform (FFT), short-time Fourier transform (STFT), discrete Fourier transform (DFT), etc.) thereto. The frequency domain version of the first microphone signal <NUM>(<NUM>) is referred to as frequency domain front microphone signal <NUM> (Yomni or Y1).

To obtain the phase information, the combinatory processing module <NUM>(B) includes a phase extraction block <NUM>. The phase extraction block <NUM> receives the frequency domain front microphone signal <NUM> and extracts phase information therefrom. This phase information, sometimes referred to herein as a phase signal, extracted from the frequency domain front microphone signal <NUM> is represented in <FIG> by arrow <NUM>. Although <FIG> illustrates the reconstruction of the phase information from only the first microphone signal <NUM>(<NUM>), it is to be appreciated that other embodiments may also or alternatively obtain the phase from second microphone signal <NUM>(<NUM>), in a similar or different manner.

In an example not falling within the scope of the claims, the phase reconstruction could be admitted (e.g., directly use the output of the square root). <FIG> illustrates one such an arrangement. In <FIG>, a device <NUM>(C) (e.g., auditory prosthesis, mobile phone, laptop, headset, headphones etc.) is similar to device <NUM>(A) of <FIG> and comprises microphone array <NUM>, directional pre-processing module <NUM>, inverse Fourier transform block <NUM>, and frequency filter <NUM>. Device <NUM>(C) also comprises a combinatory processing module <NUM>(C) which is similar to combinatory processing module <NUM>(A), except that, in <FIG>, blocks <NUM> and <NUM>, as well as the phase reconstruction are all omitted. To this end, the output of block <NUM>, intermediate signal <NUM>, is provided to the IFFT <NUM> for subsequent processing.

There are a number of unique attributes of the combinatory microphone techniques presented herein. For example, in certain embodiments, the microphones (e.g., microphones <NUM>(<NUM>) and <NUM>(<NUM>)) are temporally symmetrical. This means that the Front - Rear delay is the inverse of the Rear - Front delay. Additionally, the Fourier transform window (e.g., the length of FFTs <NUM>(<NUM>) and <NUM>(<NUM>)) needs to be of sufficient length to provide sufficient frequency resolution so that lower frequencies are not amplitude modulated due to a phase shift. For example, FFT lengths of <NUM> may operate with frequencies right down to as low as, for example, <NUM>. In another example, FFT lengths of <NUM> may operate with lower frequencies right down to as low as, for example, <NUM>. As such, the Fourier transform window provides sufficient spectral resolution to mitigate any modulation and aliasing problems. Additionally, low frequency FFT bins (channels) which are expected to have aliasing can be dealt with in a number of ways. One way is to produce a high-pass filter to remove the aliased frequencies, and combine this with a low pass signal not processed by the combinatory processing. Another way is to apply the combinatory processing to FFT bins above a certain point, and not to process the low frequency <NUM>, <NUM>, <NUM>, <NUM>, or <NUM> frequency bins, for instance.

As noted, <FIG> and <FIG> generally illustrate arrangements in which directional pre-processing module <NUM> generates two first order directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) that are processed by the combinatory processing modules <NUM>(A) and <NUM>(B). However, it is to be appreciated that any directional microphone signal, such omnidirectional microphones, second order directional microphone signals, third order directional microphones, or even a combinatory directional microphone signal could be input into a combinatory processing module (i.e., processed in accordance with the combinatory microphone techniques presented herein). <FIG> and <FIG> are polar plots illustrating the results of processing of different combinations of directional microphone signals using the combinatory microphone techniques presented herein.

In the examples of <FIG> and <FIG>, the various illustrated microphone patterns are generated from microphones forming part of a microphone array, where the microphones are disposed on a microphone axis. In each polar plot of <FIG> and <FIG>, the microphone axis connects the zero (<NUM>) degree and the one hundred and eight (<NUM>) degree points, where the <NUM> degree point is defined as the front (e.g., the front of the head of the recipient, the direction the recipient is looking, etc.) and the <NUM> degree point is defined as the back (e.g., the back of the head of the recipient, the direction that is directly opposite to the direction the recipient is looking, etc.).

Referring first to <FIG>, shown is a polar plot illustrating a polar pattern <NUM> associated with a combinatory directional microphone signal in accordance with certain embodiments presented herein. As such, the polar pattern <NUM> is referred to as a combinatory directional microphone pattern <NUM>, and illustrates the directionality or pickup pattern of the associated combinatory directional microphone signal generated as described above (e.g., in a combinatory processing module, such as combinatory processing modules <NUM>(A) or <NUM>(B)).

In the example of <FIG>, the combinatory directional microphone pattern <NUM> is generated, using the techniques described above with reference to <FIG>, from a front-facing cardioid microphone signal and a rear-facing cardioid microphone signal. The front-facing cardioid microphone signal and the rear-facing cardioid microphone signal may each be generated, for example, at a directional pre-processing module (e.g., module <NUM>).

As shown in <FIG>, the polar pattern associated with the front-facing cardioid microphone signal, referred to as front cardioid pattern <NUM>, has a greatest sensitivity to the front of the recipient (i.e., at <NUM> degrees). Also as shown in <FIG>, the polar pattern associated with the rear-facing cardioid microphone signal, referred to rear cardioid pattern <NUM>, has a greatest sensitivity to the rear of the recipient (i.e., at <NUM> degrees). Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

In the example of <FIG>, use of the front-facing cardioid microphone signal and the rear-facing cardioid microphone signal to generate the combinatory directional microphone signal results in the combinatory directional microphone pattern <NUM> that has greatest sensitivity in opposing directions that are each substantially orthogonal to the microphone axis. That is, in <FIG>, the combinatory directional microphone pattern <NUM> has greatest sensitivity at approximately ninety (<NUM>) degrees and approximately two-hundred and seventy (<NUM>) degrees. Stated differently, the combinatory directional microphone pattern <NUM> is a figure-infinity directionality pattern that is oriented substantially orthogonal to the microphone axis. As used herein, a "figure-infinity" pattern has the same general shape as a figure-<NUM> directional microphone pattern, but with sensitivity to <NUM> and <NUM> degrees instead of <NUM> and <NUM> (hence the name figure-infinity, given the shape of the infinity symbol in contrast the number <NUM>). In a cardioid analysis, it can be seen that the combinatory processing, in generating pattern <NUM>, in effect, visually splits the single null of the front and rear cardioid patterns.

Referring next to <FIG>, shown is a polar plot illustrating a polar pattern <NUM> associated with a combinatory directional microphone signal in accordance with certain embodiments presented herein. As such, the polar pattern <NUM> is referred to as a combinatory directional microphone pattern <NUM>, and illustrates the directionality or pickup pattern of the associated combinatory directional microphone signal generated as described above (e.g., in a combinatory processing module, such as combinatory processing modules <NUM>(A) or <NUM>(B)).

In the example of <FIG>, the combinatory directional microphone pattern <NUM> is generated, using the techniques described above with reference to <FIG>, from a front-facing cardioid microphone signal and a figure-<NUM> (bidirectional) microphone signal. The front-facing cardioid microphone signal and the figure-<NUM> microphone signal may each be generated, for example, at a directional pre-processing module (e.g., module <NUM>).

As shown in <FIG>, the polar pattern associated with the front-facing cardioid microphone signal, referred to as front cardioid pattern <NUM>, has a greatest sensitivity to the front of the recipient (i.e., at <NUM> degrees). Also as shown in <FIG>, the polar pattern associated with the figure-<NUM> microphone signal, referred to as figure-<NUM> or bidirectional pattern <NUM>, has dual-sensitivity (i.e., to the front and back) along the microphone axis. Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

In the example of <FIG>, use of the front-facing cardioid microphone signal and the figure-<NUM> microphone signal to generate the combinatory directional microphone signal results in the combinatory directional microphone pattern <NUM> that has a forward-facing microphone with two rear lobes. In a cardioid analysis, it can be seen that the pattern <NUM> is splitting the single null between two different locations, at <NUM> degrees and at <NUM> degrees. The effect is therefore analogous to sharing the null ability in a first order system between two locations through the process.

In the example of <FIG>, the combinatory directional microphone pattern <NUM> is generated, using the techniques described above with reference to <FIG>, from a front-facing cardioid microphone signal and a super cardioid microphone signal. The front-facing cardioid microphone signal and the super cardioid microphone signal may each be generated, for example, at a directional pre-processing module (e.g., module <NUM>).

As shown in <FIG>, the polar pattern associated with the front-facing cardioid microphone signal, referred to as front cardioid pattern <NUM>, has a greatest sensitivity to the front of the recipient (i.e., at <NUM> degrees). Also as shown in <FIG>, the polar pattern associated with the super cardioid microphone signal, referred to as super cardioid pattern <NUM>, has is similar to the front-facing cardioid, but with a figure-<NUM> contribution, leading to a tighter area of front sensitivity (i.e., at <NUM> degrees) and a small lobe of rear sensitivity (e.g., at <NUM> degrees). Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

In the example of <FIG>, use of the front-facing cardioid microphone signal and the super cardioid microphone signal to generate the combinatory directional microphone signal results in the combinatory directional microphone pattern <NUM> that has a forward-facing sensitivity with a dual-rear sensitivity. That is, as shown, the combinatory directional microphone pattern <NUM> has similar front directionality to the super cardioid pattern <NUM>, but two rear lobes that are each offset from the microphone axis (i.e., as opposed to a single rear lobe in the super cardioid pattern). This pattern <NUM> could, for example, provide superior directional properties compared to standard first order directional microphones.

In the example of <FIG>, the combinatory directional microphone pattern <NUM> is generated, using the techniques described above with reference to <FIG>, from a front-facing cardioid microphone signal and a hyper cardioid microphone signal. The front-facing cardioid microphone signal and the hyper cardioid microphone signal may each be generated, for example, at a directional pre-processing module (e.g., module <NUM>).

As shown in <FIG>, the polar pattern associated with the front-facing cardioid microphone signal, referred to as front cardioid pattern <NUM>, has a greatest sensitivity to the front of the recipient (i.e., at <NUM> degrees). Also as shown in <FIG>, the polar pattern associated with the hyper cardioid microphone signal, referred to as hyper cardioid pattern <NUM>, has is similar to the front-facing cardioid, but with a figure-<NUM> contribution, leading to a tighter area of front sensitivity (i.e., at <NUM> degrees) and a small lobe of rear sensitivity (e.g., at <NUM> degrees). Relative to a supercardoid pattern, the hyper cardioid pattern <NUM> has greater rear sensitivity. Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

In the example of <FIG>, use of the front-facing cardioid microphone signal and the hyper cardioid microphone signal to generate the combinatory directional microphone signal results in the combinatory directional microphone pattern <NUM> that has a forward-facing sensitivity with a dual-rear sensitivity. That is, as shown, the combinatory directional microphone pattern <NUM> has similar front directionality to the hyper cardioid pattern <NUM>, but two rear lobes that are each offset from the microphone axis (i.e., as opposed to a single rear lobe in the hyper cardioid pattern). Of note is that, in the rear lobes, there is a <NUM> degree signal change.

In certain aspects presented herein, the combinatory microphone techniques presented herein utilize the polarity change between a directional microphone signal and the square root property making negative numbers into imaginary numbers. Such embodiments create a directional microphone signal for part of the input directionality, and a sigmoidal driven noise cancelation process for the remainder of the input directionality. This results in an aperture-specific sinusoid-driven noise cancelation. A directional input basis decision can be made regarding which signals will be processed on a standard directional microphone basis, and which ones will have the addition of noise cancelation. The process changes the absolute calculation and only makes the real part of the signal, as shown below in Equation <NUM>.

<FIG> generally illustrates a portion of a device <NUM> (e.g., e.g., auditory prosthesis, mobile phone, laptop, headset, headphones etc.) configured to generate a combinatory directional microphone signal in accordance with Equation <NUM>, above. More specifically, device <NUM> is similar to device <NUM>(A) of <FIG>, in that it also comprises microphone array <NUM>, directional pre-processing module <NUM>, inverse Fourier transform block <NUM>, and frequency filter <NUM>. Device <NUM> also comprises a combinatory processing module <NUM>, which is similar to combinatory processing module <NUM>(A), except that, in <FIG>, the processing block <NUM> (i.e., the absolute value block that calculates an absolute value of the intermediate signal <NUM>), is replaced by processing block <NUM>. More specifically, as explained above, a square root of the cross-power signal <NUM> is calculated at processing block <NUM> to generate an intermediate signal <NUM>. In the example of <FIG>, at block <NUM>, any imaginary parts of the intermediate signal <NUM> are removed by computing the real part of the intermediate signal <NUM> (as in Equation <NUM>, above). The result is conversion of the intermediate signal <NUM> into an amplitude domain signal <NUM>.

<FIG> illustrates an example of an aperture-specific, sinusoidal driven, combinatory directional microphone signal generated, in accordance with Equation <NUM> and <FIG>, from a front-facing cardioid and a super cardioid. More specifically, <FIG> illustrates a polar pattern <NUM> associated with a combinatory directional microphone signal in accordance with certain embodiments presented herein. As such, the polar pattern <NUM> is referred to as a combinatory directional microphone pattern <NUM>, and illustrates the directionality or pickup pattern of the associated combinatory directional microphone signal generated as described above (e.g., in a combinatory processing module, such as combinatory processing modules <NUM>(A) or <NUM>(B)).

As noted, in the example of <FIG>, the combinatory directional microphone pattern <NUM> is generated, using the techniques described above with reference to <FIG>, from a front-facing cardioid microphone signal and a super cardioid microphone signal. The front-facing cardioid microphone signal and the super cardioid microphone signal may each be generated, for example, at a directional pre-processing module (e.g., module <NUM>).

As shown in <FIG>, the polar pattern associated with the front-facing cardioid microphone signal is referred to as front cardioid pattern <NUM>, while the polar pattern associated with the super cardioid microphone signal is referred to as super cardioid pattern <NUM>. Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

In the example of <FIG>, use of the front-facing cardioid microphone signal and the super cardioid microphone signal to generate the combinatory directional microphone signal results in the combinatory directional microphone pattern <NUM> that has a forward-facing sensitivity with a minor dual-rear sensitivity. That is, as shown, the combinatory directional microphone pattern <NUM> has similar front directionality to the super cardioid pattern <NUM>, is very forward focused, but from <NUM> degrees to <NUM> degrees the noise cancelation heavily removes any signal. This pattern <NUM> is a hybrid between combinatory directional microphone signals in the front angles, and a form of direction or arrival driven noise cancellation in the rear angles.

<FIG> illustrates another example of an aperture-specific, sinusoidal driven, combinatory directional microphone signal generated, in accordance with Equation <NUM> and <FIG>, from a front-facing cardioid and a hyper cardioid. More specifically, <FIG> illustrates a polar pattern <NUM> associated with a combinatory directional microphone signal in accordance with certain embodiments presented herein. As such, the polar pattern <NUM> is referred to as a combinatory directional microphone pattern <NUM>, and illustrates the directionality or pickup pattern of the associated combinatory directional microphone signal generated as described above (e.g., in a combinatory processing module, such as combinatory processing modules <NUM>(A) or <NUM>(B)).

As noted, in the example of <FIG>, the combinatory directional microphone pattern <NUM> is generated, using the techniques described above with reference to <FIG>, from a front-facing cardioid microphone signal and a hyper cardioid microphone signal. The front-facing cardioid microphone signal and the hyper cardioid microphone signal may each be generated, for example, at a directional pre-processing module (e.g., module <NUM>).

As shown in <FIG>, the polar pattern associated with the front-facing cardioid microphone signal is referred to as front cardioid pattern <NUM>, while the polar pattern associated with the hyper cardioid microphone signal is referred to as hyper cardioid pattern <NUM>. Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

In the example of <FIG>, use of the front-facing cardioid microphone signal and the hyper cardioid microphone signal to generate the combinatory directional microphone signal results in the combinatory directional microphone pattern <NUM> that has a forward-facing sensitivity with a minor dual-rear sensitivity. That is, as shown, the combinatory directional microphone pattern <NUM> has similar front directionality to the hyper cardioid pattern <NUM>, is very forward focused, but from <NUM> degrees to <NUM> degrees the noise cancelation heavily removes any signal. This pattern <NUM> is a hybrid between combinatory directional microphone signals in the front angles, and a form of direction or arrival driven noise cancellation in the rear angles. Relative to the example of <FIG> (i.e., front with super cardioid), the example of <FIG> (i.e., front with hyper cardioid) is more forward directional, with a slight increase in the rear lobes. The aperture of noise cancelation is also wider in this implementation of <FIG>, relative to that of <FIG>.

It should be noted that the aperture specific noise reduction may be determined in other ways and the phase reversal of a signal may also be dealt with in other ways than presented in <FIG>. For instance, the output for positive signals may be processed with an absolute operator, where negative signals may be set to <NUM>.

It is to be appreciated that the techniques presented herein could be used in an iterative process where one or more combinatory directional microphone signals are used at the inputs to the combinatory processing (e.g., as the directional signal inputs to a combinatory processing module). For example, <FIG> illustrates an example in which two combinatory directional microphones could be used to share four quarter nulls over four locations.

More specifically, <FIG> illustrates a polar pattern <NUM> associated with a combinatory directional microphone signal generated, using the techniques described above with reference to <FIG>, from a first order figure-<NUM> microphone signal and a figure-infinity combinatory directional microphone signal. The first order figure-<NUM> microphone signal may be generated, for example, at a directional pre-processing module (e.g., module <NUM>). The figure-infinity combinatory directional microphone signal may generated by a preliminary combinatory processing module (e.g., modules <NUM>(A), <NUM>(B), etc.).

As shown in <FIG>, the polar pattern associated with the figure-<NUM> cardioid microphone signal is referred to as figure-<NUM> cardioid pattern <NUM>, while the polar pattern associated with the figure-infinity combinatory directional microphone signal is referred to as first combinatory directional microphone pattern (figure-infinity combinatory directional pattern) <NUM>. Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

In accordance with certain embodiments presented herein, a very strong directional microphone signal can be created through "power" combinatory processing techniques, s shown below in Equation <NUM>.

In Equation <NUM>, unlike the above examples which have amplitude domain outputs and normal acoustic signal loudness growth and generally have no speech distortion, this class would have a different loudness growth and some similar distortions to noise reduction processing, but would have enhanced directionality. A simple example is a power combinatory directional microphone with inputs as two front-facing cardioids. This gives a cardioid with the same pattern as a second order directional microphone, but with some noise and speech distortion similar to noise reduction.

For example, as shown in <FIG>, a polar pattern associated with the front-facing cardioid microphone signal is referred to as front cardioid pattern <NUM> and a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1). <FIG> also illustrates pattern <NUM> associated with a power combinatory directional microphone signal generated, using the techniques described above with reference to Equation <NUM>.

Additionally <FIG> illustrates a power combinatory microphone signal generated from a front-facing cardioid signal and a hyper cardioid signal, which almost only has signals from the front half. More specifically, shown in <FIG> is a polar pattern <NUM> associated with a combinatory directional microphone signal generated, using the techniques described above with reference to Equation <NUM>, from a front-facing cardioid signal and a hyper cardioid signal. As shown in <FIG>, the polar pattern associated with the front-facing cardioid signal is referred to as front cardioid pattern <NUM>, while the polar pattern associated with the hyper cardioid signal is referred to as hyper cardioid microphone pattern <NUM>. Also shown in <FIG> is a front omnidirectional pattern <NUM>, which is associated with the raw omnidirectional output of the front microphone in a microphone array (e.g., corresponds to y1).

It is important to note that, for power combinatory directional microphones, the phase information will be calculated to minimize any audible distortions. Additionally, it would be expected that a gain control system would be utilized to present short time power combinatory microphone signals at longer time amplitude signal levels.

While magnitude combinatory directional microphones are able to maintain normal signal loudness, and power combinatory directional microphones provide enhanced directionality, a range of implementations between these two are possible. In certain embodiments, a magnitude combinatory directional microphone uses a square root <NUM> to convert the signal into the magnitude domain. A square root is the same as an exponent of a half (<NUM>), and leaving the signal in the power domain is the same as an exponent of one (<NUM>). A range of implementations are possible with functional exponents between, but not including <NUM> and <NUM>, at <NUM> is possible, which would have share characteristics between maintaining normal loudness and enhanced directionality. In fact, even exponents outside this range may be used, such as <NUM>, <NUM>, and <NUM> are possible.

It is to be appreciated that the above polar plots of <FIG> and <FIG> illustrate patterns in accordance with idealized (free-field) conditions (e.g., patterns while the microphones are not in proximity to an objects recipient's head). However, as noted above, the techniques presented herein may be implemented, for example in a hearing prosthesis that is worn on the head of a recipient. As such, in practice, the various polar patterns shown in <FIG> and <FIG> will be affected by the presence of the recipient's head adjacent to the microphones, commonly known as the head-shadow effect. For example, with an auditory prosthesis, the prosthesis (and thus the microphones) may be positioned on, for example, the right side of the recipient's head when in use. In such an example, the microphone polar patterns for the right half (i.e., between <NUM> and <NUM> degrees) will look similar to the idealized patterns shown in <FIG> and <FIG>, but the left half (i.e., between <NUM> and <NUM> degrees) will look quite different. In particular, the polar patterns will, in practice, each, have reduced sensitivity to the spatial regions on the left (opposite) side of the head. The practical effect is that the combinatory processing techniques presented herein increase sensitivity to sounds received on the same side of the head as which the hearing prosthesis is located/worn.

For those skilled in the art, it will be evident these processes can be carried out in the time domain or in the frequency domain. Although the process has been described above for the combinatory directional microphone module <NUM>(a) and <NUM>(b) in the frequency domain, they could similarly be implemented in the time domain. For instance, convolution theorem states that element wise multiplication in the frequency domain (as described in <NUM>(a) and <NUM>(b)) is equivalent to convolution in the time domain. Similarly, element wise multiplication in the frequency domain with the complex conjugate of one signal is the same as cross-correlation in the time domain as described by cross-correlation theorem. It is also intended in this description to describe the use of elementwise multiplication of frequency domain signals being either their frequency domain representation or the complex conjugate of their frequency domain representation, which may have advantageous properties under some circumstances. Similarly, for those skilled in the art, convolution represents a range of convolutions such as linear or circular, and similarly FFT also represents a range of FFT transforms as described including with and without zero padding.

There is a class of first order directional microphones, known as adaptive beamformers, which are able to steer their null depending on the location of the noise. In the same way that adaptive beamformers are able to steer their single null to the direction of the largest noise location, a system using the combinatory microphone techniques presented herein may steer two half nulls from the input directional microphone signals to maximally reduce the noise. For example, shown in <FIG>, shown is a portion of a device <NUM> (e.g., auditory prosthesis, mobile phone, laptop, headset, headphones etc.) that includes a directional pre-processing module <NUM> and a combinatory processing module <NUM>. The combinatory processing module <NUM> may be implemented similar one of the embodiments of <FIG>, <FIG>, or <FIG> and is configured to generate a combinatory directional microphone signal <NUM> from two directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>).

The directional pre-processing module <NUM> is configured to generate the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) for processing by the combinatory processing module <NUM> from microphone signals <NUM> captured by a microphone array (not shown in <FIG>). In the example of <FIG>, the directional pre-processing module <NUM> comprises two adaptive beamformers <NUM>(<NUM>) and <NUM>(<NUM>). As a result of the two adaptive beamformers <NUM>(<NUM>) and <NUM>(<NUM>), which each remove noise from different directions, the directional microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) will point to different noises or areas, so that there is two independent inputs into the combinatory processing module <NUM> (i.e., ensures that the two directional signals are not directed to the same point/target).

While the use of adaptive beamformers and even multiple beamformers as inputs into a combinatory processing module are able to steer the direction of the null, they may not, in certain examples, be able to steer the direction of the most sensitive direction. In the typical close spaced arrangements, the most sensitive direction is at zero (<NUM>) degrees and one hundred and eighty (<NUM>) degrees. A combinatory directional microphone signal producing the figure-infinity signal is not most sensitive to zero (<NUM>) or one hundred and eighty (<NUM>) degrees. With the use of a combinatory directional microphone signal such as that which produces the figure-infinity polar pattern (pattern <NUM> in <FIG>), or generally with sensitivity substantially orthogonal to the microphone axis, a full range of most sensitive listening directions is possible. This can be achieved using a combinatory directional processing module with inputs such as a figure-infinity signal and a forward-facing signal and a forward facing directional microphone signal. This may also be achieved by simple mixing of the two microphone signals. But changing the inputs (where at least one of the inputs is not zero (<NUM>) or one hundred and eighty (<NUM>) degrees sensitive), either into a combinatory processing module or by mixing, the most sensitive direction can be changed. For hearing aids, this provides an adaptive listening direction, which can be steered to any direction.

<FIG> is a flowchart of a method <NUM>, in accordance with an example not falling within the scope of the claims. Method <NUM> begins at <NUM> where a plurality of first frequency components associated with a first directional microphone signal are determined. At <NUM>, a plurality of second frequency components associated with a second directional microphone signal are determined. At <NUM>, the first frequency components are multiplied with the second frequency components to generate a cross-power signal. At <NUM>, the cross-power signal is converted to an amplitude domain to generate an amplitude domain combinatory microphone signal. In certain embodiments, a phase of the amplitude domain combinatory microphone signal may reconstructed from a phase signal.

<FIG> is a flowchart of a method <NUM>, in accordance with certain embodiments presented herein. Method <NUM> begins at <NUM> where sound signals are received at a microphone array comprising first and second microphones positioned along a microphone axis. At <NUM>, first and second directional signals are generated from the sound signals received at the microphone array. At <NUM>, a frequency element wise cross power spectrum of the first and second directional microphone signals is computed in the frequency domain. At <NUM>, a magnitude signal is generated from the frequency element wise cross power spectrum and, at <NUM>, a phase of the magnitude signal is reconstructed to generate a combinatory microphone signal. In certain embodiments, the combinatory microphone signal is associated with a microphone pickup pattern that has at least one area of broad-side sensitivity.

While adaptive beamformers and adaptive listening direction are able to steer their null depending on the noise location or steer their most sensitive direction respectively, both are typically implemented on a single close spaced microphone array. There are other automation systems which use multiple close spaced inputs to determine the systems operations. These systems typically consist of sound feature extraction, environmental classification, and then technology selection. In an automation with multiple inputs, with one being a close spaced array, the system (using at least one close spaced array) determine the type of listening environment or direction of main source and/or determine appropriate technologies to use in that listening environment.

For hearing aids, two hearing aids are often worn and wirelessly share information, creating a multiple close spaced array system with two close spaced arrays (one on each ear). Combinatory microphone signals from one or both close spaced arrays could monitor signals from specific directions. For instance, a figure-infinity combinatory microphone signal may be used to monitor the auditory scene from both sides of the listener. Another example is where a combinatory microphone signal, such as <NUM> in <FIG>, is used to monitor signals from one direction in the left ear, a further combinatory microphone signal, such as <NUM> in <FIG>, is used to monitor signals from another direction in the left ear, and a third combinatory microphone signal, such as signal <NUM>, is used to monitor signals from another direction in the right ear. Any number of directional microphone signals, omnidirectional microphone signals, with the addition of a combinatory directional microphone signal in a system may be used to monitor signals from a range of directions to assess the listening environment. This is a representation of a combinatory directional microphone scene classification system.

For other systems such as cochlear implants, mobile phones and computers, combinatory microphone signals from at least one close spaced array in the system could be used to monitor signals from a range of directions to assess the listening environment.

In any combinatory directional microphone monitoring and classification system, specific signals may be selected to represent the environment or specific technologies may be applied to the signals or selection of signals to improve the signal. The signal of interest may use one or more signals from the monitored signals in the auditory scene classification process, or other signals not used in the auditory scene classification process. The monitoring system may also be used to adapt the null direction in the case of a directional microphone system, or the most sensitive direction in the case of an adaptive listening direction system.

For hearing aids and other hearing devices, a hearing device on each ear is often worn, providing information to both ears, and are often linked wirelessly. These systems can provide important information about the sound environment contained in the interaural timing difference (ITD) or interaural level difference (ILD). The ITD and ILD are important in providing the listener information regarding the location or direction of sounds. In some cases due to the microphone locations or due to the processing of the signal or due to the presentation of the signal to the listener, the original timing or loudness of the signal may be changed, obscured or lost.

Combinatory directional microphones with greatest sensitivity substantially orthogonal to the microphone axis would provide improved sensitivity to each side of the listener, particularly when worn on the head. This would provide improvements segregation of signals at both ears compared to a range of directional microphones including forward facing directional microphone patterns and omnidirectional microphone patterns. The greater segregation of signals between the two ears with the use of combinatory directional microphones could be used advantageously in improving ITDs and ILDs.

One way to improve ITDs and/or ILDs is to use off-axis combinatory directional microphones to process signals for each ear. Another way to improve ITDs and/or ILDs is to use off-axis signal processing to enhance would be to process the signal on each ear independently to enhance the timing or level attributes of the signal. This may be done with processing any number of directional microphone signals obtained from one ear. For instance, processing an omnidirectional microphone signal and an off-axis microphones signal together to enhance the level of timing information in the signal. A third method would be to share information regarding the signal in each ear with the other ears signals to enhance the timing or level presented to one or both ears.

As noted above, the techniques presented herein may be implemented in a number of different devices that include a plurality of microphones, such as laptops, mobile phones, headsets, auditory prosthesis, etc. For example, with in one illustrative auditory prosthesis scenario, the techniques presented herein could be used to enable a recipient to hear a person seated next to them (e.g., in a car). In another example, an automation system may use the techniques presented herein to determine the location of noise. In yet another example, a chip manufacturer could use the techniques presented herein to make their MEMS microphone system with multiple independent microphones point is a specific direction. <FIG>, in particular, is a functional block diagram of one example arrangement for a bone conduction device <NUM> configured to implement embodiments presented herein. As shown, bone conduction device <NUM> is positioned at (e.g., behind) the ear of a recipient. The bone conduction device <NUM> comprises a microphone array <NUM>, an electronics module <NUM>, a transducer <NUM>, a user interface <NUM>, and a power source <NUM>.

The microphone array <NUM> comprises first and second microphones <NUM>(<NUM>) and <NUM>(<NUM>) configured to convert received sound signals (sounds) into microphone signals <NUM>(<NUM>) and <NUM>(<NUM>). The microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) are provided to electronics module <NUM>. In general, electronics module <NUM> is configured to convert the microphone signals <NUM>(<NUM>) and <NUM>(<NUM>) into one or more transducer drive signals <NUM> that activate transducer <NUM>. More specifically, electronics module <NUM> includes, among other elements, at least one processor <NUM>, a memory <NUM>, and transducer drive components <NUM>.

The memory <NUM> includes directional pre-processing logic <NUM>, combinatory processing logic <NUM>, and sound processing logic <NUM>. Memory <NUM> may comprise read only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical/tangible memory storage devices. The at least one processor <NUM> is, for example, a microprocessor or microcontroller that executes instructions for the directional pre-processing logic <NUM>, combinatory processing logic <NUM>, and sound processing logic <NUM>. Thus, in general, the memory <NUM> may comprise one or more tangible (non-transitory) computer readable storage media (e.g., a memory device) encoded with software comprising computer executable instructions and when the software is executed (at least one processor <NUM>) it is operable to perform all or part of the techniques presented herein.

Transducer <NUM> illustrates an example of a stimulator unit that receives the transducer drive signal(s) <NUM> and generates stimulation (vibrations) for delivery to the skull of the recipient via a transcutaneous or percutaneous anchor system (not shown) that is coupled to bone conduction device <NUM>. Delivery of the vibration causes motion of the cochlea fluid in the recipient's contralateral functional ear, thereby activating the hair cells in the functional ear.

<FIG> also illustrates the power source <NUM> that provides electrical power to one or more components of bone conduction device <NUM>. Power source <NUM> may comprise, for example, one or more batteries. For ease of illustration, power source <NUM> has been shown connected only to user interface <NUM> and electronics module <NUM>. However, it should be appreciated that power source <NUM> may be used to supply power to any electrically powered circuits/components of bone conduction device <NUM>.

User interface <NUM> allows the recipient to interact with bone conduction device <NUM>. For example, user interface <NUM> may allow the recipient to adjust the volume, alter the speech processing strategies, power on/off the device, etc. Although not shown in <FIG>, bone conduction device <NUM> may further include an external interface that may be used to connect electronics module <NUM> to an external device, such as a fitting system.

Claim 1:
A method, comprising:
generating first and second time domain directional microphone signals from sound signals received at a microphone array comprising first and second microphones positioned along a microphone axis;
determining a plurality of first frequency components associated with a first frequency domain directional microphone signal (<NUM>(<NUM>)) from the first time domain directional microphone signal (<NUM>(<NUM>));
determining a plurality of second frequency components associated with a second frequency domain directional microphone signal (<NUM>(<NUM>)) from the second time domain directional microphone signal (<NUM>(<NUM>));
multiplying component wise the first frequency components with the second frequency components to generate a cross-power spectrum (<NUM>); and
converting the cross-power spectrum (<NUM>) to a magnitude domain signal (<NUM>);
extracting phase information (<NUM>) from at least one of the first and second frequency domain directional microphone signals (<NUM>(<NUM>), <NUM>(<NUM>)),
generating a combinatory microphone signal (<NUM>) from the magnitude domain signal (<NUM>) and the phase information (<NUM>).