Parameter selection for audio beamforming

An audio beamformer receives signals from microphones of an array and processes the signals to produce a directional audio signal that emphasizes sound from a selected direction. The beamformer is implemented using weights or other parameters that are calculated to account for effects upon the received audio signals by the surfaces upon which the microphones are positioned.

BACKGROUND

Audio beamforming may be used in various types of situations and devices in order to emphasize sound received from a particular direction. Beamforming can be implemented in different ways, depending on system objectives.

Superdirective beamforming is a particular beamforming technique in which parameters are selected so as to maximize directivity in a diffuse noise field.

DETAILED DESCRIPTION

An audio beamformer receives audio signals from microphones of a microphone array and processes the signals to produce a directional audio signal that emphasizes sound from a selected direction. A superdirective beamformer is a particular type of beamformer that is implemented so as to maximize directivity in a diffuse noise field.

The microphones of a microphone array are positioned on a solid, rigid surface that produces diffraction and scattering of a received sound wave. In described embodiments, the effects of the diffraction and scattering upon captured audio signals are determined for multiple frequencies and directions either by experimentation or by mathematical modelling. Parameters of a superdirective beamformer are then calculated based on the determined diffraction and scattering effects.

FIGS. 1 and 2show an example of a device100that implements audio beamforming to produce a directional audio signal emphasizing sound that originates from a selected direction relative to the device100. The device100comprises a cylinder102or other rigid body having a planar, circular top surface104. A microphone array is formed by multiple input microphones or microphone elements106on the top surface104.

In the illustrated example, each of the microphones106comprises an omnidirectional or non-directional microphone that responds equally to sounds originating from different horizontal directions. One of the input microphones106is positioned at the center of the top surface104. Six other microphones106are arranged symmetrically around the periphery of the top surface104in a circular or hexagonal pattern, so that they are equidistant from each other.

FIG. 3Aillustrates logical components of an example superdirective beamformer300that may be used to perform audio beamforming in a system or apparatus such as the device100. In a device that includes M microphones106, the beamformer300receives M time domain audio signals xm(t) captured by multiple microphones106(0) through106(M−1). The nomenclature xm(t) indicates a time domain signal corresponding to the mthmicrophone of the array, wherein the signal xm(t) has a value that is a function of time t. The time-domain signals x0(t) through xM-1(t) are converted to frequency domain signals x0(ω) through xM-1(ω) by fast Fourier transforms (FFTs)302. The nomenclature xm(ω) indicates a frequency domain signal corresponding to the mthmicrophone of the array, wherein the signal xm(ω) has a value that is a function of the frequency ω. The frequency domain signal has multiple frequency components, corresponding to different frequencies ω.

The frequency components of each frequency domain signal xm(ω) are multiplied by corresponding weights wm(ω,θd) by a filter or weighting function304. The filter weights wm(ω,θd) are calculated as function of a selected direction θdfrom which sounds are to be emphasized by the beamformer. The direction θdis referred to as the focus direction of the beamformer.

The resulting filtered or weighted signals are then summed at306to produce a directional frequency domain signal y(ω, θd), which is converted to the time domain by an inverse fast Fourier transform (IFFT)308to produce a directional time-domain audio signal y(t,θd) that emphasizes sounds received from the focus direction θd.

The objective of superdirective beamforming is to maximize the output signal-to-noise ratio (SNR) under the condition that the noise field is spherically diffuse, in order to provide maximum directivity across all frequencies. In order to achieve this objective, the weights W(ω,θd) for the microphones are calculated as

W⁡(ω,θd)=(ΨN⁢⁢NDiff)-1⁢v⁡(ω,θd)vH⁡(ω,θd)⁢(ΨN⁢⁢NDiff)-1⁢v⁡(ω,θd)Equation⁢⁢1
where ΨNNDiffis a normalized noise correlation matrix for spherically diffuse noise and v(ω, θd) is an array manifold vector for the selected direction θdfrom which sound will be emphasized by the beamformer. The superscript −1 indicates an inverse matrix operation.

The superscript H indicates a Hermitian matrix transposition operation, which is performed by taking the regular transpose of a matrix and computing the complex conjugate of each element of the transposed matrix. Mathematically, the Hermitian transform of a matrix A is conj (AT), where the “conj” operator indicates the complex conjugate of ATand the superscript T indicates the regular matrix transpose operation.

FIG. 3Billustrates an example of a superdirective beamformer310implemented in the time domain. In the time-domain implementation, each of the time-domain microphone signals xm(t) is convolved by coefficients or parameters hm(t,θd) by a convolution function or operation312, wherein the coefficients or parameters hm(t, θd) are calculated by taking the inverse fast Fourier transform of the weights wm(ω, θd). The results are summed at316to produce the directional time-domain audio signal y(t, θd).

FIG. 4shows a Cartesian coordinate system relative to a circular microphone array400such as may be positioned on the top surface of the device100. The x and y axes correspond to orthogonal horizontal directions. The z axis corresponds to a vertical direction.

FIG. 5illustrates a spherical coordinate system that may be defined relative to the Cartesian coordinate system ofFIG. 4in order to specify an arbitrary point500in three-dimensional (3D) space relative to the microphone array400. In the spherical coordinate system, r is the radial distance of the point500from the Cartesian origin, which may be defined to coincide with the center microphone106. The angle θ, called the polar angle, is the angle between the z axis and a line from the Cartesian origin to the point500. The angle φ, called the azimuth angle, is the angle between the x axis and the projection onto the x-y plane of the line from the Cartesian original to the point500. The mapping from the spherical coordinate system to the 3D Cartesian coordinate system is as follows:
x=rsin(θ)cos(φ)   Equation 2
y=rsin(θ)sin(φ)   Equation 3
z=rcos(θ)   Equation 4

The position of the mthmicrophone of an array consisting of M microphones is denoted herein as pm. The acoustic signal acquired at the mthmicrophone at time t is denoted as f(t,pm). The signal acquired by a microphone array of M microphones can be expressed as

For a sound source located along the direction of Θ{θ, φ}, the unit vector pointing toward the direction Θ is
u=[sin θ cos φ sin θ sin φ cos θ]   Equation 6

For a monochromatic plane wave arriving from a source located along u, the wavenumber can be expressed as

k=-2⁢πλ⁢uEquation⁢⁢7
where λ is the wavelength of the plane wave.

Under free-field and far-field conditions, and for an ideal omnidirectional microphone array, the signal captured by the mthmicrophone can be expressed as
f(t,pm)=Aexp{j(ωt−kTpm)}   Equation 8
where A, in general, is complex valued. The superscript T indicates a matrix transposition operation.

Based on Equation 8, the basis function for a propagating plane wave can be expressed as
fBasis(t,p)=exp{j(ωt−kTp)}=exp(jωt)·exp(−jkTp)   Equation 9

In general, then, it may be said that

The array manifold vector of Equation 11 incorporates all of the spatial characteristics of the microphone array, based on free-field and far-field assumptions. Because the wavenumber k captures both frequency and direction components, v(k) can also be referred to as v(ω, Θ). vm(ω, Θ) indicates the mthelement of v(ω, Θ), which corresponds to the microphone at position pm. Θ indicates a direction relative to device100and/or its microphone array.

Because the microphones in the device100are surface mounted, the free-field and far-field assumptions upon which Equation 11 are based break down. In fact, the top surface may result in frequency and angle dependent diffraction and scattering effects. Thus, for a propagating plane wave, the signal observed by the microphones106on the top surface of the cylinder102is not accurately represented by Equation 11.

The effects of diffraction and scattering on a propagating plane wave impinging a surface at the position pmof the mthmicrophone from a direction Θ can be represented as a correction vector Am(ω, Θ) as follows:
Am(ω,Θ)=am(ω,Θ)ejφm(ω,Θ)Equation 12
where am(ω, Θ) represents the magnitude of diffraction and scattering effects at the mthmicrophone for the frequency ω and arrival direction Θ and φm(ω, Θ) represents the phase of the diffraction and scattering effects at the mthmicrophone for the frequency ω and arrival direction Θ. Under ideal free-field and far-field conditions, am(ω, Θ) would be equal to unity. The elements of the correction value Am(ω, Θ) can be determined by experiment or by mathematical modelling.

The surface effects represented by am(ω, Θ) and φm(ω, Θ) can be accounted for in the array manifold vector as follows:
{tilde over (v)}m(k){tilde over (v)}m(ω,Θ)Am(ω,Θ)exp(−jkTpm).   Equation 13
where k is the wavenumber corresponding to the frequency ω and direction Θ.

The corrected array manifold vector is:

v~⁡(k)⁢=Δ⁢v~⁡(ω,Θ)⁢=Δ⁢[A0⁡(ω,Θ)⁢exp⁡(-j⁢⁢kT⁢p0)A1⁡(ω,Θ)⁢exp⁡(-j⁢⁢kT⁢p1)⋮AM-1⁡(ω,Θ)⁢exp⁡(-j⁢⁢kT⁢pM-1)]Equation⁢⁢14
or

Equation 1 may be modified or corrected to calculate weights W for a superdirective beamformer by substituting the corrected array manifold vector {tilde over (v)}(ω, Θ) for the ideal manifold vector v(ω, Θ) as follows:

W⁡(ω,Θd)=(Ψ~NNDiff)-1⁢v~⁡(ω,Θd)v~H⁡(ω,Θd)⁢(Ψ~NNDiff)-1⁢v~⁡(ω,Θd)Equation⁢⁢16
where θdis the focus direction from which sounds are emphasized by the resulting beamformer. The weight vector wm(ω, Θ), comprising weights corresponding to single microphone m for a focus direction Θd, is corrected and calculated as follows:

Weights calculated in this manner may be used in the beamformer300to account for the diffraction and scattering effects of the surface upon which the microphones are mounted.

FIG. 6shows an example method600of determining weights for use in a beamformer such as a superdirective beamformer that receives input signals corresponding respectively to microphones of a microphone array, where each microphone m is at a position pmon an acoustically reflective surface.

An action601comprises selecting the focus direction Θdof the beamformer, which is the direction from which sounds will be emphasized by the beamformer.

An action602comprises determining diffraction and scattering effects604caused by the surface at each microphone position pm, for multiple frequencies ω and multiple angles of incidence Θ of an impinging sound wave. The diffraction and scattering effects604may include a magnitude a and a phase φ for each of the multiple frequencies and angles of incidence. The diffraction and scattering components may be indicated as am(ω, Θ) for each position pmand φm(ω, Θ) for each position pm, where ω is the frequency of an impinging sound wave and Θ is the direction from which the impinging sound wave originates.

Determining the diffraction and scattering effects may be performed by mathematically modeling physical characteristics of the device100with respect to sound waves of different frequencies arriving from different directions. Alternatively, the diffraction and scattering effects may be determined by experiment, observation, and/or measurement.

An action606comprises calculating a correction vector608corresponding to each microphone position pm. The correction vector comprises individual correction values corresponding respectively to multiple frequencies, each of which indicates magnitude differences and phase differences of the input signal caused by the surface upon which the microphone is positioned, in comparison to a free-field input signal that would be produced by a microphone in free space in response to a sound wave arriving from the focus direction Θd.

An action610comprises calculating a corrected array manifold vector612that accounts for the effects of diffraction and scattering by the surface upon which the microphones are positioned. The corrected array manifold vector {tilde over (v)} comprises multiple elements {tilde over (v)}m, each of which corresponds to a position pm:

An action614comprises calculating weights616, based on the corrected array manifold vector {tilde over (v)}, corresponding respectively to each of the microphones of the microphone array. For example, weights wm(ω), corresponding to the microphone at position pm, may be calculated as

An action618comprises providing or implementing an audio beamformer using the calculated weights616. The weights as calculated above result in what is referred to as a superdirective beamformer.

FIG. 7illustrates an example method700of beamforming. The method700implements the technique shown inFIG. 3A. An action702comprises receiving microphone signals generated by multiple microphones of a microphone array. An action704comprises performing FFT to convert the microphone signals to the frequency domain. An action706comprises multiplying the frequency components of the microphone signals by the weights calculated in the method600. An action708comprises summing the weighted frequency components corresponding to the multiple microphones. An action710comprises converting the weighted and summed frequency components back to the time domain using an IFFT, resulting in an audio signal that emphasizes sound from the selected focus direction Θd.

The operation of a superdirective beamformer in the frequency domain may be represented as follows:

The normalized noise correlation matrix ΨNNDiffused in the above calculations is determined in the context of an M-channel microphone array immersed in a spherically-diffuse noise field. The noise component of the mthmicrophone signal in the frequency domain can be represented as Nm(ω). A noise vector, having noise components for each of the M microphones, is represented as N(ω)=[N0(ω)N1(ω) . . . NM-1(ω)]T. The normalized noise correlation matrix for spherically diffuse noise is then defined as

Ψ~NNDiff⁡(ω)=E⁢{N⁡(ω)⁢NH⁡(ω)}E⁢{Nr⁡(ω)2}Equation⁢⁢21
where the E( ) is the statistical expectation operation and E{|Nr(ω)|2} is the noise energy measured by a reference omni-directional microphone.

Although the preceding description assumes the implementation of a superdirective beamformer in the frequency domain, similar techniques may be used to implement superdirective beamforming in the time domain, while accounting for diffraction and scattering effects caused by a rigid surface upon which the microphones are positioned. In addition, the described techniques may be used to determine weights and other parameters of different types of beamformers, not limited to superdirective beamformers.

FIG. 8shows relevant components of a computing device800that may be configured to implement the techniques described herein. For example, a computing device such as this may be used to calculate the weights or other parameters of a beamformer as described above. As another example, a computing device such as this may be used to implement superdirective beamforming. More specifically, the actions shown inFIGS. 6 and 7may be implemented by the computing device800or a similar device. In some cases, the device100ofFIG. 1may implement or contain the computing device800.

The computing device800has a processor802and memory804. The processor802may include multiple processors, or a processor having multiple cores. The processor802may comprise or include various different types of processors, including digital signal processors, graphics processors, etc.

The memory804may contain applications and programs in the form of computer-executable instructions806that are executed by the processor802to perform acts or actions that implement the methods and functionality described above. The memory804may be a type of non-transitory computer-readable storage media and may include volatile and nonvolatile memory. Thus, the memory804may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology. The memory804may also include type of memory that are commonly used to transfer or distribute programs or applications, such as CD-ROMs, DVDs, thumb drives, portable disk drives, and so forth.