PARTIALLY ADAPTIVE AUDIO BEAMFORMING SYSTEMS AND METHODS

Partially adaptive audio beamforming systems and methods are provided that enable improved acoustic echo cancellation of sound played on a loudspeaker that is in close proximity to a microphone array in an audio device. A stored beamformer parameter, such as an inverse covariance matrix, can be utilized by a frequency domain beamformer to generate a beamformed signal. The overall performance and resource usage by the audio device can be optimized.

TECHNICAL FIELD

This application generally relates to audio beamforming. In particular, this application relates to partially adaptive audio beamforming systems and methods usable in audio devices having a microphone array and a loudspeaker in close proximity, and enables improved acoustic echo cancellation of sound played on the loudspeaker through the use of a frequency domain beamformer having a stored beamformer parameter.

BACKGROUND

Conferencing environments, such as conference rooms, boardrooms, video conferencing applications, and the like, can involve the use of microphones for capturing sound from various sound sources that are active in such environments. Such sound sources may include humans talking, for example. The captured sound may be disseminated to a local audience in the environment through amplified speakers (for sound reinforcement), and/or to others remote from the environment (such as via a teleconference and/or a webcast). The types of microphones and their placement in a particular environment may depend on the locations of the sound sources, physical space requirements, aesthetics, room layout, and/or other considerations. For example, in some environments, the microphones may be placed on a table or lectern near the sound sources. In other environments, the microphones may be mounted overhead to capture the sound from the entire room, for example. Accordingly, microphones are available in a variety of sizes, form factors, mounting options, and wiring options to suit the needs of particular environments.

Microphone arrays having multiple microphone elements can provide benefits such as steerable coverage or pick-up patterns having lobes and/or nulls, which allow the microphones to focus on desired sound sources and reject unwanted sounds such as room noise and other undesired sound sources. The ability to steer audio pick-up patterns provides the benefit of being able to be less precise in microphone placement, and in this way, microphone arrays are more forgiving. Moreover, microphone arrays provide the ability to pick up multiple sound sources with one microphone array or unit, again due to the ability to steer the pick-up patterns.

Beamforming is used to combine signals from the microphone elements of microphone arrays in order to achieve a certain pick-up pattern having one or more lobes and/or nulls. However, even though the lobes of a pick-up pattern may be steered to detect sounds from desired sound sources (e.g., a talker in the local environment), the lobes may also detect sounds from undesired sound sources. The detection of sounds from undesired sound sources may be particularly exacerbated when a loudspeaker is in close physical proximity to the microphone elements of a microphone array, e.g., in audio devices such as speakerphones. For example, the microphone elements may pick up the sound from a remote location (e.g., the far end of a teleconference) that is being played on the loudspeaker. In this situation, the audio transmitted to the remote location may therefore include an undesirable echo, e.g., sound from the local environment as well as sound from the remote location.

Acoustic echo cancellation systems may be able to remove such echo that is picked up by the microphone array before the audio is transmitted to the remote location. However, a typical acoustic echo cancellation system may work poorly and have suboptimal performance if it needs to constantly readapt and/or is overwhelmed, such as when the sound from a physically proximate loudspeaker is being continually detected by the microphone array. For example, the echo-to-signal ratio in such a situation may be greater than 30 dB, while an adaptive filter in a typical acoustic echo cancellation system may remove the linear portion of the echo by up to 20 dB. The echo-to-signal ratio of the adaptive filter's output may therefore be greater than 10 dB, which can be difficult for a non-linear processor to handle without distorting desired sound sensed by the microphone array. As such, the sound from the loudspeaker (which may include audio from the remote location) may not be completely cancelled by a typical acoustic echo cancellation system and may be transmitted to the remote location.

Furthermore, existing beamforming techniques may be able to attenuate only certain portions of an echo signal, e.g., linear portions, without distorting the audio of desired sound sources in the local environment, and/or more fully attenuate the echo signal while distorting the audio of desired sound sources. In order to more fully attenuate the echo signal without distorting the audio of desired sound sources, existing beamforming techniques may be computationally and memory resource intensive and therefore difficult to implement in certain types of audio devices.

Accordingly, there is an opportunity for audio beamforming systems and methods that enable improved acoustic echo cancellation of a signal played on a loudspeaker that is in close proximity to a microphone array.

SUMMARY

The techniques of this disclosure are intended to solve the above-described problems by providing audio beamforming systems and methods that are designed to, among other things: (1) generate a beamformed audio signal from microphone audio signals using a frequency domain beamforming technique with a stored beamforming parameter associated with a loudspeaker, such as an inverse covariance matrix; (2) utilize a different beamforming technique to process the microphone audio signals when voice activity is not detected in the sound played on the loudspeaker; (3) update beamformer coefficients of the frequency domain beamforming technique when voice activity is not detected in the sound played on the loudspeaker; (4) improve and enhance the performance of downstream processing, such as acoustic echo cancellation, by generating the beamformed audio signal to attenuate the sound played on the loudspeaker while minimizing distortion of desired sound picked up by the microphones; and (5) reduce the use of computational and memory resources by avoiding real-time calculation of a beamforming parameter used by the frequency domain beamforming technique.

In an embodiment, an audio device includes a plurality of microphones configured to generate a plurality of audio signals, a loudspeaker configured to play back a reference signal, and a first beamformer configured to generate a first beamformed signal. The first beamformed signal may be based on the plurality of audio signals and a set of beamformer coefficients associated with a steering vector. The first beamformer may be configured to process the plurality of audio signals using a frequency domain beamforming technique with a stored beamforming parameter associated with the loudspeaker.

In another embodiment, a method includes receiving a plurality of audio signals from a plurality of microphones, receiving a reference signal for playback on a loudspeaker, and generating a first beamformed signal, using a first beamformer. The first beamformed signal may be generated based on the plurality of audio signals and a set of beamformer coefficients associated with a steering vector, and may include processing the plurality of audio signals using a frequency domain beamforming technique with a stored beamforming parameter associated with the loudspeaker.

These and other embodiments, and various permutations and aspects, will become apparent and be more fully understood from the following detailed description and accompanying drawings, which set forth illustrative embodiments that are indicative of the various ways in which the principles of the invention may be employed.

DETAILED DESCRIPTION

It should be noted that in the description and drawings, like or substantially similar elements may be labeled with the same reference numerals. However, sometimes these elements may be labeled with differing numbers, such as, for example, in cases where such labeling facilitates a more clear description. Additionally, the drawings set forth herein are not necessarily drawn to scale, and in some instances proportions may have been exaggerated to more clearly depict certain features. Such labeling and drawing practices do not necessarily implicate an underlying substantive purpose. As stated above, the specification is intended to be taken as a whole and interpreted in accordance with the principles of the invention as taught herein and understood to one of ordinary skill in the art.

The audio beamforming systems and methods described herein can enable audio devices having a microphone array and a loudspeaker in close proximity to attain improved acoustic echo cancellation (AEC) processing of audio captured by the microphone array. The systems and methods may generate a beamformed audio signal from audio signals of the microphone array by using a frequency domain beamforming technique with a stored beamforming parameter associated with the loudspeaker, such as an inverse covariance matrix. Even for a non-linear loudspeaker, the undesired sound generated by such a loudspeaker may be linearly related to the audio signals from the microphone array. Hence, the systems and methods may more completely attenuate the undesired sound played on the loudspeaker while minimizing distortion of the desired sound captured by the microphone array, e.g., speech from a talker in the local environment.

Furthermore, the frequency domain beamforming technique may be executed using less computational resources by avoiding the continuous calculation of the beamforming parameter in real time. As such, computational resources can be preserved for use by a downstream processing module that may operate on the beamformed audio signal. The downstream processing module may include an adaptive filter for acoustic echo cancellation of residual echo, a non-linear processor to remove residual non-linear echo, and/or automatic gain control. The performance of the downstream processing module may accordingly be enhanced and improved since the beamformed signal may include less undesired sound to be removed, e.g., sound from a remote location that is played on a loudspeaker.

When voice activity is not present in the sound played on the loudspeaker, the beamformed audio signal may be generated from the audio signals of the microphone array using a different beamforming technique that is more simplified and less resource intensive than the frequency domain beamforming technique. The coefficients of the frequency domain beamforming technique may be updated based on a steering vector that points towards a desired sound source, when there is no voice activity in the sound played on the loudspeaker.

FIG.1is a block diagram of an audio device100including a loudspeaker102, a microphone array104, and a beamforming system106. In embodiments, the loudspeaker102and the microphone array104may be in close physical proximity to one another and/or located in the same housing, such as when the audio device100is a speakerphone. The audio device100may receive a reference signal108, such as the sound from remote participants at the far end of a teleconference. The reference signal108may be played on the loudspeaker102so that local participants at the near end of the teleconference may hear the sound from the remote participants and/or to play other sounds. Various components included in the audio device100may be implemented using software executable by a computing device with a processor and memory, and/or by hardware (e.g., discrete logic circuits, application specific integrated circuits (ASIC), programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

The audio device100may be utilized in a conference room or boardroom and be placed on a table, lectern, desktop, etc., for example, where the sound sources may be one or more human talkers and/or other desirable sounds. Other sounds may be present in the environment which may be undesirable, such as sounds from loudspeakers (e.g., sound from a remote location of a teleconference), noise from ventilation, other persons, audio/visual equipment, electronic devices, etc. In a typical situation, the sound sources may be seated in chairs at a table, although other configurations and placements of the sound sources are contemplated and possible.

Each of the microphone elements104a,b, z in the microphone array104may detect sound and convert the sound to an audio signal. Components in the audio device100, such as analog to digital converters, processors, and/or other components, may process the audio signals and ultimately generate one or more digital audio output signals. In other embodiments, the microphone elements104a, b, c, . . . , zmay output analog audio signals so that other components and devices (e.g., processors, mixers, recorders, amplifiers, etc.) external to the audio device100that may process the analog audio signals.

The microphone elements104a, b, c, . . . , zmay be arranged in any suitable layout, including in concentric rings and/or be harmonically nested. The microphone elements104a, b, c, . . . , zmay be arranged to be generally symmetric or may be asymmetric, in embodiments. In further embodiments, the microphone elements104a, b, c, . . . , zmay be arranged on a substrate, placed in a frame, or individually suspended, for example. In an embodiment, the microphone elements104a, b, c, . . . , zmay be arranged on the perimeter of the audio device100and the loudspeaker102may be disposed in the center of the audio device100. In embodiments, the microphone elements104a, b, c, . . . , zincluded in the audio device100may be of a sufficient quantity to have enough degrees of freedom to suppress the echo (e.g., from the reference signal108) while minimizing the distortion of the sound from the desired sound source that is sensed by the microphone array104.

FIG.2is a block diagram of the beamforming system106in the audio device100ofFIG.1. The beamforming system106may receive the audio signals from the microphone array104and the reference signal108in order to form pick-up patterns so that the sound from the sound sources is more consistently detected and captured. In particular, the beamforming system106may generate a processed beamformed signal110associated with one or more lobes steered towards the desired sound source location in the environment, as described in more detail below.

The beamforming system106may include a partially adaptive beamformer202and a secondary beamformer204that both receive audio signals from the microphone elements104a, b, c, . . . , z. The partially adaptive beamformer202may use a frequency domain beamforming technique to create a beamformed signal203associated with one or more lobes steered towards desired sound source locations. The frequency domain beamforming technique of the partially adaptive beamformer202may create the beamformed signal203using coefficients that are based on the steering vector for the location of a desired sound source, as well as using a stored beamformer parameter214. In embodiments, the frequency domain beamforming technique utilized by the partially adaptive beamformer202may be a minimum variance distortionless response (MVDR) beamforming technique and/or another appropriate beamforming technique. Other types of appropriate beamforming techniques may include those included in an adaptive beamformer that utilizes an inverse covariance matrix, such as a linearly constrained minimum variance (LCMV) beamformer, a generalized sidelobe canceller (GSC) beamformer, or a Wiener beamformer. The secondary beamformer204may use a time domain beamforming technique or a frequency domain beamforming technique, such as a delay and sum beamforming technique and/or another appropriate beamforming technique, to create a beamformed signal205associated with one or more lobes steered towards desired sound source locations. The secondary beamformer204may be configured to attenuate noise and interference in the environment.

The stored beamformer parameter214used by the partially adaptive beamformer202may be an inverse covariance matrix that is associated with the loudspeaker102, in embodiments. The inverse covariance matrix may be representative of the amount of undesired sound, e.g., the echo from sound playing on the loudspeaker102. The size of the inverse covariance matrix can be based on the number of microphone elements104a, b, c, . . . , zin the microphone array104. As such, calculating the inverse covariance matrix in real time can be computationally intensive, as would be done in a traditional MVDR beamformer. Furthermore, a covariance matrix has to be estimated when there is no near-end signal present (e.g., talking by local participants), which can be difficult to determine when there is a high echo-to-signal ratio due to the proximity of the loudspeaker102to the microphone array104.

In contrast, the inverse covariance matrix used by the partially adaptive beamformer202may be determined and stored during the manufacture, installation, and/or calibration of the audio device100, prior to regular usage of the audio device100. The inverse covariance matrix may be determined and stored in this fashion because the loudspeaker102and the microphone array104are in close physical proximity to one another in the audio device100. Therefore, the structure of the acoustic field generated by the loudspeaker102may be spatially constant and may not be significantly influenced by the environment where the audio device100is located. By using a stored beamformer parameter214, e.g., an inverse covariance matrix, the partially adaptive beamformer202may be able to reduce and mitigate the echo generated by the direct path between the loudspeaker102and the microphone array104. In embodiments, the inverse covariance matrix may be determined by the audio device100following the initial manufacture, installation, and/or calibration, in order to attain a more optimal inverse covariance matrix that takes into account the particular environment where the audio device100is located.

The steering vector for the location of a desired sound source may be determined or configured as a particular three-dimensional coordinate relative to the location of the audio device100, such as in Cartesian coordinates (i.e., x, y, z), or in spherical coordinates (i.e., radial distance r, polar angle θ (theta), azimuthal angle φ (phi)), for example. In embodiments, the steering vector for the location of a desired sound source may be determined by an audio activity localizer or other suitable component(s) that can determine the location of audio activity in an environment based on the audio signals from the microphone elements104a, b, c, . . . , z. For example, the audio activity localizer may utilize a Steered-Response Power Phase Transform (SRP-PHAT) algorithm, a Generalized Cross Correlation Phase Transform (GCC-PHAT) algorithm, a time of arrival (TOA)-based algorithm, a time difference of arrival (TDOA)-based algorithm, or another suitable sound source localization algorithm. In embodiments, the audio activity localizer may be included in the audio device100, may be included in another component, or may be a standalone component. In other embodiments, the steering vectors for the location of a desired sound source may be determined programmatically or algorithmically using automated decision-making schemes, manually configured by a user, and/or adaptively determined.

The beamforming system106shown inFIG.2may also include a switch206that can select either the beamformed signal203(generated by the partially adaptive beamformer202) or the beamformed signal205(generated by the secondary beamformer204) for transmission to the downstream processing module208. The switch206may be a signal selection mechanism that selects the beamformed signal203or the beamformed signal205based on whether voice activity is detected in the reference signal108by a voice activity detector212. The beamformed signal203or the beamformed signal205may be processed by the downstream processing module208to generate a processed beamformed signal110, as described in more detail below.

A voice activity detector210may also detect whether there is voice activity in the audio signals of the microphone array104. In an embodiment, one of the audio signals of the microphone array104, e.g., the audio signal from microphone element104a,may be in communication with the voice activity detector210, as shown inFIG.2. In other embodiments, the voice activity detector210may detect whether there is voice activity in more than one audio signal of the microphone array104. As described below in more detail with respect toFIG.4, the detection of voice activity by the voice activity detector210may be utilized to determine whether to update a steering vector pointed towards the desired sound source in the environment. In embodiments, the voice activity detectors210,212may be implemented by analyzing the spectral variance of an audio signal, using linear predictive coding, applying machine learning or deep learning techniques to detect voice, and/or using well-known techniques such as the ITU 6.729 VAD ETSI standards for voice activity detection calculation included in the GSM specification, or long-term pitch prediction.

As shown inFIG.3, the downstream processing module208may include components that can process the beamformed signal203or205, such as an acoustic echo canceller302, a non-linear processor304, and/or an automatic gain control module306. The downstream processing module208may also include other types of processing, in some embodiments, such as noise reduction or feedback reduction.

In embodiments, the acoustic echo canceller302in the downstream processing module208may remove the echo that may remain in the beamformed signal203generated by the partially adaptive beamformer202, e.g., echo that is primarily due to reflections in the environment. The acoustic echo canceller302may be implemented using an adaptive filter running a least mean square (LMS) algorithm, a normalized LMS algorithm, a recursive least squares (RLS) algorithm, or another suitable algorithm. When in use, the acoustic echo canceller302may be able to use a greater amount of computational resources of the audio device100due to the partially adaptive beamformer202using a stored beamformer parameter214instead of needing to calculate a beamformer parameter in real time.

The non-linear processor304in the downstream processing module208may remove residual echo in the beamformed signal203that is not removed by the adaptive filter in the acoustic echo canceller302, and also attenuate noise and interference in the environment. The residual echo removed by the non-linear processor304may include the non-linear component of the echo signal, e.g., the portion that has no linear relationship with the reference signal108. In embodiments, the non-linear processor304may be implemented as a deep neural network, or be based on standard speech enhancement algorithms, for example.

As an example, the echo-to-signal ratio in the audio device100may be greater than 30 dB, and the partially adaptive beamformer202may remove about 20 dB of echo, leaving an echo-to-signal ratio of 10 dB at the output of the partially adaptive beamformer202. The acoustic echo canceller302running an LMS algorithm may remove about 20 dB of echo, which leaves an echo-to-signal ratio of −10 dB at the output of the acoustic echo canceller302. The non-linear processor304can more easily remove this amount of residual echo with minimal distortion of the desired sound sensed by the microphone array104.

The automatic gain control module306in the downstream processing module208may adjust the level of an audio signal, e.g., beamformed signal203or205, to be more balanced and consistent before generating and outputting the processed beamformed signal110. For example, the automatic gain control module306may compensate for input level differences due to, for example, loud or soft talkers and/or talkers who are located nearer or farther from the audio device100.

In embodiments, the downstream processing module208may process the beamformed signal203from the partially adaptive beamformer202or the beamformed signal205from the secondary beamformer204by using one, some, or all of the acoustic echo canceller302, the non-linear processor304, and the automatic gain control module306. The processed beamformed signal110from the downstream processing module208may be transmitted to a remote location (e.g., a far end of a teleconference) and/or played in the local environment for sound reinforcement. In other embodiments, the beamformed signal203and/or the beamformed signal205may be transmitted to components or devices external to the audio device100and/or to a remote location, in addition to or in lieu of the processed beamformed signal110from the downstream processing module208. In this way, the processed beamformed signal110may be, for example, transmitted to a remote location without the undesirable echo of persons at the remote location hearing their own speech and sound.

An embodiment of a method400is shown inFIG.4for the beamforming of audio signals of a plurality of microphones using the beamforming system106of the audio device100. The method400may be utilized to generate a processed beamformed signal110that is associated with lobes that are steered towards a desired sound source location while also attenuating the echo from a reference signal108being played on a loudspeaker102. The processed beamformed signal110may be derived from a beamformed signal203generated by a partially adaptive beamformer202using a frequency domain beamforming technique with a stored beamforming parameter associated with the loudspeaker102, when there is voice activity in the reference signal108. When there is no voice activity in the reference signal108(e.g., half duplex near end periods), the processed beamformed signal110may be derived from a beamformed signal205generated by a secondary beamformer204. In embodiments, the method400may be performed when the audio device100is in regular usage, e.g., when a user is conducting a teleconference with the audio device100.

One or more processors and/or other processing components (e.g., analog to digital converters, encryption chips, etc.) within or external to the audio device100may perform any, some, or all of the steps of the method400. One or more other types of components (e.g., memory, input and/or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be utilized in conjunction with the processors and/or other processing components to perform any, some, or all of the steps of the method400.

At step402, the reference signal108and the audio signals from the microphone elements104a, b, c, . . . , zmay be received at the beamforming system106. The reference signal108may include sound from remote participants at the far end of a teleconference, for example, and be received by a voice activity detector212and the downstream processing module208. One or more of the audio signals from the microphone elements104a, b, c, . . . , zmay be received by the partially adaptive beamformer202, the secondary beamformer204, and the voice activity detector210.

At step404, it can be determined whether there is voice activity in the reference signal108, such as by the voice activity detector212. Voice activity may be present in the reference signal108when participants at the far end of a teleconference are speaking, for example. If it is determined that there is voice activity in the reference signal108at step404(“YES” branch of step404), then the method400may continue to step406. At step406, the beamformed signal203may be generated by the partially adaptive beamformer202, based on the audio signals received from the microphone elements104a, b, c, . . . , zat step402, the stored beamformer parameter214, and beamformer coefficients (that are based on the steering vector for a desired sound source). The beamformed signal203may be associated with a lobe that is steered towards the desired sound source. The stored beamformer parameter214may include an inverse covariance matrix associated with the loudspeaker102, in embodiments. The beamformer coefficients may be updated when there is no voice activity detected in the reference signal108by the voice activity detector212. The method400may continue to step416after step404, as described below.

Returning to step404, if it is determined that there is no voice activity in the reference signal108(“NO” branch of step404), then the method400may continue to step408. At step408, it can be determined whether there is voice activity in one or more of the audio signals from the microphone elements104a, b, c, . . . , z, such as by the voice activity detector210. Voice activity may be present in the audio signals from the microphone elements104a, b, c, . . . zwhen participants in the local environment (e.g., at the near end of a teleconference) are speaking, for example. If it is determined that there is voice activity in one or more of the audio signals from the microphone elements104a, b, c, . . . z, at step408(“YES” branch of step408), then the method400may continue to step410.

At step410, the steering vector pointing towards the desired sound source may be updated. The steering vector may be updated when the desired sound source has changed locations and/or if the audio device100has changed locations, for example. The updated steering vector generated at step410may be utilized at step412to update coefficients for the partially adaptive beamformer202. The updated steering vector generated at step410may also be utilized by the secondary beamformer204at step414to generate the beamformed signal205. The method400may continue to step412following step410, and also following step408if it is determined that there is not voice activity in one or more of the audio signals from the microphone elements104a, b, c, . . . , z, (“NO” branch of step408).

At step412, the coefficients for the partially adaptive beamformer202may be updated based on the stored beamformer parameter214and based on the steering vector that points towards the desired sound source. In embodiments, the coefficients may be updated one frequency bin per time frame, in order to further reduce the use of computational resources of the audio device100. As previously described, the coefficients may be used by the partially adaptive beamformer202to generate the beamformed signal203at step406when voice activity has been detected in the reference signal108. The method400may continue to step414following step412.

At step414, the beamformed signal205may be generated by the secondary beamformer204, based on the audio signals received from the microphone elements104a, b, c, . . . , zat step402, and based on the steering vector for a desired sound source generated at step410. The beamformed signal205may be associated with a lobe that is steered towards the desired sound source. The method400may continue to step416following step412, and also following step406.

At step416, it can be determined whether there is voice activity in the reference signal108, such as by the voice activity detector212. In embodiments, step416may utilize the result of step404described above. If it is determined that there is voice activity in the reference signal108at step416(“YES” branch of step416), then the method400may continue to step418. At step418, the switch206may select the beamformed signal203from the partially adaptive beamformer202for transmission to the downstream processing module208. The downstream processing module208may process the beamformed signal203at step418to generate the processed beamformed signal110.

If it is determined that there is no voice activity in the reference signal108at step416(“NO” branch of step416), then the method400may continue to step420. At step420, the switch206may select the beamformed signal205from the secondary beamformer204for transmission to the downstream processing module208. The downstream processing module208may process the beamformed signal205at step420to generate the processed beamformed signal110. As compared to the beamformed signals203,205, the processed beamformed signal110that is generated at step418and step420by the downstream processing module208may be processed to remove residual echo, to balance its audio level, and/or be subject to other processing.

An embodiment of a method500is shown inFIG.5for the generation and storage of an inverse covariance matrix for use with a frequency domain beamformer, such as the partially adaptive beamformer202ofFIG.2. The method500may be utilized to generate and store the inverse covariance matrix as the stored beamformer parameter214that is used by the partially adaptive beamformer202. In some embodiments, the method500may be performed when the audio device100is not in regular usage, such as during manufacture, installation, or calibration of the audio device100. In other embodiments, the method500may be performed when the acoustic echo canceller302of the audio device100is not performing optimally (e.g., when the audio device100has been moved to a new location that has different reflections in the environment), as described below in relation to the method600ofFIG.6. The inverse covariance matrix may be used by the partially adaptive beamformer202at step406of the method400described above, for example, when generating the beamformed signal203.

At step502, a calibration audio signal may be played on the loudspeaker102of the audio device100. The calibration audio signal may include white noise and/or another appropriate type of sound, e.g., broadband sound that covers the frequency spectrum for a sufficient amount of time, such as speech or music. The calibration audio played on the loudspeaker102may be received and sensed by the microphone array104at step504. Based on the calibration audio sensed by the microphone array104at step504, an inverse covariance matrix can be generated at step506. The inverse covariance matrix may be associated with the loudspeaker102and may represent an amount of undesired sound, such as the echo from sound playing on the loudspeaker102. In embodiments, the inverse covariance matrix can be generated at step506for each frequency bin.

At step508, the inverse covariance matrix generated at step506may be stored as the beamformer parameter214for use by the partially adaptive beamformer202. In embodiments, the method500for generating an inverse covariance matrix may be performed for each particular audio device100since there may be differences in the positioning of the loudspeaker102and the microphone array104due to manufacturing tolerances and the like.

An embodiment of a method600is shown inFIG.6for the regeneration of the inverse covariance matrix based on the performance of an acoustic echo canceller. The method600may be performed by the audio device100continuously, periodically, and/or be manually activated by a user. The method600may determine whether the acoustic echo canceller302of the audio device100is not performing optimally, and then regenerate the inverse covariance matrix based on the current conditions of the audio device100(e.g., based on the current environment where the audio device100is located). The regenerated inverse covariance matrix resulting from the method600may therefore be more optimal for the current conditions of the audio device100.

At step602, the performance of the acoustic echo canceller302may be monitored, such as by monitoring metrics of the acoustic echo canceller302. For example, the echo return loss enhancement (ERLE) metric may be monitored at step602. The ERLE metric may indicate how much echo has been attenuated from an audio signal from the ratio of the reference signal108and the measured echo in the processed beamformed signal110.

At step604, it may be determined whether the performance of the acoustic echo canceller302is acceptable, based on the monitoring of step602. For example, the performance of the acoustic echo canceller302may be deemed acceptable at step604if the ERLE metric satisfies a certain criteria, e.g., if the metric is lower than a particular threshold. If the performance of the acoustic echo canceller302is acceptable at step604, then the method600may return to step602and continue the monitoring. However, if the performance of the acoustic echo canceller302is not acceptable at step604, then the method600may continue to step606. At step606, the inverse covariance matrix may be regenerated, such as by performing the method500ofFIG.5described above. In embodiments, a user may be notified at step606to recalibrate the audio device100to regenerate the inverse covariance matrix.