Patent Description:
Obtaining high quality audio at both ends of a conference call is difficult to manage due to, but not limited to, variable room dimensions, dynamic seating plans, roaming participants, unknown microphone locations, different microphone sensitivities, known steady state and unknown dynamic noise, and variable desired sound source levels. This results in audio sound sources having wide dynamic range within the ambient sound environment. Because of the complex needs and requirements, solving the problems has proven difficult and insufficient within the current art.

In the currently known art there have been various approaches to solving the complex issue of managing wide dynamic range audio signals with acceptable ambient sound level performance from multi-location based sound and signal sources. Typically, this is accomplished using heuristic-based automatic gain control techniques to enhance audio conferencing system performance in a multi-user room. Automatic gain control is used to bring the desired signal, which in this case may be but is not limited to a speaking participant in the room, to within an acceptable dynamic range to be transmitted to remote participants through third party telephone, network and/or teleconference software such as Microsoft Skype, for example. If automatic gain control was not implemented the conversations would be hard to hear with the sound volume levels swinging from very low level to very loud levels. The communication system may not be able to manage the signal properly, with too little signal strength to be heard clearly or too much signal strength, which would overdrive the system resulting in clipping of the signal and adding significant distortion. Either scenario would not be acceptable in an audio conference situation. If the signal is within a sufficient range to propagate through the system, the resulting dynamic range swings would require the remote participants to continually adjust their volume control to compensate for the widely variable level differences that would be present for each individual speaking participant. An unwanted byproduct of typical automatic gain control circuits is the ambient sound levels also tracking in proportion to volume changes by the remote participant.

Automatic gain control is typically applied as a post-processing function within a variable gain amplifier or after the analog digital converter in a digital signal processor isolated from the microphone processing logic. The automatic gain control does not know a key parameter such as the position of the sound source <NUM>, which means the automatic gain control will need to operate on heuristic principals, assumptions, and configuration limits. This is problematic because the automatic gain control solutions have to work on heuristic principals because the actual location of the sound and ambient sound sources are not known, which means the performance of the automatic gain control is not deterministic. This results in serious shortcomings by not being able to adapt to and provide consistent performance and acceptable end user experiences. Automatic gain control systems which need to deal with large dynamic range signals end up having to adjust the gain of the system, which can show up as sharp unexpected changes in background ambient sound levels. The automatic gain control will appear to hunt for the right gain setting so there can be a warbling and inconsistent sound levels making it difficult to understand the person speaking. The automatic gain control is trying to normalize to preset parameters that may or may not be suitable to the actual situation, as designers cannot anticipate all scenarios and contingencies that an automatic gain control function must handle. Third party conference and phone software such as but not limited to Microsoft Skype, for example, have specifications that need to be met to guarantee compatibility, certifications, and consistent performance. Automatic gain controls in the current art do not know the distance and the actual sound levels of the sound source <NUM> (e.g., Participant <NUM> in <FIG>) that they are trying to manage, resulting in inconsistent sound volume when switching sources and fluctuating ambient sound level performance. This makes for solutions that are not deterministic and do not provide a high level of audio performance and user experience.

Thus, the current art is not able to provide consistent performance in regards to a natural user experience regarding desired source signal level control and consistent ambient sound level performance.

An approach in the prior art is to utilize various methods to determine source location targeting parameters to determine Automatic Gain Control (AGC) settings. However, the systems in the prior art address a gain adjustment method that does not adequately manage the ambient noise levels to a consistent level, regardless of targeted AGC parameters, which is problematic for maintaining a natural audio listening experience with consistent ambient noise levels for conference participants.

<CIT> discloses multiport digital conference arrangements wherein speech samples of selected speakers are summed for distribution to the conferees. The embodiment controls the level of speech represented by information samples to be included in an output sample for distribution to the ports, and equalizes the speech level between speakers to reduce speech level contrast heard by the conferees. In addition, a speech detector for each port and microprocessor-controlled switching hardware also adjust the signal level represented by samples received on the ports to effect speaker selection. Furthermore, gain coefficients for a port may be incrementally adjusted during a predetermined period of time to avoid noticeable signal level changes when implementing speaker selection.

<CIT> discloses a system and method for automatically adjusting the gain of an audio system as a speaker's head moves relative to a microphone includes using a video of the speaker to determine an orientation of the speaker's head relative to the microphone and, hence, a gain adjust signal. The gain adjust signal is then applied to the audio system that is associated with the microphone to dynamically and continuously adjust the gain the audio system.

<CIT> describes methods and systems for adjusting audio gain levels for multi-talker audio. In one example, an audio system monitors an audio stream for the presence of a new talker. Upon identifying a new talker, the system determines whether the new talker is a first-time talker. For a first-time talker, the system executes a fast-attack/decay automatic gain control (AGC) algorithm to quickly determine a gain value for the first-time talker. The system additionally executes standard AGC techniques to refine the gain for the first-time talker while the first-time talker continues speaking. When a steady state within a decibel threshold is attained using standard AGC for the first-time talker, the system stores the steady state gain for the first-time talker to storage. Upon identifying a previously-identified talker, the system retrieves from storage the steady state gain for the talker and applies the steady state gain to the audio stream.

<CIT> describes using a camcorder which includes a camera section receiving a subject image subject through a zoom lens, converting the subject image to a video signal, and generating a corresponding wide/tele signal representing the position of the zoom lens, an audio processing part including a plurality of microphones receiving input sounds from the subject and converting the input sounds into a recordable audio signal, and a recorder/reproducer which records and reproduces the video signal and the recordable audio signal onto video tape. The audio processing part includes a plurality of analog elements. The audio processing part continuously amplifies the input audio signal using the analog elements in response to the wide/tele signal and outputs the recordable audio signal which corresponds to perceived distance from the camcorder to the subject. The analog elements may be transistors, wherein the dynamic resistance of each transistor is continuously varied responsive to the wide/tele signal.

<CIT> describes a gain adjusting system for adjusting a gain of a sound signal in an audio system, and includes a first detecting unit for capturing images of one or more faces of users and determining the number of faces and the size of the faces present in the images; a controller for receiving face data from the first detecting unit for comparing the sizes of faces in subsequently captured images with an initial face size and accordingly deciding and outputting a first decision signal; and a gain regulator coupled to the controller for adjusting the gain level of the sound signal according to the first decision signal.

<CIT> describes a method were the overall loudness of an audio track is calculated by combining a number of weighted loudness measures for segments of the audio track. The weight applied to each individual loudness measure is a function of the loudness measure. By comparing the original overall loudness measure to a desired overall loudness measure, a gain can be determined that will adjust the loudness level to the desired value. Also disclosed is a dynamic compression method that analyzes the dynamic characteristics of an audio track and determines appropriate compressor parameters. Additionally, the loudness of a post-compressor audio track can be estimated for any given compressor parameters, thus permitting post-compression loudness matching to be done even if the compression is performed in real-time.

<CIT> discloses a technique in which signals collected by multiple microphones are time delayed to account for their distance from a source, and normalised by removing an output sum from all the microphones.

<CIT> discloses a system in which sound source direction measurements from two microphone arrays are used to localize a sound source.

<CIT> discloses a method for acquiring a multi-channel sound that includes estimating positions of sound sources corresponding to mixed sound source signals received at a microphone array. A multi-channel sound source signal is generated by compensating for the sound source signals, based on differences between the estimated positions of the sound sources and a position of a virtual microphone array substituting for the microphone array.

<CIT> discloses a method for improving the quality of received specific acoustic signals with acoustic rake receivers.

An object of the present embodiments is to allow for a consistent volume of the sound source no matter where it is located in the range of the system, while keeping the background ambient sounds at a constant level.

In one embodiment of the present invention, the dynamically measured position of the sound source (from a position processor or like process) is used.

Utilizing the positional coordinate information, a system having a Channel Audio Processor can calculate and control the individual microphone gain and selection of the microphone array utilizing derived repeatable gain values, based on known path loss calculations, to overcome the limitations of a heuristic post processing automatic gain control system. One advantage of this embodiment is that it operates deterministically and can use known sound pressure level propagation formulas over distance, to account for signal path loss situations on an individual basis, deriving the appropriate required gain adjustment for each sound source relative to the microphone array. Because the gain is preferably managed on an individual sound source location basis, the disadvantages of a broad-based automatic gain control circuit of the whole signal chain is not incurred, resulting in a consistent volume with stable ambient signal performance held to unity gain values, without the typical up and down normalizing and hunting that is typical of automatic gain control functions.

Typical solutions in the current art base the amplification or compression solely on the audio signal strength. This simple approach is subject to extreme ambient sound fluctuations. As the source signal goes down in level, the automatic gain control will increase the gain to compensate. This has the effect of bringing the relative ambient sound up as well. A natural extension of this is when there is no source signal present, the automatic gain control goes to max gain to bring up a signal that is not present, which greatly increases the ambient sound in the system. This situation is avoided within the presently preferred embodiments as there is preferably no controlling the gain compensation based on sound source level, but instead on position and path loss; if there is no sound source, the preferred embodiments will not artificially try and raise the ambient sound level. According to the preferred embodiments, there needs to be a signal present and located to derive the gain values.

The present invention is directed to apparatus and methods that enable groups of people (and other sound sources, for example, recordings, broadcast music, Internet sound, etc.), known as "participants", to join together over a network, such as the Internet or similar electronic channel(s), in a remotely-distributed real-time fashion employing personal computers, network workstations, and/or other similarly connected appliances, often without face-to-face contact, to engage in effective audio conference meetings that utilize large multi-user rooms (spaces) with distributed participants.

Advantageously, embodiments of the present apparatus and methods provide an ability to provide remote participants an end user experience having all sound sources at a consistent volume level, regardless of their location with respect to the microphone array, while maintaining consistent ambient sound and ambient sound source levels at all times.

A notable challenge to picking up sound clearly in a room, cabin, or confined space is the dynamic nature of the sound sources, resulting in a wide range of sound pressure levels, while maintaining realistic and consistent ambient sound levels for the remote participant(s).

A "device" in this specification may include, but is not limited to, one or more of, or any combination of processing device(s) such as, a cell phone, a Personal Digital Assistant, a smart watch or other body-borne device (e.g., glasses, pendants, rings, etc.), a personal computer, a laptop, a pad, a cloud-access device, a white board, and/or any device capable of sending/receiving messages to/from a local area network or a wide area network (e.g., the Internet), such as devices embedded in cars, trucks, aircraft, household appliances (refrigerators, stoves, thermostats, lights, electrical control circuits, the Internet of Things, etc.).

An "engine" is preferably a program that performs a core function for other programs. An engine can be a central or focal program in an operating system, subsystem, or application program that coordinates the overall operation of other programs. It is also used to describe a special-purpose program containing an algorithm that can sometimes be changed. The best known usage is the term search engine which uses an algorithm to search an index of topics given a search argument. An engine is preferably designed so that its approach to searching an index, for example, can be changed to reflect new rules for finding and prioritizing matches in the index. In artificial intelligence, for another example, the program that uses rules of logic to derive output from a knowledge base is called an inference engine.

As used herein, a "server" may comprise one or more processors, one or more Random Access Memories (RAM), one or more Read Only Memories (ROM), one or more user interfaces, such as display(s), keyboard(s), mouse/mice, etc. A server is preferably apparatus that provides functionality for other computer programs or devices, called "clients. " This architecture is called the client-server model, and a single overall computation is typically distributed across multiple processes or devices. Servers can provide various functionalities, often called "services", such as sharing data or resources among multiple clients, or performing computation for a client. A single server can serve multiple clients, and a single client can use multiple servers. A client process may run on the same device or may connect over a network to a server on a different device. Typical servers are database servers, file servers, mail servers, print servers, web servers, game servers, application servers, and chat servers. The servers discussed in this specification may include one or more of the above, sharing functionality as appropriate. Client-server systems are most frequently implemented by (and often identified with) the request-response model: a client sends a request to the server, which performs some action and sends a response back to the client, typically with a result or acknowledgement. Designating a computer as "server-class hardware" implies that it is specialized for running servers on it. This often implies that it is more powerful and reliable than standard personal computers, but alternatively, large computing clusters may be composed of many relatively simple, replaceable server components.

The servers and devices in this specification typically use the one or more processors to run one or more stored "computer programs" and/or non-transitory "computer-readable media" to cause the device and/or server(s) to perform the functions recited herein. The media may include Compact Discs, DVDs, ROM, RAM, solid-state memory, or any other storage device capable of storing the one or more computer programs.

<FIG> illustrates a room <NUM> with a microphone array <NUM>, which comprises a plurality of microphones <NUM>. This diagram illustrates the various configuration zones that are available for the microphone array <NUM>.

For the purpose of this embodiment, the microphone array <NUM> is positioned against a wall; however the position of the microphone array <NUM> can be against any wall in the room <NUM>. There are notionally three participants illustrated in the room, Participant <NUM><NUM>, Participant <NUM><NUM> and Participant <NUM><NUM>. Participant(s) and sound source(s) can and will be used interchangeably and in this context mean substantially the same thing. Each Participant illustrates, but is not limited to, an example of the variability of position <NUM> within a room <NUM>. The embodiments are designed to adjust for and accommodate such positions (stationary and/or moving). For example, each Participant may be moving, and thus have varying location coordinates in the X, Y, and Z directions. Also illustrated is an ambient sound <NUM>, which may be present and propagated throughout the room, such that it is relatively constant for each participant <NUM>, <NUM>, <NUM> locations. For example, the room ambient noise may be one or more of HVAC noise, TV noise, outside noise, etc..

Also illustrated in <FIG> is a Minimum Threshold Distance (MTD) <NUM> and a Configurable Threshold Distance (CTD) <NUM>. The area inside the CTD <NUM> is the microphone array <NUM> configuration zone. In that zone, utilizing the specific distance P2 d(m) (e.g., distance in metric) <NUM> of the participant <NUM><NUM>, the array will be configured for individual gain and microphone selection to stabilize the array <NUM> volume output and ambient sound level <NUM> relative to the Participant <NUM> location <NUM>. Within the CTD <NUM> there is enough positional <NUM> resolution of the system to utilize distance path loss <NUM> to tune the array <NUM> for individual microphone <NUM> gain-weighted measurements. Within the zone of the CTD <NUM> and the MTD <NUM>, the microphone array <NUM> is dynamically configured to utilize between <NUM>-<NUM> of the microphones <NUM>, based on the position <NUM> of the sound source <NUM>.

For participants <NUM> outside the CTD <NUM>, all microphones <NUM> are used. As the sound source <NUM> gets further from the CTD <NUM>, its perceived volume will drop off. This is done because it is undesirable to pick up people far away and have them sound as if they are in the room.

For participants <NUM> in the zone between the MTD <NUM> and the CTD <NUM>, the system will pick the n+<NUM> microphones <NUM> which are closest to the location <NUM> of the sound source <NUM> to act as the microphone array (e.g., one of them will only be fractionally on) and the remainder are turned off.

When a participant <NUM> is within the MTD <NUM>, the system will select a pair of microphones <NUM> in the array <NUM>, so that the ambient sound level <NUM> can be maintained with one microphone <NUM> fully on and one fractionally on, e.g., <NUM>%, <NUM>%, <NUM>%, <NUM>%, <NUM>%, <NUM>%, <NUM>%, <NUM>%, <NUM>%, <NUM>%, or any value between <NUM>% and <NUM>%. When the participant <NUM> gets within the MTD of the closest microphone, the array will no longer use that microphone. Instead, the system uses one or more other microphones further away, that are outside the closest-microphone MTD in order to control the gain of the sound source <NUM>. If the microphones are spaced close enough, there will usually exist a microphone in the range where n=<NUM>. The maximum microphone spacing allowed is preferably (sqrt(<NUM>)-<NUM>) * MTD.

Beyond the CTD <NUM>, all <NUM> microphones (or however many microphones are in the array, e.g., any number between <NUM> and <NUM>; and the "array" may be a one-dimensional array, a two-dimensional matrix array, or a three-dimensional linear or matrix array having certain microphones at different distances from a Z-axis baseline) <NUM> of the microphone array <NUM> are
sequentially enabled as the positional information <NUM> (obtained from the system) becomes too granular and the best performance is realized with all <NUM> microphones in operation. Both the MTD <NUM> and the CTD <NUM> are system-configurable parameters that are set based on the microphone array <NUM> parameters and the room <NUM> parameters.

<FIG> illustrates the system Position Processor <NUM> and the automatic gain control Channel Processor <NUM>. Although one Channel Processor <NUM> is shown the embodiments, the implementation may utilize a plurality of channel processors <NUM>, resulting in multiple audio channels <NUM> with individual microphone array <NUM> gain control capabilities running in parallel. This allows for unique microphone array tunings for each sound source <NUM> position <NUM>, <NUM>, <NUM> with known positional coordinates <NUM>. Each Channel Audio Processor includes at least one Gain Weight Processor <NUM> and at least one Delay Processor <NUM>. Each "processor" may comprise one or more processor chips or boards, which may be co-located or remotely located with respect to each other. The presently preferred embodiments contemplate at least one Field Programmable Gate Array (FPGA) as the Position Processor <NUM>, and a Digital Signal Processor (DSP) as the Gain Weight Processor <NUM>. However, these processors may comprise one or more circuits and/or applications installed in one or more personal computers and/or Application Specific Integrated Circuits (ASICs). These processors run program code permanently stored therein or stored in removable media. The program code comprises one or more modules and/or engines to perform the various functions described herein.

<FIG> shows a microphone array <NUM> (comprising a plurality of microphones <NUM>) which is connected to a Position Processor <NUM>. The Position Processor may for example be designed as the processor described and depicted in <CIT>, BUBBLE PROCESSOR. See also <CIT>, A SYSTEM FOR PROCESSING AUDIO; <CIT>, VIRTUAL POSITIONING IN A SOUND SPACE.

The Position Processor <NUM> utilizing the Microphone Array signals <NUM> preferably determines the substantially exact positional location <NUM> (X,Y,Z) coordinates of the sound source <NUM> with the highest processing gain. This is the sound source <NUM> that the microphone array will focus on. The Position Processor <NUM> runs independent of the Channel Processor <NUM>. The Position Processor <NUM> communicates the positional information <NUM> to the Channel Processor <NUM>, which comprises the Delay Processor <NUM> and the Gain Weight Processor <NUM>. The Channel Processor preferably runs at the required sample rates (e.g., <NUM>) to support the desired frequency response specifications, meaning the sample rates are not limited by the invention implementation in the embodiments.

The sound pressure level (SPL) of the sound wave follows a very predictable loss pattern where the SPL is inversely proportional to the distance P2 d(m) <NUM> from the source Participant <NUM><NUM> to the microphone array <NUM>. Since the positional information <NUM> derived from the Position Processor <NUM> is known, the distance P2 d(m) <NUM> can be calculated, and the Gain Weight Processor calculates the gain required, on a per microphone <NUM> basis, based on the distance <NUM> to each microphone <NUM> of the microphone array <NUM>. Once the Gain Weight parameters <NUM> Alpha (α = the multiplication factor to be applied to each of the fully-on microphone signals. F α = the multiplication factor to be applied to the fractionally-on microphone signal (f is preferably a value between <NUM> and <NUM>)); and the f*Alpha parameters have been calculated, they are multiplied <NUM> with the individual Microphone <NUM> signals <NUM>, resulting in weighted output parameters <NUM> that have been gain-compensated based on the actual distance <NUM> from the microphone <NUM> in the microphone array <NUM>. This process accomplishes the specific automatic gain control function, which adjusts the microphone levels <NUM> that are sent to the delay elements.

The delays in the microphone array <NUM> are calculated using the positional information <NUM> from the Position Processor <NUM> in the Delay Processor <NUM>. The Delay Processor <NUM>
calculates the individual path loss delays d(m) in milliseconds for each microphone <NUM> relative to the sound source <NUM> location <NUM>. It then adds the extra DELAY into each microphone path of D-d(m) so that the overall DELAY between the sound source <NUM> and the summer <NUM> through all the microphone paths is preferably a constant D. The value constant D would typically be the delay through the longest path between a microphone <NUM> and a position monitored by the position processor <NUM>, measured in milliseconds. For example if the longest distance between the <NUM> antennas and the <NUM> points monitored by the position processor is <NUM>, then then the value of D would be that distance converted into a delay, about <NUM>. The result is that signals from all microphones <NUM> are aligned in the time domain, allowing for maximum natural gain of all direct signal path signals to the microphone array <NUM>. All of the output signals <NUM> are summed at the Summer <NUM> and output for further system processing. The resulting delays are applied to all of the microphones whether they will be used by the Gain Weight Processor <NUM> or not.

To provide gain control of the desired signal without affecting the ambient sound level is accomplished through the following methods. This is accomplished by controlling the processing gain of the microphone array <NUM>. Processing gain is how much the array <NUM> boosts the desired signal source relative to the undesired signal sources. As illustrated with a linear microphone array <NUM>, the processing gain is roughly the square root of the number of microphones in use ( <MAT> if we use all <NUM> microphones). When it is desired to reduce the volume of the focused signal without affecting ambient levels <NUM>, the microphones <NUM> in the array <NUM> are turned off to reduce the gain and provide the proper scaling constants to keep the ambient sounds <NUM> at the same level. For example, if half the microphones are turned off, the gain drops to <MAT>, or a 3dB drop from <NUM> microphones.

In this embodiment, the maximum gain that can be achieved with all <NUM> microphones is <NUM>, and the minimum gain (when reduced to a single microphone) is <NUM>. This gives a <NUM>. 8dB gain range. The CTD <NUM> is where to set the desired signal levels with all <NUM> microphones <NUM> on. Below the CTD <NUM>, the microphones in the array <NUM> are individually turned off to maintain a consistent sound level. Beyond the CTD <NUM>, the system typically cannot produce more gain, so the sound level will drop off with the inverse distance law.

It is not preferred to just switch microphones <NUM> in and out, since this may cause undesirable jumps in the sound volume. To make the adjustments continuous, some number of microphones <NUM> are fully turned on and one microphone <NUM> to be partially turned on. The partially turned-on microphone <NUM> allows a smooth transition from one set of microphone(s) to another, and to implement any arbitrary gain within the limits.

Calculation of microphone gain parameters. To determine a specific gain, Gfocus, for the focused signal while keeping the background gain, Gbg, at unity, the system
turns n microphones <NUM> on fully and have one microphone <NUM> on fractionally with a constant that is somewhere between <NUM> and <NUM>. Each microphone signal is weighted by the common constant α. Given the assumptions that the background signals are orthogonal so they add by power when combined, and that the levels of the signals arriving at each microphone <NUM> are equal, the rms gain of n signal with a gain of α and one signal with a gain of f α is: <MAT>.

Setting Gbg to unity to keep it constant gives: <MAT>.

The array <NUM> is designed to combine the focused source coherently so the signals from this source add by amplitude. The coherent gain of the focused source is: <MAT>.

Substituting (<NUM>) into (<NUM>) gives: <MAT>.

For a given Gfocus, first assume that f=<NUM> and find the largest integer n that give a result less than or equal to Gfoctus <MAT>.

Equation (<NUM>) can be solved for f using the standard quadratic equation and picking the solution where <NUM> ≤ f < <NUM>. Then compute α from equation (<NUM>).

<FIG> illustrates the microphone arrangement <NUM> and the gain weight values α when a participant <NUM> is located outside of the CTD <NUM>. The Figure shows a structure (one or more circuits) comprising the microphone arrangement <NUM>, Gain Weight Multipliers <NUM>, and the Summer <NUM>. The MTD <NUM> for this embodiment has been set to <NUM>, and the CTD has been set to <NUM>. The position of the participant <NUM> has been determined by the Position Processor <NUM>, and the Gain Weight Processor <NUM> has determined the distance <NUM> to be <NUM>. This positions the participant <NUM> outside of the CTD <NUM>. Based on the embodiment calculations per the above discussion, the calculated Gain Value used to set the Channel Processor AGC <NUM> to is <NUM> in this embodiment. All microphones <NUM> are enabled, n= <NUM>, and the per microphone α gain value is <NUM>. Since all microphones <NUM> are fully enabled there is no fractional gain value and f=<NUM>.

<FIG> illustrates the microphone arrangement <NUM> and gain weight values α when a participant <NUM> is located inside of the CTD <NUM> but not within the MTD <NUM>. The Figure shows the circuit comprising the microphone arrangement <NUM>, Gain Weight Multipliers <NUM>, and the Summer <NUM>. The MTD <NUM> has been set to <NUM> in this embodiment, and the CTD has been set to <NUM>. The position of the participant <NUM> has been determined by the Position Processor <NUM>, and the Gain Weight Processor <NUM> has determined the distance <NUM> to be <NUM> in this embodiment. This positions the participant <NUM> within the CTD <NUM>. Based on the calculations described above, the calculated Gain Value used in this embodiment to set the Channel Processor AGC <NUM> to is <NUM>. Only some of the microphones <NUM> are enabled, n=<NUM>, and the per microphone a gain value is <NUM>. One Microphone is partially turned on with a fractional value of f=<NUM>. The microphone(s) <NUM> selected are based on the closest proximity to the participant <NUM>.

<FIG> illustrates the microphone arrangement <NUM> and gain weight values α when a participant <NUM> is located inside of the MTD <NUM>. The Figure shows the circuit comprising the microphone arrangement <NUM>, Gain weight multipliers <NUM>, and the Summer <NUM>. The MTD <NUM> has been set to <NUM> and the CTD has been set to <NUM>. The position of the participant <NUM> has been determined by the Position Processor <NUM>, and the Gain Weight Processor <NUM> has determined the distance <NUM> to be <NUM> in this embodiment. This positions the participant <NUM> within the CTD <NUM>. As this distance may be too close for the system to control the gain, a microphone further from the source (e.g., <NUM>) is selected to be the primary on microphone. Based on the calculations described earlier, the calculated Gain Value required to set the Channel Processor AGC <NUM> to is <NUM> in this embodiment. Only some of the microphones <NUM> are enabled, n=<NUM>, and the per microphone a, gain value is <NUM>. One microphone is partially turned on with a fractional value of f=<NUM>. The microphone(s) <NUM> are selected based on determining the microphones <NUM> that are located outside of a distance equal to the MTD <NUM>. In this embodiment, the microphone(s) <NUM> selected are <NUM> away from the participant, which is a distance greater than the MTD109 of <NUM>.

<FIG> illustrates a flow chart outlining the logic to derive the processing gain to identify the position of the sound source <NUM>. The purpose of the system is to create an improved sound output signal <NUM> by combining the inputs from the individual microphone elements in the array in a way that increases the magnitude of the direct sound <NUM> received at the microphone array relative to the reverb and noise components. If the magnitude of the direct signal <NUM> is doubled relative to the reverb and noise signals, it will have roughly the same effect as halving the distance between the microphones <NUM> and the sound source <NUM>. The signal strength when the array is focused on a sound source <NUM> divided by the signal strength when the array is not focused on any sound source <NUM> (such as ambient background noise, for example) is defined as the processing gain of the system. The system preferably sets up thousands of listening positions within the room and simultaneously measures the processing gain at each of these locations. The virtual listening position with the largest processing gain is substantially the location of the sound source <NUM>. Of course, the processing of these flowcharts may be performed in any of the devices, servers, computers, FPGAs, DSPs, and/or ASICs described above.

To derive the processing gains <NUM>, the volume of the room where sound pickup is desired is preferably divided into a large number of virtual microphone positions. When the array is focused on a given virtual microphone, then any sound source within a close proximity of that location will produce an increased processing gain at that virtual microphone. The volume around each virtual microphone in which a sound source will produce maximum processing gain at that point, may be defined as a bubble. Based on the location of each microphone and the defined 3D location for each virtual microphone, and using the speed of sound which can be calculated given the current measured room temperature, the system can determine the expected propagation delay from each virtual microphone to each microphone array element <NUM>.

The flow chart in <FIG> illustrates the signal flow within the processing unit. This example monitors <NUM> bubbles simultaneously. The sound from each microphone element <NUM> is sampled at the same time as the other elements within the microphone array <NUM> and at a fixed rate of <NUM>. Each sample is passed to a microphone element processor <NUM>. The microphone element processor <NUM> conditions and aligns the signals in time and weights the amplitude of each sample so they can be passed on to the summing node <NUM>.

The signal components <NUM> from the microphones element processors <NUM> are summed at node <NUM> to provide the combined microphone array signal for each of the <NUM> bubbles. Each bubble signal is preferably converted into a power signal at node <NUM> by squaring the signal samples. The power signals are then summed over a given time window by the <NUM> accumulators at node <NUM>. The sums represent the signal energy over that time period. The processing gain for each bubble is preferably calculated at node <NUM> by dividing the energy of each bubble by the energy of an ideal unfocused signal <NUM>. The unfocused signal energy is preferably calculated by summing at <NUM> the energies of the signals from each microphone element <NUM> over the given time window, weighted by the maximum ratio combining weight squared. This is the energy that would be expected if all of the signals were uncorrelated. The processing gain <NUM> is preferably calculated for each bubble by dividing the microphone array signal energy by the unfocused signal energy <NUM>.

Processing Gain is achieved because signals from a common sound source all experience the same delay before being combined which results in those signals being added up coherently, meaning that their amplitudes add up. If <NUM> equal amplitude and time aligned direct signals <NUM> are combined the resulting signal will have an amplitude 12x higher, or a power level 144x higher. Signals from different sources and signals from the same source with significantly different delays, as the signals from reverb and noise do not add up coherently and do not experience the same gain. In the extremes, the signals are completely uncorrelated and will add up orthogonally. If <NUM> equal amplitude orthogonal signals are added up, the signal will have roughly 12x the power of the original signal or a <NUM>. 4x increase in amplitude (measured as nns). The difference between the 12x gain of the direct signal <NUM> and the <NUM>. 4x gain of the reverb and noise signals is the net processing gain (<NUM> or 11dB) of the microphone array when it is focused on the sound source <NUM>. This makes the signal sound as if you have moved the microphone <NUM><NUM>. 4x closer to the sound source. This example uses a <NUM> microphone array but it could be extended to an arbitrary number (N) resulting in a maximum possible processing gain of sqrt(N) or <NUM> log (N) dB.

The bubble processor system preferably simultaneously focuses the microphone array <NUM> on <NUM> points in <NUM>-D space using the method described above. The energy level of a short burst of sound signal (<NUM>-<NUM>) is measured at each of the <NUM> virtual microphone bubble points and compared to the energy level that would be expected if the signals combined orthogonally. This gives the processing gain <NUM> at each point. The virtual microphone bubble that is closest to the sound source should experience the highest processing gain and be represented as a peak in the output. Once determined, the location is known.

Node <NUM> searches through the output of the processing gain unit <NUM> for the bubble with the highest processing gain. The (x,y,z) location <NUM> of the virtual microphone corresponding to that bubble can then be determined by looking up the index in the original configuration to determine the exact location of the sound source. The parameters <NUM> maybe communicated to various electronic devices to steer and focus them to the identified sound source position.

After deriving the location of the sound source, focusing the microphone array on that sound source can be accomplished after achieving the gain. The bubble processor is preferably designed to find the sound source quickly enough so that the microphone array can be focused while the sound source is active, which can be a very short window of opportunity. The bubble processor system is preferably able to find new sound sources in less than <NUM>. Once found, the microphone array focuses on that location to pick up the sound source signal and the system reports the location of the sound through the Identify Source Signal Position <NUM> to other internal processes and to the host computer, so that it can implement sound sourced location based applications.

The embodiments described in this application have been presented with respect to use in one or more conference rooms preferably with multi users. However, the present invention may also find applicability in other environments such as: <NUM>. Commercial transit passenger and crew cabins such as, but not limited to, aircraft, busses, trains and boats. All of these commercial applications can be outfitted with microphones and can benefit from consist desired source volume and control of the ambient sound conditions which can vary from moderate to considerable; <NUM>. Private transportation such as cars, truck, and mini vans, where command and control applications and voice communication applications are becoming more prominent; <NUM>. Industrial applications such as manufacturing floors, warehouses, hospitals, and retail outlets to allow for audio monitoring and to facilitate employee communications without having to use specific portable devices; and <NUM>. Drive through windows and similar applications, where ambient sounds levels can be quite high and variable, can be controlled to consistent levels within the scope of the invention. Also, the processing described above may be carried out in one or more devices, one or more servers, cloud servers, etc..

Claim 1:
A method of controlling microphones (<NUM>) in a multi-microphone array (<NUM>) to provide a consistent volume for each desired sound source and a consistent ambient sound level, based on measured distance information from a sound source (<NUM>) to the multi-microphone array, in a shared physical space having the multi-microphone array and the sound source, the method comprising:
determining, using one or more processors (<NUM>), the measured distance information (<NUM>) corresponding to the sound source in the shared space; and
receiving, using the one or more processors (<NUM>), outputs from the microphones in the multi-microphone array and providing a combined sound signal by summing the received outputs, wherein the received outputs are time aligned and weighted based on the measured distance information to the sound source from the microphones in the shared space;
characterised in that the method further comprises:
controlling, using the one or more processors, the microphones of the multi-microphone array, wherein the controlling comprises:
selecting and enabling the microphones of the multi-microphone array, using the one or more processors, based on zones defined by the measured distance information of the sound source from the microphones of the multi-microphone array such that:
(i) when the sound source is outside of a configurable threshold distance (<NUM>), all microphones of the multi-microphone array are enabled;
(ii) when the sound source is between the configurable threshold distance and a minimum threshold distance (<NUM>), at least two microphones of the multi-microphone array are selected, and the at least two selected microphones are enabled differently depending on their individual distances from the sound source; and
(iii) when the sound source is within the minimum threshold distance, at least two microphones of the multi-microphone array are selected, and the at least two selected microphones are enabled such that at least one of the at least two selected microphones is enabled fractionally.