Abstract:
An energy based technique to estimate the positions of people speaking from an ad hoc network of microphones. The present technique does not require accurate synchronization of the microphones. In addition, a technique to normalize the gains of the microphones based on people&#39;s speech is presented, which allows aggregation of various audio channels from the ad hoc microphone network into a single stream for audio conferencing. The technique is invariant of the speaker&#39;s volumes thus making the system easy to deploy in practice.

Description:
BACKGROUND 
       [0001]    Typical audio conferencing systems use an array of microphones that are fixed in location relative to each other and are synchronized in order to capture the audio of a meeting. In this configuration, sound source localization (SSL) techniques can readily be used to determine the location of a person speaking. Once the sound source is localized, beam-forming can be used to output higher quality audio than if a single microphone was used. Additionally, if a camera is associated with the microphone array, the speaker&#39;s video can be displayed in conjunction with the captured audio. 
         [0002]    Often, however, the locations of microphones in a meeting room are not fixed or known. For example, meeting participants bring laptops or other computing devices with built-in microphones to a meeting. These laptops or other computing devices are usually wireless network enabled, so they can form an ad hoc network. Compared to traditional microphone array devices, these ad hoc microphone arrays are spatially distributed and the microphones in general are closer to the meeting participants. Thus, higher audio quality can be expected in capturing audio from a speaker (e.g., a person talking), assuming that the microphones used in the mobile computing devices and those in the fixed array devices have the same quality. On the other hand, microphones in an ad hoc arrangement present many challenges. For example, these microphones are not synchronized and the location of these microphones and associated computing devices such as laptop computers is unknown. Additionally, the microphones have different and unknown gains, and their quality is different (i.e., they have different signal to noise ratios). These factors present a problem in capturing a high quality audio recording of a meeting. 
       SUMMARY 
       [0003]    The present energy-based sound source localization and gain normalization technique for ad hoc microphones is an energy-based technique for locating speakers (e.g. people talking) when microphones are positioned in an ad hoc manner. This technique does not require accurate time synchronization. In fact, the present energy-based sound source localization and gain normalization technique only uses the average energy of the meeting participants&#39; speech signals to obtain a position estimate of a person speaking. 
         [0004]    Given that the microphones in associated computing devices configured in an ad hoc network are spatially distributed, a person speaking in a room is usually relatively close to one of the microphones. Therefore, the present energy-based sound source localization technique selects the signal from the microphone that is closest to the speaker (the signal that has the best signal to noise ratio (SNR)) when determining the locations of the microphones and the people speaking in the room. The present energy-based sound source localization and gain normalization technique estimates the relative gains of the microphones using meeting participants&#39; speech signals. In general, one embodiment of the present energy-based sound source localization and gain normalization technique first computes the speakers&#39; and microphone positions as well as the gain of the microphones, assuming that every person speaking has a computing device with a microphone. Then the technique is used to determine the position of people speaking that do not have their own associated computing device. The coordinates of the microphones/people and the gain of each microphone can then be used to improve the audio or video of the meeting. Purposes may include, for example, beam-forming to improve the quality of the audio, sound source localization in order to locate the speaker and display associated video or contact information, and aggregating various audio channels from the ad hoc microphone network into a single gain normalized stream for audio conferencing. 
         [0005]    It is noted that while the foregoing limitations in existing sound source localization schemes described in the Background section can be resolved by a particular implementation of the present energy-based sound source localization and gain normalization technique, this is in no way limited to implementations that just solve any or all of the noted disadvantages. Rather, the present technique has a much wider application as will become evident from the descriptions to follow. 
         [0006]    This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. 
         [0007]    In the following description of embodiments of the present disclosure reference is made to the accompanying drawings which form a part hereof, and in which are shown, by way of illustration, specific embodiments in which the technique may be practiced. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present disclosure. 
     
    
     
       DESCRIPTION OF THE DRAWINGS 
         [0008]    The specific features, aspects, and advantages of the disclosure will become better understood with regard to the following description, appended claims, and accompanying drawings where: 
           [0009]      FIG. 1  is a diagram depicting a general purpose computing device constituting an exemplary system for a implementing a component of the present energy-based sound source localization and gain normalization technique. 
           [0010]      FIG. 2  is a diagram depicting an exemplary environment wherein the present energy-based sound source localization and gain normalization technique could be employed. 
           [0011]      FIG. 3  is a block diagram depicting one embodiment of the architecture of the present energy-based sound source localization and gain normalization technique&#39;s architecture. 
           [0012]      FIG. 4  is a block diagram depicting one embodiment of the architecture of the present energy-based sound source localization and gain normalization technique that determines the location of people speaking that have an associated computing device with a microphone. 
           [0013]      FIG. 5  is a block diagram depicting one embodiment of the present energy-based sound source localization and gain normalization technique that determines the location of people speaking that do not have an associated computing device with a microphone. 
           [0014]      FIG. 6  is a flow diagram depicting one exemplary embodiment of the present energy-based sound source localization and gain normalization technique wherein each speaker has a computing device with a microphone. 
           [0015]      FIG. 7  is a flow diagram depicting one exemplary technique of segmenting the input audio streams to determine which speaker is associated with which microphone. 
           [0016]      FIG. 8  is a flow diagram depicting one exemplary embodiment of the present sound source localization technique wherein each speaker does not have a computing device with a microphone. 
           [0017]      FIG. 9  is a block diagram depicting another embodiment of the architecture of the present energy-based sound source localization and gain normalization technique&#39;s architecture wherein the position of any give speaker is computed without assuming that the speakers and the microphones are co-located. 
           [0018]      FIG. 10  is a flow diagram depicting one exemplary embodiment of the present sound source localization technique wherein the position of each speaker is computed without assuming that the speakers and microphones are co-located. 
       
    
    
     DETAILED DESCRIPTION 
     1.0 The Computing Environment 
       [0019]    Before providing a description of embodiments of the present energy-based sound source localization and gain normalization technique, a brief, general description of a suitable computing environment in which portions thereof may be implemented will be described. The present technique is operational with numerous general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. 
         [0020]      FIG. 1  illustrates an example of a suitable computing system environment. The computing system environment is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the present sound source localization technique. Neither should the computing environment be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment. With reference to  FIG. 1 , an exemplary system for implementing the present energy-based sound source localization and gain normalization technique includes a computing device, such as computing device  100 . In its most basic configuration, computing device  100  typically includes at least one processing unit  102  and memory  104 . Depending on the exact configuration and type of computing device, memory  104  may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. This most basic configuration is illustrated in  FIG. 1  by dashed line  106 . Additionally, device  100  may also have additional features/functionality. For example, device  100  may also include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape. Such additional storage is illustrated in  FIG. 1  by removable storage  108  and non-removable storage  110 . Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Memory  104 , removable storage  108  and non-removable storage  110  are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by device  100 . Any such computer storage media may be part of device  100 . 
         [0021]    Device  100  may also contain communications connection(s)  112  that allow the device to communicate with other devices. Communications connection(s)  112  is an example of communication media. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. The term computer readable media as used herein includes both storage media and communication media. 
         [0022]    Device  100  has a microphone and may also have other input device(s)  114  such as keyboard, mouse, pen, voice input device, touch input device, and so on. Output device(s)  116  such as a display, speakers, a printer, and so on may also be included. All of these devices are well known in the art and need not be discussed at length here. 
         [0023]    Device  100  can include a camera as an input device  114  (such as a digital/electronic still or video camera, or film/photographic scanner), which is capable of capturing a sequence of images, as an input device. Further, multiple cameras could be included as input devices. The images from the one or more cameras can be input into the device  100  via an appropriate interface (not shown). However, it is noted that image data can also be input into the device  100  from any computer-readable media as well, without requiring the use of a camera. 
         [0024]    The present energy-based sound source localization and gain normalization technique may be described in the general context of computer-executable instructions, such as program modules, being executed by a computing device. Generally, program modules include routines, programs, objects, components, data structures, and so on, that perform particular tasks or implement particular abstract data types. The present energy-based sound source localization and gain normalization technique may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices. 
         [0025]    The exemplary operating environment having now been discussed, the remaining parts of this description section will be devoted to a description of the program modules embodying the present sound source localization technique. 
         [0026]    2.0 Energy-based Sound Source Localization and Gain Normalization Technique 
         [0027]    The following paragraphs discuss an exemplary operating environment, overviews of exemplary systems and processes employing the energy-based sound source localization and gain normalization technique, and details regarding the various embodiments. 
         [0028]    2.1 Exemplary Operating Environment 
         [0029]      FIG. 2  depicts an exemplary environment in which the present energy-based sound source localization and gain normalization technique can be practiced. The present energy-based sound source localization and gain normalization technique is typically used in a meeting room environment. As in a typical meeting, meeting participants  202  can sit around a table  204 , or can sit away from the table. Each meeting participant  202  can have a notebook computer  206  or other computing device  208  equipped with a microphone and the capability to link to a wireless network via a wireless access point  210 . Alternately the computers can plug into a standard network. Audio streams of each person speaking are captured by the microphones in the laptops or other computing devices  206 ,  208  and are transferred to a computer  212  for processing. The computer  212  can be controlled by a keyboard  214  or other input device  216  and typically has a display  218 . It can also have inputs from a camera  220  and other devices. The computer  212  can also be networked with other computers and can send and receive data to and from these other devices. The energy-based sound source localization and gain normalization technique can compute the location of the microphones in the laptop computer and computing devices, the locations of each person speaking and the gains of the microphones. These parameters can be used in beam-forming to improve the quality of the audio recorded or to locate a speaker and display his image on the display  218 , or transmit his image over a network, if associated video is available. Other uses of this data include gain normalization and displaying the speaker&#39;s contact information, among others. 
         [0030]    2.2 System Overview 
         [0031]      FIG. 3  provides an overview of one embodiment of the present energy-based sound source localization and gain normalization system  300 . The modules shown are typically resident on computer  212 . In general, one embodiment of the present energy-based sound source localization and gain normalization technique has a first module  302  that can compute the speakers&#39; and microphone positions as well as the gain of the microphones, assuming that every person speaking has a computing device with a microphone, by using the average energy of an audio segment for each person speaking. The technique can also employ a second module  304  that determines the position of people speaking that do not have their own associated computing device, again employing the average energy for each person speaking. Finally, the coordinates of the microphones, speakers and the gain of one or more of the microphones are then used to improve the audio or video of the meeting by an application module  306 . Such applications may include, for example, beam-forming to improve the quality of the audio, sound source localization in order to locate the speaker and display associated video or contact information, and aggregating various audio channels from the ad hoc microphone network into a single gain normalized stream for audio conferencing. It should be noted that modules  302  and  304  can be employed alone, without the other module. 
         [0032]    More specifically, as shown in  FIG. 4 , in one exemplary embodiment of the present energy-based sound source localization and gain normalization technique, it is assumed that the location of a person speaking and their computing device with microphone are co-located. Audio streams from all people speaking in the room are input. The system employs a segmentation module  402  to segment the received audio streams from each person in the room to find the average energy of an audio segment for each of the people speaking and to identify which microphone/computing device corresponds to a given person. Another module, the attenuation and gain computation module  404 , computes the attenuation of a person&#39;s speech when it reaches each of the microphones and can also compute the gain of each of the microphones. A distance determination module  406  uses these attenuations to find the distance between each microphone relative to the other microphones. A coordinate determination module  408  then uses the distances between the microphones to find the coordinates of each microphone, which are also assumed to be the coordinates of the person associated with that microphone/computing device. The coordinates of the microphones/people and the gain of each microphone can then be used by the application module  306  which applies the locations of the microphones, speakers and gains for some purpose. 
         [0033]    The present energy-based sound source localization and gain normalization technique also can compute the location of people speaking that do not have their own computing device with a microphone. In one embodiment, shown in  FIG. 5 , this is done by employing an average energy ratio computing module  502  that computes the ratio of the average energy of the audio stream of a speaker that is not closest to a microphone to a first microphone over the average energy of the audio stream of the speaker that is not closest to a microphone to a second microphone. This average energy ratio is then input into an attenuation ratio computing module  504  where it is used to compute an attenuation ratio, the ratio of the attenuation of the audio stream of the speaker that is not closest to a microphone to a first microphone over the attenuation of the audio stream of the speaker that is not closest to a microphone to a second microphone. Once the attenuation ratio is found it is input into a distance ratio computation module  506  where it is used to find a distance ratio, the ratio of the distance of the speaker that is not closest to a microphone to a first microphone over the distance of the speaker that is not closest to a microphone to a second microphone. Finally, the distance ratio is used to find the coordinates of the speaker that is not closest to a microphone in a coordinate finding module  508 . 
         [0034]    2.3 Exemplary Energy-based Sound Source Localization and Gain Normalization Process 
         [0035]      FIG. 6  provides a flow diagram of one exemplary embodiment of the present energy-based sound source localization and gain normalization process  600  where each person in the room has a laptop computer or other computing device with a microphone. Audio streams y are input from i microphones and j speakers, as shown in block  602 . The audio streams are segmented in order to find the average energy of an audio segment from each microphone i that corresponds to the jth person&#39;s speech, as shown in block  604 . For all speaker&#39;s, the attenuation C ij  of person J&#39;s speech when it reaches each microphone i, is computed (block  606 ). Optionally the gain of each microphone can also be computed (block  606 ). The attenuation C ij  is then used to find the distance D ij  between each microphone (block  608 ). The distance D ij  between each microphone is then used to find the coordinates of each microphone (block  610 ). 
         [0036]      FIG. 7  provides an exemplary segmentation process that can be employed by the present energy-based sound source localization and gain normalization process. As shown in  FIG. 7 , block  702 , each person speaking corresponding to each computing device is recorded in an audio file. The audio files are divided into segments by detecting the first speech frame through simple thresholding, and the audio segments are aligned across audio files (block  704 ). Speaker segmentation is performed for each audio segment by finding the audio file that has the highest signal to noise ratio (SNR) (block  706 ). The speaker of each audio segment is then associated with the microphone that captured the audio file that produces the highest SNR for that segment. 
         [0037]    Mathematically the above exemplary embodiment of the present energy-based sound source localization and gain normalization process can be described as follows. As mentioned above, it is assumed that meeting participants are in a room and have their laptops or other computing device in front of them. It is further assumed that each laptop or other computing device has an associated microphone and that the laptops are connected by a network. 
         [0038]    It is assumed there are m laptops, and for ease of description, that each person speaks once. Let y i (t),i=1, . . . , m denote the audio stream captured by the ith laptop. Let a ij  denote the average energy of the audio segment in y i (t) that corresponds to jth person&#39;s speech. Let s j  denote the average energy of jth person&#39;s original speech which is unknown and let c ij  denote the attenuation of person j&#39;s speech when it reaches microphone i. Let m i  denote the gain of the microphone on laptop i. It is possible to model the average energy of the audio segment in y i (t), a ij , that corresponds to the jth person&#39;s speech, as the gain of microphone, m i , times j&#39;s original speech, s j  , times c ij , the attenuation of person j&#39;s speech when it reaches microphone i, as shown below: 
         [0000]      a ij =m i s j c ij    (1) 
         [0039]    The assumption is made that each speaker and the associated computing device/laptop are at the same location. Thus, c ij =c ji , and c ii =1. 
         [0040]    From equation (1), the ratio of a ij /a ii  can be calculated as 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         a 
                         ij 
                       
                       
                         a 
                         ii 
                       
                     
                     = 
                     
                       
                         
                           
                             m 
                             i 
                           
                            
                           
                             s 
                             j 
                           
                            
                           
                             c 
                             ij 
                           
                         
                         
                           
                             m 
                             i 
                           
                            
                           
                             s 
                             i 
                           
                         
                       
                       = 
                       
                         
                           
                             s 
                             j 
                           
                            
                           
                             c 
                             ij 
                           
                         
                         
                           s 
                           i 
                         
                       
                     
                   
                    
                   
                     
 
                   
                    
                   and 
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       a 
                       jj 
                     
                     
                       a 
                       ji 
                     
                   
                   = 
                   
                     
                       
                         
                           m 
                           j 
                         
                          
                         
                           s 
                           j 
                         
                       
                       
                         
                           m 
                           j 
                         
                          
                         
                           s 
                           i 
                         
                          
                         
                           c 
                           ji 
                         
                       
                     
                     = 
                     
                       
                         s 
                         j 
                       
                       
                         
                           s 
                           i 
                         
                          
                         
                           c 
                           ji 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Multiplying equations (2) and (3), one obtains 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           a 
                           ij 
                         
                          
                         
                           a 
                           jj 
                         
                       
                       
                         
                           a 
                           ii 
                         
                          
                         
                           a 
                           ji 
                         
                       
                     
                   
                   = 
                   
                     
                       s 
                       j 
                     
                     
                       s 
                       i 
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Substituting equation (4) into (2), one can obtain c ij , the attenuation of person j&#39;s speech when it reaches microphone i, as 
         [0000]    
       
         
           
             
               
                 
                   
                     c 
                     ij 
                   
                   = 
                   
                     
                       
                         
                           a 
                           ij 
                         
                         
                           a 
                           ii 
                         
                       
                        
                       
                         
                           
                             
                               a 
                               ii 
                             
                              
                             
                               a 
                               ji 
                             
                           
                           
                             
                               a 
                               ij 
                             
                              
                             
                               a 
                               jj 
                             
                           
                         
                       
                     
                     = 
                     
                       
                         
                           
                             a 
                             ij 
                           
                            
                           
                             a 
                             ji 
                           
                         
                         
                           
                             a 
                             ii 
                           
                            
                           
                             a 
                             jj 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Equation (5) has the following properties: it is independent of the gains of the microphone in computing device or laptop i, and it is invariant of the scaling of the speech energy. For example, if a ji  and a ii  are multiplied by the same value, the right hand side remains the same. 
         [0041]    If d ij  denotes the Euclidean distance between laptop i and j, clearly the attenuation of person j&#39;s speech when it reaches microphone i, c ij , is a function of d ij . Theoretically speaking, audio energy is inversely proportional to the square of the distance between the sound source and the microphone. However, empirical data taken in several meeting rooms has shown that d ij  is approximately a linear function of 1/c ij . It is believed that there is a linear relationship because of room reverberation, environmental and sensor noises, occlusions, and the relatively small distances between the microphones and people speaking. Based on this observation, one may set the distance between the sound source (the person speaking) and the microphone equal to the inverse of d ij =1/c ij , thus obtaining the distance between each pair of microphones. Metric Multidimensional Scaling (MDS) may then be used to obtain the 2D coordinates for each microphone. 
         [0042]    The present energy-based sound source localization and gain normalization technique then can also compute the location of people speaking that do not have their own computing device with a microphone. In one embodiment, shown in  FIG. 8 , as shown in block  802 , this is done by first computing an average energy ratio which is the ratio of the average energy of the audio stream of a speaker that does not have a microphone to a first microphone over the average energy of the audio stream of the speaker that does not have a microphone to a second microphone. This average energy ratio, as shown in block  804 , is then used to compute an attenuation ratio, the ratio of the attenuation of the audio stream of the speaker that does not have a microphone to a first microphone over the attenuation of the audio stream of the speaker that does not have a microphone to a second microphone. Once the attenuation ratio is found it is used to find a distance ratio, the ratio of the distance of the speaker that is not closest to a microphone to a first microphone over the distance of the speaker that is not closest to a microphone to a second microphone (block  806 ). Finally, the distance ratio is used to find the coordinates of the speaker that is not closest to a microphone (block  808 ). 
         [0043]    This situation where there are meeting participants who do not have laptops or another computing device with a microphone, can mathematically be described as follows. Let one assume speaker k has no computing device with a microphone. Note that one cannot apply equation (5) to compute the attenuation of person k&#39;s speech when it reaches microphone i, c ik , because the average energies a ki  and a kk  are not available. But for any given two laptops i and j, one can compute the ratio c ik /c jk . Given two microphones i and j, in order to estimate speaker k&#39;s position, one computes the distance ratio using the attenuation of speaker k&#39;s audio to microphone i over speaker k&#39;s attenuation to microphones. 
         [0044]    From equation (1), one obtains the average energy ratio: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       a 
                       ik 
                     
                     
                       a 
                       jk 
                     
                   
                   = 
                   
                     
                       
                         
                           m 
                           i 
                         
                          
                         
                           s 
                           k 
                         
                          
                         
                           c 
                           ik 
                         
                       
                       
                         
                           m 
                           j 
                         
                          
                         
                           s 
                           k 
                         
                          
                         
                           c 
                           jk 
                         
                       
                     
                     = 
                     
                       
                         
                           m 
                           i 
                         
                          
                         
                           c 
                           ik 
                         
                       
                       
                         
                           m 
                           j 
                         
                          
                         
                           c 
                           jk 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Thus, one may compute the ratio of the attenuation of person k&#39;s speech when it reaches microphone i, c ik , to the attenuation of person k&#39;s speech when it reaches microphone j, c ik , as a function of the average energy ratio and the ratio of the gain of microphones to the gain of microphone i, that is: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       c 
                       ik 
                     
                     
                       c 
                       jk 
                     
                   
                   = 
                   
                     
                       
                         a 
                         ik 
                       
                       
                         a 
                         jk 
                       
                     
                      
                     
                       
                         m 
                         j 
                       
                       
                         m 
                         i 
                       
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Again from equation (1), one obtains 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       a 
                       ji 
                     
                     
                       a 
                       ii 
                     
                   
                   = 
                   
                     
                       
                         
                           m 
                           j 
                         
                          
                         
                           s 
                           i 
                         
                          
                         
                           c 
                           ji 
                         
                       
                       
                         
                           m 
                           i 
                         
                          
                         
                           s 
                           i 
                         
                       
                     
                     = 
                     
                       
                         
                           m 
                           j 
                         
                          
                         
                           c 
                           ji 
                         
                       
                       
                         m 
                         i 
                       
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Therefore, the ratio of the gain of microphone j to the gain of microphone i: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       m 
                       j 
                     
                     
                       m 
                       i 
                     
                   
                   = 
                   
                     
                       
                         a 
                         ji 
                       
                       
                         a 
                         ii 
                       
                     
                      
                     
                       1 
                       
                         c 
                         ji 
                       
                     
                   
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Substituting equation (9) into (7), one obtains 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       c 
                       ik 
                     
                     
                       c 
                       jk 
                     
                   
                   = 
                   
                     
                       
                         a 
                         ik 
                       
                       
                         a 
                         jk 
                       
                     
                      
                     
                       
                         a 
                         ji 
                       
                       
                         a 
                         ii 
                       
                     
                      
                     
                       1 
                       
                         c 
                         ji 
                       
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
         [0045]    Notice that c ji  can be computed from equation (5). Thus, one is able to compute c ik /c jk  by using equation (10). Therefore the distance ratio is obtained by 
         [0000]    
       
         
           
             
               
                 
                   
                     
                        
                       jk 
                     
                     
                        
                       ik 
                     
                   
                   = 
                   
                     
                       
                         a 
                         ik 
                       
                       
                         a 
                         jk 
                       
                     
                      
                     
                       
                         a 
                         ji 
                       
                       
                         a 
                         ii 
                       
                     
                      
                     
                       1 
                       
                         c 
                         ji 
                       
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
         [0046]    Let P i  and P j  denote the coordinates of laptop i and j, respectively. Notice that P i  and P j  can be computed by using the method described in the previous section. Let P k  denote the unknown coordinate of speaker k. Then one obtains 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                            
                           
                             
                               P 
                               k 
                             
                             - 
                             
                               P 
                               j 
                             
                           
                            
                         
                         2 
                       
                     
                     
                       
                         
                            
                           
                             
                               P 
                               k 
                             
                             - 
                             
                               P 
                               i 
                             
                           
                            
                         
                         2 
                       
                     
                   
                   = 
                   
                     
                        
                       jk 
                     
                     
                        
                       ik 
                     
                   
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
           
         
       
     
         [0047]    If there are m laptops. There are 
         [0000]    
       
         
           
             
               ( 
               
                 
                   
                     m 
                   
                 
                 
                   
                     2 
                   
                 
               
               ) 
             
               
           
         
       
     
         [0000]    equations. When m≧3, one obtains enough equations to solve for the two coordinates of speaker k. The system of equations in (12) can be solved by a nonlinear least square solver. 
       2.4 Gain Normalization 
       [0048]    Equation (9) is a formula to compute the gain ratios between any two microphones. To normalize the gains across the microphones, one only needs to pick one of the microphones, say, microphone  1 , as the reference microphone, and multiply the audio signal of the j&#39;th microphone by 
         [0000]    
       
         
           
             
               
                 
                   m 
                   1 
                 
                 
                   m 
                   j 
                 
               
             
             . 
           
         
       
     
       2.5 Alternate Embodiments 
       [0049]    It should be noted that numerous other alternate embodiments to the above described energy-based sound source localization and gain normalization procedure are possible. For example, Equation (12) can also be used to estimate the coordinates of a speaker with a laptop or another computing device with a microphone, thus without assuming that the speaker and their microphone are co-located. In this embodiment of the present energy-based sound source localization and gain normalization technique has a first module  902  that computes one or more speakers&#39; positions by using the average energy of an audio segment for each person speaking. The coordinates of one or more speakers can then be used to improve the audio or video of the meeting by an application module  904 . More specifically, as shown in  FIG. 10 , block  1002 , this can be done for a given speaker by computing an average energy ratio which is the ratio of the average energy of the audio stream of a speaker to a first microphone over the average energy of the audio stream of the speaker to a second microphone. This average energy ratio, as shown in block  1004 , is then used to compute an attenuation ratio, the ratio of the attenuation of the audio stream of the speaker to the first microphone over the attenuation of the audio stream of the speaker to the second microphone. Once the attenuation ratio is found it is used to find a distance ratio, the ratio of the distance of the speaker to the first microphone over the distance of the speaker to the second microphone (block  1006 ). Finally, the distance ratio is used to find the coordinates of the speaker (block  1008 ). 
         [0050]    Additionally, it is possible to find the position of a person by employing the log domain. For example, by using the following equation in the log domain the location of a speaker can be approximated: 
         [0000]    
       
         
           
             
               
                 
                   
                     ln 
                      
                     
                         
                     
                      
                     
                       a 
                       ij 
                     
                   
                   = 
                   
                     
                       ln 
                        
                       
                           
                       
                        
                       
                         m 
                         i 
                       
                     
                     + 
                     
                       ln 
                        
                       
                           
                       
                        
                       
                         s 
                         j 
                       
                     
                     + 
                     
                       ln 
                        
                       
                           
                       
                        
                       λ 
                     
                     - 
                     
                       ln 
                        
                       
                         
                           
                             
                               ( 
                               
                                 
                                   u 
                                   i 
                                 
                                 - 
                                 
                                   x 
                                   j 
                                 
                               
                               ) 
                             
                             2 
                           
                           + 
                           
                             
                               ( 
                               
                                 
                                   v 
                                   i 
                                 
                                 - 
                                 
                                   y 
                                   j 
                                 
                               
                               ) 
                             
                             2 
                           
                         
                       
                     
                     + 
                     
                       N 
                        
                       
                         ( 
                         
                           0 
                           , 
                           
                             σ 
                             i 
                             2 
                           
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where (u i , v i ) are the coordinates of the microphones, (x j , y j ) are the coordinates of speaker j, a ij  is the energy of the audio segment in y i (t) that corresponds to the jth person&#39;s speech, s j  denote the average energy of j th person&#39;s original speech, m, denotes the gain of the microphone i. The term 1n λ is a scalar that can be estimated using empirical data and the term N(0,σ i   2 ) represents the noise of measurements with microphone i. The unknown parameters {(u i ,v i ),(x j ,y j ),m i ,s j } can be estimated by minimizing the following weighted sum of squares: 
         [0000]    
       
         
           
             
               
                 
                   
                     ∑ 
                     
                       i 
                       , 
                       j 
                     
                   
                    
                   
                     
                       
                         1 
                         
                           σ 
                           i 
                           2 
                         
                       
                        
                       
                         [ 
                         
                           
                             ln 
                              
                             
                               ( 
                               
                                 
                                   m 
                                   i 
                                 
                                  
                                 
                                   s 
                                   j 
                                 
                                  
                                 λ 
                               
                               ) 
                             
                           
                           - 
                           
                             ln 
                              
                             
                               ( 
                               
                                 
                                   a 
                                   ij 
                                 
                                  
                                 
                                   
                                     
                                       
                                         ( 
                                         
                                           
                                             u 
                                             i 
                                           
                                           - 
                                           
                                             x 
                                             j 
                                           
                                         
                                         ) 
                                       
                                       2 
                                     
                                     + 
                                     
                                       
                                         ( 
                                         
                                           
                                             v 
                                             i 
                                           
                                           - 
                                           
                                             y 
                                             j 
                                           
                                         
                                         ) 
                                       
                                       2 
                                     
                                   
                                 
                               
                               ) 
                             
                           
                         
                         ] 
                       
                     
                     2 
                   
                 
               
               
                 
                   ( 
                   14 
                   ) 
                 
               
             
           
         
       
     
         [0000]    One of m i &#39;s, e.g., m 1 , should be set to an arbitrarily positive number, say 1, since one can only obtain microphones&#39; relative gains. The term σ I   2  is the variance of the noise measurements with microphone i. 
         [0051]    It should also be noted that any or all of the aforementioned embodiments throughout the description may be used in any combination desired to form additional hybrid embodiments.