Abstract:
An automated security surveillance system ideally determines a location of a possible disturbance and adjusts its cameras to record video footage of the disturbance. In one embodiment, a disturbance can be determined by recording audio of the nearby area. A system, coupled to a camera, may include an arrangement of at least four audio sensors configured record audio of the nearby area to produce independent outputs. The system further may include a processing module configured to determine an angle and distance of an audio source relative to a location of the arrangement of the at least four audio sensors. The system can then adjust the camera by rotation along an azimuth or elevation angle and adjusting the zoom level to record video of the audio source. Through use of the system, a surveillance system can present an image of a source of possible disturbance to an operator more rapidly and precisely than through manual techniques.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    Security systems can employ cameras to secure or monitor a specific area. Some security systems provide manual controls for a user to rotate or zoom cameras to monitor an area. Other security systems have cameras rotate or zoom on a schedule to monitor different locations periodically. Based on the video footage, security personnel can deploy to an area to stop a current threat or disturbance. 
       SUMMARY OF THE INVENTION 
       [0002]    In one embodiment, a system may include an arrangement of at least four audio sensors configured to produce independent outputs. The system further may include a processing module configured to determine an angle and distance of an audio source relative to a location of the arrangement of the at least four audio sensors. 
         [0003]    In another embodiment, the processing module may be further configured to orient a camera lens of a camera system to the audio source. The camera system may be operationally coupled to the processing module. The processing module may be further configured to instruct the camera system to cause the camera lens to adjust its zoom to a zoom factor as a function of the distance of the audio source. In another embodiment, the processing module may be further configured to calculate an azimuth angle from the audio source to the arrangement and an elevation angle from the audio source to the arrangement. 
         [0004]    In another embodiment, the arrangement of audio sensors may include a central audio sensor and three surrounding audio sensors. Each of the three surrounding audio sensors may be positioned on a respective axis orthogonal to each other axis with an origin located at the central audio sensor. Distances between the audio sensors of the arrangement may be at least one order of magnitude smaller than the distance of the arrangement of audio sensors to the audio source. 
         [0005]    In another embodiment, the independent outputs of the at least four audio sources may be audio signals. The arrangement of at least four audio sources may be configured to produce a combined output of the distance of the audio source to the arrangement by correlating the audio signals received at the at least four audio sources. 
         [0006]    In another embodiment, method may include producing independent outputs from audio sensors from an arrangement of at least four audio sensors. The method may further include determining an angle and distance of an audio source relative to a location of the arrangement of the at least four audio sensors. 
         [0007]    In another embodiment, a non-transitory computer-readable medium can be configured to store instructions for locating an audio source. The instructions, when loaded and executed by a processor, may cause a system coupled to the processor to receive independent outputs from an arrangement of at least four audio sensors. The instructions may further cause the system to determine an angle and distance of an audio source relative to a location of the arrangement of the at least four audio sensors. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]    The foregoing will be apparent from the following more particular description of example embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments of the present invention. 
           [0009]      FIG. 1A  is a block diagram illustrating an example embodiment of a camera configured with an arrangement of audio sensors and a processing module to determine a location of an audio source and adjust the camera to rotate and zoom to the audio source. 
           [0010]      FIG. 1B  is a diagram illustrating an example embodiment of the processing module coupled with the camera. 
           [0011]      FIG. 2  is a diagram illustrating an embodiment employing audio sensors, an intra-sensor distance, and an audio source located away from the audio sensors by a distance to audio source, respectively. 
           [0012]      FIG. 3  is a diagram illustrating employing three audio sensors to find the location of the audio source. 
           [0013]      FIG. 4  is a block diagram illustrating an example embodiment of four audio sensors employed to determine the location of an audio source. 
           [0014]      FIG. 5  is a block diagram illustrating a camera and an audio source. 
           [0015]      FIG. 6  is a block diagram illustrating an example embodiment of a process employed by the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0016]    A description of example embodiments of the invention follows. 
         [0017]    An arrangement of at least four audio sensors can be configured to localize a location of an audio source. The direction of the audio source can be detected with three audio sensor; however, three audio sensors can only narrow the possible directions to two. A fourth sensor can isolate the location of the audio sensor. Four sensors can correctly identify the location of the audio source, but also the distance from the audio source to the arrangement of audio sensors as well. A video surveillance Pan/Tilt/Zoom (PTZ) camera installed with three audio sensors can be directed to the audio source, but if the audio source is far away from the camera, the camera cannot see the details of what caused the sound. If the distance to the audio source is known, then the camera can be zoomed-in according to the distance detected to see the details of an object or subject producing the audio source within the video. 
         [0018]      FIG. 1A  is a block diagram  100  illustrating an example embodiment of a camera  104  configured with an arrangement of audio sensors  102  and a processing module  112  to determine a location of an audio source  106  and adjust the camera to rotate and zoom to the audio source  106 . The camera  104  is generally located at a high location, for example, on top of a building  110 . The camera  104  can also be located on top of a pole or fence or connected to a wall. The system is configured to determine the location of the audio source  106  located on the ground  108 . A height of the camera  104  relative to the ground  108  is known (e.g., known to the processing module  112  and/control system  118 ). 
         [0019]    The camera  104  can be used for surveillance of areas around the building  110 . The camera  104  can, upon hearing audio from the audio source  106 , determine the location of the audio source  106 , rotate to point at the audio source  106  and zoom and/or focus on audio source  106  to acquire video. 
         [0020]    The camera  104  is operatively coupled with the processing module  112 , which is configured to receive data from the arrangement of audio sensors  102  of the camera  104  and output processed data  114  that indicates the location of the audio source  106 . The processed data  114  can be in the form of an azimuth (and/or pan) angle, an altitude (and/or tilt) angle, and/or the distance to the target. The processing module  112  also outputs a zoom or position command  122  to the camera  104 . The zoom or position command  122  can cause the camera  104  to rotate along the azimuth and/or elevation angle and zoom to the audio source  106 . 
         [0021]    In one embodiment, the processed data  114  is transmitted over a network  116  to a control system  118 . An operating user  120  can operate the control system  118  and see the location of the target relative to the camera. In another embodiment, the control system can output the location of the target relative to the building  110 . The operating user  120  can then act appropriately by, for example, dispatching security to the location of the audio source  106 . 
         [0022]      FIG. 1B  is a diagram  150  illustrating an example embodiment of the processing module  112  operatively coupled with the camera  104 . The camera  104 , as in  FIG. 1A , includes an arrangement of audio sensors  102 . The camera also includes a camera lens  152  and a motor  154 . Upon receiving a zoom or position command  122  from the processing module  112 , the motor  154  adjusts the camera  104  along an azimuth adjustment  156  and/or an elevation adjustment  158 , or both. Further, if the zoom or rotation command  122  is a zoom command, the camera  104  adjusts its camera lens  152  to zoom on the audio source. The camera  104  sends audio data from the arrangement of audio sensors  150  to the processing module  112 . The processing module, in response to the audio data  150 , sends the zoom or rotation command  122 . 
         [0023]      FIG. 2  is a diagram  200  illustrating an embodiment employing audio sensors  202  (M 1 ) and  204  (M 2 ), an intra-sensor distance d ( 206 ), and an audio source  214  located away from M 1  and M 2  with distance to audio source  210   a  (D 1 ) and  210   b  (D2), respectively. The difference in distance  212  (ΔD) between D 1  and D 2  can be computed by performing correlation (e.g., correlation or autocorrelation) on the audio signals received from the audio sensors  202  and  204  (M 1  and M 2 ). Both D 1  and D 2  are at least an order of magnitude larger than d (e.g., D 1 &gt;&gt;d and D 2 &gt;&gt;d). Under this assumption, the line from the audio source  214  to audio sensor  202  (M 1 ) and the line from the audio source  214  to audio sensor  204  (M 2 ) are approximately parallel. Therefore, angle Φ 1    208   a  is approximately congruent to angle Φ 2   208   b , or φ 1 ≈φ 2 , which can be both be represented by a common symbol φ because the two angles are equal. The angle, φ, can be computed by the formulae as follows: 
         [0000]    
       
         
           
             
               
                 cos 
                  
                 
                   ( 
                   φ 
                   ) 
                 
               
               ≈ 
               
                 
                   Δ 
                    
                   
                       
                   
                    
                   D 
                 
                 d 
               
             
             = 
             
               
                  
                 
                   
                     D 
                     1 
                   
                   - 
                   
                     D 
                     2 
                   
                 
                  
               
               d 
             
           
         
       
       
         
           
             φ 
             = 
             
               
                 cos 
                 
                   - 
                   1 
                 
               
                
               
                 ( 
                 
                   
                      
                     
                       
                         D 
                         1 
                       
                       - 
                       
                         D 
                         2 
                       
                     
                      
                   
                   d 
                 
                 ) 
               
             
           
         
       
     
         [0024]      FIG. 3  is a diagram illustrating employing three audio sensors  302  (M 1 ),  304  (M 2 ), and  306  (M 3 ) to find the location of the audio source  214 . Audio sensors  302  (M 1 ) and  304  (M 2 ) are positioned along an X-axis  350  and audio sensors  302  (M 1 ) and  306  (M 3 ) are on the Z-axis  354 . A distance to audio source  320   a  (D 1 ) represents the distance from the audio source  214  to audio sensor  302  (M 1 ). A distance to audio source  320   b  (D 2 ) represents distance from audio source  214  to audio sensor  304  (M 2 ). A distance to audio source  320   a  (D 3 ) represents distance from the audio source  214  to audio sensor  306  (M 3 ). Intra sensor distance  312  (d 2 ) represents the distance between audio sensor  302  (M 1 ) and audio sensor  304  (M 2 ). Intra-sensor distance  314  (d 3 ) represents distance between audio sensor  302  (M 1 ) and audio sensor  304  (M 3 ). 
         [0025]    Distances to audio source  320   a ,  320   b , and  320   c  (D 1 , D 2 , and D 3 , respectively) are approximately congruent (e.g., D 1 ≈D 2 ≈D 3 ) and are each represented collectively by D, such that D is at least an order of magnitude greater than d 2  and d 3  (e.g., D&gt;&gt;d 2 , and D&gt;&gt;d 3 ). Azimuth  310   a  (φ 1 ), is approximately congruent to azimuth  310   b  (φ 2 ) (e.g., φ 1 ≈φ 2 ) and both azimuths  310   a - b  are represented collectively by φ. Elevation angle  308   a  (θ 1 ) is approximately congruent to elevation angle  308   b  (θ 2 ) (e.g., φ 1 ≈φ 2 ) and are represented collectively by θ. The three lines from audio source  214  to audio sensors  302 ,  304 , and  306  (M 1 , M 2 , and M 3 , respectively) are approximately parallel. 
         [0026]    Azimuth (e.g., pan or horizontal) angles  310   a - b  and elevation (e.g., tilt or vertical) angle  308   a - b  are represented by the symbols φ and Θ, respectively. Azimuth and altitude can be computed by using similar formulae. The formulae are as follows: 
         [0000]    
       
         
           
             
               
                 cos 
                  
                 
                   ( 
                   φ 
                   ) 
                 
               
               ≈ 
               
                 
                   Δ 
                    
                   
                       
                   
                    
                   
                     D 
                     2 
                   
                 
                 
                   d 
                   2 
                 
               
             
             = 
             
               
                  
                 
                   
                     D 
                     1 
                   
                   - 
                   
                     D 
                     2 
                   
                 
                  
               
               
                 d 
                 2 
               
             
           
         
       
       
         
           
             φ 
             = 
             
               
                 cos 
                 
                   - 
                   1 
                 
               
                
               
                 ( 
                 
                   
                      
                     
                       
                         D 
                         1 
                       
                       - 
                       
                         D 
                         2 
                       
                     
                      
                   
                   
                     d 
                     2 
                   
                 
                 ) 
               
             
           
         
       
       
         
           
             
               
                 cos 
                  
                 
                   ( 
                   θ 
                   ) 
                 
               
               ≈ 
               
                 
                   Δ 
                    
                   
                       
                   
                    
                   
                     D 
                     3 
                   
                 
                 
                   d 
                   3 
                 
               
             
             = 
             
               
                  
                 
                   
                     D 
                     1 
                   
                   - 
                   
                     D 
                     3 
                   
                 
                  
               
               
                 d 
                 3 
               
             
           
         
       
       
         
           
             θ 
             = 
             
               
                 cos 
                 
                   - 
                   1 
                 
               
                
               
                 ( 
                 
                   
                      
                     
                       
                         D 
                         1 
                       
                       - 
                       
                         D 
                         3 
                       
                     
                      
                   
                   
                     d 
                     
                       3 
                        
                       
                           
                       
                     
                   
                 
                 ) 
               
             
           
         
       
     
         [0027]    φ and Θ correspond to the pan and tilt angles of the PTZ camera, respectively. 
         [0028]      FIG. 4  is a block diagram  400  illustrating an example embodiment of four audio sensors  304 ,  306 ,  302 , and  408  employed to determine the location of an audio source  214 . The system employed in  FIG. 4  is similar to the system of  FIG. 3 , except for the employment of the audio sensor  408  (M4), located a distance to audio source  420  (D4). Audio sensor  408  is employed to determine the precise location of the audio source  214 . Without the fourth audio sensor  408 , the location of the audio source  214  is known to be one of two locations. The fourth audio sensor  408  narrows the location down to one particular location. 
         [0029]      FIG. 5  is a block diagram  500  illustrating a camera  510  and an audio source  506 . The camera and sensors can be installed on top of a pole or on top of a building. The camera can be facing down to the ground, a common location of audio sources. H represents a height of the camera relative to the ground. The camera is angled toward a ground  512  at an elevation angle  508  θ. The formulae to compute a distance  504  (D) from camera to the audio source are as follows: 
         [0000]    
       
         
           
             
               sin 
                
               
                 ( 
                 θ 
                 ) 
               
             
             = 
             
               H 
               D 
             
           
         
       
       
         
           
             D 
             = 
             
               H 
               
                 sin 
                  
                 
                   ( 
                   θ 
                   ) 
                 
               
             
           
         
       
     
         [0030]    The system can also perform zoom adjustment. If the targeted object is an individual and the individual stands a distance from the camera such that zoom is needed to see the individual clearly, the system can zoom the camera to focus on the individual. 
         [0031]    Θ 0  represents the tilt angle of the camera when the camera centers on the person. Z 0  represents zooming factor of the camera needed to see the person clearly. D 0  represents a distance from the camera to the person. 
         [0032]    In actual detection and tracking, the pan angle (φ), tilt angle (Θ) and distance (D) from the camera to the audio source can be computed by using the equations described above. The zooming factor (Z) can be computed by the either of the following formulae: 
         [0000]    
       
         
           
             
               Z 
               
                 Z 
                 0 
               
             
             = 
             
               
                 D 
                 
                   D 
                   0 
                 
               
               = 
               
                 
                   sin 
                    
                   
                     ( 
                     
                       θ 
                       0 
                     
                     ) 
                   
                 
                 
                   sin 
                    
                   
                     ( 
                     
                       θ 
                       0 
                     
                     ) 
                   
                 
               
             
           
         
       
       
         
           or 
         
       
       
         
           
             Z 
             = 
             
               
                 Z 
                 0 
               
               · 
               
                 
                   sin 
                    
                   
                     ( 
                     
                       θ 
                       0 
                     
                     ) 
                   
                 
                 
                   sin 
                    
                   
                     ( 
                     θ 
                     ) 
                   
                 
               
             
           
         
       
     
         [0033]    The zooming factor controls the pan-tilt-zoom camera such that the size of the object seen in the image stays constant. Z 0 , D 0 , and θ 0  are determined in a calibration stage. Z 0  represents a zooming factor of an object at distance D 0  and with a tilt angle θ 0 . For instance, during the calibration stage, the system computes the tilt angle θ 0  of a person standing at a known location away from the camera with a known distance D 0  and using the height of the camera (H) and D 0  according to the following formula: 
         [0000]    
       
         
           
             
               sin 
                
               
                 ( 
                 
                   θ 
                   0 
                 
                 ) 
               
             
             = 
             
               H 
               
                 
                   
                     H 
                     2 
                   
                   + 
                   
                     D 
                     0 
                     2 
                   
                 
               
             
           
         
       
     
         [0034]    The system adjusts the camera to center at the person and adjusts the zooming factor to a number, Z 0 , which is a zooming factor where the person fills the image. These are used as the calibration variables for the system. 
         [0035]    Three audio sensors can isolate the correct audio source location to two possibilities but cannot eliminate one of the two possible audio source locations. Four audio sensors can identify the one location. Assuming the three audio sensors are on the X and Z axes, the following analysis applies to determining the location with three audio sensors; however, the three audio sensors can be on any combination of axes and similar analyses can apply. From  FIG. 3 , if (D 3 −D 1 )&gt;0 (or D 3 &gt;D 1 ), then the audio source is located below the X-Y plane (e.g., z&lt;0), and otherwise is located above the X-Y plane (z&gt;0). If (D 2 −D 1 )&gt;0 (or D 2 &gt;D 1 ), then the audio source is located at the left side of Y-Z plane (x&lt;0), and otherwise is located on the right side of Y-Z plane (x&gt;0). The audio source can be narrowed to two possible solutions by employing three audio sources: either y&gt;0 or y&lt;0. A fourth audio sensor positioned on the Y-axis can resolve whether y&gt;0 or y&lt;0. The result out of these four sensors is the direction of audio source in relation to the sensors. An arrangement of the sensors on different axes can change the direction of the uncertainty of the audio sources position, but the problem remains the same, and a fourth sensor is needed to resolve the position of the audio source with no ambiguity. Further, if an arrangement of three or more sensors is installed on each camera, and if two cameras include the arrangement of sensors, then the physical location of the audio source can be identified. 
         [0036]      FIG. 6  is a block diagram  600  illustrating an example embodiment of a process employed by the present invention. The process first records an audio source using an arrangement of audio sensors ( 602 ). Then, the process correlates audio signals from audio sensors to a distance of the audio sensor from the audio source. In one embodiment, the correlation can be autocorrelation. The process can further correlate all of the audio signals from the audio sensors with all of the other signals to determine the distance from each sensor. The process then calculates an azimuth (pan) and altitude (tilt) angles based on correlated distance and known positions of the audio sensors ( 606 ). The positions of the audio sensors are known relative to each other and also known relative to their positions in the real world, for example, being on top of a building indicating the height of the audio sensors. 
         [0037]    Then, the system calculates distance to the audio source based on a height of the arrangements of sensors and an altitude angle ( 608 ). Then, the system calculates the zoom factor based on distance or altitude angle, and height ( 610 ). Then, the system zooms the camera to the audio source ( 612 ). The system then rotates the camera to the audio source ( 614 ). The system then takes a picture and/or records video using the camera correctly oriented at the audio source. 
         [0038]    Embodiments or aspects of the present invention may be implemented in the form of hardware, software, or firmware. If implemented in software, the software may be any form of software capable of performing operations consistent with the example embodiments disclosed herein. The software may be stored in any non-transient computer readable medium, such as RAM, ROM, magnetic disk, or optical disk. When loaded and executed by processor(s), the processor(s) are configured to perform operations consistent with the example embodiments disclosed herein. The processor(s) may be any form of processor(s) capable of being configured to execute operations as disclosed herein. 
         [0039]    While this invention has been particularly shown and described with references to example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.