Patent Publication Number: US-2009219389-A1

Title: Detection of Smoke with a Video Camera

Description:
The invention relates to a method and a device for detection of smoke by analysis of at least one video image recorded by a video camera monitoring an area. 
     In recent times there have been efforts made to make use of the video systems available in any event for security monitoring in buildings, tunnels etc. for detection of smoke. Since the video images are very often of little interest to an observer and in addition smoke would only make small changes to the video image, there is no question of monitoring by the personnel at the screens. If it can be done at all, the monitoring can only be undertaken by an automatic evaluation of the video images. With a known method for automatically investigating video images for the occurrence of smoke the intensity values of the individual pixels in consecutive images are compared to one another. If intensity values are measured which are representative of a brighter image caused by the presence of smoke, it is concluded that smoke is present and an alarm is triggered. 
     One of the problems which arises with this method is that smoke is not detected against a bright background and even fire which generates only a little smoke is not detectable. In addition changes in brightness, as might be caused for example by a person moving through the field of view of the camera, can trigger a false alarm. Attempts have been made to resolve this problem by investigating a further outer area in addition to the actual monitoring area and, on changes in this outer area, interrupting the observation of the monitoring area. The disadvantage of this method is that a fire is not detected in certain circumstances until after a certain delay, and that sources of smoke in the outer area provided in addition to the monitoring area are not detected. 
     The object of the present invention is to be seen as proposing an efficient option for the detection of smoke by means of at least one video image recorded by a video camera monitoring an area. 
     The inventive object is achieved in each case by the subject matter of the independent claims. Further developments of the invention are specified in the subclaims. 
     A key element of the invention is to be seen as the detection of smoke by analysing at least one video image recorded by a video camera monitoring an area. An area in such cases can be a room, a tunnel (section), a car park, a street or a section of a street etc. Basically a first step checks, by determining the direction and the size for a moving area of the at least one video image, a probability of smoke being present in the moving area. If a moving area yields a positive test result, there is thus a degree of probability that smoke is present. Thereafter at least a part of the moving area is evaluated for the presence of smoke, depending on at least one item of information characteristic for smoke. Inventively information characteristic of smoke is seen as the speed of the smoke, the number of pixels in the video image which describe this movement, the luminance change (brightness change) of the at least one video image in relation to the background, the change of colour of the moving smoke and the movement of the smoke. 
     An advantage of the inventive method or the inventive device is to be seen as the ability to detect smoke in an efficient manner. This is obtained particularly by the two-part evaluation and by the suitable selection of the characteristic information for the smoke. 
    
    
     
       The invention will be explained in greater detail on the basis of an exemplary embodiment shown in a figure. The figures show 
         FIG. 1  a block diagram according to the invention for detection of smoke, 
         FIG. 2  a simplified presentation of a video image, 
         FIG. 3  a decision diagram for the detection of smoke, 
         FIG. 4  an inventive device. 
     
    
    
       FIG. 1  shows a block diagram in accordance with the invention for detection of smoke. At least one intensity image [X ij (t)] is obtained from at least one video image, which was created with a particular frequency. The video image can in this case have a size of 352×288 pixels for example. The next step is pre-processing. The aim of the pre-processing is that the areas which are of interest for the detection of smoke are filtered out of the video image. To this end a background accumulation matrix [B ij (t)] is first created. The background accumulation matrix [B ij (t)] is obtained from the intensity images [X ij (t)] weighted with a weighting factor, with the weighting factor specifying how strongly the intensity images flow into the accumulation matrix [B ij (t)] 
     The accumulation matrix is determined as follows: 
         Bij ( t )=α Bij ( t− 1)+(1−α) Xij ( t ), α=weighting factor 
     Next a subtraction matrix D ij (t)=/B ij (t)−X ij (t)/is computed for at least one moving area. Finally the colour-weighted subtraction matrix [S ij (t)] is obtained from the colour weighting of the subtraction matrix D ij (t). 
     This subtraction matrix [S ij (t)] is computed from 
         S   ij ( t )=Luma{ D   ij ( t )}×{1−|Chroma U{D   ij ( t )}−Chroma V{D   ij ( t )}|}, 
     with Luma {D ij } being the luma component of D ij , ChromaU (D ij ) the U-chroma component of D ij  and ChromaU (D ij ) the V-chroma component of D ij . 
     The probable presence of smoke at location (i, j) is finally determined for example by the projection of the colour-weighted subtraction matrix [S ij (t)] onto the x/y axis of a Cartesian coordinate system. 
     The projection onto a Cartesian coordinate system appears as follows: 
       [ i   m   ,j   m ]( t )={( i,j )| i =max { x  projection of  S   ij ( t )}, 
         j =max { y  projection of  S   ij ( t )}} 
     x projection of S ij (t): P xi (t)=S i0 (t)+S i1 (t)+S i2 (t)+ . . . +S IV (t)
 
y-projection of S ij (t): p yj (t)=S 0j (t)+S 1j (t)+S 2j (t)+ . . . +S Hj (t)
 
     The size of S ij  in this example is H×V (H=speed of the smoke×the movement of the smoke=V). Obviously any given coordinate system can be chosen. Thus spherical coordinates, cylindrical coordinates etc. could also be used for example. 
     A probable presence of smoke in a moving area of the video image can then be checked with the aid of the colour-weighted subtraction matrix [S ij (t)]. For a probable presence of smoke a region of interest (ROI) in the video image reduced by comparison with the original image is defined. Obviously more than one ROI can be defined in a video image or for a number of channels. By reducing the data to around 1:100, the size of the ROI can in such cases be 8×128 pixels for example; the processor load for the actual analysis or evaluation is significantly reduced. Whether smoke is present in a moving area of the recorded video image is clarified with reference to at least one item of characteristic information for smoke. In the present example the five following items of information are used to increase the security of detection. 
     Viewed as characteristic information for smoke are the speed of the smoke (movement of the smoke), the number of pixels (active pixels) which describe this movement, the luminance change of the at least one video image in relation to the background, the colour change of the moving smoke and the movement of the smoke (y position in the histogram). 
     The following information characteristic of smoke is now computed for each ROI:
         The smoke movement of SROI(t): v(t)=time correlation of the y-projection of SROI (t), for example pyj(t),   The variance of BROI(t) and XROI (t), which is used for determining the change in brightness relative to the (normal) background: 1(t)=1−var{BROI (t)}/var{BROI (t)},   Active pixels of S ROI  (t): a(t)=number of pixels of S ROI (t) with a value greater than 0,   Colour change: c(t)=number of pixels with {1|ChromaU (D ROI (t))−ChromaV (D ROI  (t)  1 }&lt;threshold value,   y position in the histogram: h(t)=values of the y-projection of S ROI  (t), for example p yj (t) is used to create a histogram with 64 channels.       

     Thereafter the information characteristic for smoke v(t), l(t), a(t), c(t) and h(t) is integrated over a specific time and thereby over a number of images. The function appears as follows for example: 
     
       
         
           
             
               
                 F 
                 x 
               
               = 
               
                 X 
                 = 
                 
                   
                     ∑ 
                     
                       
                         t 
                         0 
                       
                       &lt; 
                       t 
                       &lt; 
                       
                         t 
                         n 
                       
                     
                   
                    
                   
                       
                   
                    
                   
                     x 
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                 
               
             
              
             
                 
             
           
         
       
       
         
           with 
         
       
       
         
           
             
               X 
               = 
               V 
             
             , 
             L 
             , 
             A 
             , 
             C 
             , 
             H 
           
         
       
     
     The respective average is determined for the information integrated over time. 
     Average of smoke movement F V =V
 
Average of brightness change F L =L
 
Average of active pixels F A =A
 
Average of colour change F C =C
 
Average of y position in the histogram F H =H
 
     Thereafter the probability of the presence of smoke is computed for each of these averages. This is done by pattern detection. For each average a discriminator value Ψ is determined. A threshold value δ (or also a probability function) can for example define the discriminator in the following manner: for the change in brightness 
     
       
         
           
             
               Ψ 
               L 
             
             = 
             
               
                 Γ 
                  
                 
                   ( 
                   
                     F 
                     L 
                   
                   ) 
                 
               
               = 
               
                 { 
                 
                   
                     
                       
                         
                           
                             F 
                             L 
                           
                           &gt; 
                           
                             δ 
                             L 
                           
                         
                         , 
                       
                     
                     
                       
                         
                           then 
                            
                           
                               
                           
                            
                           
                             ψ 
                             L 
                           
                         
                         = 
                         1 
                       
                     
                   
                   
                     
                       
                         
                           F 
                           L 
                         
                         &gt; 
                         
                           δ 
                           
                             L 
                             , 
                           
                         
                       
                     
                     
                       
                         
                           then 
                            
                           
                               
                           
                            
                           
                             ψ 
                             L 
                           
                         
                         = 
                         0 
                       
                     
                   
                 
               
             
           
         
       
     
     or 0≦Γ(F L )≦1, with Γ(x) as probability function The smoke pattern is defined by the product of all discriminators 
     
       
         
           
             
               ℵ 
                
               
                 ( 
                 t 
                 ) 
               
             
             = 
             
               
                 
                   ∏ 
                   
                     i 
                     = 
                     Information 
                   
                 
                  
                 
                     
                 
                  
                 
                   Ψ 
                    
                   
                       
                   
                    
                   i 
                 
               
               = 
               
                 { 
                 
                   
                     Ψ 
                     v 
                   
                   · 
                   
                     Ψ 
                     L 
                   
                   · 
                   
                     Ψ 
                     A 
                   
                   · 
                   
                     Ψ 
                     C 
                   
                   · 
                   
                     Ψ 
                     H 
                   
                 
                 } 
               
             
           
         
       
     
     or as the average of all discriminators 
     
       
         
           
             
               ℵ 
                
               
                 ( 
                 t 
                 ) 
               
             
             = 
             
               
                 
                   1 
                   / 
                   
                     N 
                     F 
                   
                 
                  
                 
                   
                     ∑ 
                     
                       i 
                       = 
                       Information 
                     
                   
                    
                   
                       
                   
                    
                   
                     Ψ 
                      
                     
                         
                     
                      
                     i 
                   
                 
               
               = 
               
                   
               
                
               
                 
                   { 
                   
                     
                       Ψ 
                       v 
                     
                     + 
                     
                       Ψ 
                       L 
                     
                     + 
                     
                       Ψ 
                       A 
                     
                     + 
                     
                       Ψ 
                       C 
                     
                     + 
                     
                       Ψ 
                       H 
                     
                   
                   } 
                 
                 / 
                 
                   N 
                   F 
                 
               
             
           
         
       
     
     with N F =5 being the number of items of information. 
     Finally the decision is made as to whether the moving area of the video image involves an image of smoke. To this end an integrator I(t), which increases or decreases by a value of σ is determined. 
                                                I(t=0) = 0;               If   (t) = 1   then I(t) = I(t−1) + σ  +                 (added to S +  if I (t) &gt; S + )               else I(t) = I(t−1) − σ  −                 (added to S −  (usually 0) if I(t) &lt; S − )                        
with σ + ,σ −  usually assuming the value +1
 
     Smoke is detected and for example an alarm triggered, if I(t) exceeds a critical value K: 
     
       
         
           
               
               
               
             
               
                   
                   
               
             
            
               
                   
                 if I(t) &gt; K 
                 then smoke 
               
               
                   
                   
                 else no smoke 
               
               
                   
                   
               
            
           
         
       
     
       FIG. 2  shows a simplified diagram of a video image VB. The image contains a moving area, which is intended to represent smoke. The video image VB also shows a ROI which has been determined in accordance with the description for  FIG. 1 . 
       FIG. 3  shows a decision diagram for the detection of smoke, as described under  FIG. 1 . If I(t) exceeds a specific threshold value K, an alarm is triggered and it is highly probable that smoke has been detected. So that I(t) does not increase to infinity and thereby the reaction time for smoke detection is unnecessarily reduced, a maximum value I T  is defined. The time until an alarm is triggered is designated as the critical time. This time should be as short as possible. 
       FIG. 4  shows an inventive device VR with a receiver unit E and a transmitter unit S for communicating for example with other units such as sensors, central units etc. and a processing unit V to execute the method in accordance with  FIG. 1 . In such cases the device can be integrated into a video camera, a central unit etc. or can represent a separate unit.