Abstract:
A smoke detection method for identifying, in a current input image, an area indicative of the presence of smoke, there being a sequence of two or more input images, the method comprising the steps of: storing a background estimation for a current input image; and comparing the current input image with the background estimation to detect a partial obscuring of the background estimation indicative of the presence of smoke in the current input image.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     1. Field of the Invention  
         [0002]     This invention relates to smoke detection.  
         [0003]     2. Description of the Prior Art  
         [0004]     Smoke detection systems are well known. One of the most common methods of detecting smoke (and the most frequently used within buildings, such as a person&#39;s home) is to have a local detector that physically detects smoke particles in the air. Such smoke detectors are suited to small indoor environments, where the amount of air to be sampled is relatively small. For a large indoor environment, such as a warehouse, multiple such smoke detectors are required to enable detection of smoke in a sufficiently short time. This is a costly solution and is often not easy to deploy. Furthermore, such smoke detectors are not very well suited to detecting smoke in an outdoor environment, such as a park, a forest or a car park. This is due to a variety of reasons, such as: the vast quantity of air present; the lack of a vertical restraint on the movement of the air; the size of the area to be monitored; and potential air flow dynamics that direct smoke away from one or more of the detectors.  
         [0005]     Detection of smoke by video/image processing techniques has also been proposed. For example, areas of an image can be compared with known smoke characteristics via pattern matching techniques to detect smoke. For example, smoke plumes may be detected in this manner. Another proposed method of using video based smoke detection is to detect the diffusion of light from light sources and/or bright objects within the video images to identify a pattern consistent with the slow accumulation of smoke.  
       SUMMARY OF THE INVENTION  
       [0006]     According to an aspect of the invention, there is provided a smoke detection method for identifying, in a current input image, an area indicative of the presence of smoke, there being a sequence of two or more input images, the method comprising the steps of: storing a background estimation for a current input image; and comparing the current input image with the background estimation to detect a partial obscuring of the background estimation indicative of the presence of smoke in the current input image.  
         [0007]     Embodiments of the invention make use of the fact that smoke is partially transparent, i.e. smoke partially obscures the scene behind the smoke. An estimate of what constitutes the background of the scene being captured by a video camera (i.e. what would be behind some smoke) is formed. By comparing this background estimate with a current input image, areas of the current input image that are covered by partially transparent smoke can be identified. This provides a smoke detection system with several advantages: early smoke detection is achieved (due to detecting partially transparent smoke); the smoke detection is remote (due to using video processing techniques); and the smoked detection does not rely on specific characteristics of smoke formation (such as plume shape or diffusion of light from a bright source) which may not actually occur (for example, due to physical factors such as wind, buildings, etc.).  
         [0008]     Further respective aspects of features of the invention are defined in the appended claims. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0009]     The above and other objects, features and advantages of the invention will be apparent from the following detailed description of illustrative embodiments which is to be read in connection with the accompanying drawings, in which:  
         [0010]      FIG. 1  schematically illustrates a smoke detection system according to an embodiment of the invention;  
         [0011]      FIG. 2  schematically illustrates an overview of the video processing performed to detect smoke;  
         [0012]      FIG. 3  is a schematic flowchart of the video processing performed to detect smoke;  
         [0013]      FIG. 4  illustrates example images generated by the video processing according to the flowchart shown in  FIG. 3 ; and  
         [0014]      FIGS. 5 and 6  schematically illustrate a method of updating a background estimate.  
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0015]      FIG. 1  schematically illustrates a smoke detection system according to an embodiment of the invention. Three video cameras  100 A,  100 B,  100 C are connected to a video processing unit  102  which analyses the video captured by the video cameras  100  to determine whether the scene  103  that one of the video cameras  100  is arranged to capture contains smoke  105 . If the video processing unit  102  determines that the scene  103  contains smoke  105 , then the video processing unit  102  triggers an alarm  104 . The alarm  104  may be an audible alarm, a visual alarm or an audible and visual alarm. An electronic alarm, such as a pager signal, an email or a short message service (SMS) or voice-based mobile phone message could be used. The smoke detection system shown in  FIG. 1  may be arranged such that a human operator is alerted to the possibility of smoke  105  being present in the scene  103  being captured by one of the video cameras  100 , in response to which the human operator performs a visual verification himself prior to setting off another (main) alarm, for example to call the emergency services. Additionally, or alternatively, the video processing unit  102  may be connected to a fire extinguisher system  106 . The fire extinguisher system  106  may be a fully automatic fire extinguisher system or may be under the control of a human user. The fire extinguisher system  106  may use information provided to it by the video processing unit  102 , concerning the location of the smoke  105  within the scene  103 , so that a fire generating the smoke  105  may be extinguished.  
         [0016]     The smoke detection system shown in  FIG. 1  is particularly suitable to outdoor environments where it would be impractical to fit standard smoke detectors which operate by detecting particles of smoke in the air. For example, the smoke detection system shown in  FIG. 1  may be used in a car park or around the perimeter of a property as shown in  FIG. 1 .  
         [0017]     The video cameras  100  shown in  FIG. 1  may be any ordinary video cameras and need not necessarily be special video cameras such as ultraviolet video cameras or infrared video cameras, i.e. the video cameras  100  may be video cameras that capture light in the visible spectrum. As such, the video cameras  100  may be video cameras of a closed circuit television (CCTV) system that already exists for surveillance purposes, the video outputs of the video cameras  100  being routed to the video processing unit  102  as well as to a pre-existing video surveillance unit (not shown in  FIG. 1 ).  
         [0018]     It will be appreciated that the smoke detection system shown in  FIG. 1  may make use of any number of video cameras  100 .  
         [0019]      FIG. 2  schematically illustrates an overview of the video processing performed by the video processing unit  102  to detect smoke. A current input image  200  from one of the video cameras  100  is received by the video processing unit  102 . The video processing unit  102  maintains an estimate  202  of the background of the current input image  200 . The background estimate  202  is updated on a regular basis, for example for every input image  200  received by the video processing unit  102 . The background estimate  202  is an estimation of the scene  103  as viewed by the video camera  100  when no smoke  105  is present. Therefore, when no smoke  105  is present in the scene  103  being captured by the video camera  100 , the current input image  200  should be approximately the same as the background estimate  202 .  
         [0020]     When smoke  105  is present in the scene  103  being captured by the video camera  100 , the current input image  200  will be approximately the same as the background estimate  202  except that some of the areas of the background estimate  202  will be covered by an area representing the partially transparent smoke  105 . The video processing unit  102  therefore compares the current input image  200  with the background estimate  202  to try to detect areas of the background estimate  202  that have been covered by an area representing partially transparent smoke  105 . This results in a prediction  204  of where smoke  105  may be present in the current input image  200 . Given this information, the background estimate  202  may be updated from the current input image  200  but with the smoke  105  removed.  
         [0021]      FIG. 3  is a schematic flowchart of the video processing performed by the video processing unit  102 . The video processing unit  102  makes use of a current input image  im  (corresponding to the current input image  200  of  FIG. 2 ) and a background estimate  b  (corresponding to the background estimate  202  of  FIG. 2 ). The basic relationship between the current input image  im , the background estimate  b , a smoke color  s  and a smoke density a is given in Equation 1 below.
     im = b *α+ s * (1−α)  Equation 1 
         [0022]     In this embodiment, Equation 1 uses  im ,  b  and  s  to represent values for a pixel location in a respective color image and, as such,  im ,  b  and  s  are vector values having three components. These three components could correspond to red, green and blue components or a luminance and two chrominance components for example. However it will be appreciated that the smoke detection could operate on a single component such as a luminance component in a black and white image. For clarity, the rest of this description will assume that the images being referred to are color images with three color planes. The smoke color  s  may vary across the image to accommodate spatially changing colors of smoke. Additionally the smoke density a may vary on a pixel-by-pixel basis to accommodate changes in smoke density or thickness. The value of the smoke density a ranges from zero (which represents totally opaque smoke) to a value of one (which represents totally transparent smoke, i.e. no smoke, the background estimate  b  being identical to the current input image  im ). However, for a given pixel location, the smoke density α is assumed to be constant across the color planes.  
         [0023]     The background estimate  b  is initialised to an image of the scene  103  known not to contain smoke  105 , for example a video frame from the video camera  100  when the scene  103  is known not to contain smoke  105 .  
         [0024]     At a step S 300 , the current input image  im  and the background estimate  b  are used to produce an initial estimate of the smoke color  s . This may be formed in a variety of ways, but in preferred embodiments the value for a pixel location of the smoke color  s  is set to a value of 1 if the corresponding pixel in the current input  im  is greater than the corresponding pixel value in the background estimate  b ; otherwise the value for the pixel location in the smoke color  s  is set to a value of 0. Note that for the purposes of this description, pixel values lie in the range from 0 to 1. This initial estimation for the smoke color  s  is derived from Equation 1, namely: when a pixel value of the current input image  im  is greater than the corresponding pixel value in the background estimate  b , then the corresponding smoke color  s  must be greater than both of these values, and setting the corresponding smoke color  s  to 1 is certain to meet this criteria; whereas when a pixel value of the current input image  im  is less than or equal to the corresponding pixel value in the background estimate  b  then the corresponding smoke color  s  must be less than or equal to both of these values, and setting the corresponding smoke color  s  to 0 is certain to meet this criteria.  
         [0025]     At a step S 302 , the initial estimate for the smoke color  s  is low pass filtered to remove any high frequencies from the initial estimate for  s . This is performed as it is assumed that the color of the smoke  105  is largely constant and will only change slowly over the current input image  im .  
         [0026]     At a step S 304 , Equation 1 is used to calculate an initial value for the smoke density α. To do this, Equation 1 can be re-arranged into Equation 2 below.  
       Equation   ⁢           ⁢   2   ⁢     :         
       α   =         im   _     -     s   _           b   _     -     s   _             
 
         [0027]     As Equation 1 uses  im ,  b  and  s  as three dimensional vectors, a value for the smoke density α is calculated for each of the corresponding color planes. As it is assumed that the smoke density α will be consistent across all color planes, a single value for the smoke density α is calculated from each of the initial color plane specific values for the smoke density α, for example by averaging them.  
         [0028]     At a step S 306 , the high frequencies are removed from the smoke density α. This is performed as it is assumed that the smoke density a will only very slowly across the image.  
         [0029]     At this stage, the smoke color  s  has currently only been estimated very crudely. Therefore, at a step S 308 , it is determined whether the smoke color  s  needs to be updated. If the smoke color  s  needs to be updated (i.e. this is the first time that the processing has reached the step S 308  for this current input image  im ) then processing proceeds to a step S 310  at which an improved estimate for the smoke color  s  is generated using Equation 3 below (which is a re-arrangement of Equation 1).  
       Equation   ⁢           ⁢   3   ⁢     :         
         s   _     =         im   _     -       b   _     *   α         1   -   α           
 
         [0030]     Processing then resumes at the step S 302  so that the improved estimate for the smoke color  s  is low pass filtered (at the step S 302 ), a new estimate for the smoke density α is generated (at the step S 304 ) and then the high frequencies are removed from the newly generated smoke density α (at the step S 306 ).  
         [0031]     It will be appreciated that embodiments of the invention may by-pass the removal of the high frequencies from one or more of: the initial estimate for the smoke color  s ; the initial smoke density α; the improved estimate for the smoke color  s ; and the new estimate for the smoke density α.  
         [0032]     When processing returns to the step S 308 , the smoke color  s  no longer needs to be updated and processing continues at a step S 312 . At the step S 312 , a correlation map c between the current input image  im  and the background estimate  b  is generated. As smoke mainly effects the low frequencies in an image, the correlation is calculated using the high frequency components of the current input image  im  and the background estimate  b . The correlation map c is calculated according to Equation 4 below.  
       Equation   ⁢           ⁢   4   ⁢     :         
       c   =       ∑     col   =   1     3     ⁢         f   ⁡     (         im   _     hp     *       b   _     hp       )             f   ⁡     (         im   _     hp     *       im   _     hp       )       *     f   ⁡     (         b   _     hp     *       b   _     hp       )             3           
           im   _     hp     =       im   _     -     f   ⁡     (     im   _     )             
     where     
           b   _     hp     =       b   _     -     f   ⁡     (     b   _     )             
       f   =     low   ⁢           ⁢   pass   ⁢           ⁢   filter         
 
         [0033]     The summation is across the three color components in this example.  
         [0034]     Processing continues at a step S 314  at which a probability map p (i.e. a set of probability values, such as one per pixel position) is generated. The probability map p is generated according to Equation 5 below.  
       Equation   ⁢             ⁢             ⁢   5   ⁢     :         
       p   =       c   2     *     (     1   -   α     )     *     (     1   -     abs   ⁡     (     c   -   α     )         )     *       ∑     col   =   1     3     ⁢     (       s   _     -     im   _       )             
 
         [0035]     As can be seen, the probability map p will assume a large value at a given pixel location if: 
        1) there is a large degree of correlation between the current input image  im  and the background estimate  b  (as represented by a large value of the correlation map c); and     2) the smoke density α is close to zero; and     3) the correlation map c is close to the smoke density α; and     4) the current input image  im  is sufficiently different from the smoke color  s .        
 
         [0040]     It will be appreciated that a different version of Equation 5 may be used. For example, one or more of the four multiplication terms may be omitted from Equation 5. Additionally, one or more of the terms may be weighted in order to provide a greater degree is significance to one or more specific factors, given the particular requirements of the smoke detection system being employed.  
         [0041]     As can be seen from Equation 5, there are several competing factors contributing to the probability map p. For example, as the smoke density α decreases the (1−α) term becomes larger whilst the correlation map c will be reduced (as there is less correlation between the background estimate  b  and the current input image  im ). Additionally, for almost opaque smoke, the value of the smoke density α must be close to 0, which means that the current input image  im  becomes close to the smoke color  s  (Equation 1). However this conflicts with the requirements that the current input image  im  must be sufficiently different from the smoke color  s . These competing factors are required though to ensure that: 
        a) the current input image  im  is sufficiently similar to the background estimate  b  so that any differences are know not to be a non-transparent object; and     b) the current input image  im  is sufficiently different to the background estimate  b  so that a degree of certainty can be achieved that there is really some smoke  105  present in the scene  103  being captured by the video camera  100 .        
 
         [0044]     At a step S 316 , the background estimate  b  is updated. The process of updating the background estimate  b  will be described in more detail later.  
         [0045]     The values of the probability map p may then be compared to a threshold probability so that if one or more (or at least a sufficient number) of these values exceeds a threshold probability, then the video processing unit  102  activates the alarm  104  and/or the fire extinguishing system  106 .  
         [0046]      FIG. 4  illustrates example images generated by the processing according to the flowchart shown in  FIG. 3 . An area of smoke  400  in the current input image  im  is clearly visible when compared to the background estimate  b .  
         [0047]      FIG. 5  schematically illustrates a method of updating the background estimate  b . The background estimate  b  is updated in dependence upon the current background estimate  b , the current input image  im  and a reconstructed background  rb . This is performed according to Equation 6 below.
     b ′=u* im +ν* rb +w* b u+ν+w= 1  Equation 6 
         [0048]     The reconstructed background  rb  is generated from Equation 7 below (which is a rearrangement of Equation 1).  
       Equation   ⁢           ⁢   7   ⁢     :         
         rb   _     =         im   _     -       s   _     *     (     1   -   α     )         α         
 
         [0049]     As can be seen, the updated background estimate is a linear combination of the reconstructed background estimate  rb , the current input image  im  and the current background estimate  b . In preferred embodiments, the contributions from the current input image  im  and the reconstructed background  rb  are smaller than the contribution from the current background estimate  b . This causes the background estimate  b  to be updated slowly. The reason for doing this is that the scene  103  behind the smoke  105  can be assumed to be largely static. The reason for not simply setting the updated background estimate  b′  to be the reconstructed background  rb  is that there may be moving objects in the foreground which could cause the updating to diverge, i.e. the background estimate  b  would become worse and worse.  
         [0050]     Account must be taken of the situation when a new object appears in the scene  103  or an object is removed from the scene  103 . Due to the slowly updating nature of the background estimate  b , this newly appearing or disappearing object can appear to be smoke  105 .  
         [0051]     Preferred embodiments address this problem by using two or more background estimates  b , each of which updates at a different rate to the other background estimates.  
         [0052]      FIG. 5  shows three columns of images: the left column represents a time series of current input images  im ; the middle column represents a time series of background estimates  b     f    for a fast updating background estimate; and the right column represents a time series of background estimates  b     s    for a slowly updating background estimate. The fast updating background estimates  b     f    and the slow updating background estimates  b     s    are calculated using versions of Equation 6 with appropriate multiplication constants. Preferred embodiments use Equations 8 and 9 below for updating the fast updating background estimates  b     f    and the slow updating background estimates  b     s    respectively.
     b ′= 0.0475 * im + 0.0025*   rb + 0.95*   b     Equation 8     b ′= 0.00095*   im + 0.00005*   rb + 0.999*   b     Equation 9 
         [0053]     Whilst  FIG. 5  shows the use of two background estimates, it will be appreciated that more than two background estimates may be used to address the problem that newly introduced or removed objects appear to be smoke  105 .  
         [0054]     The use of multiple background estimates updating at different rates works as an object is either entirely visible in one of the background estimates and fading in another or is entirely removed from one of the background estimates and fading in another. When generating the probability map p, each of the background estimates is used and the minimum probability is taken on a pixel-by-pixel basis.  
         [0055]      FIG. 5  shows how using multiple background estimates works in practice, when removing an object  500  from the scene  103 . At a time t 1 , the object  500  is present in the scene  103  being captured by the video camera  100  and consequently appears in the corresponding current input image  im . As the object  500  has been present in the scene for some time, the object  500  also appears in the background estimates  b     f    and  b     s    at time t 1 . At the next frame (at time t 2 ), the object  500  has been removed from the scene  103  and is therefore no longer present in the current input image  im . As the contribution from the current input image  im  is larger when updating the fast updating background estimate  b     f    than when updating the slow updating background estimate  b     s    (see Equations 8 and 9), the object  500  now appears as a faint object  501  in the fast updating background estimate  b     f    whilst it appears as a more solid object  502  in the slow updating background estimate  b     s    . At the next frame (at time t 3 ), the object  501  has disappeared from fast updating background estimate  b     f    whilst the object  500 ,  502  appears as a faint object  503  in the slow updating background estimate  b     s   . At the next frame (at time t 4 ), the object  500  has disappeared entirely from both of the background estimates  b     f   ,  b     s   .  
         [0056]     When computing the probability map p at time t 2 , the presence of the solid object  502  in the slow updating background estimate  b     s    will result in a low probability for smoke detection and therefore prevents the faint object  501  in the fast updating background estimate  b     f    from providing a high probability of smoke. Similarly, at the time t 3 , the complete absence of the object  500  in the fast updating background estimate  b     f    will result in a low probability of smoke detection, thereby avoiding a higher probability of smoke detection caused by the faint object  503  in the slow updating background estimate  b     s   .  
         [0057]      FIG. 6  schematically illustrates a method of updating the background estimate  b  when an object  600  is introduced into the scene  103 . At a time t 1 , the object  600  is not present in the scene  103  being captured by the video camera  100  and consequently does not appear in the corresponding current input image  im . The object  600  therefore does not appear in the background estimates  b     f    and  b     s    at time ti 1 . At the next frame (at time t 2 ), the object  600  has been introduced into the scene  103  and is therefore present in the current input image  im . As the contribution from the current input image  im  is larger when updating the fast updating background estimate  b     f    than when updating the slow updating background estimate  b     s    (see Equations 8 and 9), the object  600  now appears as a faint object  601  in the fast updating background estimate  b     f    whilst it hardly appears at all in the slow updating background estimate  b     s   . At the next frame (at time t 3 ), the object  601  now appears as a more solid object  602  in fast updating background estimate  b     f    whilst the object  600  appears as a faint object  603  in the slow updating background estimate  b     s   . At the next frame (at time t 4 ), the object  600  appears as a more solid object  604 ,  605  in both of the background estimates  b     f   ,  b     s   .  
         [0058]     When computing the probability map p at time t 2 , the absence of any object in the slow updating background estimate  b     s    will result in a low probability for smoke detection and therefore prevents the faint object  601  in the fast updating background estimate  b     f    from providing a high probability of smoke. Similarly, at the time t 3 , the presence of the more solid object  602  in the fast updating background estimate  b     f    will result in a low probability of smoke detection, thereby avoiding a higher probability of smoke detection caused by the faint objection  603  in the slow updating background estimate  b     s   .  
         [0059]     Preferred embodiments perform one or more extra stages of processing in order to help improve the smoke detection results. One of these stages includes masking (or excluding) certain pixels from the smoke detection calculations. For example, in order to remove the adverse effects that saturated pixel values can have on the smoke detection calculations, pixel values taking a maximum or a minimum possible value are excluded from the smoke detection calculation. It will be appreciated that pixel values at or near the maximum or the minimum possible pixel value could also be excluded. Other pixels could also be excluded for other reasons. For example, the background estimate  b  could be analysed to determine areas of low detail, these areas being excluded from the smoke detection calculation. It will be appreciated that the masking could be performed based on pixel values either in the current input image  im  or the background estimate  b .  
         [0060]     Another extra processing stage which preferred embodiments apply is gamma correction. This is performed to remove all gamma effects from the current input image  im  so that the processing is performed in the linear light domain. Gamma correction is performed according to Equation 10 below.
 
 im   out = im   in   2.2   Equation 10
 
         [0061]     Another processing stage which preferred embodiments apply is contrast correction. It is often the case that the video camera  100  performs automatic contrast adjustment, for example when the sun moves behind a cloud. The general form of the equation for correcting contrast is given in Equation 11 below.
 
   im     out   =k   contrast   * im     in   Equation 11
 
         [0062]     An estimate for the contrast adjustment parameter k contrast  is generated from the current input image  im  and the background estimate  b  according to Equation 12 below.  
       Equation   ⁢           ⁢   12   ⁢     :         
         k   constrast     =       ∑     (     c   *       ∑     col   =   1     3     ⁢       im   _       b   _           )         3   ⁢     ∑   c             
 
         [0063]     In Equation 12, the summation where col ranges from 1 to 3 is across the color planes; the other summations are across all pixels in the correlation map c. Preferred embodiments also reject pixels where k contrast  is not approximately equal across all 3 color planes.  
         [0064]     The reason for including the correlation map c in Equation 12 is that this weights areas of the current input image  im  more heavily where it correlates with the background estimate  b . This prevents k contrast  becoming overly affected by new objects appearing in the scene  103 .  
         [0065]     Finally, the smoke detection results produced by embodiments of the invention may be combined with fire/flame detection probabilities output by a fire detection system. An example of a suitable fire detection system is provided in co-pending application number 0514706.1. This fire detection system outputs a probability map for whether a current input image  im  represents a fire/flame. This probability map may be combined with the probability map p to provide an overall smoke-and-flame-probability-map (for example by simple multiplication of the two probability maps).  
         [0066]     The smoke detection performed by the video processing apparatus  102  may be undertaken in software, hardware or a combination of hardware and software. Insofar as the embodiments of the invention described above are implemented, at least in part, using software control data processing apparatus, it will be appreciated that a computer program providing such software control and a storage medium by which such a computer program is stored, are envisaged as aspects of the invention.  
         [0067]     Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.