Abstract:
The present invention provides a method and apparatus for detecting fire in a monitored area even if the flames are hidden behind obstructing objects. The steps of this method include: (a) detecting and capturing, at a prescribed frequency, video images of the monitored area, (b) converting the captured images into two-dimensional bitmaps of the temporally varying brightness values observed in the captured images, wherein the spatial resolution of this bitmap is determined by the number of pixels comprising the bitmaps, (c) specifying for any two of the pixels in the bitmaps a temporal coherence factor whose magnitude is a measure over a prescribed time of the similarities observed in the temporal variations of the brightness values being captured at each of the bitmap&#39;s pixels, (d) computing this temporal coherence factors for each of the prescribed set of pixels in the captured images, and (e) analyzing the computed temporal coherence factors to identify those sets of pixels that have temporal coherence factors whose values are above a prescribed threshold value, wherein this prescribed threshold value is set so as to identify those pixels that correspond to regions of the monitored area whose temporal variations in brightness indicate that the source of this brightness is a fire in the monitored area.

Description:
CROSS-REFERENCE TO RELATED APPLICATION  
       [0001]     This application claims the benefit of U.S. Provisional Patent Application No. 60/572,056, filed May 18, 2004 by the same inventor.  
     
    
     BACKGROUND OF THE INVENTION  
       [0002]     1. Field of the Invention  
         [0003]     The present invention generally relates to electrical, condition responsive systems and methods. More particularly, this invention relates to a method and apparatus for detecting fire in a monitored area by detecting the light emission patterns of flames in the monitored area as a result of the detailed, computational analysis of a sequence of digitized images of the monitored area.  
         [0004]     2. Description of the Related Art  
         [0005]     Fire detectors are very important safety devices that can provide early warning of fire in a monitored area. Considerable efforts have been devoted to improving upon the technology used in fire detectors as a means of increasing their usefulness and reliability.  
         [0006]     One of the most commonly used methodologies for fire detection is based on the light scattering properties of smoke or suspended aerosol particles that are generated by a fire. However, a disadvantage of this approach is that it assumes that smoke from a fire will find its way to a sensor at an instant in the fire&#39;s progression when there is still time for action to be taken to put out the fire before excessive damage occurs.  
         [0007]     Another common fired detection method tries to overcome this limitation by directly measuring the light that is radiated by a fire. Such methods involve direct light emission measurements with the goal of identifying the variations of illumination intensities that are attributable to the turbulent flickering of flames.  
         [0008]     Furthermore, it has been found that the reliability of this method can be greatly enhanced if such illumination intensities are measured in spectral or wavelength ranges that are characteristic of particular combustion processes. In particular, often the range around 4.5 μm is used because of the emission of CO radicals at this wavelength. This technique allows for reducing the sensitivity of such a system to light sources that are not caused by combustion (e.g., electric light, reflections of sunlight).  
         [0009]     However, a possible disadvantage of this approach is that its sensitivity is optimized for only particular types of fuels. Thus, use of devices based on this methodology is usually limited to environments where probable fires could be caused by only a limited number of materials (i.e., natural gas, oil, methanol). To address this limitation, it has been suggested that a plurality of sensors be used wherein each sensor works at only a specific wavelength of interest. See King et al., U.S. Pat. No. 5,995,008, and Shuler, U.S. Pat. No. 5,804,825.  
         [0010]     Another disadvantage of this approach is that the sensors that are used with such methodologies are highly sensitive to the distance between the source of radiation (flames) and the actual sensor element. Thus, the areas that can be monitored by these devices can be quite limited in size (i.e., since making their sensitivity high so as to cover a greater area can result in false alarms). It is common practice to place these devices only in close proximity to installations that pose high fire risks. Additionally, the relatively high cost of such sensors prevents the use of a great number of them to monitor large production and commercial spaces.  
         [0011]     An alternative approach for fire detection utilizing television cameras was suggested by Okamoto et al. in U.S. Pat. No. 6,680,671. In this approach, the averaged intensity of all pixels in a monitored area is analyzed in search of particular flame-indicative, flickering patterns. However, this approach does not utilize the reliability enhancing technique of analyzing such data in spectral or wavelength ranges that are characteristic of particular combustion processes. Thus, the performance of this method is below that of other traditional methods.  
         [0012]     Another, alternative approach is to utilize the sophisticated imaging techniques to detect fire by identifying its characteristic light patterns within the digitized images of monitored areas with the use of specialized software algorithms; see Privalov et al., U.S. Pat. No. 6,184,792. This method provides a means of early fire detection within the framework of standard CCTV surveillance systems, and it is insensitive, within a monitoring camera&#39;s allowable range, to the distance between the camera and a light source which may be due to the presence of a fire. However, a significant disadvantage of this method is that it can be only employed when flames are directly visible by the imaging device. Any obstruction in the line of site between camera and flames will prevent a fire from being detected.  
         [0013]     Despite the considerable prior art relating to radiation fire detectors, there is still a need for fire detector methods and systems that can more effectively detect fire independent of the types of combustibles being consumed, while also providing uniform monitoring sensitivity across an entire observation area. Inexpensive technologies that can provide such fire detection improvements can significantly affect public safety in general.  
       OBJECTS AND ADVANTAGES  
       [0014]     There has been summarized above, rather broadly, the prior art that is related to the present invention in order that the context of the present invention may be better understood and appreciated. In this regard, it is instructive to also consider the objects and advantages of the present invention.  
         [0015]     It is an object of the present invention to provide apparatus and methods that are effective in detecting fire within the entire volume of a monitored area.  
         [0016]     It is another object of the present invention to provide apparatus and methods that are effective at detecting fire in industrial petrochemical installations.  
         [0017]     It is a further object of the present invention to provide apparatus and methods that can overcome the deficiencies of prior optical fire detection systems that are associated with their performance limitations when obstructions are in the line of site between camera and any flames in a monitored area.  
         [0018]     It is an object of the present invention to provide apparatus and methods that can operate within the framework of the ordinary Closed Circuit Television (CCTV) surveillance systems that are used to monitor commercial, outdoor, industrial and residential areas.  
         [0019]     It is yet another object of the present invention to demonstrate how existing security surveillance equipment may be combined into unique systems which provide an improved best means for addressing the detection of fire in the industrial, commercial and residential installations.  
         [0020]     It is a further object of the present invention to provide a means for providing notification of fire within a monitored area to remote operators who are using closed circuit television to remotely monitor the area.  
         [0021]     These and other objects and advantages of the present invention will become readily apparent as the invention is better understood by reference to the accompanying summary, drawings and the detailed description that follows.  
       SUMMARY OF THE INVENTION  
       [0022]     Recognizing the need for the development of improved fire detection systems and methods, the present invention is generally directed to satisfying the needs set forth above and overcoming the disadvantages identified with prior art devices and methods.  
         [0023]     In accordance with the present invention, the foregoing need can be satisfied by providing an improved method for detecting fire in a monitored area. The steps of this method include: (a) detecting and capturing, at a prescribed frequency, video images of the monitored area, (b) converting the captured images into two-dimensional bitmaps of the temporally varying brightness values observed in the captured images, wherein the spatial resolution of this bitmap is determined by the number of pixels comprising the bitmaps, (c) specifying for any two of the pixels in the bitmaps a temporal coherence factor whose magnitude is a measure over a prescribed time of the similarities observed in the temporal variations of the brightness values being captured at each of the bitmap&#39;s pixels, (d) computing this temporal coherence factors for each of the prescribed set of pixels in the captured images, and (e) analyzing the computed temporal coherence factors to identify those sets of pixels that have temporal coherence factors whose values are above a prescribed threshold value, wherein this prescribed threshold value is set so as to identify those pixels that correspond to regions of the monitored area whose temporal variations in brightness indicate that the source of this brightness is a fire in the monitored area.  
         [0024]     A further specific, preferred embodiment of this method of fire detection includes the situation in which the sets of pixels are pairs of adjacent pixels along the horizontal and vertical axes defining the bitmap.  
         [0025]     A still further specific, preferred embodiment of this method of fire detection includes the situation in which the temporal coherence factor of any two arbitrary pixels in the bitmaps is the convolution of the time derivative of the brightness values of the pixels.  
         [0026]     The present invention discloses the mathematical treatment of the video images which allow one to identify the regions of the image exhibiting the properties typical for objects that reflect the light radiated by flames. Operating a computer algorithm that utilizes the above-mentioned treatment, one can generate a fire alarm when a fire threat has been detected.  
         [0027]     Thus, there has been summarized above, rather broadly, the present invention in order that the detailed description that follows may be better understood and appreciated. There are, of course, additional features of the invention that will be described hereinafter and which will form the subject matter of the claims to this invention.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0028]      FIG. 1  shows a block diagram of a preferred embodiment of the fire detection method and apparatus of the present invention.  
         [0029]      FIG. 2  shows the algorithm of a preferred embodiment of the fire detection method and apparatus of the present invention.  
         [0030]      FIG. 3  shows the layout of a FIFO buffer and how the coherence map vectors are calculated.  
         [0031]      FIG. 4  illustrates the typical reflective patterns in the room as they are identified using the present invention.  
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0032]     Before explaining at least one embodiment of the present invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting.  
         [0033]      FIG. 1  shows a preferred embodiment of the fire detection method and apparatus of the present invention that can operate within the framework of the ordinary Closed Circuit Television (CCTV) surveillance system for commercial, outdoor, industrial and residential installations.  
         [0034]     In a preferred embodiment, the present invention monitors the images being collected from a monitored area so as to form of a series of matrices (bitmaps) collected at a prescribed time interval. Each matrix consists of (n×m) values that represent a brightness value at each point (x, y) within a bitmap.  
         [0035]     An open flame, present in the vicinity of the observation area, and not necessarily within direct line of sight of the video camera, creates reflections off the objects present at the scene. These objects may include, but are not limited to walls, desks, shelves, equipment, etc.  
         [0036]     Unlike the reflection of other lights sources, the reflections of the light originated from flames has a very specific flickering emission pattern. I have observed that this pattern is characterized by seemingly chaotic modulation of light intensity within the frequency range from 2 to 8 Hz.  
         [0037]     Such flickering reflection may be too weak to be visible to the human eye, particularly when the amplitude of flickering reflections are comparable to the noise level of any camera that being used to collect the images of the monitored area. However, unlike any temporally random noise, the reflected flickering flame light is observed to be temporally coherent, since it comes from one source. This coherence results in the flickering intensity from each pixel on all reflective surfaces having similarities in their temporal behaviors or patterns. I have discovered that these temporal similarities can be used to identify an out-of-sight flame.  
         [0038]     The fire detection system of the present includes a video imaging and capture device (e.g., a television camera)  1 , which is preferably sensitive to radiation in the infrared (IR) range, capable of converting the captured images into two-dimensional bitmaps whose spatial resolution is determined by the number of pixels comprising the bitmaps, and with a field of view that covers the monitored area that may include a flame source (a), obstruction (b) and wall (c). See  FIG. 1 . It is important to note that this arrangement is given for exemplary purpose only and is not intended to limit the application of the disclosed method.  
         [0039]     The captured video images are analyzed with a computer system  2  with adequate analytical capabilities, including the algorithms  3  disclosed herein, to compute a temporal coherence factor for the brightness values present at each pair of adjacent pixels. This computed factor is used to identify the regions of the bitmap or the corresponding monitored area where these coherence factors are predominantly positive.  
         [0040]     In a first preferred embodiment, the coherence factor of arbitrary pair of pixels x and y is defined as the convolution of the product of the time derivatives of the brightness of respective pixels:  
         Ψ   ⁡     (     x   ,   y     )       =     ∫           ∂       Y   x     ⁡     (   t   )           ∂   t       ·       ∂       Y   y     ⁡     (   t   )           ∂   t         ⁢     ⅆ   t             
 
 where Y x (t) and Y y (t) is a brightness value of the given pixels x and y at any given time t. 
 
         [0041]     To identify the parts of the image belonging to such a reflective area, it is sufficient to compute only two coherence factors for each pixel of coordinate (x, y): a first using the next pixel along x-axis, and a second using the next adjacent pixel along y-axis.  
         [0042]     Therefore, for an entire image of size m×n pixels, the matrix of the size m−1×n−1 is generated with each element of this matrix having two values: the first value is the coherence factor between the pixel (x, y) and (x+1, y), and the second value is the coherence factor between pixel (x, y) and (x, y+1).  
         [0043]     It should be noted that the analytical technique described above is only one of the many that could be used to identify a surface that is reflecting radiation from a flame that is outside the line-of-sight of a monitoring camera. Thus, the exemplary technique given above should not be interpreted as a limiting analytical technique of the present invention.  
         [0044]     The software which expresses the algorithms and analytical procedures  3  of the present invention and which is running on this computer system scans these coherence matrices to identify isolated regions where values are positive and exceed a prescribed threshold value. Those regions are identified as surfaces which are reflecting flame radiation.  FIG. 4  illustrates the typical reflective patterns in the room as they are identified using the present invention.  
         [0045]     Additionally, it is possible to highlight these regions in the image and further analyze them for their shape, area, time consistency, etc. as a step prior to utilizing an alarm means  4  to send a signal which alerts someone of the presence of a fire in the monitored area. If all such secondary, appropriate conditions are met, proper alarm signals can be issued.  
         [0046]     The flowchart of the software algorithm  3  controlling the computer system is presented at  FIG. 2 . The main cycle of the algorithm starts ( 1 ) with acquiring the next frame ( 2 ) from the video capturing device in form of the matrix. Each element of this matrix corresponds to the brightness level at a particular pixel. The difference matrix is calculated ( 3 ) to reflect the ongoing, instantaneous temporal changes in these brightness level (i.e., the difference in brightness level from a newly acquired image and the previous image). The difference matrix is stored ( 5 ) in a FIFO buffer of prescribed size for further computation.  
         [0047]     In one representative embodiment of this algorithm, the size of FIFO is set to be sixteen, but that does not preclude other values. The current frame image is also stored ( 4 ) in a special buffer to be used in the next cycle of computing temporal brightness changes.  
         [0048]     A coherence matrix is compiled each time FIFO is filled ( 6 ). Each element of the coherence matrix has two values Mx and My computed as sum across each i-th member of FIFO:  
         Mx     x   ,   y       =         ∑   i     ⁢         Δ     x   ,   y     i     ·     Δ       x   +   1     ,   y     i       ⁢           ⁢     My     x   ,   y           =       ∑   i     ⁢       Δ     x   ,   y     i     ·     Δ     x   ,     y   +   1       i               
 
 where Δ x,y  are difference values for pixel at position (x, y). 
 
         [0049]     After a coherence matrix is compiled, FIFO is reset ( 7 ). Isolated clusters are computed from a coherence matrix using one of the standard computer graphics algorithms, e.g., floodfill or line sweep.  
         [0050]     Each cluster that is larger than a prescribed size is considered a candidate for being flame reflection artifact. In some embodiments however, additional tests can be conducted ( 8 ) that may include, but are not limited to: consistency in appearance of the clusters for prescribed period of time, shape of the cluster and its particular position relative to other objects in the scene. If all the criteria are met, an alarm is sounded or raised ( 9 ).  
         [0051]     Other embodiments of the present invention may provide a variety of additional functions, including, but not limited to: feeding video images to a remote operator, highlighting the regions that caused the alarms in these video images, raising sound and light alarms, issuing e-mails and emergency calls to appropriate response units. The steps ( 7 - 9 ) are performed for each isolated cluster found ( 10 ).  
         [0052]      FIG. 3  illustrates in more details how a coherence matrix can be calculated. FIFO buffer contains k copies of difference images. Each pair of adjacent pixels in such a difference image is used to compute a desired coherence factor. For every pixel position the coherence factor can be compiled using the next pixel on the right and next pixel below. In some embodiments, pixels on the left and above may be used instead. Diagonal patterns also can be employed in yet other embodiments.  
         [0053]     It is important to note that it is not necessary to maintain FIFO to compute the coherence factors. In yet another embodiment, the coherence factors can be computed by using a recursive formula: 
 
 Mx   x,y   =Mx   x,y +Δ x,y ·Δ x+1,y  
 
 My   x,y   =My   x,y +Δ x,y ·Δ x,y+1  
 
         [0054]     Once the prescribed number of frames is processed, the entire scene is being analyzed, and the matrix is reset to zero.  
         [0055]     Alternatively, another embodiment may constantly maintain the coherence factor without need of resetting. The coherence values may be simply updated recursively at every cycle using only current difference image: 
 
 Mx   x,y =(1− a )· Mx   x,y   +a·Δ   x,y ·Δ x+1,y  
 
 My   x,y =(1− a )· My   x,y   +a·Δ   x,y ·Δ x,y+1  
 
 where α is a filtering factor, usually a small number α&lt;&lt;1. 
 
         [0056]     In this embodiment, the analysis of coherence matrix can be performed at a much slower rate than update cycle. It is recommended that this analysis not be earlier then once each 1/α period of the frame rate. Other embodiment however, may have the analysis period deviating from above.  
         [0057]     Alternatively, some embodiments may employ other ways to calculate the difference image. For example, instead of using strictly a difference in two consecutive frames, one can use the difference between the current frame and a time-averaged frame.  
         [0058]     In yet another embodiment, the coherence factors can be calculated directly from a Fourier transform of the brightness of each pixel using a restricted form of the Weiner-Khinchin theorem that says that convolution of two functions is an inverse Fourier transform of a product of the Fourier response of the first function and a complex conjugate of the Fourier response of the second function:  
           ∫     -   ∞     ∞     ⁢         a   ⁡     (     t   +   τ     )       ·     b   ⁡     (   t   )         ⁢           ⁢     ⅆ   t         ≡       1       2   ⁢   π         ⁢       ∫     -   ∞     ∞     ⁢         exp   ⁡     (     -   ⅈωτ     )       ·     A   ⁡     (   ω   )       ·       B   *     ⁡     (   ω   )         ⁢           ⁢     ⅆ   ω               
 
 where i={square root}{square root over (−1)}, A(ω) is frequency response for function a(t), B*(ω) is complex conjugate of frequency response of function b(t). For coherence factor time shift τ will be equal to zero while exp(−iωr) will become 1, and integration over the frequency range can be substituted with the sum:  
         M     a   ,   b       =       1       2   ⁢   π         ⁢       ∑   j     ⁢     {         F   j     ⁡     [       ∂       y   a     ⁡     (   t   )           ∂   t       ]       ·       F   j   *     ⁡     [       ∂       y   b     ⁡     (   t   )           ∂   t       ]         }             
 
 where y a (t) and y b (t) are brightness values of pixels a and b and F i [] denotes a j-th Fourier term. 
 
         [0059]     This equation can be further expanded using the rule for Fourier transform of a derivative: 
 
 F   j   [f ′( t )]≡2π ·i·j·F   j   [f ( t )]
 
 Thus, the coherence factors will become:  
         M     a   ,   b       =       8     ⁢     π     3   /   2       ⁢       ∑   j     ⁢       〈     i   ·   j   ·       F   j     ⁡     [       y   a     ⁡     (   t   )       ]         〉     ·     〈     i   ·   j   ·       F   j   *     ⁡     [       y   b     ⁡     (   t   )       ]         〉               
 
 Ignoring constant terms, the coherence factors for pixel (x, y) become:  
         Mx     x   ,   y       =       ∑     j   =   1     n     ⁢       [     i   ·   j   ·       Y   j     ⁡     (     x   ,   y     )         ]     ·     [     i   ·   j   ·       Y   i   *     ⁡     (       x   +   1     ,   y     )         ]             
         My     x   ,   y       =       ∑     j   =   1     n     ⁢       [     i   ·   j   ·       Y   j     ⁡     (     x   ,   y     )         ]     ·     [     i   ·   j   ·       Y   j   *     ⁡     (     x   ,     y   +   1       )         ]             
 
 where Y i (x, y) is a i-th Fourier term of the brightness y x,y (t):  
           Y   i     ⁡     (     x   ,   y     )       =       1       2   ⁢   π         ⁢     ∫       ⅇ       -   2     ⁢     j   ·   π   ·   f   ·   ⅈ   ·   t         ⁢       y     x   ,   y       ⁡     (   t   )       ⁢     ⅆ   t               
 
 where f is a frame rate, while Y i * (x, y) is complex conjugate of abovementioned Fourier term. 
 
         [0060]     It is important to note that this summation ignores a zero Fourier term, leaving out constant bias in brightness values. Due to limited frame rate, only a few Fourier terms will be sufficient for determining the coherent flickering artifacts of a flame radiation.  
         [0061]     In yet another embodiment, a Fourier transform can be performed directly on the brightness. When calculating the convolution simply by including only higher Fourier terms, one will eliminate a constant background bias:  
         M     x   ,   y       =       ∑     j   =   m     n     ⁢       Y   a     ·     Y   b   *             
 
 where Y j (x) and Y j (y) are j-th terms of Fourier transform of the brightness values of the pixels x and y; m-denotes the lower frequency cutoff term and n-denotes the higher frequency cutoff term. The advantage of this approach is that it may be less sensitivity to the pixel level noise to which the time derivative is known to be sensitive. 
 
         [0062]     With respect to the above description then, it is to be realized that the optimum dimensional relationships for the parts of the invention, to include variations in size, materials, shape, form, function and manner of operation, assembly and use, are deemed readily apparent and obvious to one skilled in the art, and all equivalent relationships to those illustrated in the drawings and described in the specification are intended to be encompassed by the present invention.  
         [0063]     Therefore, the foregoing is considered as illustrative only of the principles of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope of the invention as hereinafter set forth in the claims.