Abstract:
An image segmentation method has a training phase and a segmentation phase. In the training phase, a frame of pixellated data from a camera is processed using information on camera characteristics to render it camera independent. The camera independent data are processed using a chosen value of illuminant spectral characteristics to derive reflectivity data of the items in the image. Pixels of high reflectivity are established. Then, using data from the high reflectivity pixels, the actual illuminant spectral characteristics are established. The illuminant data are then processed to determine information on the illumination of the scene represented by the frame of pixellated data to derive reflectivity data of the scene. The segmentation phase comprises operating on a subsequent frame of pixellated data to render it camera independent and using the determined illumination information to process the camera independent data to determine reflectivity data of the scene to derive a foreground mask.

Description:
RELATED APPLICATIONS 
       [0001]    The present application is a National Phase entry of PCT Application No. PCT/GB2009/002829, filed Dec. 7, 2009, which claims priority from Great Britain Application Number 0822953.6, filed Dec. 16, 2008, the disclosures of which are hereby incorporated by reference herein in their entirety. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The present invention relates to the field of image processing. 
         [0003]    Embodiments relate to methods and devices for object segmentation in images. One example application is to video surveillance systems. 
       BACKGROUND OF THE INVENTION 
       [0004]    Prior art documents exist in which camera output is treated as a direct measure of reflectance; this can be true where chromaticity and intensity of illumination are invariant, and where a single camera is used. An example is U.S. Pat. No. 6,763,136. In fact, as known to those skilled in the art, camera output is a combination of illumination (which itself depends on illuminant), on camera sensitivity and on reflectivity. 
         [0005]    An aim is to determine reflectance or reflectivity of surfaces represented in a scene which is subject to variations, and not constrained as above. 
         [0006]      FIG. 6  illustrates a block diagram of an automated video surveillance system  1  which includes serially connected stages. These stages include a video acquisition stage  10 , a background-foreground object segmentation stage  11 , a tag detection stage  12 , a skin detection stage  13 , an object localization stage  14 , an object tracking stage  15 , and a movement classification stage  16 . 
         [0007]    The video acquisition stage, and frame grabber  10  acquire an image of a scene. The foreground object segmentation stage  11  segment foreground objects from the surrounding information, by comparison with a background model  21 , and this can be cropped out leaving a new image with a foreground object or predefined tagged object. The tag detection stage  12  and skin detection stage  13  help identify an object e.g. tag or head within the image by comparison with a tag model  22  and a skin model  23 . 
         [0008]    Once these features have been identified, the object localization stage  14  determines the 2D location of foreground objects, heads and tags within the camera field of view. Then detected object locations may be tracked using an object tracking stage  15 . Where the requirement of a video surveillance system is to detect movement, a movement classification module  16  is required to analyze the trajectory of each object, and compare it with preset trajectories or behaviors. 
         [0009]    Background-Foreground segmentation stage  11 , tag detection stage  12  and skin detection stage  13  are different types of object segmentation algorithms. 
         [0010]    The goal of object segmentation algorithms is to ultimately group pixels into regions which represent objects. The output of these algorithms may be a binary mask, or a set of connected pixels which correspond to a certain group. 
         [0011]    The object segmentation problem is divided into three main problems: selection of image representation, statistical modelling, and threshold selection. 
         [0012]    1. Selection of Image Representation 
         [0013]    The main question is what is the optimal image representation for object classification? 
         [0014]    Developers have used different color spaces, e.g. RGB, normalized RGB, HSV, YCbCr, CIELAB and RGB color ratio. However normalized RGB and HSV are the most common color spaces used. It has been shown that these color spaces are more tolerant of minor variations in the illuminant. 
         [0015]    2. Statistical Modelling 
         [0016]    Object segmentation systems typically build a statistical model in order to define a decision rule which discriminates between the pixels of the target object and those of other objects. Statistical models used in existing object segmentation approaches are divided into parametric and non-parametric approaches. Parametric approaches use a specific functional form with adjustable parameters chosen to fit the model to the data set; examples are: Gaussian models such as mixtures of Gaussians, and elliptic boundary model. Examples of non-parametric approaches are: normalized lookup table, Bayes classifier, and self organizing map. 
         [0017]    3. Threshold Selection 
         [0018]    A threshold in the decision rule determines if a pixel in the test image corresponds to a target object or not. Due to camera noise and illumination variations, pixels are liable to be classified as a target object even if they do not belong to it: a high value of the threshold allows maintaining a low number of false detections. On the other hand, objects presenting a low contrast with respect to the target risk will be eliminated if the threshold is too high. 
         [0019]    Referring to  FIG. 7 , the first step of a typical object segmentation algorithm  600  is video acquisition  601  followed by image conversion  604 . Data of a current frame  602  is applied to a foreground segmentation step  606  that receives background model data  603  as a second input. The foreground segmentation step  606  has an output, a foreground pixel map  607  of the current frame that is fed to a noise removal step  608 . The noise removal step  608  has an output  609  that is an enhanced foreground pixel map, and this is input to a connected component analysis stage  610  whose output is a set of labelled foreground regions  611 . The set of labelled foreground regions  611  is fed to a further post-processing stage  612 , having an output  613  that is fed to an object and feature extraction stage  614  and to an update feedback path to a background modelling stage  620 , that is fed with data  605  from the image conversion stage  604  during an initialization phase. 
         [0020]    The algorithm starts by acquiring a number (N), of training images and converting the acquired training images into a convenient image representation; e.g. a specific color space such as normalized RGB. As noted above normalized RGB can be used to reduce the effect of illumination change. The second step is to build the background model  607  for the target object. A threshold selection module has the role of choosing the appropriate thresholds (r) to use in later comparison between a current frame and the background model. 
         [0021]    Test images are then acquired and analyzed to determine if a pixel in the test image corresponds to the background model  607  or not. The post-processing stage  608  filters out noises before making the final decision. Finally, to update the background model, model maintenance is performed. 
         [0022]    The goal of the foreground object segmentation algorithm is to determine which pixels belong to the foreground, even before classifying the remaining changed pixels into different foreground classes. 
         [0023]    Object segmentation is a well known desideratum. Conventional techniques may be computationally intensive, and not be capable of implementation in real-time. 
         [0024]    A device for and a method of image processing that can enable segmentation to be carried out in a less computationally-intensive fashion would be desirable. Such a device could use an assessment of reflectivity to assist in this aim. 
         [0025]    In order to extract the reflectivity of different parts of an image, an illumination estimation algorithm is required. Several approaches have been used to build illumination estimation algorithms, in contexts other than that of image analysis. 
         [0026]    Referring to  FIG. 8 , the output of a camera depends on three factors: 
         [0027]    1. Illuminant (light source) (E)  800   
         [0028]    2. Spectral response characteristics (Q) of camera (sensors)  840   
         [0029]    3. Reflectivity (S) of the object  830   
         [0030]    For an RGB camera, the R G and B outputs are related to the illuminant, the camera response and the reflectivity by Equation (1) 
         [0000]        R=w   d   ∫E (λ)· S (λ)· Q   R (λ) dλ+{tilde over (w)}   s   ∫E (λ)· Q   R (λ) dλ 
 
         [0000]        G=w   d   ∫E (λ)· S (λ)· Q   G (λ) dλ+{tilde over (w)}   s   ∫E (λ)· Q   G (λ) dλ 
 
         [0000]        B=w   d   ∫E (λ)· S (λ) ·Q   B (λ) dλ+{tilde over (w)}   s   ∫E (λ)· Q   B (λ) dλ   Eq. 1
 
         [0000]    where λ is the wavelength (the visible range is approximately from 400 nm to 700 nm), E(λ) is the illuminant spectral intensity of radiation, S(λ) is the spectral reflectivity of the object, Q R (λ) is the red camera sensor spectral response characteristics, Q G (λ) is the green camera sensor spectral response characteristics, Q B (λ) is the camera sensor spectral response characteristics, w d  the parameter for the diffuse reflection  820  {tilde over (w)} s  component, the geometrical parameter for the specular reflection  810  component. 
         [0031]    The reflectivity of an object is a measure of the amount of light reflected by an object, or radiance, relative to the amount of incident light shone on the object, or irradiance, and is indicative of the reflectance or intrinsic brightness of an object. 
         [0032]    The reflectivity of an object can be used as a signature of the object. Hence it may be used to segment the object with respect to the remainder of an image. The reflectivity of an object is composed of diffuse (Lambertian, body) and specular (surface) components. 
         [0033]    The described embodiments deal with diffuse-reflecting materials. 
         [0034]    Humans have the ability to separate the illumination power spectral distribution from the surface reflectance function when judging object appearance; such ability is called color constancy. 
       SUMMARY OF THE INVENTION 
       [0035]    In embodiments there is provided a device for processing an image, comprising: processing circuitry configured to receive data of a pixellated current image, data indicative of at least one parameter indicative of the spectral characteristics of the illumination of the scene shown in the current image, and operable to determine the reflectivity of at least one surface of at least one object shown in the current image; modelling circuitry configured to derive a background model from information of at least one previous image of the scene; and decision circuitry configured to compare said reflectivity with a reflectivity of a part of the background model. 
         [0036]    In embodiments there is provided a method of processing an image; the method comprising: processing a pixellated current image, using at least one parameter indicative of the spectral characteristics of the illumination of the scene shown in the current image, to determine the reflectivity of at least one surface of at least one object shown in the current image; and comparing said reflectivity with a reflectivity of a part of a background model of the scene shown in the image, the background model being derived from information of at least one previous image of the scene. 
         [0037]    The device and method may be applied to foreground object segmentation. 
         [0038]    An embodiment includes acquiring an image of a scene including an object of interest, estimating the scene illumination characteristics, calculating values of the reflectivity of the objects in the scene by using the estimated illumination characteristics, and using the reflectivity to segment the object of interest, also referred to herein as a foreground object. 
         [0039]    In embodiments, an image representation describes the physical properties of object surfaces, as a step in achieving robust segmentation for foreground objects. A color appearance representation is based on the estimation of illumination spectral characteristics and the knowledge of camera sensor spectral response. An illumination estimation algorithm, a calculation of the physics-based model for image formation (in a described example assuming diffuse-only materials) and transformation from a camera output to the required parameters can be performed in real-time. 
         [0040]    In embodiments, segmentation between foreground and background objects is done by matching between the spectral reflectivity of a reference background frame and each new frame over the full spectrum of the visible wavelengths. The technique uses a robust physics-based algorithm for foreground-object segmentation, which performs better than known statistics-based algorithms. 
         [0041]    There is disclosed a device for processing an image, comprising: processing circuitry configured to receive data of a pixellated current image, and to determine from that image data at least one parameter indicative of the characteristics of the illuminant; the processing circuitry further configured to process the illuminant data to provide data indicative of at least one parameter indicative of the spectral characteristics of the illumination of the scene shown in the current image, and to process the image data thereby to determine the reflectivity of at least one surface of at least one object shown in the current image; modelling circuitry configured to derive a background model from information of at least one previous image of the scene; and decision circuitry configured to compare said reflectivity with a reflectivity of a part of the background model. 
         [0042]    In the device, the processing circuitry may be configured to adaptively separate the spectral characteristics of the illumination from the determined surface reflectivity. 
         [0043]    The processing circuitry may be operable to derive the weights of spectral basis functions indicative of illumination or surface reflectivity for pixels in the current image. The weights may be a direct estimate of true parameters of the reflectivity. The decision circuitry may include thresholding circuitry whereby only reflectivity differences greater than a threshold are taken into account. The processing circuitry may be configured to normalize the spectral similarity measure to enable selection of pixels indicative of the illumination spectral characteristics. 
         [0044]    Estimating circuitry may estimate the spectral characteristics of the illumination of the scene. The estimating circuitry may be operable to determine highlight regions of an input image, and estimate spectral characteristics of the illuminant of the scene therefrom. 
         [0045]    The estimating circuitry may be operable to determine highlight regions of an input image, and estimate the correlated color temperature of the illuminant of the scene 
         [0046]    The device may further comprise circuitry for determining the spectral characteristics of the illumination using data from the current image and estimating therefrom the spectral characteristics of the illumination. The modelling circuitry may be operable to derive the background model using statistical parameters of surface reflectivity and illuminant characteristics. The modelling circuitry may include update circuitry whereby a change in illuminant triggers an update of model parameters. 
         [0047]    The device may further comprise mask-production circuitry operable to use the result of said decision circuitry to create a segmented mask. The device may further comprise a mask processing stage. The mask processing stage may be configured to apply dilation and erosion processes to the segmented mask. A feedback to the modelling circuitry may enable update of the background model using information from the current image. 
         [0048]    There is disclosed object segmentation apparatus comprising a device as recited above, and a surveillance system comprising the object segmentation apparatus. 
         [0049]    There is also disclosed a method of processing an image; the method comprising: 
         [0050]    processing pixellated image data to determine from that image data at least one parameter indicative of the characteristics of the illuminant; processing image data to derive at least one parameter indicative of the spectral characteristics of the illumination of the scene shown in the current image, processing image data to determine the reflectivity of at least one surface of at least one object shown in the current image; and comparing said reflectivity with a reflectivity of a part of a background model of the scene shown in the image, the background model being derived from information of at least one previous image of the scene. 
         [0051]    The processing step may comprise deriving the weight of spectral basis functions indicative of surface reflectivity or illumination for pixels in the current image. The comparing step may comprise a threshold comparison. 
         [0052]    The method may comprise calculating a ratio between the autocorrelation of the pixels of an image and the autocorrelation of a mean background image. 
         [0053]    The method may comprise using the result of said comparing step to create a segmented mask. Dilation and erosion processes may be used. A model maintenance process may enable the background model to be updated using information from the current image. 
         [0054]    There is also disclosed an image segmentation method comprising a training phase, and a segmentation phase, wherein the training phase comprises operating on a frame of pixellated data from a camera using information on camera characteristics to render it camera-independent, and processing the camera independent data using a chosen value of illuminant spectral characteristics to derive reflectivity data whereby pixels of high reflectivity are established, using data from the high reflectivity pixels, determining data on the actual illuminant spectral characteristics and processing the illuminant data to determine information on the illumination of the scene represented by the frame of pixellated data to derive reflectivity data of the scene; the segmentation phase comprises operating on a subsequent frame of pixellated data to render it camera-independent and using the determined illumination information to process the camera independent data to determine reflectivity data of the scene to derive a foreground mask. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0055]    An embodiment of the invention will now be described, by way of example only with respect to the accompanying drawings, in which:— 
           [0056]      FIG. 1  shows a schematic diagram of an embodiment of an image processing device; 
           [0057]      FIG. 2  shows a schematic diagram of reflectivity processing circuitry; 
           [0058]      FIG. 3  shows a schematic diagram of comparison circuitry; 
           [0059]      FIG. 4  shows a schematic diagram of an illumination estimation stage; 
           [0060]      FIG. 5  shows an exemplary algorithm for segmenting an object; 
           [0061]      FIG. 6  shows a block diagram of an automated video surveillance system; 
           [0062]      FIG. 7  shows a known algorithm for object segmentation; and 
           [0063]      FIG. 8  is a diagram showing how illumination and the image picked up by a camera are related. 
       
    
    
     DETAILED DESCRIPTION 
       [0064]    In representing the spectral reflectance of objects, the described embodiment that follows uses Parkkinen basis functions by way of example. Other representations may of course be used. For example instead of Parkkinen basis functions, embodiments may use other spectral basis functions, an appropriate set of eigenvectors of the statistical distribution of the spectral reflectances of Munsell color chips, or of natural surfaces, or an appropriately selected set of Fourier basis functions. 
         [0065]    Referring to  FIG. 1 , an image processing device  1000  for object segmentation of a current image has a camera  100  having an output  101  providing the input to a data converter  102 . The data converter  102  has an output  103  to reflectivity processing circuitry  104  also receiving a second input  105  from a first memory  106 . The output  107  of the reflectivity processing circuitry  104  provides a first input to decision circuitry  110  that receives a second input  109  from a background modelling circuit  108 . The output  111  of the decision circuitry  110  provides the input to mask production circuitry  112 . The output  113  of the mask production circuitry  112  is input to a mask processing circuit  114  providing a first output  115  and a second output  117 . The first output is provided as the processing output of the device  1000 , and the second output  117  provides an input to a feedback and model-update circuit  118 , having an output  119  to the background modelling circuit. 
         [0066]    Current image information is captured by the camera in pixellated form, and passed to the data converter  102  where the data are converted to RGB output data  103 . The RGB output data  103  together with correlated color temperature data  105  indicative of the illumination of the scene depicted in the current image are processed in the reflectivity processing circuitry  104  such that the output  107  is a set of data indicative of the reflectivity of the surfaces of the objects in the scene represented by the pixels of the current image. This output data set  107  is compared with a set of background data  109  indicative of the reflectivity of surfaces of a background image to the scene of the current image. Typically this set of background data is derived by means of a training phase during which no motion occurs, and/or a series of images to take into account newly appearing objects that may become static. 
         [0067]    The correlated color temperature data  105  in this embodiment is predetermined, since the embodiment is for a fixed situation, such as an industrial inspection system. 
         [0068]    In other embodiments, spectral characteristic data is derived adaptively from input image data by determining the surface reflectivity of objects represented by the image, and separating out the spectral data from the surface reflectivity. 
         [0069]    The comparison in this embodiment requires that surface reflectivity represented by a pixel must differ from that represented by a background pixel by more than a particular threshold, rather than merely being greater or less in reflectivity. The output  111  of the decision circuitry  110  is binary and formed into a segmented mask  113  by mask production circuitry  112 . The mask  113  in this embodiment is fed to post-processing circuitry  114  that carries out dilation and erosion processes to remove noise in the foreground pixels. Once the filtered foreground pixels are obtained, connected regions are then found by using a connected-component labelling algorithm. The labelled regions may contain near but disjoint regions due to defects in the foreground segmentation process. It has been experimentally found to be effective to merge these overlapping isolated regions. Also, some relatively small regions caused by environmental noise are eliminated in a region-level post-processing step, followed by object and feature extraction stages. 
         [0070]    In other embodiments, morphological processes other than, or additional to, dilation and erosion are effected. One output  115  of the post processing circuitry  114  provides the object segmenting mask that is used to segment desired objects from the current image. The other output  117  is processed by the feedback circuitry  118  and passed back to the background modelling circuitry to update the background model. 
         [0071]    The decision circuitry  110  operates on a pixel-by-pixel basis and classifies those pixels that are sufficiently different from the statistical model (by more than the threshold) as being representative of a target object, and those that are not as being other objects. 
         [0072]    Due to camera noise and illumination variations, a certain number of pixels are likely to be classified as a target object even if they do not belong to it: a high value of the threshold allows maintaining a low number of false detections. On the other hand, objects presenting a low contrast with respect to the target risk to be eliminated if the threshold is excessively high. Accordingly, in some embodiments, a statistical model is used to modify the thresholds in threshold selection process stage. 
         [0073]    Turning to  FIG. 2 , reflectivity processing circuitry  104  processes the input image data  103  also using sensor spectral response data  121  of the camera to calculate the weights of Parkkinen basis functions  122  on the basis of a known or calculated illumination color temperature  105 , as a direct estimate of the reflectivity characteristics. The output  107  is a set of weight data representing the reflectivity of the surfaces of the objects in the input image in isolation from the effects caused by the camera characteristics and the effects caused by illumination, and changes in illumination. 
         [0074]    In order to build a computational physical model, equation 1 is used for the three basis functions of Parkkinen (n=3), so the reflectivity function is represented with 
         [0000]    
       
         
           
             
               
                 
                   
                     S 
                      
                     
                       ( 
                       λ 
                       ) 
                     
                   
                   ≈ 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       n 
                     
                      
                     
                       
                         w 
                         i 
                       
                        
                       
                         
                           
                             φ 
                             i 
                           
                            
                           
                             ( 
                             λ 
                             ) 
                           
                         
                         . 
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   2 
                 
               
             
           
         
       
     
         [0075]    For example, from the manufacturer&#39;s data sheet, the actual characteristics of the sensors of the camera  100  are obtained, as the sensor spectral responses  121 . 
         [0076]    The aim now is to calculate the weights of Parkkinen basis functions, to obtain the spectral reflectivity of the object represented by the pixels of the current image. 
         [0077]    The model is rewritten as: 
         [0000]        R=w   d   w   1   ∫E (λ)·φ 1 (λ)· Q   R (λ) d λ+w   d   w   2   ∫E (λ)·φ 2 (λ)· Q   R (λ) dλ+w   d   w   3   ∫E (λ)·φ 3 (λ)· Q   R (λ) dλ+{tilde over (w)}   s   ∫E (λ) Q   R (λ) dλ 
 
         [0000]        G=w   d   w   1   ∫E (λ)·φ 1 (λ)· Q   G (λ) dλ+w   d   w   2   ∫E (λ)·φ 2 (λ)· Q   G (λ) dλ+w   d   w   3   ∫E (λ)·φ 3 (λ)· Q   G (λ) dλ+{tilde over (w)}   s   ∫E (λ) Q   G (λ) dλ 
 
         [0000]        B=w   d   w   1   ∫E (λ)·φ 1 (λ) ·Q   B (λ) dλ+w   d   w   2   ∫E (λ)·φ 2 (λ)· Q   B (λ) dλ+w   d   w   3   ∫E (λ)·φ 3 (λ)· Q   B (λ) dλ+{tilde over (w)}   s   ∫E (λ) Q   B (λ) dλ   Eq. 3
 
         [0000]      or 
         [0000]        R={tilde over (w)}   1   ∫E (λ)·φ 1 (λ)· Q   R (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   R (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   R (λ) dλ+   ∫E (λ) Q   R (λ) dλ 
 
         [0000]        G={tilde over (w)}   1   ∫E (λ)·φ 1 (λ)· Q   G (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   G (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   G (λ) dλ+   ∫E (λ) Q   G (λ) dλ 
 
         [0000]        R={tilde over (w)}   1   ∫E (λ)·φ 1 (λ)· Q   B (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   B (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   B (λ) dλ+   ∫E (λ) Q   B (λ) dλ 
 
         [0000]      where 
         [0000]        {tilde over (w)}   1   =w   d   w   1   ,{tilde over (w)}   2   =w   d   w   2  and  {tilde over (w)}   3   =w   d   W   3    Eq.  5 
         [0000]    Knowing that the first basis function of Parkkinen is constant, so φ 1 (λ)=k φ , so 
         [0000]        R={tilde over (w)}   1   ·k   φ   ∫E (λ)·φ 1 (λ)· Q   R (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   R (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   R (λ) dλ+   ∫E (λ) Q   R (λ) dλ 
 
         [0000]        R={tilde over (w)}   1   ·k   φ   ∫E (λ)·φ 1 (λ)· Q   G (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   G (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   G (λ) dλ+   ∫E (λ) Q   G (λ) dλ 
 
         [0000]        R={tilde over (w)}   1   ·k   φ   ∫E (λ)·φ 1 (λ)· Q   B (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   B (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   B (λ) dλ+   ∫E (λ) Q   B (λ) dλ   Eq. 6
 
         [0000]    which then could be merged with the specular component to give: 
         [0000]        R={tilde over (w)}   1   ·k   φ+{tilde over (w)}   s   ∫E (λ)·φ 1 (λ)· Q   R (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   R (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   R (λ) dλ+   ∫E (λ) Q   R (λ) dλ 
 
         [0000]        R={tilde over (w)}   1   ·k   φ+{tilde over (w)}   s   ∫E (λ)·φ 1 (λ)· Q   G (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   G (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   G (λ) dλ+   ∫E (λ) Q   G (λ) dλ 
 
         [0000]        R={tilde over (w)}   1   ·k   φ+{tilde over (w)}   s   ∫E (λ)·φ 1 (λ)· Q   B (λ) dλ+{tilde over (w)}   2   ∫E (λ)·φ 2 (λ)· Q   B (λ) dλ+{tilde over (w)}   3   ∫E (λ)·φ 3 (λ)· Q   B (λ) dλ+   ∫E (λ) Q   B (λ) dλ   Eq. 7
 
         [0000]      by taking 
         [0000]        X   i   =∫E (λ)·φ i (λ)· Q   R (λ) dλ 
 
         [0000]        Y   i   =∫E (λ)·φ i (λ)· Q   R (λ) dλ 
 
         [0000]        Z   i   =∫E (λ)·φ i (λ)· Q   R (λ) dλ 
 
         [0000]    These integrations are calculated to obtain the transformation matrix 
         [0000]    
       
         
           
             
               
                 
                   
                     [ 
                     
                       
                         
                           R 
                         
                       
                       
                         
                           G 
                         
                       
                       
                         
                           B 
                         
                       
                     
                     ] 
                   
                   = 
                   
                     
                       
                         [ 
                         
                           
                             
                               
                                 X 
                                 1 
                               
                             
                             
                               
                                 X 
                                 2 
                               
                             
                             
                               
                                 X 
                                 3 
                               
                             
                           
                           
                             
                               
                                 Y 
                                 1 
                               
                             
                             
                               
                                 Y 
                                 2 
                               
                             
                             
                               
                                 Y 
                                 3 
                               
                             
                           
                           
                             
                               
                                 Z 
                                 1 
                               
                             
                             
                               
                                 Z 
                                 2 
                               
                             
                             
                               
                                 Z 
                                 3 
                               
                             
                           
                         
                         ] 
                       
                        
                       
                         [ 
                         
                           
                             
                               
                                 
                                   
                                     
                                       w 
                                       ~ 
                                     
                                     1 
                                   
                                   · 
                                   
                                     k 
                                     φ 
                                   
                                 
                                 + 
                                 
                                   
                                     w 
                                     ~ 
                                   
                                   s 
                                 
                               
                             
                           
                           
                             
                               
                                 
                                   w 
                                   ~ 
                                 
                                 2 
                               
                             
                           
                           
                             
                               
                                 
                                   w 
                                   ~ 
                                 
                                 3 
                               
                             
                           
                         
                         ] 
                       
                     
                     . 
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   9 
                 
               
             
           
         
       
     
         [0000]    Now the weights of the basis functions can be obtained from RGB values by: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       [ 
                       
                         
                           
                             
                               
                                 
                                   
                                     w 
                                     ~ 
                                   
                                   1 
                                 
                                 · 
                                 
                                   k 
                                   φ 
                                 
                               
                               + 
                               
                                 
                                   w 
                                   ~ 
                                 
                                 s 
                               
                             
                           
                         
                         
                           
                             
                               
                                 w 
                                 ~ 
                               
                               2 
                             
                           
                         
                         
                           
                             
                               
                                 w 
                                 ~ 
                               
                               3 
                             
                           
                         
                       
                       ] 
                     
                     = 
                     
                       
                         
                           [ 
                           
                             
                               
                                 
                                   X 
                                   1 
                                 
                               
                               
                                 
                                   X 
                                   2 
                                 
                               
                               
                                 
                                   X 
                                   3 
                                 
                               
                             
                             
                               
                                 
                                   Y 
                                   1 
                                 
                               
                               
                                 
                                   Y 
                                   2 
                                 
                               
                               
                                 
                                   Y 
                                   3 
                                 
                               
                             
                             
                               
                                 
                                   Z 
                                   1 
                                 
                               
                               
                                 
                                   Z 
                                   2 
                                 
                               
                               
                                 
                                   Z 
                                   3 
                                 
                               
                             
                           
                           ] 
                         
                         
                           - 
                           1 
                         
                       
                        
                       
                         [ 
                         
                           
                             
                               R 
                             
                           
                           
                             
                               G 
                             
                           
                           
                             
                               B 
                             
                           
                         
                         ] 
                       
                     
                   
                    
                   
                     
 
                   
                    
                   or 
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   10 
                 
               
             
             
               
                 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 w 
                                 d 
                               
                               · 
                               
                                 w 
                                 1 
                               
                               · 
                               
                                 k 
                                 φ 
                               
                             
                             + 
                             
                               
                                 w 
                                 s 
                               
                               · 
                               
                                 k 
                                 s 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               w 
                               d 
                             
                             · 
                             
                               w 
                               2 
                             
                           
                         
                       
                       
                         
                           
                             
                               w 
                               d 
                             
                             · 
                             
                               w 
                               3 
                             
                           
                         
                       
                     
                     ] 
                   
                   = 
                   
                     
                       
                         
                           [ 
                           
                             
                               
                                 
                                   X 
                                   1 
                                 
                               
                               
                                 
                                   X 
                                   2 
                                 
                               
                               
                                 
                                   X 
                                   3 
                                 
                               
                             
                             
                               
                                 
                                   Y 
                                   1 
                                 
                               
                               
                                 
                                   Y 
                                   2 
                                 
                               
                               
                                 
                                   Y 
                                   3 
                                 
                               
                             
                             
                               
                                 
                                   Z 
                                   1 
                                 
                               
                               
                                 
                                   Z 
                                   2 
                                 
                               
                               
                                 
                                   Z 
                                   3 
                                 
                               
                             
                           
                           ] 
                         
                         
                           - 
                           1 
                         
                       
                        
                       
                         [ 
                         
                           
                             
                               R 
                             
                           
                           
                             
                               G 
                             
                           
                           
                             
                               B 
                             
                           
                         
                         ] 
                       
                     
                     . 
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   11 
                 
               
             
           
         
       
     
         [0000]    As a special case for diffuse-only reflection w s =0 then: 
         [0000]    
       
         
           
             
               
                 
                   
                     [ 
                     
                       
                         
                           
                             
                               w 
                               ⃛ 
                             
                             1 
                           
                         
                       
                       
                         
                           
                             
                               w 
                               ⃛ 
                             
                             2 
                           
                         
                       
                       
                         
                           
                             
                               w 
                               ⃛ 
                             
                             3 
                           
                         
                       
                     
                     ] 
                   
                   = 
                   
                     
                       
                         [ 
                         
                           
                             
                               
                                 X 
                                 1 
                               
                             
                             
                               
                                 X 
                                 2 
                               
                             
                             
                               
                                 X 
                                 3 
                               
                             
                           
                           
                             
                               
                                 Y 
                                 1 
                               
                             
                             
                               
                                 Y 
                                 2 
                               
                             
                             
                               
                                 Y 
                                 3 
                               
                             
                           
                           
                             
                               
                                 Z 
                                 1 
                               
                             
                             
                               
                                 Z 
                                 2 
                               
                             
                             
                               
                                 Z 
                                 3 
                               
                             
                           
                         
                         ] 
                       
                       
                         - 
                         1 
                       
                     
                      
                     
                       [ 
                       
                         
                           
                             R 
                           
                         
                         
                           
                             G 
                           
                         
                         
                           
                             B 
                           
                         
                       
                       ] 
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   12 
                 
               
             
           
         
       
     
         [0000]    where          =w d ·w 1 ·k φ ,          =w d ·w 2  and          =w d ·w 3 . 
         [0078]    By using this transformation, the RGB image is represented by basis function weights as an expression of the reflectivity of the surfaces represented in the current image. 
         [0079]    Turning now to  FIG. 3 , given their illumination parameters, to compare two objects (whether they have the same surface material and color or not), their spectral reflectivity functions  123 ,  124  can be compared by cross-correlation. 
         [0080]    This may be achieved by the correlation between the Parkkinen basis functions in a statistical computing module  300  of the decision circuitry  110 . 
         [0000]    
       
         
           
             
               
                 
                   
                     [ 
                     
                       
                         
                           
                             K 
                             11 
                           
                         
                       
                       
                         
                           
                             K 
                             22 
                           
                         
                       
                       
                         
                           
                             K 
                             33 
                           
                         
                       
                       
                         
                           
                             K 
                             12 
                           
                         
                       
                       
                         
                           
                             K 
                             13 
                           
                         
                       
                       
                         
                           
                             K 
                             23 
                           
                         
                       
                     
                     ] 
                   
                   = 
                   
                     [ 
                     
                       
                         
                           
                             Corr 
                              
                             
                               ( 
                               
                                 
                                   φ 
                                   1 
                                 
                                 , 
                                 
                                   φ 
                                   1 
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             Corr 
                              
                             
                               ( 
                               
                                 
                                   φ 
                                   2 
                                 
                                 , 
                                 
                                   φ 
                                   2 
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             Corr 
                              
                             
                               ( 
                               
                                 
                                   φ 
                                   3 
                                 
                                 , 
                                 
                                   φ 
                                   3 
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             Corr 
                              
                             
                               ( 
                               
                                 
                                   φ 
                                   1 
                                 
                                 , 
                                 
                                   φ 
                                   2 
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             Corr 
                              
                             
                               ( 
                               
                                 
                                   φ 
                                   1 
                                 
                                 , 
                                 
                                   φ 
                                   3 
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             Corr 
                              
                             
                               ( 
                               
                                 
                                   φ 
                                   2 
                                 
                                 , 
                                 
                                   φ 
                                   3 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   13 
                 
               
             
           
         
       
     
         [0081]    If the first surface has weights  123  given by          ,           and          , while the second surface has weights  124  given by          ,           and          , then the correlation  125 , indicating the degree of similarity between two different surfaces becomes: 
         [0000]    
       
         
           
             
               
                 
                   
                     Corr 
                      
                     
                       ( 
                       
                         
                           I 
                           1 
                         
                         , 
                         
                           I 
                           2 
                         
                       
                       ) 
                     
                   
                   = 
                   
                     
                       
                         
                           w 
                           ⃛ 
                         
                         1 
                       
                        
                       
                         
                           w 
                           ⃛ 
                         
                         1 
                         ′ 
                       
                        
                       
                         K 
                         11 
                       
                     
                     + 
                     
                       
                         
                           w 
                           ⃛ 
                         
                         2 
                       
                        
                       
                         
                           w 
                           ⃛ 
                         
                         21 
                         ′ 
                       
                        
                       
                         K 
                         22 
                       
                     
                     + 
                     
                       
                         
                           w 
                           ⃛ 
                         
                         3 
                       
                        
                       
                         
                           w 
                           ⃛ 
                         
                         3 
                         ′ 
                       
                        
                       
                         K 
                         33 
                       
                     
                     + 
                     
                       
                         ( 
                         
                           
                             
                               
                                 w 
                                 ⃛ 
                               
                               1 
                             
                              
                             
                               
                                 w 
                                 ⃛ 
                               
                               2 
                               ′ 
                             
                           
                           + 
                           
                             
                               
                                 w 
                                 ⃛ 
                               
                               2 
                             
                              
                             
                               
                                 w 
                                 ⃛ 
                               
                               1 
                               ′ 
                             
                           
                         
                         ) 
                       
                        
                       
                         K 
                         12 
                       
                     
                     + 
                     
                       
                         ( 
                         
                           
                             
                               
                                 w 
                                 ⃛ 
                               
                               1 
                             
                              
                             
                               
                                 w 
                                 ⃛ 
                               
                               3 
                               ′ 
                             
                           
                           + 
                           
                             
                               
                                 w 
                                 ⃛ 
                               
                               3 
                             
                              
                             
                               
                                 w 
                                 ⃛ 
                               
                               1 
                               ′ 
                             
                           
                         
                         ) 
                       
                        
                       
                         K 
                         13 
                       
                     
                     + 
                     
                       
                         ( 
                         
                           
                             
                               
                                 w 
                                 ⃛ 
                               
                               2 
                             
                              
                             
                               
                                 w 
                                 ⃛ 
                               
                               3 
                               ′ 
                             
                           
                           + 
                           
                             
                               
                                 w 
                                 ⃛ 
                               
                               3 
                             
                              
                             
                               
                                 w 
                                 ⃛ 
                               
                               2 
                               ′ 
                             
                           
                         
                         ) 
                       
                        
                       
                         K 
                         23 
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   14 
                 
               
             
           
         
       
     
         [0082]    Referring now to  FIG. 4   a , an illumination estimation algorithm starts by segmenting areas in the image which represent high specularities (highlights)  701 . To do this a modified version  1041  of the illumination estimation stage  104  is supplied with one background frame BG, and the data from this is processed using camera spectral response data  121 . At present, there is likely to be no calculated value of illuminant, so a value  1042  for the illuminant correlation color temperature (CCT) is chosen depending on the available knowledge of the environment. In the embodiment, initially the color temperature is set at T=4400° K for Fluorescent F 2 . 
         [0083]    The illumination spectral characteristic is then calculated in processing step  1043  using Planck&#39;s formula: 
         [0000]    
       
         
           
             
               
                 
                   
                     E 
                      
                     
                       ( 
                       λ 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         c 
                         1 
                       
                       · 
                       
                         λ 
                         
                           - 
                           5 
                         
                       
                       · 
                       
                         
                           ( 
                           
                             
                                
                               
                                 
                                   c 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                                 
                                   λ 
                                   · 
                                   T 
                                 
                               
                             
                             - 
                             1 
                           
                           ) 
                         
                         
                           - 
                           1 
                         
                       
                     
                     ≅ 
                     
                       
                         c 
                         1 
                       
                       · 
                       
                         λ 
                         
                           - 
                           5 
                         
                       
                       · 
                       
                          
                         
                           
                             
                               - 
                               c 
                             
                              
                             
                                 
                             
                              
                             2 
                           
                           
                             λ 
                             · 
                             T 
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   15 
                 
               
             
           
         
       
     
         [0000]    where E(λ) is the illumination spectral power distribution in Watts/m 2 /wavelength interval; λ is the wavelength in m; c 1 =3.74183.10 −16  Watt·m 2 ; c 2 =1.4388.10 −2  m·° K; T is the color temperature of the illuminant, in ° K. 
         [0084]    Finally the reflectivity  1045  (per pixel) is calculated in step  1044  using the processes set out above with respect to  FIG. 2 . 
         [0085]    Turning to  FIG. 4   b , in order to segment the areas with high specularities (Highlight Mask )  701 , the reflectivity value  1045  is then subject to autocorrelation in a statistical computing module  600  for all pixels of the current frame. The autocorrelation outcome is thresholded, in this embodiment using a threshold value of 0.5, to provide a set of pixels of high specularity, as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     Highlight 
                     mask 
                   
                   = 
                   
                     { 
                     
                       
                         
                           0 
                         
                         
                           
                             
                               
                                 Corr 
                                  
                                 
                                   ( 
                                   
                                     BG 
                                     , 
                                     BG 
                                   
                                   ) 
                                 
                               
                               &lt; 
                               0.5 
                             
                             , 
                           
                         
                       
                       
                         
                           1 
                         
                         
                           otherwise 
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   16 
                 
               
             
           
         
       
     
         [0086]    In other embodiments and for other data sets different values of threshold may be appropriate. 
         [0087]    The segmented pixels with high specularities  701  are converted to CIE xyz and used to calculate the actual correlated color temperature  703  of the illuminant, using a process  702  based upon McCamy&#39;s method, according to the following two equations: 
         [0000]    
       
         
           
             
               
                 
                   n 
                   = 
                   
                     
                       x 
                       - 
                       
                         x 
                         e 
                       
                     
                     
                       
                         y 
                         e 
                       
                       - 
                       y 
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   17 
                 
               
             
           
         
       
     
         [0000]    where 
         [0000]        x   e =0.3320 and  y   e =0.1858 
         [0000]        CCT= 449 ·n   3 +3525· n   2 +6823.3 ·n+ 5520.33  Eq. 18
 
         [0088]    The value of CCT  703  of the illuminant may now be applied to the memory  106  for use by the reflectivity processing circuitry  104  (see  FIG. 1 ) in handling later image data. McCamy&#39;s method, used in this embodiment, is able to estimate the CCT from CIE 1931 chromaticity coordinates x and y. McCamy&#39;s method has a maximum absolute error of less than 2° K for color temperatures ranging from 2,856 to 6,500° K (corresponding to CIE illuminants A through D65). These errors are negligible over the range of interest in this patent application. This method proves useful for implementation in real-time applications. It was derived from the assumption that CCT may be represented by a third-order polynomial function of the reciprocal of the slope of the line from specular pixels to the chromaticity of the light and the fact that the isotemperature lines for CCTs of principal interest nearly converge towards a point on the chromaticity diagram. 
         [0089]    Turning to  FIG. 5 , the algorithm  500  starts with a training phase  501  where training images  502  are acquired using an image acquisition stage, here including a full color video camera, to produce a composite video output stream at a conventional frame rate. 
         [0090]    The image acquisition stage may include a variety of types of video cameras and in general, any suitable mechanism for capturing an image of a subject may be used as the image acquisition stage. The image acquisition stage may, alternatively, be an interface to a storage device, such as a magnetic storage medium or other components for storing videos or image data. The stored videos include compressed and uncompressed formats. 
         [0091]    As used herein, “image data” refers to data which define or characterize an image including color spaces such as RGB, YCbCr, YUV, and CIE XYZ, etc. . . . , as well as other associated data, such as luminance values and chrominance values. 
         [0092]    The video output stream is received by a frame grabber, which serves to latch frames of the composite video input stream and to convert the composite video analogue signal into a digitised output signal, which is then processed by the remainder of the system. Typical conventional video cameras produce an analog video output stream of 30 frames per second, and thus the image acquisition is configured to capture and digitize image frames as this video rate. 
         [0093]    The video camera need not be limited to composite-video, analog output, or 30 frames per second; the video camera could output digital video, at frame rates other that 30 frames per second. 
         [0094]    The image acquisition produces a digitised frame output signal. The camera itself can digitize acquired images, and hence the frame grabber stage can be integrated directly into the camera. 
         [0095]    In order to start the background modelling phase, this embodiment acquires a sequence of N input images  504  which represent relatively static background frames. These frames are then converted into reflectivity weights  520  in a reflectivity calculation step  508  that is based upon an illumination estimation step  506  providing a calculated CCT. Then the mean reflectivity of the background training images is calculated in a manipulation step  510  as a reference reflectivity 
         [0000]      BG mean , 
         [0000]      BG mean =mean(BG)  Eq. 19
 
         [0096]    Next, in the manipulation step  510  the autocorrelation is calculated, without normalization of its magnitude, as 
         [0000]      BG Corr   Corr (BG mean ,BG mean ).  Eq. 20
 
         [0097]    In the described embodiment, the reflectivity correlation is calculated using Eq. 14. 
         [0098]    Then the manipulation step  510  passes through all background training frames BG l  and calculates the ratio C BG  between the autocorrelation of the reflectivity of each frame and the autocorrelation of the mean background reflectivity of background frames, BG mean , 
         [0000]    
       
         
           
             
               
                 
                   
                     C 
                     BG 
                   
                   = 
                   
                     
                       Corr 
                        
                       
                         ( 
                         
                           
                             BG 
                             mean 
                           
                           , 
                           
                             BG 
                             mean 
                           
                         
                         ) 
                       
                     
                     
                       Corr 
                        
                       
                         ( 
                         
                           
                             BG 
                             i 
                           
                           , 
                           
                             BG 
                             i 
                           
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   21 
                 
               
             
           
         
       
     
         [0000]    where i represents the background frame number from 1→N, where N is the number of background frames used to estimate the reflectivity of the background. 
         [0099]    Finally, the manipulation step calculates the maximum C max  and minimum C min  of C BG  as a statistical model of the background, 
         [0000]        C   min =min( C   BG ) 
         [0000]        C   max =max( C   BG ).  Eq. 22
 
         [0100]    To perform object segmentation, after building the statistical model of the background  540 , the segmentation phase  530  starts by capturing a new frame  532 , and then the reflectivity calculation module  532  converts the new frame to basis function weights representing its reflectivity NF, using the illumination parameter  512  calculated in the illumination calculation stage  506  of the training phase. Then a statistical computing module  514  calculates the autocorrelation NF Corr  of the reflectivity of the new frame as 
         [0000]        NF   Corr   =Corr(   NF,NF ).  Eq. 23
 
         [0101]    Then the ratio C between the correlation of the reflectivity of the new frame and the autocorrelation of the mean reflectivity of the background frames BG mean , is calculated as 
         [0000]    
       
         
           
             C 
             = 
             
               
                 
                   BG 
                   Corr 
                 
                 
                   NF 
                   Corr 
                 
               
               . 
             
           
         
       
     
         [0102]    In a change detection stage  516 , a threshold creates an initial binary foreground mask  535  defined as FG Mask , using the maximum C max  and minimum C min  values of correlation ration C BG  measured from the pool of background frames. 
         [0000]    
       
         
           
             
               
                 
                   
                     FG 
                     Mask 
                   
                   = 
                   
                     { 
                     
                       
                         
                           0 
                         
                         
                           
                             
                               
                                 
                                   τ 
                                   min 
                                 
                                  
                                 
                                   C 
                                   min 
                                 
                               
                               &lt; 
                               C 
                               &lt; 
                               
                                 
                                   τ 
                                   max 
                                 
                                  
                                 
                                   C 
                                   max 
                                 
                               
                             
                             , 
                           
                         
                       
                       
                         
                           1 
                         
                         
                           otherwise 
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   24 
                 
               
             
           
         
       
     
         [0103]    The thresholds used here in this described embodiment are τ max =1.2 and τ min =0.8, other datasets may require other thresholds. 
         [0104]    In other embodiments, the ratio of reflectivity autocorrelation used in this embodiment to detect the foreground mask can be replaced by at least one measure indicative of the reflectivity change of object surface. For example, the cross-correlation between the mean reflectivity of background frames BG mean  and the reflectivity parameter of each background frame can be calculated in the background modelling phase as Corr(BG mean , BG l ) and Eq. 22 applied to find the bounds C max  and C min  of this parameter. In the object segmentation phase, the cross-correlation between the mean reflectivity of background frames BG mean  and the reflectivity of each new frame NF can be calculated as Corr(BG mean ,NF) and used as C in Eq. 24 to detect pixels which are significantly different from the background. 
         [0105]    After the initial foreground mask  535  has been created from the new frame by the change detection stage  516 , a post-processing step  540  applies a dilation step which performs a dilation operation thereon to expand or thicken the initial foreground mask  535 . The dilation operation is the union of all translations of the mask  535  by the structuring element SE. 
         [0106]    Then an erosion step performs an erosion operation on the resultant mask, which reduces the size of an object by eliminating area around the object edges. After the erosion operation has been completed; the detection stage performs a connected component blob analysis on the mask  535 . The erosion operation removes noise and eliminates foreground mask details smaller than the structuring element SE, where the size of the structuring element for both dilation and erosion, in this embodiment, is: 
         [0000]      SE=2 pixels.  Eq. 25
 
         [0107]    There are many conventional techniques which can be used in place of dilation followed by erosion. 
         [0108]    The result is a final foreground mask  537 . 
         [0109]    The scene background model  540  is derived by observing the scene during a training period; this model may be updated continuously, especially where change of illumination occurs, or where objects may enter the scene and remain static for substantial time periods. The embodiments update the background model  540  through a model maintenance phase  539 , using an update rate α, according to the following formulas: 
         [0000]      BG mean (FG Mask )=(1−α)·BG mean (FG Mask )+α· NF (FG Mask )  Eq 26
 
         [0000]        C   max ( C   Mask   max )=(1−α)· C   max ( C   Mask   max )+α· NF ( C   Mask   max )  Eq. 17
 
         [0000]        C   min ( C   Mask   max )=(1−α)· C   min ( C   Mask   max )+α· NF ( C   Mask   max )  Eq. 28
 
         [0000]      where 
         [0000]        C   Mask   max   =C&gt;τ   max   C   max , 
         [0000]        C   Mask   max   =C≧τ   max   C   max ,  Eq. 29
 
         [0000]    The value of the update rate α used for the background model in this embodiment is α=0.001, although other datasets may require other values. 
         [0110]    The modelling circuitry may include update circuitry operable to rapidly update if a rapid change in illuminant occurs, for example if a shadow moves across part or the entire image, or if an illuminating light source turns off. 
         [0111]    The mask output  537  shows very good segmentation results for foreground pixels, the selection of thresholds is easy. The use of the full spectrum of the intrinsic spectral reflectivity is effective in order to differentiate between foreground and background objects. 
         [0112]    The algorithm can work in real-time and each frame is executed in average 0.178 seconds using a Xeon personal computer, (processor: Intel Xeon 3 GHz, memory: 2 GB DDR RAM), an implementation in Matlab (version: 7.4, release: R2007a). 
         [0113]    An implementation uses the value of correlation as the metric of the similarity between the pixels in the new frame with corresponding pixels in the background. 
         [0114]    In the described embodiment account is made only of diffuse reflections, and it is assumed that illumination characteristics follow Planck&#39;s formula in estimating color temperature. However the method of the invention is more generally applicable, with the same or a similar algorithm being useable for both diffuse and specular reflections, and the use of other techniques to estimate the illumination. 
         [0115]    The invention is not limited to the described features of the embodiments, and extends to the full scope of the appended claims.