Abstract:
A video imaging system including: a low resolution colour digital video camera and a high resolution monochromatic digital video camera operably connected to a digital processing system.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS  
       [0001]    This application claims priority from International Patent Application Number PCT/CA2011/050666 filed on 24 Oct. 2011 which claims priority from U.S. Patent Application Ser. No. 61/405,941 filed 22 Oct. 2010. 
     
    
     FIELD OF THE INVENTION  
       [0002]    This invention relates to camera imaging. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS  
         [0003]      FIG. 1  is a schematic diagram of a dual video image system according to one or more embodiments of the present invention; 
           [0004]      FIG. 2  is a schematic diagram of consecutive image frames according to one or more embodiments of the present invention; 
           [0005]      FIG. 3  is a schematic diagram of a two camera system according to one or more embodiments of the present invention; 
           [0006]      FIG. 4  is a schematic diagram of a two sensor single lens system according to one or more embodiments of the present invention; and 
           [0007]      FIG. 5  is a schematic diagram of a rotating half mirror system according to one or more embodiments of the present invention. 
       
    
    
     SUMMARY  
       [0008]    According to one or more embodiments of the present invention, a video imaging system comprising a low resolution colour digital video camera and a high resolution monochromatic digital video camera operably connected to a digital processing system. The system can further comprise an object motion module for detecting objects moving within the fields of view of the cameras, and an object position module for determining the position of an object in the overlapping field of view of the cameras. 
         [0009]    According to one or more embodiments of the present invention, a method comprising providing an image frame from a low resolution colour digital video camera and a corresponding image frame from high resolution monochromatic digital video camera and fusing the two image frames to obtain a colour image having higher resolution than the image frame from the low resolution colour digital video camera. The method can further comprise providing a three dimensional coordinate system for determining the position of a moving object in the overlapping fields of view of the cameras whereby the two dimensional position of the moving object is determined according its position in the images, whereas the distance from the cameras to the object in the axis perpendicular to the plane of the images is derived from the parallax error between the two image frames to be fused. 
         [0010]    According to one or more embodiments of the present invention, a camera imaging system comprising a low resolution colour digital sensor chip, a high resolution monochromatic digital sensor chip, a beam splitter, and a lens, wherein the lens gathers incident light towards the beam splitter, and the beam splitter splits the light towards the two sensor chips. The system further comprises a digital processing system which fuses a low resolution colour image from the colour sensor and a high resolution monochromatic image from monochromatic sensor to produce a high resolution colour image. 
       DETAILED DESCRIPTION  
       [0011]    Referring to  FIG. 1 , an overall system configuration for a dual video imaging system according to an embodiment of the present invention comprises a colour digital video camera  2  having lens  20  and a monochromatic digital video camera  4  having lens  22 . The cameras  2  and  4  each generate a digital signal of scene  6 , which is then transmitted to digital processing system (“DPS”)  12 . 
         [0012]    The cameras  2  and  4  employ charge-coupled device (“CCD”) sensors or complementary metal-oxide-semiconductor (“CMOS”) sensors. Camera  2  is a low resolution colour (“LC”) video camera while camera  4  is a high resolution monochrome (“NM”) video camera. Cameras  2  and  4  are capable of providing streaming video signals as part of a security, surveillance or monitoring system. It will be understood, however, that the applications for the cameras  2  and  4  are not limited to such systems. 
         [0013]    Camera  2  has a field of view defined by light rays  8  while camera  4  has a field of view defined by light rays  10 . Colour camera  2  and monochrome camera  4  produce separate streaming video signals which are then supplied to the DPS  12 . The cameras  2  and  4  are adjacent and can be housed together in a single camera housing (not shown). 
         [0014]    The low resolution colour streaming video signals from camera  2  are fused by image fusing module (“FM”)  26  in processor  12  with corresponding high resolution monochrome streaming video signals from camera  4  to produce a fused high resolution colour streaming video signal (“HC”)  28 . Fusing the colour and monochrome video signals provides the dual camera system with improved sensitivity capable of acquiring high resolution colour video signals under poor lighting conditions due to the inclusion of the high resolution signal from the monochrome camera and the colour signal from the colour camera. 
         [0015]    The colour and monochrome video signals are comprised of individual image frames. Corresponding pairs of video image frames from cameras  2  and  4  are isolated and then fused. Various methods for fusing the frame pairs can be used. For example, image fusion methods for fusing a low resolution multispectral satellite images with high resolution panchromatic satellite images are known in the field of remote sensing and can be adapted to fuse video image frames from cameras  2  and  4 . One such fusion method is disclosed in U.S. Pat. No. 7,340,099 (Zhang) which is incorporated herein by reference in its entirety. Other image fusion methods used for satellite imagery include arithmetic based, statistics based, ratio based and wavelet based methods. By substituting colour and monochrome video image frame pairs according to the present invention for multispectral and panchromatic images respectively, prior art image fusing methods can be adapted to fuse video image frames acquired by camera  2  with video image frames acquired by camera  4 . 
         [0016]    In a further aspect, referring to  FIG. 2 , a moving object  30  in the scene  6  can be detected by both video cameras  2  and  4  based on finding changes in the consecutive image frames  32  of each video camera. If there is no moving object in the scene  6 , the images in the consecutive frames will be the same. If there is a moving object  30  in scene  6 , the images in the consecutive frames will be not the same. The changed area between two adjacent frames is the location of the moving object on the image. The changed areas can be found by comparing images in adjacent frames. Suitable conventional methods, techniques and algorithms for comparing consecutive image frames and finding changes in such image frames can be adopted for this system to find moving objects in consecutive images frames from each of the two cameras  2  and  4 . 
         [0017]    In a still further aspect, referring to  FIG. 3 , the position of objects  01  and  02  in the scene  6  is provided in a three dimensional coordinate system. Since cameras  2  and  4  are adjacent, not overlaid with each other, the light rays from lens  20  and lens  22  to any object in the scene  6  are not parallel. The closer the object to the two lenses, the larger the angle between the two light rays from the two lenses to the object. For example, the object  02  is closer to the two lenses  20  and  22  than is object  01 . The angle A 2  is therefore larger than angle A 1 . The distance from lenses  20  and  22  to any object in the scene  6  can be calculated according to the base line distance between lenses  20  and  22  and the viewing angle between the two light rays. This distance gives the coordinate along the Z-axis of the three-dimensional coordinate system. Using the base line distance between the two lenses and the viewing angle between the two light rays to determine the distance from an airborne sensor to a ground object is well known in the fields of photogrammetry and computer vision. Such methods can be adapted to determine the distance to objects in the scene  6  because cameras  2  and  4  view essentially the same scene, but there is a parallax between the field of view  8  of camera  2  and the field of view  10  of camera  4 . 
         [0018]    The plane of the image frames  32  in  FIG. 2  is defined by an X-Y coordinate system which is used to position objects (such as object  30 ) in two dimensions on the image frames  32 . The X-Y axis position of an object plus its Z-axis position provides the object&#39;s position in three dimensions relative to cameras  2  and  4 . The X-Y-Z positions are provided to a position calculating module in the DPS  12  which calculates the position of objects in the scene  6 . The position calculating module is programmed with suitable computer algorithms based on prior art photogrammetric or computer vision methods as described above. 
         [0019]    In one or more embodiments, the dual camera system according to the present invention provides colour video with improved sensitivity compared with a conventional video camera, the detection of moving objects, and the three dimensional position of the objects in the common field of view of the cameras  2  and  4 . 
         [0020]    According to one or more embodiments of the present invention, methods of the present invention can be applied to image frames from two corresponding still cameras. 
         [0021]    In a still further aspect, referring to  FIG. 4 , the dual imaging system described above with initial reference to  FIG. 1  is modified by replacing the dual cameras and dual lenses with a dual sensor single lens camera. Lens  22  is omitted and a light splitter  40  is added. The light beam splitter  40  splits the incoming light into two directions. Camera  2  is re-configured with its low resolution digital colour sensor  42  towards one of the split light beams and camera  4  is re-configured with its high resolution monochrome digital sensor  44  towards the other split beam. 
         [0022]    Cameras  2  and  4  are positioned such that when the light splitter  40  splits the incoming light into two directions, about half of the incident light is directed towards the colour digital sensor  42  and about the other half of the incident light is directed towards the monochromatic digital sensor  44 . In this embodiment, the capacity of detecting distance from the camera to a moving object is reduced. 
         [0023]    Separate streaming video signals from sensors  42  and  44  are then supplied to the DPS  12  in a similar manner to the signals from cameras  2  and  4  in the system described with initial reference to  FIG. 1 . 
         [0024]    Low resolution colour streaming video signals from sensor  42  are fused by the FM  26  in processor  12  with corresponding high resolution monochrome streaming video signals from sensor  44  to produce a fused high resolution colour streaming video signal (“HC&#39;)  28  using the methods described herein. 
         [0025]    In a still further embodiment, referring to  FIG. 5 , a rotating half mirror  50  or vibrating mirror (not shown) can be used in place of the splitter  40  of  FIG. 4 . The half mirror  50  rotates between a first position (P 1 ) where incident light from the lens  20  passes through the empty half of the mirror directly reaching the sensor  44  and a second position (P 2 ) where incident light passing through the lens  20  is directed by the half mirror to the sensor  42 . The mirror  50  rotates between the first and second positions sufficiently quickly to obtain a suitable image pair of a scene. 
         [0026]    It is understood that other devices can be used in place of splitter  40  or a mirror  50 , as long as they can direct incident light from lens  20  towards both sensor  42  and sensor  44  simultaneously.