Patent Document

FIELD OF THE INVENTION 
       [0001]    Disclosed embodiments relate generally to an image capturing device and, more particularly, to a motion detection technique for an image capturing device. 
       BACKGROUND OF THE INVENTION 
       [0002]    There are several algorithms related to image processing which rely on motion detection. For example, the method described in co-pending application Ser. No. 11/509,712, filed Aug. 25, 2006, entitled METHOD, APPARATUS AND SYSTEM PROVIDING ADJUSTMENT OF PIXEL DEFECT MAP, uses motion detection to separate real defects from small elements of the scene in determining the location of defects in the pixel array. As another example, passive continuous auto-focus algorithms use motion detection to control their refocusing procedures. 
         [0003]    Simple algorithms for motion detection compare an image of the current frame with a reference frame image and count the number of pixels which are different between the frames. This type of motion detection algorithm, however, requires a large frame memory buffer and, therefore, cannot easily be implemented in system-on-a-chip (SOC) imager systems which have circuit area limitations. Moreover, the large frame memory adds cost to the imaging device. 
         [0004]    Accordingly, there exists a need for an improved method and system for motion detection within an imager. There further exists a need for a motion detection system and method which may be implemented in an SOC system. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0005]      FIG. 1  illustrates image frames between which motion may be detected in accordance with disclosed embodiments. 
           [0006]      FIG. 2  is a flowchart illustrating the steps of a method for using statistics data for motion detection in accordance with disclosed embodiments. 
           [0007]      FIG. 3  is a block diagram of a system-on-a-chip (SOC) imager construction implementing disclosed embodiments. 
           [0008]      FIG. 4  depicts a camera system, for example, a digital still or video camera, which may employ an imager constructed in accordance with disclosed embodiments. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0009]    In the following detailed description, reference is made to the accompanying drawings which form a part hereof, and in which are shown, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to make and use them, and it is to be understood that structural, logical or procedural changes may be made. Particularly, in the description below, processes are described by way of flowchart. In some instances, steps which follow other steps may be reversed, be in a different sequence or be in parallel, except where a following procedural step requires the presence of a prior procedural step. The disclosed processes may be implemented by an image processing pipeline or other circuit which processes an array of pixel values. The image processing can be implemented using hardware circuits, programmed processors, or a combination of the two. 
         [0010]    Disclosed embodiments provide a motion detection method, system and apparatus for detecting motion based on statistics data that already exists within an imager for other purposes. Disclosed embodiments base the determination of whether or not there is motion in the scene on the relative values of the statistics data between two frames of the imaged scene. This allows the disclosed embodiments to be easily implemented with minimal expense. 
         [0011]    One type of statistics data that is readily available within most imagers is a sharpness score. The sharpness score is a number representing the degree of contrast between adjacent areas within a zone. In a digital camera, sharpness scores are determined by measuring the difference between the intensities of adjacent areas of a pixel array. The pixel array is divided into multiple zones, and a sharpness score for each zone is calculated after an analysis of adjacent areas within each zone. In a red/green/blue (RGB) pixel array, the intensity of each pixel may be calculated by converting the RGB values into a hue/saturation/value (HSV) system, where the value V variable is used to represent intensity. In this way, intensity values for each pixel may be calculated, and differences in intensities of multiple pixels may be calculated. These differences in intensities represent sharpness scores. Most cameras, including digital cameras, have an automatic focus feature in which scenes viewed through the camera can be focused automatically. When an image is out of focus, the sharpness score for the image decreases; the sharpness score is used to implement the auto-focus functionality (by changing the distance between lens and sensor based on the sharpness score). 
         [0012]    Another type of statistics data that is readily available within most imagers is luma. Luma represents the brightness in an image and may also be referred to as a brightness value. Each pixel of a pixel array has an associated brightness value, which is expressed as a combination of multi-bit digital data values (e.g., one each for red, green and blue). If eight bits are used to describe each of red, green and blue, then the value of each may range from 0 to 255. Therefore, the brightness value may also range from 0 to 255 for such an RGB, 8-bit imager. 
         [0013]      FIG. 1  illustrates two image frames (Frame A and Frame B), each comprising a plurality of windows  30  arranged in a 4×4 grid. A conventional auto-focus statistics engine (e.g., statistics engine  122 ,  FIG. 3 ) operates with a set of windows  30  arranged in an N×M grid and provides a sharpness score and an average brightness (luma) for each window  30 . Each window  30  comprises a plurality of pixels. Disclosed embodiments implement motion detection using these data (sharpness score and brightness) by calculating, for each window, changes in the statistics data received for two different frames. The plurality of windows  30  of Frame A corresponds with the plurality of windows  30  of Frame B, such that window A 1  corresponds with window B 1 , window A 2  corresponds with window B 2 , etc. 
         [0014]    In conventional auto-focus operations, the sharpness score and brightness are calculated for each window  30 . These values are the statistics data for the window  30  and are used in the disclosed embodiments for motion detection. Two thresholds are stored in memory for use in motion detection: (1) a threshold TH_ratio for an acceptable amount of change between frames A, B for a single window (e.g., between A 1  and B 1 ) and (2) a threshold TH_motion_cnt for an acceptable amount of windows  30  with change above the threshold TH_ratio. The statistics data for two frames A, B of each window  30  are obtained and the change between frames A, B is determined for each window  30 . Then, a determination of how many windows  30  have statistics values that have changed more than the TH_ratio threshold is made. If the determined number of windows having statistics that have changed more than the TH_ratio threshold is larger than the TH_motion_cnt threshold, then motion is detected. 
         [0015]    One disclosed embodiment is now described in more detail with reference to  FIGS. 1 and 2 .  FIG. 2  is a flowchart illustrating the steps of a method  205  of using statistics data for motion detection in accordance with disclosed embodiments. 
         [0016]    At step  210 , the array of statistics data from Frame A, the array of statistics data from Frame B, the windows data change threshold TH_ratio and the maximum number of changed windows threshold TH_motion_cnt are obtained. In this embodiment, the statistics data includes sharpness scores and/or brightness for each window  30  of each frame A, B. These input parameters are previously stored within or determined by the image flow processor  110  ( FIG. 3 ). At step  220 , motion counter motion_cnt is initialized; e.g., it is set to zero. At step  230 , the relative difference value for the first window, D 1 , is calculated as shown in Equation (1): 
         [0000]        D   i = |A     i     −B     i     | / max(A     i     ,B     i)   ,   (1) 
         [0000]    where D i  is the difference value for window i, A i  is the value of the statistics data for window i of frame A, B i  is the value of the statistics data for window i of frame B, and max(A i , B i ) is the higher of the statistics data values for window i of frames A and B. 
         [0017]    It should be noted that disclosed embodiments may operate using only the change in sharpness score or only the change in average brightness (luma) or the results from these two statistics values could be combined in making a decision about motion. If the combination of the two statistics values is being used to make a decision about motion, then D i  is determined as the sum of the absolute values for each of sharpness score and average brightness divided by the sum of the max value of each of sharpness score and average brightness. 
         [0018]    At step  240 , if D 1  is greater than the windows data change threshold TH_ratio, the counter motion_cnt is incremented at step  250 , as shown in Equation (2): 
         [0000]      If  D   i   &gt;TH _ratio, then motion_cnt=motion_cnt+1   (2) 
         [0000]    otherwise, the counter motion_cnt does not change. 
         [0019]    At step  260  a determination is made as to whether window i is the last window  30  of the frame. If window i is not the last window (e.g., i is not equal to Nwin), then i is incremented at step  270  and the next difference value (D i ) is calculated at step  230 , again using Equation (1). The counter motion_cnt is incremented if necessary at step  250 , using Equation (2), and i is again incremented until D i  has been calculated for all windows in the frame. Once a difference value has been calculated for each window  30 , a determination of whether or not there was motion is made at step  280 . This determination is made in accordance with Equation (3): 
         [0000]      If motion_cnt&gt;TH_motion_cnt, then M=1, else M=0;   (3) 
         [0000]    where M=1 (true) corresponds to detected motion and M=0 (false) corresponds to no detected motion. 
         [0020]    Disclosed embodiments use statistics data that is already collected within the imager for use in other image processing functions, such as, e.g. focus. The method of the disclosed embodiments provides an additional benefit in that the costs for adding a motion detection feature are low since the method uses information for motion detection that is already available within the image flow processor  110 . 
         [0021]    In an additional disclosed embodiment, the motion detection may be weighted such that changes in a certain portion of the frame are more important than changes in a different portion of the frame, by using different weights for different windows. For example, if it is desired to make the motion detection in a central part of an image more sensitive than in the peripheral portions, windows at the center of the frame will be given a higher weight than windows at the periphery. A weighted motion score can be calculated in accordance with Equation (4): 
         [0000]    
       
         
           
             
               
                 
                   
                     Motion_score 
                     = 
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         
                           i 
                           = 
                           Nwin 
                         
                       
                        
                       
                         
                           Weight 
                           i 
                         
                         * 
                         
                           Motion 
                           i 
                         
                       
                     
                   
                   ; 
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where Motion_score represents the total amount of motion in the image (and is similar to motion_cnt), Motion i  is a variable representing the amount of motion sensed in a particular window i of the imager array, Weight i  is the weight given to window i, and N win  is the total number of windows. Motion i  is set to be equal to one if the change in the statistics data is more than threshold TH_ratio for window i and is equal to zero if it is not. 
         [0022]    Once the Motion_score is calculated, it is compared with predefined threshold TH_motion_cnt. If Motion_score is greater than TH_motion_cnt, motion is detected. If Motion_score is less than TH_motion_cnt, motion is not detected. 
         [0023]      FIG. 3  illustrates a block diagram of a system-on-a-chip (SOC) imager constructed in accordance with disclosed embodiments. The system-on-a-chip may use any type of imager technology, CCD, CMOS, etc. The system-on-a-chip also includes the motion detection capabilities of the disclosed embodiments. 
         [0024]    The imager  100  comprises a sensor core  200  that communicates with an image flow processor  110  that is connected to an output interface  130 . Sensor core  200  may include a pixel array. A phase-locked loop (PLL)  244  is used as a clock for the sensor core  200 . The image flow processor  110 , which is responsible for image and color processing, includes interpolation line buffers  112 , decimator line buffers  114 , and a color processing pipeline  120 . The color processing pipeline  120  includes, among other things, a statistics engine  122 . One of the functions of the image flow processor  110  is the performance of motion detection in accordance with disclosed embodiments. The sharpness scores and brightness values for the individual windows ( FIG. 1 ) are determined by the statistics engine  122 . 
         [0025]    The output interface  130  includes an output first-in-first-out (FIFO) parallel buffer  132  and a serial Mobile Industry Processing Interface (MIPI) output  134 , particularly where the imager  100  is used in a camera in a mobile telephone environment. The user can select either a serial output or a parallel output by setting registers in a configuration register within the imager  100  chip. An internal bus  140  connects read only memory (ROM)  142 , a microcontroller  144 , and a static random access memory (SRAM)  146  to the sensor core  200 , image flow processor  110 , and output interface  130 . 
         [0026]    Disclosed embodiments may be implemented as part of an image flow processor  110 , by a processor executing a program, by hardware circuits with a processing pipeline, or by a combination of both, etc. The method  205  may be implemented as computer instructions and stored on a computer readable storage medium. 
         [0027]    Disclosed embodiments may be implemented as part of a camera such as e.g., a digital still or video camera, or other image acquisition system.  FIG. 4  illustrates a processor system as part of, for example, a digital still or video camera system  800  employing a system-on-a-chip imager  100  as illustrated in  FIG. 3 , which imager  100  provides for motion detection as described above. The processing system includes a processor  805  (shown as a CPU) which implements system, e.g. camera  800 , functions and also controls image flow through the system. The processor  805  is coupled with other elements of the system, including random access memory  820 , removable memory  825  such as a flash or disc memory, one or more input/out devices  810  for entering data or displaying data and/or images and imager  100  through bus  815  which may be one or more busses or bridges linking the processor system components. The imager  100  receives light corresponding to a captured image through lens  840  when a shutter release button  842  is depressed. 
         [0028]    While described embodiments have been described in detail, it should be readily understood that the embodiments are not limited to those disclosed. Rather the embodiments can be modified to incorporate any number of variations, alterations, substitutions or equivalent arrangements not heretofore described.

Technology Category: h