Patent Abstract:
The present invention implements an automated blank time function, which calculates a horizontal and/or a vertical blank time for a desired integration time and window size. Input data and control signals for predetermined integration time and window size are provided to a register interface, which generates configuration signals. The configuration signals are applied to machines, which generate reset control signals and read control signals from the configuration signals for controlling the imager.

Full Description:
[0001]    This application claims the benefit of U.S. Provisional Patent Application Serial No. 60/290,964 filed on May 16, 2001. 
     
    
     
       FIELD OF THE INVENTION  
         [0002]    The present invention relates to electronic imaging devices, such as CCD and CMOS imagers, and is directed more particularly to the user-controlled interface of such devices.  
         BACKGROUND OF THE INVENTION  
         [0003]    In any type of image capture system that uses the rolling shutter technique to capture video images there is a fundamental limitation on the maximum exposure time. In rolling shutter image acquisition, the row Reset and Read sequences need to increment through the frame at the same rate in order to preserve a constant integration/exposure time on a row-by-row basis. If the integration time is larger than the frame readout time, the row Reset sequence can wrap around to the first row before the Read sequence for that row has begun, thereby destroying the proper image information.  
           [0004]    [0004]FIG. 1 is a simplified illustration of the Reset and Read sequence in rolling shutter, where the horizontal axis represents time, and the vertical axis represents the row number.  
           [0005]    If the Read sequence is moved to the left on the time axis, integration time is decreased, and increased if moved to the right. However, if the Read sequence is moved too far along to the right of the time axis, such that T int &gt;T frame , the Reset sequence repeats prior to reading, destroying the original frame information. This yields an actual integration time of T int, act =T int −T frame  as shown in FIG. 2. Also, if the row or column frame size is decreased, T frame  and the maximum integration time will also be decreased. As the column window size, illustrated in FIG. 6, is increased, the slope ΔRow Number/ΔTime decreases, which in turn increases the frame time T frame . Likewise, decreasing the column window size increases the slope, which decreases the frame time. Therefore, the smaller the window size the smaller the maximum available integration time.  
           [0006]    A non-zero vertical blank time T vblank , consists of a time-delay inserted between the end of the current reset sequence and the initiation of a new reset sequence. This will produce a delay between consecutive frames as shown in FIG. 3.  
           [0007]    A non-zero horizontal blank time T hblank , consists of a time-delay inserted between the end of the reset sequence for a particular row number, and the initiation of the reset sequence in the next subsequent row. This will decrease the slope- ΔRow Number/ΔTime as shown in FIG. 4. Increasing either T hblank  or T vblank  will increase the frame time T frame  thereby increasing the maximum integration time T int .  
           [0008]    Maximizing the integration time T int  for small window sizes can be quite complicated, since many calculations may be required. However, one solution is to capture one frame at a time or just set the horizontal T hblank  and vertical blank T vblank  times to large values. Since many applications require high frame rates, both these solutions are undesirable and not very flexible.  
           [0009]    There are a few simple solutions already known in the art, as can be demonstrated from the data sheet for the LM9627 Color CMOS Image Sensor VGA 30 FPS imager, produced by National Semiconductor, which is incorporated herein by reference. In this system, the user must select values for the various time delays and variables which affect the operation of the imager. The simplified description of this method is to insert a vertical blank time T vblank  equal to the integration time T int  before the Reset and Read sequence repeats. This is illustrated in FIG. 5. Because a blank time T vblank  equal to the integration time T int  is inserted between frames, the problem of the Reset “wrap-around” prior to the row Read is completely eliminated and the imaging cycle will get the proper integration time no matter what the window size is.  
           [0010]    Usually, the user determines the integration time that is desired and can calculate the additional values, or use a look up table to find the appropriate values and then enter them into the imager&#39;s memory.  
           [0011]    The equation used to calculate these values is generally understood to be:  
             T   int &lt;[(Δ Col   *T   col )+ T   row     —     blank   +T   hblank ]*Δ Row   +T   vblank    
           [0012]    T col  is the period at which a column or columns are read out T row     —     blank  is defined generally here as the delay time from sampling and latency in the imager, it is related to the time that it would take to do a single read, and other similar delay times. The Δ Col  value means the number of columns that a particular window size will use, as the window size varies the Δ Col  value will change accordingly. The Δ Row  value is the number of rows that a particular window size will use; as the window size varies the Δ Row  value will change accordingly. The horizontal blank time, T hblank , and the vertical blank time, T vblank , are defined values that are entered into the imager and will vary depending on the desired settings the user wishes to accomplish, both in terms of integration time T int , and other settings. In order for a user to determine an appropriate integration time for a particular window size (Δ Col  and Δ Row ) the T vblank , and T hblank  times have to be adjusted in order to create the appropriate environment for the image to be captured, optimization is often difficult. As can be seen from the above equation, four variables must be adjusted and optimized for changing window sizes.  
           [0013]    These methods are both time consuming and require an intimate knowledge of the imager&#39;s functionality. It is apparent from this that an automatic blanking time mode needs to be developed which will allow the user and controlling software to have an simple way to adjust the blanking time or the frame readout time, without performing any calculation or manually entering a number of controlling variables.  
           [0014]    Therefore, there is a need for apparatus and a method of automatically setting vertical and horizontal blanking times for a selected integration time and window size.  
         SUMMARY OF THE INVENTION  
         [0015]    The invention is directed to a method and apparatus for implementing an automated blank time function, which calculates a horizontal and/or avertical blank time for a desired integration time and window size. The method comprises providing input data and control signals for predetermined integration time and window size for autoblank mode, generating configuration signals from the input signals, and generating reset control signals and read control signals from the configuration signals for controlling the imager. Autoblank mode is used to represent automatic blank time mode. The method may further include producing reset counter values to control the reset cycle and read counter values to control the read cycle, as well as a signal for synchronizing the read control signals and the reset control signals.  
           [0016]    The apparatus comprises a register interface for receiving input data and control signals for a predetermined integration time and window size and for generating configuration signals from the input signals. The apparatus further includes a device for generating reset control signals and read control signals from the configuration signals for controlling the imager.  
           [0017]    In accordance with an aspect of the invention, the device for generating reset control signals and read control signals comprises finite state machines wherein a reset finite state machine produces reset control signals in response to configuration signals, and a read finite state machine produces read control signals in response to the configuration signals. The reset finite state machine may produce the reset control signals, reset counter values to control the reset cycle, whereas the read finite state machine may produce the read control signals and counter values to control the read cycle. Further a signal is generated to synchronize the reset and the read finite state machines.  
           [0018]    In accordance with another aspect of the invention, the method for controlling an imager by automatically setting vertical and horizontal blank times for a selected integration time and window size is stored on a computer readable memory element as instructions or statements for use in its execution in a computer.  
           [0019]    With regard to another aspect of the invention,  
             T   int &lt;[(Δ Col   * T   Col )+T row     —     blank   +T   hblank ]*Δ Row   +T   vblank ,  
           [0020]    where:  
           [0021]    T int  is the integration time,  
           [0022]    T col  is the period at which a column or columns are read out,  
           [0023]    T row     —     blank  is the delay time from sampling and latency for a single read,  
           [0024]    T hblank  is the horizontal blank time,  
           [0025]    T vblank  is the vertical blank time,  
           [0026]    Δ Col  is the number of columns for a predetermined window size, and  
           [0027]    Δ Row  is the number of rows for a predetermined window size;  
           [0028]    and frame time may be minimized by setting:  
           [0029]    an optimal T vblank  for a fixed T hblank , or  
           [0030]    an optimal T hblank  for a fixed T vblank , or  
           [0031]    optimal T vblank  and T hblank .  
           [0032]    In accordance with another aspect of this invention, T vblank =T int .  
           [0033]    In accordance with a further aspect of this invention, T int  is smaller then but very nearly equal to {[(Δ Col *T Col )+T row     —     blank +T hblank ]*Δ Row +T vblank } to minimize frame time.  
           [0034]    Other aspects and advantages of the invention, as well as the structure and operation of various embodiments of the invention, will become apparent to those ordinarily skilled in the art upon review of the following description of the invention in conjunction with the accompanying drawings. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0035]    The invention will be described with reference to the accompanying drawings, wherein:  
         [0036]    [0036]FIG. 1 is a typical timing diagram illustrating a rolling shutter in operation;  
         [0037]    [0037]FIG. 2 is a typical timing diagram illustrating a rolling shutter where the integration time is greater than the frame time;  
         [0038]    [0038]FIG. 3 is a typical timing diagram illustrating a rolling shutter with a nonzero vertical blank time;  
         [0039]    [0039]FIG. 4 is a typical timing diagram illustrating a rolling shutter with a non-zero horizontal blank time;  
         [0040]    [0040]FIG. 5 is a typical timing diagram illustrating a rolling shutter with a vertical blank time equal to the integration time;  
         [0041]    [0041]FIG. 6 is a typical timing diagram illustrating the operation of a rolling shutter at the row activation level;  
         [0042]    [0042]FIG. 7 is a block diagram illustrating a preferred implementation of the present invention;  
         [0043]    [0043]FIG. 8 is a block diagram illustrating the flow chart for the Reset Finite State Machine of a preferred implementation of the present invention; and  
         [0044]    [0044]FIG. 9 is a block diagram illustrating the flow chart for the Read Finite State Machine of a preferred implementation of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0045]    In the preferred embodiment of the present invention, a register is placed on an imager chip and is programmed with values that represent the activation of an automatic blank mode, as well as some register values, which would embody the exposure control.  
         [0046]    Referring to FIG. 7, a preferred embodiment of the current invention is illustrated using a series of decision-making blocks, that could be implemented on or off chip, in a electronic circuit or programmed into a computer that interfaced with the decoding circuitry. The manner in which these decision making blocks are programmed will not be described in detail since one skilled in the art could create blocks which would perform the tasks that are associated with each block in a manner that is well known in the art. This invention is shown to be implemented by having the user or interface computer set configuration signal bit or bits  703  that relate to the desired setting for the imager, internal or external to the imaging chip, by means of a register interface  702 . The register  702  receives interface signals  701  such as a series of register addresses, data and control signals  701 . These configuration signals  703  are then inputted to control circuitry including a system of Finite State Machines (FSM) and supporting circuitry, so that the row RESET or access cycle waits a full integration period to wrap around to the beginning of the frame. The Reset FSM  704  controls the RESET sequence and synchronizes the Read FSM  705  and the Read FSM  705  controls the row READ access signals as well as column sampling signals. Specifically, as illustrated in FIG. 7, the configuration signals  703  are coupled to both the Reset FSM  704  as well as the Read FSM  705 . The Reset FSM  704  outputs a set of Reset Control Signals  706  which control the imaging array, a Read FSM  705  synchronizing signal  707  which is operatively coupled to the Read FSM  705 , as Reset Counter Values  709 , which keep track of the reset cycle. The Read FSM  705  is synchronized with the Reset FSM  704  through the Read FSM Synchronizing signal  707 . The Read FSM  705  outputs Read Control Signals  708  to the imaging array, as well as Read Counter Values  710 , which keep track of the row read cycle and the column sampling. The Reset Control Signals  706  consist of a sequence of row reset signals as needed. The Read Control signals  708  are simply the controlling signals, which activate both the column and row read sequences.  
         [0047]    The preferred algorithm to program into either a combination of the Reset FSM  704 , and Read FSM  705 , or a single FSM which performs the tasks assigned to the Reset and Read FSMs, are illustrated in FIG. 8 and FIG. 9 respectively.  
         [0048]    The Reset Finite State Machine  704  algorithm is illustrated in FIG. 8. The system is initialized  801 , the current row number (row_num) is set to a system defined starting row number, this is dependent on the window size that is defined by the user, and the window of pixels that is being used in the array. The number of clocks cycles between consecutive row resets (N) is set to 0, the number of clock cycles for the integration period or the number of cycles between the end of a given row&#39;s reset and the beginning of that row&#39;s read (Q) is set to zero and the reset enable pulse (rst_en) is set to a low or an inactive value. The system defined start of the reset pulse (N RST     —     START ) and end of the reset pulse (N RST     —     STOP ) are set to their defined values. The length of time spend on a particular row is defined by T Row , this is defined by the user/system. The integration time T int  between reset and readout is also defined.  
         [0049]    The system then holds the current row number valid, and increments the N value  802 . Then the algorithm evaluates if a reset pulse should be initiated  804  by comparing N, to the N RST     —     START  and N RST     —     STOP . If N is larger than N RST     —     START , but smaller than N RST     —     STOP , it finds the condition true and places rst_en equal to a high or active value  803  and advances to block  806 , if false then it places rst_en at a low value  805  and cycles through to the next question  806 .  
         [0050]    In block  806 , the algorithm evaluates whether it has reached the end of the clock cycles that should be spent on a particular row. If false then it proceeds to increase the N count  802 . If true then it evaluates whether it has reached the end of a particular frame  807 , frame generally being understood in the art as the number of rows, defined by the row window size, that will be read by the system.  
         [0051]    If the system finds that the frame has ended, then if examines the autoblank setting, to determine if autoblank mode is activated  809 . If true, then it increases the integration count, by incrementing Q  811 . If false, then the system proceeds into the next frame  810  by reinitializing the current variables to their initial settings as in  801 .  
         [0052]    After the integration count is incremented  811 , the system compares the current Q value to the system defined T int  value  812 . If it has then it initiates the new frame by moving to block  810 . If not, then it proceeds to the  811  block and increments the Q value.  
         [0053]    This algorithm works in conjunction with the Read Finite State Machine  705  as defined in FIG. 9, as long as the rolling shutter is active. A synchronization signal is sent between the Reset Finite State Machine  704  and the Read Finite State Machine  705  in order to ensure that both start at the same time, the variables in each algorithm are local variables distinct to each machine.  
         [0054]    [0054]FIG. 9 shows the Read Finite State Machine  705  algorithm in more detail. There is an initialization block  901 , which defines the current row number (row_num) as being set to a system defined starting row number, this is dependent on the window size that is defined by the user, and the window of pixels that is being used in the array. The number of clocks cycles between valid row addresses (N) is set to 0, the number of clock cycles between the end of the valid row address and the integration of the same row (Q) is set to 0 and the read enable pulse (read_enable) is set to low or an inactive value. The system-defined start of the read pulse (N READ     —     START ) and end of the read pulse (N READ     —     STOP ) are set to their defined values. The length of time spend on a particular row is defined by T Row , this is defined by the user/system. The integration time between reset and readout T int  is also defined.  
         [0055]    The algorithm is set to monitor the integration count  902 . It does this by incrementing the integration count  902  as it evaluates the count as compared to the defined integration time T int    903 . These steps offset the row reset and the row read cycles producing the desired integration time. The synchronization is performed by starting  801  on FIG. 8 at the same time as  901  in FIG. 9. This can be achieved via a synchronization signal.  
         [0056]    When the integration time T int  is reached, the algorithm begins to proceed through the read of the pixels. The row access time is monitored  904 , and then the algorithm checks to see if a read enable pulse should be generated by comparing the current clock cycle (N) to the predefined start and stop times for the Read Enable pulse  912 . If this is true, then the algorithm places the read_enable signal high  913 , if false then the read_enable is kept low  914 .  
         [0057]    Then the algorithm compares the N value to the end of row value  905 . If false then it checks to it returns to the Row Width Count level  904 . If true, then it checks to see if the frame has ended by comparing the row number to the row stop number  907 , if false then it increments the row number  906 , and returns to the End of Row level  905 . If true, then it checks the autoblank register  908  to see if autoblank mode has been activated. If false then it moves to the next frame  909 , if true then it increases the integration count  910 , until it reaches the predefined integration time T int    911 , at which point it moves to the next frame  909 .  
         [0058]    The method presented above, namely inserting a vertical blank time equal to the integration time, will not produce a minimal frame time. Therefore, a more optimal method for implementing an automated blank time function would be to calculate the minimum horizontal or vertical blank time for a desired integration time or window size. This will solve the Reset “wrap around” effect, as well as providing a much faster frame time.  
         [0059]    While the invention has been described according to what is presently considered to be the most practical and preferred embodiments, it must be understood that the invention is not limited to the disclosed embodiments. Those ordinarily skilled in the art will understand that various modifications and equivalent structures and functions may be made without departing from the spirit and scope of the invention as defined in the claims. Therefore, the invention as defined in the claims must be accorded the broadest possible interpretation so as to encompass all such modifications and equivalent structures and functions.

Technology Classification (CPC): 7