Method for calibrating a multi-channel imaging system

In a high speed video capture and display system including an N.sub.channels channel sensor for producing N.sub.channels parallel analog video signals, N.sub.channels signal processing channels for processing the analog video signals, each of the N.sub.channels signal processing channels including an analog-to-digital converter (ADC) having gain and offset which are controlled by controlling the common mode and differential voltages across each ADC ladder network; and a central processing unit for controlling the gain and offset by means of digital control signals; the method of calibrating the system to produce a uniform image comprising the steps of. PA1 a) illuminating the sensor with a broadband and spatially uniform light source to determine L.sub.max ; PA1 b) initially setting the common mode and differential voltages of each ADC to nominal settings; PA1 c) illuminating the sensor with a spatially uniform broadband illumination N.sub.illum evenly spaced illumination levels; PA1 d) for each of the illumination levels collect for each of the N.sub.channels, N.sub.samples of spatially distributed pixels; and PA1 e) for each channel find the gain m.sub.j and the offset b.sub.j ; PA1 f) calculating the desired corrections in gain k.sub.mj and offset .DELTA..sub.bj by finding the specific channel whose gain m.sub.l represents the median gain value among the set of gains m.sub.j for all channels and for all channels computing k.sub.mj and .DELTA..sub.bj ; PA1 g) calculating and applying the desired correction in the digital common mode and span controls over all channels; and PA1 h) repeating steps b-g until convergence is reached.

FIELD OF INVENTION 
This invention relates in general to high frame rate imaging systems and 
relates more particularly to high frame rate video capture and display 
system using a multi-channel sensor in which gain and offset of each 
channel is controlled to optimize system response. 
BACKGROUND OF THE INVENTION 
There are many applications which require the recording of hundreds or 
thousands of images per second. Such applications include automobile crash 
testing, missile range instrumentation and fault detection in high speed 
machinery. Typical high frame rate video cameras employ a multi-channel 
sensor to minimize channel bandwidth and data rate. (See, e.g., U.S. Pat. 
No. 4,322,752, issued Mar. 30, 1982, inventor Bixby). One of the 
difficulties to be dealt with in using such a sensor is the requirement 
that the transfer characteristic, relating output signal amplitude to 
sensor illumination, be closely matched across all channels. Under uniform 
sensor illumination, matching errors larger than approximately 1% can be 
readily detected by the human eye. If the transfer function is linear, 
uniformity can be achieved by adjusting individual channel gain and 
offset. 
In a known color fast frame imaging system, a sixteen channel sensor is 
employed, wherein the sensor is read out as sixteen parallel channels of 
analog video signals. The system electronics supports the sensor with 
sixteen independent channels of analog and digital processing including an 
analog-to-digital converter (ADC). Two controls control the respective 
channels gain and offset by adjusting the top and bottom ladder electrical 
potentials of the respective channel's ADC. In the known procedure, each 
channel's offset control was adjusted while the sensor was capped (kept in 
darkness) to balance each channel's average video level to the system 
black setup value. Subsequently, each channel's gain control was adjusted 
to balance each channel's average video value to the global average video 
level in the presence of a broad-band flat field light source. 
A drawback to this procedure is that it assumes a linear response, but 
indirect evidence has shown that a non-linear response may be occurring. 
When left uncompensated for, this lack of linearity in the sensor response 
has implications beyond a non-linear rendering of intensity values. The 
greatest impact of the non-linearity is in regards to its effects on the 
color reproduction. Any departure from the ideal response immediately 
begins to color the rendering of neutrals in the scene for typical 
lighting situations. This is because such departures can be looked at as 
an offset from the ideal response. This offset is multiplied by the white 
balance gains of which there are three different gains for each of the 
three color components. Thus, this singular offset now behaves as three 
different offsets for each of the color planes. When these offsets are 
added to the rendering of a neutral scale, the result is a false 
coloration of the scene. Furthermore, the very nature of the non-linearity 
has been seen to vary from channel to channel. Thus, the false coloration 
of neutrals can appear to vary abruptly across channel boundaries. 
SUMMARY OF THE INVENTION 
According to the present invention, there is provided a solution to the 
needs of the prior art. 
According to one aspect of the present invention, there is provided a high 
speed video capture and display system including an N.sub.channels sensor 
for producing N.sub.channels parallel analog video signals, N.sub.channels 
signal processing channels for processing the analog video signals, each 
of the N.sub.channels signal processing channels including an 
analog-to-digital converter (ADC) having gain and offset which are 
controlled by controlling the common mode and differential voltages across 
each ADC ladder network; and a central processing unit for controlling the 
gain and offset by means of digital control signals; the method of 
calibrating the system to produce a uniform image comprising the steps of: 
a) illuminating the sensor with a broadband and spatially uniform light 
source to determine L.sub.max ; 
b) initially setting the common mode and differential voltages of each ADC 
to nominal settings; 
c) illuminating the sensor with a spatially uniform broadband illumination 
at N.sub.illum evenly spaced illumination levels; 
d) for each of the illumination levels collect for each of the 
N.sub.channels, N.sub.samples of spatially distributed pixels; and 
e) for each channel find the gain m.sub.l and the offset b.sub.j ; 
f) calculating the desired corrections in gain k.sub.mj and offset 
.DELTA..sub.bj by finding the specific channel whose gain m.sub.l 
represents the median gain value among the set of gains m.sub.j for all 
channels and for all channels computing k.sub.mj and .DELTA..sub.bj ; 
g) calculating and applying the desired correction in the digital common 
mode and span controls over all channels; and 
h) repeating steps b-g until convergence is reached. 
ADVANTAGEOUS EFFECT OF THE INVENTION 
The present invention has the following advantages. 
1. Existing gain and offset controls are optimized to achieve a balanced 
neutral scale for the linear portion of the response. 
2. Rendering errors are redistributed towards the shadows rather than being 
distributed throughout the response.

DETAILED DESCRIPTION OF THE INVENTION 
Referring now to FIG. 1, there is shown a block diagram of an imaging 
system incorporating the present invention. As shown, a multichannel CCD 
image sensor 10 receives illumination from flat field light source 12. 
Sensor 10 produces a multichannel image signal which is processed by 
multichannel signal processor 14. The processed signal is stored and 
displayed on display 16. A central processing unit (CPU) 18 transmits and 
receives control and data signals to components 10,12,14,16 over bus 20. A 
photodiode 22 receives light from light source 12 and is controlled by CPU 
18. 
Sensor 10 is a multichannel CCD sensor having a matrix of photosites 
divided into channels numbered 1 to N.sub.channels channels. Sensor 10 can 
be a black and white sensor, wherein each photosite is read out as a black 
and white pixel. Sensor 10 can also be a color CCD sensor, as shown in 
FIG. 4, having a color filter array, wherein each pixel is read out as a 
single color R, G, B corresponding to a color filter array filter element. 
As shown in FIG. 4, the color filter array is formed of repeating GR/BG 
arrays. The single color pixels are transformed into RGB tricolor pixels 
through spatial transfer processing in signal processor 14. As shown, each 
channel numbered 1 to N.sub.channels includes five rows of photosites. It 
will be understood that any other color matrix and other colors such as 
cyan, magenta, and yellow can be used. 
Referring to FIG. 2, there is shown a block diagram of the signal processor 
of FIG. 1. As shown, signal processor 14 includes among other components 
signal processing circuits 30 for channels numbered 1 to N.sub.channels 
for processing N.sub.channels image signals from sensor 10. Each circuit 
30 includes an ADC 32 having a lower ADC ladder potential input 34 and an 
upper ADC ladder potential input 36. The potentials at inputs 34 and 36 
are controlled respectively by CPU 20 through digital control signals sent 
over serial I/O daisy chain 38 to SPAN DAC and COMMON MODE DAC 42 which 
convert the digital control signals to analog-electrical potentials. Adder 
44 receives signals from DACs 40 and 42. The digital video from each ADC 
32 is stored in Frame Store 44. 
FIG. 3 is a graphical diagram of light intensity versus image pixel value. 
Depicted are exemplary light source illumination values (including 
L.sub.max) used in the calibration method described below. 
The calibration method of the present invention will now be described in 
detail having reference to FIGS. 1-4. 
Goal of the channel balancing method 
The goal of the channel balancing method is to ensure the proper 
reproduction of color when the sensor is operated in an area where its 
response curve is linear. To accomplish this goal, the gains must be 
adjusted such that this linear portion of the response curve for the 
channel in question has an equivalent slope to that of the remaining 
channels. Furthermore, the offsets must be adjusted such that the linear 
portion of the curve, when extrapolated out to the condition of no 
illumination, is equivalent to the system black setup level for all 
channels of the sensor. 
The basic channel balancing method and derivation 
The basic idea: 
Attempt to balance the gain and offsets of all channels such that their 
"pseudo-linear" portions of their response curves have a 
least-means-squared fit to the function: 
EQU y=mx+b 
x.epsilon.[0 . . . 1] 
where: 
y.epsilon.[0 . . . 255] 
b=black.sub.-- setup 
The points sampled: 
Select N.sub.illum evenly spaced points 
##EQU1## 
for i.epsilon.[1 . . . N.sub.illum ]. 
Each of these xi corresponds in the experiment to a spatially uniform 
broadband illumination of i*L.sub.max /N.sub.illum for i.epsilon.[1 . . . 
N.sub.illum ] (L.sub.max must be suitably chosen such that there is no 
clipping of the video when the algorithm runs.) 
The data collected: 
From these x.sub.i, collect from each of the N.sub.channels channels 
N.sub.samples spatially distributed green samples of the resultant pixel 
values y.sub.ijk for i.epsilon.[1 . . . N.sub.illum ], j.epsilon.[1 . . . 
N.sub.channels ], and k.epsilon.[1 . . . N.sub.samples ]. 
The data computed: 
For each channel j.epsilon.[1 . . . N.sub.channels ], find m.sub.j and 
b.sub.j for the least means squared fit to a linear curve: 
Solve for m.sub.j and b.sub.j for each channel j.epsilon.[1 . . . 
N.sub.channels ] over i.epsilon.[1 . . . N.sub.illum ] illumination levels 
and k.epsilon.[1 . . . N.sub.samples ] samples at each illumination level, 
with the digital common mode and span controls set to nominal (optional, 
but recommended for the first iteration of the algorithm; however, 
successive iterations must not set these controls to nominal) via the 
following system of linear equations: 
##EQU2## 
whose solution is: 
##EQU3## 
Choose a specific channel as a reference gain channel: 
Over all channels j.epsilon.[1 . . . N.sub.channels ], select a specific 
channel l whose gain term m.sub.l represents the median value amongst the 
set of gains m.sub.j, where j.epsilon.[1 . . . N.sub.channels ]. 
Calculate the desired corrections k.sub.mj and .DELTA..sub.bj to each 
channel's gain m.sub.j and offset b.sub.j terms: 
Over all channels j.epsilon.[1 . . . N.sub.channels ], compute 
##EQU4## 
[Note: The 
##EQU5## 
correction term may seem odd here. It comes about because, unlike the 
b.sub.j term, the b term represents a value relating to conditions not as 
they exist now, but after this algorithm is run. Therefore, one must take 
into account the very likely possibility that the span will change in 
addition to the offset, and as a change in span alone will cause any 
non-zero offset to move as well, this must be corrected for.] 
Determine the changes necessary in the bottom A/D ladder potential 
##EQU6## 
and its top to bottom span 
##EQU7## 
to effect the above corrections, and ultimately to the changes in the 
control values 
##EQU8## 
and 
##EQU9## 
Over all channels j.epsilon.[1 . . .N.sub.channels ], we note that 
##EQU10## 
Note that the first of the two previous equations will change as a 
function of the implementation details. This equation relates a change of 
digital value in the common mode control, .DELTA..sub.bj, to a change in 
the bottom ladder potential, 
##EQU11## 
by a small signal gain in units of 
##EQU12## 
and 
##EQU13## 
represent the current digital control values for gain and offset, 
respectively) 
##EQU14## 
Note that the previous two equations will also change as a function of the 
implementation details. The first equation relates the digital common mode 
and span control values to the voltage of the top A/D ladder potential, 
and the second equation relates the digital common mode control to the 
voltage of the bottom A/D ladder potential. As a consequence, the 
following derivations will also vary in its numerical details as a 
function of the particular implementation. 
##EQU15## 
We now are left with the quantities 
##EQU16## 
and 
##EQU17## 
These results indicate the amount by which respectively the digital common 
mode 
##EQU18## 
and span 
##EQU19## 
controls should be adjusted to effect the desired corrections, as 
demonstrated below: 
##EQU20## 
Other implementation details Selection of N.sub.illum 
The algorithm requires that N.sub.illum, the number of illumination levels 
to be used in the calibration, be known at run time. This parameter must 
be determined via experiment on a broad number of systems and sensors. 
There are two competing influences on the selection of this number. A 
higher value of N.sub.illum allows for a greater sampling of the sensor 
response curve, but since this number also determines the total number of 
equally spaced illumination values, it also has the effect of lowering the 
smallest illumination level sampled. Thus, with increasing N.sub.illum, it 
increases the number of samples taken in the area where we suspect a 
non-linear response from the sensor. Lowering it, however, decreases the 
total number of samples, resulting in less noise averaging. We have found 
by experiment that for our system, N.sub.illum =10 results in the best 
overall results. 
Iterating the algorithm 
The algorithm is based on a feed-forward technique. That is, given a 
measured error in the output response, it computes the correction 
necessary in the control values to null the error metric based on ideal 
circuit response and component values. Thus, in any real implementation, 
there is always a finite amount of error in the final correction. The 
algorithm deals with that situation by iteration. That is, the first 
iteration takes N.sub.illum samples of the sensor response and computes 
and applies the necessary corrections. The next iteration takes another 
N.sub.illum samples of the sensor response with the controls set by the 
previous iteration and computes and applies a smaller, second set of 
corrections. The only difference in the second and future iterations is 
that the digital common mode and span control values are retained from the 
previous iteration. These iterations can be done as long as is necessary 
to achieve the desired result. In our system testing, we have found that 
two iterations is sufficient to guarantee a satisfactory result. 
A step by step example of operating the algorithm 
Note: the following operations applies in all its detail to our particular 
implementation. As was described in the dialog above, particular aspects 
of the algorithm will be numerically dependent on the particular circuit 
implementation. 
1. find L.sub.max : 
1.1 Adjust a broadband and spatially uniform light source upwards in 
intensity until the first pixels in the image start to saturate or clip. 
1.2 Slowly decrease the intensity until all of the pixels are safely out of 
saturation or clipping. 
1.3 Call this light level L.sub.max (FIG. 4), and record its level by a 
photometer 22 (FIG. 1). 
2. Initialize digital common mode and span controls (FIG. 2): 
2.1 If this is the first iteration of the algorithm, it is recommended to 
set the digital control values for the common mode and span controls to 
nominal. This, however, must not be done if this the second or successive 
iteration of a multiple iteration calibration. 
3. Take samples of the system response: 
3.1 repeat for i=N.sub.illum to 1: 
3.1.1 Set the light level via the photometer 22 (FIG. 1) to i*L.sub.max 
/N.sub.illum. 
3.1.2 Record a sufficient number of frames to eliminate any random and 
correlated temporal noise. 
3.1.3 Sum the pixels of a given channel to obtain a single figure which 
represents the sensor response to the illumination. This can be, for a CFA 
sensor, for example, only a sum of the green pixels of each channel, or a 
weighted average of the red, green and blue pixels. The choice of the 
sampling grid is a function of the application, but must take into account 
the avoidance of any known spatially correlated noise. (Our implementation 
avoids the edges of the sensor, as well as the first few columns of each 
sensor channel.) 
3.1.4 Record for each iteration the metric chosen for step 3.1.3. As an 
example of one possible implementation, we will show it as just a 
numerical sum of the measured pixels: 
##EQU21## 
3.1.5 Perform step 3.1.4 for each of the channels j. 3.2 With each 
iteration, we have the quantity 
##EQU22## 
Now sum these quantities across all iterations i as follows: 
##EQU23## 
and 
##EQU24## 
4. calculate the measured gain and offset for each channel: 
4.1 compute for each channel j the following: 
##EQU25## 
5. calculate the desired correction in gain and offset k.sub.mj and 
.DELTA..sub.bj : 
5.1 Find the specific channel l whose gain term m.sub.l represents the 
median gain value amongst the set of gains m.sub.j, where j.epsilon.[1 . . 
. N.sub.channels ] 
5.2 Over all channels j.epsilon.[1 . . . N.sub.channels ], compute: 
EQU k.sub.mj =m.sub.l /m.sub.j 
5.3 Over all channels j.epsilon.[1 . . . N.sub.channels ], compute: 
##EQU26## 
6. calculate and apply the desired correction in the digital common mode 
and span controls: 
6.1 Over all channels j.epsilon.[1 . . . N.sub.channels ] compute: 
##EQU27## 
6.2 Over all channels j.epsilon.[1 . . . N.sub.channels ], apply the 
following correction: 
##EQU28## 
6.3 Over all channels j.epsilon.[1 . . . N.sub.channels ], compute: 
##EQU29## 
6.4 Over all channels j.epsilon.[1 . . . N.sub.channels ], apply the 
following correction: 
##EQU30## 
7. Repeat from step 1 as necessary for algorithm convergence. 
8. The system should now be calibrated for proper operation. 
The invention has been described in detail with particular reference to a 
preferred embodiment thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the 
invention. 
______________________________________ 
TS LIST 
______________________________________ 
10 sensor 
12 light source 
14 signal processor 
16 display 
18 central processing unit 
20 bus 
22 photodiode 
30 circuit 
32 analog-to-digital converter 
34, 36 inputs 
38 serial I/O daisy chain 
40 SPAN DAC 
42 COMMON MODE DAC 
44 frame store 
______________________________________