Color signature sensor

A color signature sensor for color recognition or discrimination utilizing a spectral analysis system for use as a process control for automation and on-line quality assurance. An object to be observed is illuminated by a suitable light source, light collected from the object under examination is routed into a monochromator. The output of the monochromator is fed to an analog-to-digital converter and analyzed by a microprocessor.

BACKGROUND AND SUMMARY OF THE INVENTION 
This invention is directed to a field of color signature sensor used in 
process automation which performs color recognition of objects for the 
purposes of identification, sorting or matching. 
In this invention reflected (or transmitted) light from an illuminated 
object (or process) is transmitted optically to a spectral dispersive 
element in the form of a concave diffraction grating. Spectrally dispersed 
light from the grating is focussed on a photodetector array which 
generates a spatially discretized analog signal representing the color 
signature of the object. The signal is then converted to digital form and 
sent to a microprocessor for processing. The sampled and digitized signal 
is a unique and complete description of the colored object being observed. 
Cross-correlation and mean-value statistics are used by the microprocessor 
for color recognition.

DESCRIPTION 
Referring now to the apparatus shown in FIG. 1 there is shown an 
illumination source (I) or lamp 10 for properly illuminating an object 11 
to be observed. Object 11 may be, for example, individual piece parts 
moving along a conveyor belt of a production line in which it is needful 
to compare the color of each part against the color of a reference part. 
Color perception depends on the spectral energy density of the 
illumination source 10, the spectral reflectance (or transmittance) 
distribution of the object 11 and the spectral responsivity 
characteristics of the signal acquisition (the optics portion) system 
herein described. 
Diffusely reflected (or transmitted) light from an illuminated object, such 
as 11, is collected by a fiber optic probe 12 that routes the light signal 
to the input of a monochromator shown in the form of a spectral dispersive 
element 13. The fiber optic path is exemplary only and the light from the 
object may be otherwise optically directed to the spectral dispersive 
element. Additional optics may be located at the input end 14 of the fiber 
to affect signal collection efficiency, angle-of-view, and the like. 
Similarly, additional optics may be located at the output end 15 of the 
fiber to affect better spot-size matching or far-field divergence angle 
matching to the spectral dispersive element 13. 
The spectral dispersive element 13 may consist of a blazed concave 
diffraction grating as shown, or may consist of other spectral dispersive 
configurations (generally called monochromator) that are known to those 
skilled in optics. The essential function of concave diffraction grating 
13 is to image the fiber output face 15 onto the input face of a detector 
array 16 and to disperse it spectrally (that is, to angularly decompose 
the input light distribution into the continuous spectrum of pure colors, 
each described by a specific wavelength) over the spatial extent of the 
acceptance aperture of array 16. The spatial characteristics of the array 
and of the individual photodiodes of the array, as well as its physical 
alignment with respect to 15 and 13, are chosen such that the spectrally 
dispersed signal of interest falls within the acceptance aperture of the 
array. The output of each photodiode in the array corresponds to the time- 
and space-integrated intensity of a specific band of pure color 
wavelengths in the light from object 11. This spectrally dispersed light 
distribution is the color signature L(x)=L'(.lambda.) of the illuminated 
object where .lambda. represents wavelength of light and x represents the 
spatial extension of the array. For colorimetry applications L'(.lambda.) 
extends from .lambda..congruent.400 nm to .lambda..congruent.700 nm; for 
other applications, the range of .lambda. is not restricted to the visible 
band thus permitting the inclusion of invisible wavelength bands for 
various additional applications such as counterfeit detection. The 
photodetector array may be similar, for example, to the EG&G RETICON solid 
state line scanners RL128G, the Honeywell TCL chip, or the like. This 
detector array 16 consists of a linear sequence of photodetectors that 
detects the light distribution L(x)=L'(.lambda.) and converts it to a 
time-dependent electrical current I(t) on wiring 17 that is clocked out in 
response to timing signals from the microcontroller/microprocessor 18. The 
array is scanned by the processor once every five to twenty milliseconds 
depending on the application with 16.67 milliseconds being one of the 
preferred intervals. A typical spectral distribution from a standard red 
and a standard blue color chip is seen in FIG. 2. In general, the signal 
I(t) on wiring 17 is an aperiodic, analog current that represents the 
signal data from the object 11 acquired by the optics hardware described 
above. Under static object illuminations, I(t) becomes periodic with 
successive scans and represents the acquired color signature. Multiple 
samples of the color signature of the same object under the same 
illumination can be used to reduce the noise present in each signature 
alone. 
The signal I(t) first flows into a sample-and-hold analog-to-digital 
converter 19 that forms a digital word description sequence of the colored 
object 11. The digitized output from 19 is connected by wiring 20 to 
microprocessor 18. The microprocessor also interfaces with a system memory 
21 by wiring 22 and also with an input-output (I/O) buffer device 23 by 
wiring 24. The processor can remember a number (such as eight) of color 
signatures each of which uniquely represents (eight) sample/reference 
colors. Sample/reference colors can be trained by putting a 
sample/reference object in the sensor's field-of-view, selecting a channel 
with a switch and pressing a "train" button. The color signature of the 
sample/reference object is then stored in memory. Data processing can be 
free-running at an exemplary speed of approximately sixty-eight 
spectrum-comparing operations being performed per second. The 
match/no-match outputs between incoming signature data and memory 
signature data can be updated after each compare operation or they can be 
latched by a higher level controller or external trigger when an object is 
properly in the sensor's field-of-view and thus when the outputs are 
valid. Under higher level control the color signatures can be trained, 
dumped or loaded through a serial port to facilitate day-to-day or 
product-to-product reprogramming and off-line training. For convenience 
the digitized signature output at 20 can be described as an m.times.n 
matrix I.sub.mn, where m is the number of photodetectors (or pure color 
sample bands) in the array and n is the bit precision of the digitization 
process, that is, n may be thought of as the number of bits resolved in 
the analog to digital converter. As an example m may be about 24-32 and n 
may be about 8-12. Generally m and n can be traded off with one another to 
meet various performance and cost constraints. The uniqueness and 
completeness of I.sub.mn as a signature description of a colored object is 
preserved in this digitization process provided m and n are chosen 
appropriately. All systematic aberrations occurring in the optical fiber 
12 from the fiber input 14 to the fiber output 15, the diffraction grating 
13, and the detector array 16 are incorporated implicitly in I.sub.mn. It 
is necessary to account for these aberrations only if the sensor output is 
an NBS-traceable color determination. Color recognition and discrimination 
outputs do not require aberration correction or compensation because all 
the acquired color signatures contain the same systematic aberrations. Any 
time-dependent or random system variations can be accounted for by 
acquiring cross-correlation and mean-value statistics for sequential 
acquisitions of the color signature of a static object. These statistics 
give what is equivalent to an autocorrelation function for this system 
(that is, system reproducibility and system error). This autocorrelation 
function essentially determines the color discrimination threshold for any 
two or more samples as described below. The simplest and probably the most 
useful data analysis scheme is to indicate whether the current color 
signature matches one or more reference signatures. Good success has been 
had using cross-correlation and mean difference formulae. The 
cross-correlation gives a statistical indication of "how good" the match 
(primarily in hue and chroma) is between two signatures, while the 
difference between the mean-values of each signature gives an indication 
of the difference primarily in brightness (or value). 
The color sensor's basic programming is designed for process control 
monitoring for factory automation and on-line quality assurance and not 
for absolute color analysis. In many respects this mimics a human 
inspector in that the sensor can not readily describe the color that it is 
currently seeing, but by. comparing stored spectra to the current scene it 
can recognize very subtle color differences. The sensor's memory for 
colors and its speed, however, are far superior to the human eye and 
brain. Color recognition and discrimination are obtained according to the 
following description: Suppose the sampled and digitized signature from 
object 11 (sample A) is given as the sequence described collectively as 
##EQU1## 
That is, 
##EQU2## 
is a shorthand notation for the sequence of ordered numbers (a.sub.0, 
a.sub.1, a.sub.2 . . . a.sub.m-1). The a.sub.i 's are the weighted pure 
color values within the analog-to-digital converter 19 after 
sample-and-hold and analog-to-digital conversion. The a.sub.i 's represent 
the time- and space-integrated light values from each of the m 
photodetectors that constitute and collectively describe the object's 
color signature. That is, the color signature consists of an ordered 
sequence of weighted pure color bands with a one-to-one correlation of the 
color bands to the a.sub.i 's. This provides the common basis for 
measuring the similarity of one sample with respect to the next. 
FIG. 3 shows an example of the ordered sequence of numbers 
##EQU3## 
that represents the sample-and-hold values of the continuous color 
signature of the standard blue in FIG. 2. The choice of m=19 in FIG. 3 is 
exemplary only. 
Similarly suppose the sampled and digitized signature from object 11 
(sample B) is given as 
##EQU4## 
The notation and meaning of I.sub.mn (B) is similar to that described for 
I.sub.mn (A), except that I.sub.mn (B) refers to sample B and I.sub.mn (A) 
refers to sample A. FIG. 4 shows the ordered sequence of numbers 
##EQU5## 
that represents the sample-and-hold values of the continuous color 
signature of the standard red in FIG. 2. Again, the choice of m=19 in FIG. 
3 is exemplary only. 
The signature mean (or average) values are defined by 
##EQU6## 
where .SIGMA. is the summation of m weighted values. Thus .eta..sub.a and 
.eta..sub.b may also be written 
EQU .eta..sub.a =(a.sub.0 +a.sub.1 + . . . +a.sub.m-1)/m (5) 
and 
EQU .eta..sub.b =(b.sub.0 +b.sub.1 + . . . +b.sub.m-1)/m. (6) 
FIGS. 3 and 4 show the approximate values .eta..sub.a and .eta..sub.b 
corresponding to 
##EQU7## 
for blue and 
##EQU8## 
for red, respectively, in FIG. 2. The cross-correlation .psi. between 
I.sub.mn (A) and I.sub.mn (B) is given as 
##EQU9## 
The numerator can be expanded explicitly as 
##EQU10## 
and the denominator factors can be expanded explicitly as 
##EQU11## 
The mean-value difference .phi. between I.sub.mn (A) and I.sub.mn (B) is 
given as 
EQU .phi.(A,B)=.eta..sub.a -.eta..sub.b. (11) 
By definition, .psi.(A,A)=1; however, since nonzero noise sources are 
always present, .psi.(A,A')&lt;1 where I.sub.mn (A') represents any 
sequential signature sample of A. Similarly, it is generally true that 
.PHI.(A,A').noteq.0. For convenience, we can take .psi.(A,A') and 
.PHI.(A,A') jointly to represent the total system error indicative of the 
system autocorrelation. (Similarly, .psi.(B,B') and .phi.(B,B') can be 
used as well in lieu of .psi.(A,A') and .phi.(A,A').) Thus, two or more 
colors can be discriminated provided 
EQU .vertline..psi.(A,A')-.psi.(A,B).vertline.&gt;.sigma..sub.cc &gt;0 (12) 
EQU or .vertline..PHI.(A,A')-.PHI.(A,B).vertline.&gt;.sigma..sub.mvd &gt;0 (13) 
where .sigma..sub.cc and .sigma..sub.mvd are the discrimination thresholds 
for cross-correlation and mean-value difference, respectively. 
.sigma..sub.cc and .sigma..sub.mvd are small nonzero numbers. If 
EQU .vertline..psi.(A,A')-.psi.(A,B).vertline..ltoreq..sigma..sub.cc (14) 
EQU and .vertline..PHI.(A,A')-.PHI.(A,B).vertline..ltoreq..sigma..sub.mvd, (15) 
the sample colors of A and B are said to match. 
It is noted that .psi.(A,A') and .PHI.(A,A') (or .psi.(B,B') and 
.PHI.(B,B')) may be insufficiently complete descriptions of the total 
system error, since many more samples of the object A (or B) are necessary 
in general. Similar arguments apply to .psi.(A,B) and .PHI.(A,B). 
Furthermore, it is well known in signal processing that averaging multiple 
signal acquisitions reduces the level of additive (random) noise and 
improves the signal-to-noise ratio of the final acquired signal. The 
signal-to-noise ratio improves in proportion to the square root of the 
number of waveforms averaged. Thus, for example, 
##EQU12## 
where a.sub.i =(a.sub.i.sup.1 +a.sub.i.sup.2 + . . . +a.sub.i.sup.k)/k is 
the mean-value of a.sub.i over k samples, is a better indicator of 
I.sub.mn (A) in the determination of both .psi. and .PHI.. FIG. 5 shows in 
flow chart form the method steps performed by the aparatus which are 
described above. 
If the optical hardware is designed and configured properly, color 
determination can be obtained straightforwardly from I.sub.mn of any 
sample. However, optical system imperfections (due to diffraction 
limitations, nonideal assembly or component performance, or 
performance/cost tradeoffs) give rise to distorted signals for 
L'(.lambda.) and I.sub.mn that cause errors in the color specification. In 
the terms of network theory, the system transfer characteristic is 
nonideal and "blurs" the system output. However, since the system impulse 
response for any spectrally dispersive system, as exemplified in FIG. 1, 
can be measured using a tunable, monochromatic illumination source over 
the wavelength band of interest, the optical system transfer 
characteristic can be determined. Within the spatial quantization error of 
the linear array (due to the limited number of photodiodes per unit 
distance x and the active/passive photodiode area ratio), it is possible 
to deconvolve the system transfer characteristic from the acquired signal 
to improve effectively the fidelity of the acquisition system. The spatial 
quantization error can be minimized by maximizing the effective lineal 
density of photodiodes in the array. This deconvolution concept can be 
implemented in a signal "deblurring" algorithm that permits use of 
relatively imprecise/low-cost components to achieve the performance 
obtainable in precise, high-cost spectrophotometric systems. In other 
words, this deconvolution concept enables the color signature sensor to 
emulate expensive, high-performance systems, in the function of color 
determination without many of the disadvantages (e.g. high cost, low 
speed, precise alignment and calibration) associated with these 
conventional systems. 
SUMMARY 
In this invention it is thus recognized that detected color signature 
L'(.lambda.) is a characteristic and unique descriptor of the color of an 
illuminated object without need for additional processing or analysis such 
as is required, for example, for the L.sup.* a.sup.* b.sup.* 
specification. There is herein used a color discrimination algorithm based 
on signature cross-correlation and mean-value difference, without need for 
color determination as is conventional. There is also described the system 
deconvolution (or signature deblurring) to filter out or compensate for 
optical system aberrations. This allows the use of inexpensive system 
optics. Also there is shown use of system autocorrelation output to set 
recognition threshold for inter-signature cross-correlation and mean-value 
difference.