Electronic color infrared camera

An infrared digital electronic camera includes a solid state color image sensor having an array of image sensing elements and an array of color filter elements including infrared color filter elements arranged over the image sensing elements for producing a color image signal. A signal processing circuit processes the color image signals from the image sensor to produce a false color image signal.

FIELD OF THE INVENTION
 The invention relates generally to the field of photography, and in
 particular to multi-spectral photography. More specifically, the invention
 relates to an electronic color infrared camera.
 BACKGROUND OF THE INVENTION
 Infrared photography with false color materials offers potential value in a
 wide range of fields. Applications include medical, reconnaissance,
 geographical surveys, resource management (roads, buildings, and
 utilities), law enforcement, environmental and agricultural assessment,
 art authenticity analysis, forgery investigation, and pictorial
 applications.
 Traditionally, false color images are captured photographically with
 traditional photographic cameras and infrared-sensitive film. The film
 products available on the market today are: KODAK Aerochrome Infrared Film
 2443; and KODAK Aerochrome Infrared NP Film SO-134. These products are
 sensitive to light in the green (500-600 nm), red (600-700 nm), and
 infrared (700-900 nm) portions of the electromagnetic spectrum.
 It has been proposed to employ 3 monochromatic video cameras, each having a
 spectral filter (e.g. green red and infrared centered on 550 nm, 650 nm
 and 850 nm respectively) and aimed at the same scene to produce a false
 color digital image signal. See the article "An airborne multi-spectral
 video/radiometer remote sensing system for natural resource monitoring" by
 C.M.U. Neale, Thirteenth Biennial Workshop on Color Aerial Photography,
 Orlando Fla., May 6-9, 1992.
 More recently, infrared information has been captured with a
 panchromatic-infrared electronic camera. In this method, a filter wheel is
 placed before a panchromatic and infrared sensitive charge coupled device
 (CCD) array in an electronic camera. An image is acquired by sequentially
 exposing the CCD through a series of filters, which represent the desired
 spectral bands of the imagery. When imagery is acquired electronically, it
 can be easily downloaded from the camera into a computer, where it can be
 analyzed and displayed. Currently, Eastman Kodak's Professional Digital
 Camera System (DCS) Model 420 IR operates by this sequential filter wheel
 technique.
 Although these technological options exist to capture infrared-sensitive
 imagery, they are not without problems. Today's infrared-sensitive films
 are consumable media and require wet photographic processing. Detailed
 analysis of the resulting images requires photographic scanning for input
 into geographic analysis computer software or digital image processing
 routines, causing a considerable delay in preparing and analyzing
 time-sensitive data. The infrared imaging systems employing video (as
 opposed to digital) imaging technology suffer from the problems of low
 resolution, poor response to relative image/camera motion, and the
 complexity resulting from the use of a number of cameras.
 The state-of-the-art digital electronic technology described above requires
 a dedicated infrared electronic camera, with a moving filter-wheel
 assembly, as embodied in the DCS Model 420 IR camera system, with the
 Kodak Color Filter Wheel Assembly. Because the filter wheel requires
 sequential capture of the imagery bands (it acquires three bands in 40
 seconds), the camera can only be used for still photography applications
 (i.e. where there is no relative movement between the camera and the
 scene), and is therefore not useful for aerial photography. Also, the
 filter wheel increases the complexity of the camera system thereby
 decreasing the mechanical reliability of the system.
 Alternatively, an infrared sensitive electronic camera employing beam
 splitters and three linear detectors is shown in U.S. Pat. No. 4,170,987,
 issued Oct. 16, 1979 to Anselmo et al. This approach trades off the high
 cost and mechanical complexity of the filter wheel for the higher cost of
 three separate image detectors.
 From the foregoing it is seen that there is a need for an improved digital
 electronic infrared camera to capture near infrared imagery.
 SUMMARY OF THE INVENTION
 The present invention is directed to overcoming one or more of the problems
 set forth above. According to one aspect of the invention, an infrared
 digital electronic camera includes a solid state color image sensor having
 an array of image sensing elements and an array of color filter elements
 including infrared color filter elements arranged over the image sensing
 elements for producing a color image signal. A signal processing circuit
 processes the color image signals from the image sensor to produce a false
 color image signal.
 According to a further aspect of the invention, a digital electronic camera
 includes a solid state color image sensor having an array of image sensing
 elements and an array of red, green and blue color filter elements for
 producing a color image signal. A package for mounting the solid state
 color image sensor has a window that blocks blue light and passes infrared
 light. A signal processing circuit (preferably a programmed
 microprocessor, or alternatively a custom integrated circuit) processes
 the color image signals from the image sensor to produce a false color
 image signal by subtracting an infrared signal from the red and green
 signals produced by the image sensor. This aspect of the invention has the
 special advantage of being easily and cost effectively produced in a
 process that normally produces true color digital image cameras, by merely
 replacing the window in the package for mounting the image sensor and
 adding appropriate signal processing, thereby realizing economies of scale
 in the manufacturing process.
 According to a still further aspect of the invention, a digital electronic
 camera is provided, having: a solid state color image sensor with an array
 of image sensing elements and an array of color filter elements arranged
 over the image sensing elements for producing a color image signal. A
 filter mechanism having an infrared filter portion and a color filter
 portion is moveable between a first position wherein the infrared filter
 portion is located to filter light reaching the image sensor, and a second
 position wherein the color filter portion is located to filter light
 reaching said image sensor. A signal processing circuit responds to the
 filter being in the second position for processing the color image signals
 from the image sensor to produce a false color image signal. This aspect
 of the invention has the special advantage of being readily convertible
 between an infrared sensing electronic camera to a true color electronic
 camera. In a preferred embodiment of this aspect of the invention, the
 image sensor has red, green and blue sensitive elements, the color filter
 portion of the filter mechanism is a yellow filter, and the signal
 processing circuit removes an infrared component from the green and red
 signal when the yellow filter is over the image sensor. The signal
 processing circuit causes the green component of the sensed image to be
 displayed as blue, the red component to be displayed as green, and the
 infrared to be displayed as red.
 The infrared electronic camera of the present invention is advantageous in
 that all three color bands of information are collected concurrently
 without the need for a mechanical filter wheel assembly; nor does the
 camera need multiple cameras, multiple image sensors, or means for image
 registration between the images from multiple cameras or multiple image
 sensors. The infrared camera of the present invention is useful for motion
 or rapid capture applications.
 These and other aspects, objects, features and advantages of the present
 invention will be more clearly understood and appreciated from a review of
 the following detailed description of the preferred embodiments and
 appended claims, and by reference to the accompanying drawings.

DETAILED DESCRIPTION OF THE INVENTION
 Referring now to FIG. 1, an infrared camera according to the present
 invention is shown. The camera, generally designated 10, includes a two
 dimensional solid state image sensing array 12. The image sensor 12 may
 comprise any of the known configurations for solid state image sensors,
 such as frame transfer, interline transfer CCD's, or diode arrays. The
 image sensor 12 includes a color filter array 14 having an array of color
 filter elements 16 disposed over the image sensing elements of the image
 sensor 12. Conventional color image sensors are packaged in a container
 (not shown) having a window with an infrared filter to prevent exposure of
 the image sensor to infrared light. In the present invention, the image
 sensor 12 is packaged in a container with a clear glass window 17 that
 does not block the infrared wavelengths of interest.
 A lens 18 is positioned with respect to the image sensor 12 to form an
 image of a scene 20 onto the image sensor. A filter mechanism 22 includes
 an infrared filter portion 24 and a yellow filter portion 26. The filter
 mechanism may simply comprise two different filter elements that are
 individually attachable to the front of the lens 18, or may comprise a
 more sophisticated mechanical mechanism for moving the filter portions
 with respect to lens 18. Although the filter mechanism 22 is shown located
 in front of lens 18 it is to be understood that it could be located
 anywhere in the optical path of sensor 12. When the yellow filter 26 is
 positioned in front of the lens 18, the camera 10 functions as an infrared
 camera according to the present invention. When the infrared filter 24 is
 positioned in front of lens 18 and the yellow filter 26 is removed, the
 camera 10 functions as a normal electronic true color camera. Preferably,
 for imaging vegetation, the yellow filter includes a low pass infrared
 cutoff filter that cuts off the infrared at 760 nm. For other
 applications, it is preferable to include an infrared cutoff filter that
 cuts off at 900 nm.
 The output 27 of the sensor 12 is supplied to signal processing electronics
 28. When the yellow filter 26 is positioned in front of lens 18, the
 signal processing electronics 28 produce a false color signal 29 as
 described below. The signal processing electronics 28 receives a signal 30
 indicating the electronic camera is operating in the infrared sensing
 mode. Signal processing electronics 28 may comprise electronic circuitry
 in the camera, or alternatively, may be implemented in a general purpose
 digital computer such as a personal computer to which the signals
 generated by the electronic camera are sent. The signal 30 may be supplied
 automatically to the signal processing electronics 28 by the filter
 mechanism 22 or may be supplied manually by the operator of the camera.
 The operational concept of the camera when in the infrared sensing mode is
 shown in FIG. 2 where elements similar to those of FIG. 1 are similarly
 numbered. Spectral energy represented by the arrows labeled B,G,R, and IR
 is reflected or emitted from the object 20 (e.g. vegetation) in the blue
 (B), green (G), red (R), and infrared (IR) portions of the spectrum. The
 spectral information is first filtered through the yellow filter 26 in the
 camera 10, which eliminates any blue spectral information. So, only
 information in the green, red, and infrared portions of the spectrum
 remain. The remaining spectral information passes through the camera's
 lens 18 and the clear glass window 17 over the CCD sensor 12, thus
 preserving the green, red, and infrared information.
 Now, the CCD sensor 12 captures the spectral information. The "green
 sensitive" pixels (those covered by green filter elements 16) on the CCD
 array now respond to both the green light and infrared light. Similarly,
 the "red sensitive" pixels (those covered by red filter elements 16)
 respond to the red and infrared light. The "blue sensitive" pixels (those
 covered by blue filter elements 16) on the CCD array now respond to only
 the infrared light. The image sensor 12 produces a digital image signal 27
 that represent the sensed green red and infrared image information.
 The output signal 27 is provided to signal processing electronics 28 (e.g.
 personal computer 31 having a CRT display 32) for additional processing to
 prepare the image for viewing, as shown in FIG. 3. Signal processing
 electronics 28 perform a data transformation which removes the infrared
 components from what were the green and red portions of the output signal
 27. The infrared information may be removed from the green and red
 portions of the signal for example by calculating an infrared value for
 each pixel in the image and subtracting the infrared value from the red or
 green pixel values.
 A preferred method of processing the image signals to compute an infrared
 value at each pixel location in the image is shown in U.S. Pat. No.
 5,373,322, entitled "Apparatus and Method for Adaptively Interpolating a
 Full Color Image Utilizing Chrominance Gradients", issued Dec. 13, 1994,
 to C. A. Laroche and M. A. Prescott, which is incorporated herein by
 reference.
 Additional color matrixing adjustments may be made by the signal processing
 electronics 28 to correct for color balance and unwanted overlap in the
 spectral bands that could not be filtered optically. The signal processing
 electronics also manipulates the image signal so the processed green
 information that represents the original scene (G') is displayed as blue
 (B"), the red information that represents the original scene (R') is
 displayed as green (G"), and the infrared information that represents the
 original scene (IR') is displayed as red (R") on CRT 32. Although this is
 a preferred method of false color display, any other choice of false color
 display could be used within the spirit of the present invention.
 Furthermore, if it is not desired to convert the camera from false color
 infrared to true color, the filter mechanism 22 may include only the
 yellow filter element 26, and the signal processing electronics 28 does
 not need to receive operating mode signal 30.
 Although the preferred embodiment of the invention employs user-selectable
 (yellow, infrared, or other) filters over the lens, other filter locations
 and configurations are possible. For example as shown in FIG. 4, the
 yellow filter could be included in the glass window 17 in the image sensor
 package. In this embodiment, the camera is not readily convertible to a
 true color camera. Alternatively, the yellow filter can be included in or
 formed directly over or under the filter array 14 of the image sensor 12.
 FIG. 5 shows the resulting color sensitivity of the pixels 34 in an image
 sensor 12 with such an arrangement, where R+IR indicates sensitivity to
 red and infrared, etc. Alternatively, the filter array 14 of the infrared
 camera of FIG. 4 may directly filter green (G) red (R) and infrared(IR),
 as illustrated in FIG. 6. In this embodiment, the yellow filter is not
 present in the system. The filter elements 16 in FIG. 6 may be formed from
 suitable organic dyes or multi-layer dielectric color filters as is known
 in the prior art. In this embodiment, the signal processing electronics is
 not required to subtract an infrared component from the green and red
 signals.
 The invention has been described with reference to a preferred embodiment.
 However, it will be appreciated that variations and modifications can be
 effected by a person of ordinary skill in the art without departing from
 the scope of the invention.
 TS LIST
 10 camera
 12 image sensor
 14 color filter array
 16 color filter element
 17 glass window
 18 lens
 20 scene
 22 filter mechanism
 24 infrared filter portion
 26 yellow filter portion
 27 digital image signal
 28 signal processing electronics
 29 false color signal
 30 operating mode signal
 31 personal computer
 32 CRT
 34 pixels