Abstract:
A direct-view focal plane array (FPA) having a detector layer, an amplification layer, a non-uniformity correction layer and a display layer. A direct path from the detector to the display element in each pixel may be established via a configurable digitally set analog circuit that controls gain and level for non-uniformity correction. The detector layer is operative to detect an infrared image with a raw image pixel response x and convert the infrared image into an electrical signal. The electrical signal is then fed into the amplification layer for amplification and the non-uniformity layer for offset and gain correction. An offset correction coefficient b and a gain correction coefficient m are inputted to the non-uniformity layer, to transform the raw image pixel response x into a corrected pixel response y, which prevents the true scene content from bein masked by the fixed pattern.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    Not Applicable 
       STATEMENT RE: FEDERALLY SPONSORED RESEARCH/DEVELOPMENT 
       [0002]    Not Applicable 
       BACKGROUND 
       [0003]    The present invention relates in general to a focal plane array, and in more particular, to a direct-view focal plane array. 
         [0004]    Several goggle technologies have been developed to convert light or image in various infrared ranges into visible images, so as to allow the observers to see or detect images or objects in various ambient conditions. For example, night vision goggle technologies have been developed that allow imaging nighttime low light level scenes to a display for easy viewing by the human eye. Existing light is amplified and displayed on a phosphor screen without the need for high power raster scanning of a detector array or of the display. This technology is also known as image intensifying (I2) goggles. An important aspect of this technology is its light weight and low power dissipation allowing many hours of operation using only a couple commercial off-the-shelf batteries. The limitation of this technology is its inability to perform thermal imaging while maintaining the same power and weight characteristics (i.e., low power and light weight) needed by a goggle device. For example, in low light conditions (e.g., cloud cover and no ambient light), I2 goggles will not work. To make I2 goggles work in low light conditions, the wavelength response of the I2 goggle must be extended via a photocathode so that it can view thermal imagery. However, the photocathode would have to be cooled to reduce thermal generated carriers. The reason is that photocathodes used in I2 goggles are ambient temperature devices and do not have provisions for temperature control. Unfortunately, cooling of the photocathode would lead to high power dissipation and larger size and weight. 
         [0005]    Prior art thermal imagers consume too much energy for use in a goggle. Thermal imaging systems create images of heat patterns and are useful even when there is no ambient light. There are many types of detectors that are used to detect thermal infrared light that occurs with greater than approximately 3.0 microns in wavelength such as solid state photovoltaic or photoconductive infrared detectors. Unfortunately, these detectors require cooling systems such as cryogenic closed cycle types and thermoelectric types which all require too much power to be feasible for use in a goggle. Uncooled bolometer type detectors do not require any cooling nor thermal stabilization systems. Unfortunately, even these types of detectors when coupled to focal plane array readout electronics, digitizing electronics, drive electronics, digital signal processing electronics, and display electronics require too much power to be feasible for use in a goggle. State of the art in low power camera design has produced an imaging module that only requires approximately 1 W of electrical power to operate, and this does not include the display. Unfortunately, even these imaging modules would use up the batteries for night vision goggles in less than one hour. Excessive power requirements of the systems discussed above require special batteries that increase the cost, size and weight of the systems. 
         [0006]    In summary, the prior art focal plane array based systems have the desired functionality, but require too much power and are too large for man-portable goggle type applications. Therefore, there is a substantial need in the art to provide a focal plane array based system that provides the desired functionality, packaged in a small size and operates at low power. 
       BRIEF SUMMARY 
       [0007]    A direct-view focal plane array operative to detect infrared and convert the infrared light into a visible image is provided. With the direct-view focal plane array, an infrared object or spatial image can be viewed directly by human eyes or a conventional visible focal plane array. The direct-view focal plane array as provided eliminates the requirement of multiplexing and de-multiplexing processes, digitization, high-speed digital data manipulation and operations, and signal conditioning on a continuous basis. By the direct-view focal plane array, a direct path from the detector to the display element is established via a configurable digitally set analog circuit that controls gain and level for non-uniformity correction (NUC). Configurable pixels, switchable to be used for symbology (by way of example and not limitation, text on the display) or for displaying the detector signal, are formed. The ability to add functionality by adding layers allowing additional capability to multiplex images on demand for capture and transmission of images of interest is enabled. The direct-view focal plane further allows for image processing functions in the layers such as interpolation where one detector influences many higher resolution display pixels. The enabling technology may be characterized as the ability to form circuitry on layers that can be stacked while having interconnection at the pixel level that results in a vertically integrated circuit. By way of example and not limitation, there may be a detector at the input of this vertically integrated circuit, and a display element at the output of the vertically integrated circuit thereby forming one pixel. The vertically integrated pixels may be arranged in a two dimensional mosaic array organized in a rectilinear fashion, or some other pattern such as an interleaved or honeycomb pattern that determines an active electro-optical focal plane on both sides of the array. The individual pixels functions may be controlled using circuitry located at the periphery of this active pixellated region. 
         [0008]    The direct-view focal plane array may comprise a detector layer, an amplification layer, a non-uniformity correction layer and a display layer. A layer is defined in this text in terms of functionality and also in terms of physical circuitry. One or more layers of functionality may reside in one layer, or level, of physical circuitry. The detector layer is operative to detect an infrared image with a raw image pixel response x and convert the infrared image into an electrical signal. The electrical signal is then fed into the amplification layer for pre-amplification to enable detector noise limited performance and the non-uniformity layer for offset and gain correction. In the case where a simple two-point non-uniformity correction scheme is used, an offset correction coefficient b and a gain correction coefficient m are input to the non-uniformity layer, such that the raw image pixel response x can be transformed into a corrected pixel response y by the relationship of y=mx+b. Thereby, the true scene content in the image captured by the detector layer will not be masked by the fixed pattern while being displayed by the display layer. Other methods of fixed pattern noise suppression known in the art or developed in the future may also be implemented as well. 
         [0009]    The direct-view focal plane array may include a two-dimensional array of pixels, and each pixel may include a detector unit, an amplification circuit, a gain-correction circuit, and a display unit. The detector unit may be operative to detect an infrared optical signal with a raw image pixel response x and convert the infrared signal into an electrical signal. The amplification circuitry may be operative to amplify the electrical signal and to offset the raw image pixel response x by an offset correction coefficient b. The gain correction circuitry is operative to provide a gain correction coefficient m to the raw image pixel response x. The display unit may be operative to convert the amplified and corrected electrical signal into a visible signal with a corrected pixel response y, wherein y=mx+b. Preferably, both the offset and gain correction values m and b are analog to reduce the circuit complexity to perform non-uniformity correction. Digital signals are also contemplated. The amplification circuitry in the preferred embodiment may comprise a digital-to-analog converter to convert a predetermined digital offset correction value into the analog offset correction coefficient b, and the gain correction circuitry comprises a digital-to-analog converter to convert a predetermined digital gain correction value into the analog gain correction coefficient m. The display unit includes a light-emitting device such as an organic light-emitting diode (OLED) unit or a field-emission display (FED) unit. The focal plane array may further comprise a buffer to temporarily store the electrical signal before being fed to the amplification circuitry. The amplification circuitry and the gain correction circuitry can be integrated on a single or multiple layers. In addition, the detector unit, the amplification circuitry, the gain correction circuitry, and the display unit may be in electrical communication with each other and integrated by vertical integrated sensor array technology. 
         [0010]    It is contemplated to multiplex small groups of detectors with small groups of corresponding display elements to reduce the number of interconnects between each layer. For example, the FPA may be organized into smaller arrays, such as 2×2 subarrays, (that may be multiplexed at a relatively low frequency as compared to a conventional FPA, but high enough that the transitions between pixels is transparent to the human eye) through one interconnect for each subarray in the FPA, to the corresponding 2×2 subarray of display elements. In this way, only one interconnect is required for the subarray as compared to four interconnects in this case. Multiplexing the small subarrays will add some complexity and power dissipation to the circuitry. 
         [0011]    The focal plane array can be used in a goggle to provide direct view for a user or an observer under various ambient conditions. The goggle includes an objective optic for focusing infrared light onto the focal plane array. The focal plane array then converts the infrared optical signal into a visible optical signal for the user to observe. There may also be an optical element on the visible side of the focal plane array to enable easier viewing by the observer. The direct view focal plane array may also be used with a normal visible camcorder or digital camera to view a scene in the infrared in conjunction with a re-imaging optics chain with the direct view FPA at the intermediate focal plane of the optical system. 
         [0012]    The direct-view focal plane array focal plane array can be operated with a very low power and used as an infrared, or thermal, goggle, a LLL (low light level) SWIR (shortwave infrared) goggle, or a LLL (low light level) visible goggle, depending on the specific type of detector array as selected. The direct-view focal plane array may be in the form of a multi-layer semiconductor circuit. On one side of the multi-layer semiconductor circuit may be an array of detectors, and on the other side may be an array of low power display elements. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0013]    These and other features and advantages of the various embodiments disclosed herein will be better understood with respect to the following description and drawings, in which like numbers refer to like parts throughout, and in which: 
           [0014]      FIG. 1  is a diagram showing the functions normally found in a conventional time division multiplexed FPA, with the functions outlined in the dashed line of the functions not required in the Direct View FPA; 
           [0015]      FIG. 2  shows the multi-layer structure of an exemplary direct-view focal plane array; 
           [0016]      FIG. 3  shows an exemplary arrangement of pixels of the direct-view focal plane array as illustrated in  FIG. 2 ; 
           [0017]      FIG. 4  shows the circuit diagram of each pixel of the direct-view focal plane array as shown in  FIG. 2 ; and 
           [0018]      FIG. 5  shows the realization of direct view provided by the focal plane array. 
       
    
    
     DETAILED DESCRIPTION 
       [0019]    As discussed above, most of the currently available prior art focal plane array systems that have the desired functionality require too much power for man-portable goggle type applications. The functions that consume significant power are listed in Table I and outlined for the FPA alone in the functional signal flow diagram as shown in  FIG. 1 . As shown, for a typical conventional time-division multiplexed FPA based imaging system, when an electrical current is generated by the photo-generated electrons, trans-impedance amplification is performed and the signal carried by the electrical current is converted into a voltage signal and temporarily stored. According to the housekeeping circuit, high-speed switching and high-bandwidth amplification are continuously performed on the voltage signal for every pixel of every frame to provide a video output, and many functions are performed off of the focal plane array after the output of the pixels has been digitized using high power digital signal processors. Therefore, although the system of signal processing as illustrated allows flexibility in design, it is extremely inefficient during operation. 
         [0000]    
       
         
               
               
             
           
               
                 TABLE I 
               
               
                   
               
               
                 On Focal Plane Array 
                 Off Focal Plane Array 
               
               
                   
               
             
             
               
                 Detector Bias 
                 Video Output Load 
               
               
                 Detector Interface Amplifier 
                 Video Buffer, Pre-Amp 
               
               
                 Time Division Multiplexing - 
                 ASIC/FPGA and Digital Support 
               
               
                 Column Select (Slow Speed) 
                 Circuitry 
               
               
                 Circuitry 
                 Power Regulation and Distribution 
               
               
                 Analog Column Amplification 
                 Digital Drive Circuitry 
               
               
                 Column Buffer Amplifier 
                 Display Drive Circuitry - 
               
               
                 Row Select (High Speed) Circuitry 
                 Demultiplexing, Video 
               
               
                 Housekeeping Functions 
                 Amplification, Bias 
               
               
                 For Timing 
               
               
                 Video Output Amplifier 
               
               
                   
               
             
          
         
       
     
         [0020]    To resolve the power consumption issue, the continuous operation of the circuitry that provides the high-speed switching and the high-bandwidth amplification outlined in  FIG. 1  is eliminated in the embodiment as shown in  FIGS. 2-4 . As shown in  FIG. 2 , the direct-view focal plane array  10  includes a detecting layer  12  at one side and a display layer  18  at the opposite side thereof. The detecting layer  12  includes an array of detectors  12   a  operative to detect infrared light, and the display layer  18  includes an array of display elements  18   a  operative to display the image or light detected by the detecting layer  12  in the visible wavelength range. The detecting layer  12  and the display layer  18  are connected to each other through one or more circuit layers, and the VISA (vertically integrated sensor array) technology allows the layer-to-layer connection for each pixel of the focal plane array  10 . Each of the circuit layers sandwiched between the detecting layers  12  may be designed with one or more specific functions, such as amplification, non-uniformity correction, and display adjustment. In this embodiment, two circuit layers, including the amplification layer  14  and the uniformity correction layer  16  are formed and stacked between the detecting layer  12  and the display layer  18 . 
         [0021]    The direct-view focal plane array  10  as shown in  FIG. 2  includes an array of pixels arranged along the X-axis and Y-axis to provide a two-dimensional imaging. The array of pixels and conversion of light are illustrated in  FIG. 3 . As shown, each of the pixels is operative to detect an optical infrared or visible photon signal and convert this signal into a visible image directly viewable by the user. Depending on the specific requirement, the focal plane array may incorporate various detectors so that it may be used as a thermal goggle, a LLL SWIR goggle, or a LLL visible goggle. 
         [0022]    The circuit for each pixel as shown in  FIGS. 2 and 3  is illustrated in  FIG. 4 . As shown, each of the pixels may include an optical infrared or visible photon detector unit  12   a  for absorbing the optical infrared or visible photon image and a display unit  18   a  for displaying the image in the visible wavelength range. Again, depending on the specific requirement or application, the optical infrared or visible photon detector unit  12   a  may be operative to detect images at various ranges of light such as visible, NIR (near infrared), SWIR, MWIR (mid wave infrared) or LWIR (long wave infrared) depending on the type of detector chosen. To minimize the volume of the focal plane array  10 , the display unit  18   a  may be selected from a flat panel display such as organic light-emitting diode (OLED), field-emission display (FED), or other visible display device. Similar to the functions as shown in  FIG. 2 , once an image is detected by the detector unit  12   a , an electrical current is photo-generated and converted into a voltage signal in amplifier  42 , also shown as a functional amplification layer  14 . The voltage signal is then fed to uniformity correction circuit  16 , and then displayed by the display unit  18   a . As shown in  FIG. 4 , each of the pixels comprises a unit  14   a  of the amplification  14  and a unit  16   a  of the uniformity correction circuit  16 . The amplification unit  14   a  includes an amplifier  42  of which, in this embodiment, converts the detector signal into a voltage. 
         [0023]    The output of the amplifier  42  is input to unit  14   a . The non-uniformity correction unit may comprise offset correction  14   a  and gain correction  16   a . The non uniformity correction unit may comprise an offset stage, here shown as a summing amplifier  46  with the negative input connected with two resistors  44  and  45 , where the resistor  44  is connected to the buffer  42  allowing the voltage signal to be input to the offset unit  16   a , and the resistor  45  is fed with an analog offset value converted from a predetermined digital offset value converted by a digital-to-analog converter  43 . The positive input of the amplifier  46  is grounded in this embodiment and a multiplier  49  is used for gain correction. A predetermined digital gain value is converted to an analog gain value by a digital-to-analog converter  48  and input to the multiplier  49  allowing the gain to be corrected according to the digital gain value. The output of the multiplier  49 , that is, the non-uniformity correction circuit unit  16   a  is then connected to the display unit  18   a  for displaying the image captured by the detector unit  12   a  in the visible wavelength range, such that the observer or user can obtain a direct view of the infrared image as shown in  FIG. 5 . Additionally or alternatively, a video recorder or a still image recorder  50  may be placed in front of the display unit  18  such that the video recorder or still image recorder  50  can obtain a direct view of the infrared image (see  FIG. 5 ). The video recorder or still image recorder  50  may be a visible camera (e.g., camcorder, digital photographic camera, or visible film camera). 
         [0024]    The offset and gain corrections as described above are the coefficients used for non-uniformity correction. As understood, there is very low contrast in the infrared wavelength range or at low light levels in the visible wavelength range, so that any slight difference in the responses between pixels to the same input stimulus will cause a fixed pattern to appear in the image that masks the true scene content. This is called fixed pattern noise. The response curve of each pixel is modeled as a linear function of output pixel response over the input optical power incident on the pixel. By adjusting the offset level and the gain level for each pixel, all of the apparent pixel responses can be made to look exactly alike. In the prior art, the output of a focal plane array is directly applied to a digitizer and all of the pixel responses are fed to a digital signal processor with all of the non-uniformities, and the corrected image is calculated digitally using stored calibration coefficients for each pixel. These operations must be performed on every pixel for every frame. In contrast, in the embodiment as shown in  FIG. 4 , the two-point correction is merely a linear equation y=mx+b. Other methods of fixed pattern noise suppression known in the art or developed in the future may be implemented as well. This is called as a two-point correction because two calibration points are collected on the raw pixel response to describe the raw linear transfer function thereof, and two correction coefficients are generated. In the linear equation, x is the raw pixel response, b is the offset correction, m is the gain correction coefficient, and y is the corrected pixel response. These operations may be performed in analog in the pixel circuitry using digitally controlled voltage to control the gain and offset levels. As the operating temperature changes, new coefficients would have to be loaded because the detector transfer function changes as the operation temperature thereof changes. This can be accomplished at a relatively low speed because the temperature change is not likely to not occur very quickly. This saves power effectively because the digital “housekeeping” is only required at startup and every so often when the coefficients need to be updated or when the user wants to save an image. There is no continuous framing operation as in conventional prior art systems, so the direct view FPA described herein provides a continuous image to the user without time division multiplexing. This makes the system the most efficient way to convert the infrared scene to a visible scene continuously for the human eye to observe, and therefore a goggle utilizing this direct view FPA can operate from battery power for much longer periods of time than a conventional imaging system. As the life cycle cost of this type of goggle system is dominated by batteries, the life cycle cost can be reduced dramatically over a typical video system from the lower power required. 
         [0025]    In an aspect of the focal plane array discussed herein, the same may be electrically connected to peripheral circuitry. The peripheral circuitry may be connected to certain pixels for use in symbology such as text, numbers, stop sign, left turn, etc. The peripheral circuitry may be operative to send a high or low voltage based on the symbol to be displayed. By way of example and not limitation, inputs of certain pixels may be connected to a high or low voltage rail instead of the detector. 
         [0026]    In an aspect of the focal plane array discussed herein, the same may be optionally be in communication with peripheral circuitry. The peripheral circuitry may receive the information provided by the outputs of the pixels and produce a time-division multiplexed video signal for viewing by another user(s). 
         [0027]    The above description is given by way of example, and not limitation. Given the above disclosure, one skilled in the art could devise variations that are within the scope and spirit of the invention disclosed herein. Further, the various features of the embodiments disclosed herein can be used alone, or in varying combinations with each other and are not intended to be limited to the specific combination described herein. Thus, the scope of the claims is not to be limited by the illustrated embodiments.