Abstract:
An image processing system 10 includes an array (12) of detectors 14, each of which is designed to produce a current proportional to incident radiation. This system provides image processing at a viable sampling rate even for very large arrays and permits very efficient determination of single element detections. 
     The modulation functions supplied from a weighted summer (18). The weighted summer applies an invertible matrix of weights to a series of orthonormal Walsh functions defined over a predetermined sampling interval, the Walsh functions being generated by a function generator (16). 
     The modulated outputs of the array are combined by a summer (20) and distributed among parallel channels by a divider (22). Correlators (24) correlate the signal in each channel with a respective one of the original Walsh functions. The correlated outputs are digitized by analog-to-digital converters for transmission and processing by a digital processor (28). The processor can at least partially reconstruct the detected spatial distribution for output to a display (30).

Description:
BACKGROUND OF THE INVENTION 
     The present invention relates to image processing, and more particularly to a system and method for processing data from sparsely excited very large imaging arrays. 
     One application for very large imaging arrays is in staring sensors for detecting and locating the onset of a radiative event. For example, a satellite based sensor can be used to stare at a region to detect missile or spacecraft launchings or nuclear tests. 
     However, in order to provide for precise location of the exciting event, very large photo arrays are required. For the applications listed above arrays of 10,000 by 10,000 picture elements (pixels) are called for. To sample such an array at, for example, ten times per second, an overall sampling rate of 10 9  Hz is required. This creates extreme demands on the subsequent image processing. 
     While advances in component design will inevitably provide faster sampling and related processing components, imaging objectives exceed the capabilities of even these future components. Accordingly, an objective of the present invention is to provide a system and method for more efficient processing of image data from very large arrays. 
     SUMMARY OF THE INVENTION 
     The present invention provides a system and method for efficient processing of image data from large arrays by modulating pixel elements according to respective mutually orthogonal functions. The modulated outputs can be multiplexed to utilize system hardware bandwidth more efficiently and then correlated according to the original modulation functions to obtain the desired image data. The invention is particularly applicable to sparsely excited very large image arrays. 
     The modulation can be effected by a variety of means, including varying the bias across individual photodiodes or by controlling the percentage of light reaching the photodiodes by, for example, a liquid crystal shutter. The modulated signals can be summed or otherwise multiplexed into one or more channels. By correlating the multiplexed signals according to the original modulation functions, for example, by parallel mixing of the multiplexed signals with respective modulation signals and filtering the results to integrate and remove unwanted terms the desired image data may be obtained. 
     Such a system can provide for efficient detection and location of illumination or a change in intensity of a signal pixel within a defined sete of pixels. More complex illumination or change patterns can be characterized by further processing. Depending on the particular embodiment, the further processing can involve additional mathematical manipulation or subsequent sampling. 
     In accordance with the present invention, the demands on sampling hardware are greatly reduced in proportion to the reduction in channels carrying the image data. Minimal processing overhead is incurred in detecting and locating single element events. More complex events can be decoded with further processing. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a schematic of a signal detection and processing system in accordance with the present invention. 
     FIG. 2 is a schematic representation of a photodiode array in accordance with the present invention. 
     FIG. 3 is a schematic of a modulation scheme for the diodes of the array of FIG. 2. 
     FIG. 4 is an alternative schematic of the modulation scheme shown in FIG. 3. 
     FIG. 5 is a schematic showing part of a signal processing system used in conjunction with the array of FIG. 2. 
     FIG. 6 is a schematic of a modulation scheme for photodiodes in accordance with the present invention. 
     FIG. 7 is a schematic of an N-output photodiode in accordance with the present invention. 
     FIG. 8 is a schematic of a signal processing system using spatial weighting functions in accordance with the present invention. 
     FIG. 9 is a schematic of a single element detection implementation of the present invention. 
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     A signal processing system 10 includes a detector array 12 comprising a multitude of detectors 14, as shown in FIG. 1. The array 12 can be a superelement of a much larger array, similar superelements being processed sequentially in the manner described below with respect to array 12. Each detector 14 provides an output as a function of the detected value of a variable referable to an event of interest. For example, the signal processing system can be an image processor and the detectors can be photodiodes which output current as a function of the intensity of incident radiation. The pattern of radiation incident to the array 12 can indicate the source of a radiative event such as a rocket launching. 
     The signal processing system 10 includes a function generator 16 for generating a set of time functions. In the illustrated system 10, these functions are orthogonal over a predetermined time interval which is short relative to the duration of events to be detected using the array 12. Preferably, the time functions are Walsh functions or an alternative set of functions orthonormal over the predetermined time interval. 
     A weighted summer 18 accepts as input the orthogonal time functions provided by the function generator and in turn produces a set of modulation functions in the form of weighted sums of the time functions. Preferably, the weights applied by summer 18 define an invertible matrix. For complete decoding, the matrix can be a square N×N matrix, where N is the number of detectors in the array 12 and the number of functions γ i  provided by function generator 16. 
     The array 12 is designed to apply the modulation functions supplied by the weighted summer 18 to each of the detectors 14. For complete decodability, the array 12 can provide that the output of each detector 14 is modulated by a distinct modulation function. For some applications, alternative arrangements can be implemented efficiently. For example, each row of detectors 14 and each column of detectors 14 of array 12 can be assigned a distinct modulation function. In such an embodiment, the array 12 can be arranged so that the output of each detector 14 is modulated by the sum of the respective row and column modulation functions. Many alternative modulation function-to-detector mapping schemes are also provided for by the present invention. 
     A current summer 20 or alternative signal combining or multiplexing means is provided to combine the outputs of the detectors 14. Directly or indirectly, the output of the summer 20 is replicated over multiple channels by a signal divider 22 or related means. 
     The parallel outputs of the divider are directed to correlators 24. Each correlator 24 correlates a divider output with a respective one of the time functions γ i  provided by the function generator 16. The correlators have the effect of isolating components of the summed signal according to respective time functions γ i . 
     The correlator outputs can then be converted to digital form by analog-to-digital converters 26. The converters 26 form part of a means of sampling the output of correlators 24 over an interval of time over which the time-varying functions are orthogonal. The sampling of the converters 26 can be synchronized over the predetermined interval of orthogonality for the time functions. This synchronization may be accomplished using any well-known technique such as by sending appropriate control signals to the A/D converters 26 from the processor 28 over lines 29. The digitized correlator outputs can then be processed to obtain information as to the spatial variable of interest. In an embodiment providing for complete decoding, a matrix inversion can yield a complete spatial distribution. In other cases, more limited information can be obtained by pair-wise dividing selected correlator outputs. 
     In the preferred embodiment 10, both complete and partial decoding are provided for. The partial decoding, which is relatively rapid, identifies which detector has detected a change in the value of the incident variable when only one detector has detected such a change. The information, such as images, can be directed to a display 30 or other readout device. 
     Provision is made for the digital processor 28 to control the time function generator 16 via line 32. This line 32 can be used to switch certain time functions on and off, for example, to allow more complete decoding by successive samplings in cases where multiple detectors are excited concurrently. 
     In a preferred embodiment, illustrated in FIG. 2, an imaging array 212 comprises a rectangular or square array of photodiodes. The effective gain of each diode 214 in the array can be controlled as a function of the bias voltage applied by voltage function generators 216 and 217, as shown in FIGS. 3 and 4. As an exemplary alternative, one could use a variably reflective surface such as a liquid crystal shutter to modulate the light intensity before its incidence on the array. 
     For the configuration of FIG. 2, the current in a diode 214 can be approximately characterized as: 
     
         i=K.sub.0 +K.sub.1 ·v·q+f(v,q) 
    
     where i is the current, K 0  and K 1  are constants, v is the bias voltage, q the intensity of light incident the particular diode, see FIGS. 3 and 4, and f(v,q) comprises higher order terms in v, q or the combination. 
     The array 212 is subdivided into sub-arrays or superelements 240 which are sampled sequentially. In the embodiment of FIG. 2, each superelement 240 is constructed as an N×N array of pixels or photo diodes. In this case, N is even, so that i and j take on the values of -1/2(n), . . . , -1, 1, . . . 1/2(n). As indicated in FIGS. 3 and 4, generated voltage functions X(i,t) and Y(j,t) are summed at the diode at the intersection of row i and column j of array superelement 240. The resultant output current is then a function I(i,j,t) of row, column and time. Proper selection of diodes and pre-distortion of X(i,t) and Y(j,t) are used to minimize the effect of f(X+Y,q). Thus, ##EQU1## 
     Voltage biases x and Y are applied in parallel to all superelements that go to make up the total array, and N is in the range from 8 to 100. 
     The bias voltages X and Y are selected so that: ##EQU2## where α k  (i,t 0 ) satisfies orthogonality with respect to k over i for a fixed t 0 , and β o  (j,t 0 ) satisfies orthogonality with respect to l over j for a fixed t 0 . Also, α k  (i,t) and β 1  (j,t) satisfy orthogonality over a fixed interval of time T, for fixed i 0  and j 0 , and orthogonality with respect to k and l, respectively, so that one can form: 
     
         α.sub.k (i,t)=φ.sub.k (i)·γ.sub.k+1 (t) 
    
     
         β.sub.1 (j,t)=θ.sub.1 (j)·γ.sub.k+1+2 (t) 
    
     
         and make the substitution 
    
     
         φ.sub.k (i)=θ.sub.k (i). 
    
     
         Thus, 
    
     
         α.sub.k (i,t)=φ.sub.k (i)·γ.sub.k+1 (t) 
    
     
         β.sub.1 (j,t)=φ.sub.1 (i)·γ.sub.k+1 (t) 
    
     
         where, ##EQU3## 
    
     The currents from each element of each superelement are summed in a &#34;virtual ground&#34; amplifier 220, to form I T  (t), as shown in FIG. 5, where ##EQU4## 
     The output of this amplifier 220 is divided at location 222 so it feeds 2K correlators 224 and filters 225. Walsh functions are used for γ n  (t), so that the multipliers shown in FIG. 5 can be simple mixers. 
     The correlator outputs are sampled sequentially over all superelements. That is, all the filter outputs u k  are sampled from one superelement, and then all the u k  are sampled from the next superelement and so on until all of the superelements are sampled and then this cycle is repeated. 
     The output of the correlators is given by: ##EQU5## 
     In the case where only one pixel receives a sudden change in illumination and this is detected on an moving target indicator (MTI) basis, the coordinates of the affected pixel are readily obtained: 
     
         u.sub.0 =A.sub.0 ·φ.sub.0 (i)=A.sub.0 ·K.sub.0 
    
     
         u.sub.1 =A.sub.1 ·φ.sub.1 (i)=A.sub.1 ·K.sub.0 ·i 
    
     
         u.sub.2 =B.sub.0 ·φ.sub.0 (j)=B.sub.0 ·K.sub.0 
    
     
         u.sub.3 =B.sub.1 ·φ.sub.1 (j)=B.sub.0 ·K.sub.0 ·j 
    
     for the case where φ x  (i) and φ y  (j) are quantized Legendre polynomials. Therefore, the coordinates of the i, j position can be computed by forming: 
     
         i=(A.sub.0 /A.sub.1)·(u.sub.0 /u.sub.1) 
    
     
         j=(B.sub.0 /B.sub.1)·(u.sub.3 /u.sub.2) 
    
     and where: 
     
         |u.sub.0 |≧|u.sub.0 &#39;+δ| 
    
     
         |u.sub.2 |≧|u.sub.2 &#39;+δ| 
    
     where u 0  &#39; and u 2  &#39; are the measured values of u 0   and u 2   at the previous sampling period for the superelement, and where δ is the MTI threshold. 
     For this case, the sampling rate for 10  8  elements at 10 samples per second would be 10 9  samples per second using the straightforward approach. Using a 16×16 superelement, the present invention provides for a factor of 64 reduction in the sampling rate: ##EQU6## 
     For the occurrence of more than one excited element per superelement, a problem arises in that there is uncertainty in how to pair up the x and y coordinates properly. This problem can easily be resolved if we examine the superelement again, this time with the biases on some of the potential pairings removed. Thus, if we have a potential pairing that disappears, we know that was the proper pairing. For the specific case of two excited elements in an superelement, a single examination of the superelement with one of the potential pairings suppressed is sufficient to unambiguously detect the correct pairing. 
     In the embodiment of FIG. 6, the outputs of two elements 314 and 315 from a one-dimensional array of photodiodes are modulated by modulators 318 and 319 according to respective modulation functions v 1  (t) and v 2  (t). 
     The diodes are selected to provide output currents proportional to the incident light intensity so that the modulated output m k  (t) 5 for the k th  diode is proportional to v k  (t)·q k . The m k  (t) are summed by amplifier 320 to Yield: 
     
         M(t)∝v.sub.1 (t)·q.sub.1 +v.sub.2 (t)·q.sub.2 
    
     Thus, M(t) is a sum of terms, each of which is proportional to the incident light intensity and the modulation on a particular element. Assuming the incident light intensities are approximately constant over a sampling interval, since if the modulating signals v k  (t) are chosen to be orthonormal signals over this interval, the single signal M(t) can be processed to recover each q k . 
     In one aspect of the present invention, a number of spatially dependent weighting functions can be used to permit straightforward computations on sums of diode signals to determine the intensities of the light striking the array. This allows centralization of the processing of image arrays. It is described below for a one-dimensional array but is directly extendable to arrays of higher dimensionality. 
     The N-output diode element 414 of FIG. 7 consists of a photo diode generating a voltage proportional to the incident light intensity q 1 , which is then amplified by a factor of α j  (1), for the j th  of the outputs. The amplifications are effect®d by parallel amplifiers 420. 
     Consider the use of N of these N-output diode elements 514 in an Nxl array to detect the light intensity incident where the N diodes are located. The configuration and interconnection of these elements are shown in FIG. 8. As is illustrated, the signal from the j th  output of one of the N-output diode elements is summed, by a respective one of N summers 520, with the output from the j th  element of each of the other (N-1) N-output diode elements. This forms the N sums V(1), . . . , V(N), where ##EQU7## where C is a constant. 
     This set of equations can conveniently be expressed in matrix forms as: ##EQU8## 
     Thus, we have available V through measurements, A is a matrix of weights which we can choose and q is of interest. Therefore, if A is chosen to be an invertible matrix, q can be calculated in a straightforward manner: 
     
         q=A.sup.-1 ·V 
    
     In particular, for the case where N is odd, one can renumber the elements -K, . . . , 0, . . . K, where K=1/2(N-1), and choose the coefficients α j  (-k), . . . , α j  (k) as samples of the j th  order Legendre polynomials over the interval [-K,K]. Then the weight matrix A is orthogonal, and is thus easily invertible. 
     Modulation tagging of diode signals can be combined with spatial weighting so that multiple-output diodes are not required. This technique can be used to advantage in large arrays of photo diodes, where centralized processing is desired, but use of multiple output diode elements is impractical. The approach will be described for a one dimensional array, but is directly extendable to arrays of higher dimensionality. 
     As above, an Nxl array of multiple output diode elements can be used to format the signals V(1), . . . , V(N), where ##EQU9## and where C is a constant, q k  is a measure of light intensity incident on the k th  diode, and α j  (k) is the weighting applied to the jth output of the k th  multiple output diode element. As described above, q 1 , . . . , q n  can be determined from the signals V(1), . . . , V(N). 
     In the embodiment of FIG. 9, N diodes 614 are arranged in an N×1 array to measure the light intensity incident on the N photo-sensitive diodes 614. The diode outputs are modulated according to respective modulation functions v k  (t) applied by modulators 618. 
     An amplifier 620 sums modulator outputs m k  (t) to yield a combined output M(t). As described above, the illumination dependent output from the kth diode can be described as: 
     
         m.sub.k (t)=C·q.sub.k ·v.sub.k (t) 
    
     Thus, M(t) is given by: ##EQU10## 
     The modulation functions are selected to have the form: 
     
         v.sub.k (t)=α.sub.1 (k)γ.sub.2 (t)+α.sub.2 (k)γ.sub.2 (t)+ . . . +α.sub.N (k)γ.sub.N (t) 
    
     where γ 1  (t), . . . ,γ N  (t) form an orthonormal set of time functions over the interval [O,T], such as Walsh functions. Thus: ##EQU11## 
     The mixers 624 and filters 625 yield inner products between M(t) and the time functions γ j  (t). The inner product between M(t) and the j th  orthogonal time function γ j  (t) is: ##EQU12## which is identical to V(j), and the set V(1), . . . , V(N) was shown to contain all the intensity information in a recoverable form. Thus, M(t) is a single signal formed as the sum of illumination dependent signals which are appropriately modulated, and can be processed in a straightforward manner to obtain the desired illumination information. 
     If only one pixel is non-zero, we can determine its location. As above, indices range from -K to K, where K=1/2(N-1), and the Legendre polynomial approach leads to the following weight coefficients: 
     
         a.sub.jk=c.sub.j ·P.sub.j (K/K), j,k=-K, . . . ,K 
    
     where c j  is a constant. Specifically, the first two rows of matrix A are given by: 
     
         a.sub.1k =c.sub.1 
    
     
         a.sub.2k =c.sub.2 ·k 
    
     where k=-k, . . . ,0, . . . ,K. 
     If, for example, q k0  is the only non-zero reading, then q k0  and k 0  can be determined from the first two inner products, since: 
     
         V(1)=c.sub.1 ·q.sub.k0 
    
     
         V(2)=c&#39;.sub.2 ·q.sub.k0 ·k.sub.0 
    
     Thus, determination of k 0  is given by: ##EQU13## where the constant B can be easily eliminated in forming the inner products. This last division can be performed by a processor 628. 
     Thus, several embodiments of the present invention and variations thereof have been disclosed. From the foregoing it is clear that the present invention is applicable to detection systems for a wide variety of spatial distribution variables, and is not limited to photo-detection. Different modulation and processing schemes can be used. Accordingly, the present invention is limited only by the scope of the following claims.