Hadamard enhanced sensors

A method for increasing the pixel count delivered from a plurality of light detectors when scanning an object that includes positioning a mask, having a plurality of two dimensional Walsh-Hadamard filter patterns, in front of a plurality of light detectors to control the passage of light through the mask filter patterns to the light detectors. The method also includes positioning a lens assembly in front of the mask to project an image through the mask filter patterns on to the plurality of light detectors. The method also includes moving the mask, plurality of light detectors, or the image in a plane defined by a planar surface of the plurality of light detectors. The method also includes measuring the filtered image projected through the mask with the plurality of light detectors.

FIELD OF THE INVENTION

The currently described invention relates to methods and apparatus for increasing the number of pixels delivered by a focal plane detector array.

BACKGROUND

Conventional electronic imaging techniques rely on the number of detectors that can be assembled together in a single array. Current manufacturing techniques limit detector resolution and how tightly the detectors can be spaced in an array. In addition, applications for detector arrays continue to call for detector arrays having smaller physical footprints while increasing pixel density. A need therefore exists for improved methods and systems for increasing pixel density delivered by detector arrays.

SUMMARY

One embodiment is a method for increasing the pixel count delivered from a plurality of light detectors when scanning an object. The method includes positioning a mask, having a plurality of two dimensional Walsh-Hadamard filter patterns, in front of a plurality of light detectors to control the passage of light through the mask filter patterns to the light detectors. The method also includes positioning a lens assembly in front of the mask to project an image through the mask filter patterns on to the plurality of light detectors. The method also includes moving the mask, plurality of light detectors, or the image in a plane defined by a planar surface of the plurality of light detectors. The method also includes measuring the filtered image projected through the mask with the plurality of light detectors.

In some embodiments, the method includes storing a set of data representing the measured filtered image. In some embodiments, the method includes retrieving a representation of the projected image from the stored set of data by performing an inverse Walsh-Hadamard transform on the stored set of data. In some embodiments, the method includes moving the mask relative to the plurality of light detectors and the image.

In some embodiments, the method includes moving the plurality of light detectors relative to the mask and image. In some embodiments, the method includes moving the image relative to the plurality of light detectors and the mask. In some embodiments, the method includes moving one of the mask, plurality of light detectors, or the image along one or more axes.

Another embodiment is an apparatus for scanning an object. The apparatus includes a mask that includes a plurality of two dimensional Walsh-Hadamard filter patterns to control the passage of light through the mask. The apparatus also includes a plurality of light detectors located on a first side of the mask to measure light passing through the mask. The apparatus also includes a lens assembly located on a second side of the mask to project an image through the mask on to the plurality of detectors to generate a filtered image. The apparatus also includes an actuator to move the mask, plurality of light detectors, or the image.

In some embodiments, the actuator moves in a step and repeat motion. In some embodiments, the mask is coupled to the plurality of light detectors. In some embodiments, the actuator is configured to move one of the mask, plurality of light detectors, or the image along one or more axes.

In some embodiments, the actuator is configured to move the mask relative to the plurality of light detectors and the image. In some embodiments, the actuator is configured to move the plurality of light detectors relative to the mask and image. In some embodiments, the actuator is configured to move the image relative to the plurality of light detectors and the mask. In some embodiments, the apparatus includes a computer memory configured to store a set of data representing the measured filtered image. In some embodiments, the apparatus includes an image displacing element to move the image relative to the plurality of light detectors and the mask.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

Embodiments disclosed herein significantly increase the number of pixels delivered by a standard focal plane detector array. The increase is the result of applying Walsh-Hadamard masks, in the form of micro masks, to the focal plane imagery. By using the Walsh-Hadamard masks it is possible to increase the number of pixels delivered by each individual detector by a factor corresponding to the pattern mask subarray size. For example, a pattern mask having a 4×4 Walsh-Hadamard array size increases pixel density by a factor of 16 (4×4=16).

FIG. 1is a schematic illustration of an apparatus100for scanning an object, according to an illustrative embodiment. The apparatus100includes a mask104that has a plurality of two-dimensional Walsh-Hadamard filter patterns116. The filter patterns116of the mask104control the passage of light through the mask104. An actuator108is coupled to the mask104to controllably move the mask104along the direction of axis112. The apparatus100also includes a detector array132located on a first side128of the mask104to measure light passing through the mask104.

The detector array132includes a plurality of light detectors136for measuring the light passing through the mask104from the second side124of the mask104to the first side128of the mask. In this embodiment, the mask104is actuated to move along a single axis112parallel to the face148of the light detectors136. The apparatus100also includes a processor150coupled to the actuator108and the detector array132. The processor150provides commands to the actuator108to controllably move the mask104. The processor150also receives the outputs from the light detectors136of the detector array132. The processor150includes computer memory and stores the light detector136outputs in the computer memory for subsequent processing and/or retrieval.

The apparatus100also includes a lens assembly120located on the second side124of the mask104. The lens assembly104projects an image144of an object140through the mask104on to the plurality of light detectors136to generate a filtered image.

Referring toFIG. 2A, in this embodiment, sixteen (16) Walsh Hadamard filter patterns116(also referred to as pattern blocks) are used for the mask104. Each pattern block116subdivides a detector into a 4×4 subarray. Thus each detector is capable of reporting out sixteen (16) subpixels in a 4×4 subarray. The linear sequence of pattern blocks116is repeated horizontally and vertically to entirely cover the focal plane region of the detector array132. The linear sequence of pattern blocks116is evident when compared with a single linear sequence of 4×4 pattern blocks116as shown inFIG. 2B(the individual pattern blocks136are separated for clarity of illustration purposes). In this embodiment, the full mask is extended an extra sixteen detectors in width so no part of the focal plane is left unmasked when the mask104is moved along the rows of detectors.

Walsh-Hadamard (WH) matrices provide a set of orthogonal basis vectors which can be used to fully represent sampled signals which have 2nsamples. In one embodiment, the WH matrices are developed (e.g., performing a WH transform) by starting with a primitive 2×2 matrix of the following form:

H2=[111-1]EQN.⁢1
This primitive matrix is then expanded by replacing each of the terms in the primitive matrix with the primitive matrix, itself. The result of the first expansion is a type of outer product matrix of the following form:

Each of the rows in EQNS. 1 and 2 is an orthogonal WH basis vector. Thus, from EQN. 1 we have the four vectors:
h1=1,1,1,1  EQN. 3
h2=1,−1,1,−1  EQN. 4
h3=1,1,−1,−1  EQN. 5
h4=1,−1,−1,1  EQN. 6
This process can be continued to produce eight basis vectors of eight elements, sixteen basis vectors of sixteen elements, and so on.

In some embodiments, a specific type of WH basis function is used (Binary Walsh (BW) vectors) for non-coherent optical applications where it is difficult to create a mask with −1 openings. In a BW matrix (and function set), the −1 values are replaced by 0. Thus, EQNS. 3, 4, 5 and 6 convert to;
w1=1,1,1,1  EQN. 7
w2=1,0,1,0  EQN. 8
w3=1,1,0,0  EQN. 9
w4=1,0,0,1  EQN. 10
Each of these basis vectors can be arranged in any desired geometrical pattern. The basis vectors of EQNS. 7, 8, 9 and 10 could be arranged in 2×2 patterns. In this embodiment, we focus on sixteen element basis vectors for the mask104. The principles discussed here are applicable to any BW and WH basis function set of any 2nsize.

Referring toFIG. 1, in this embodiment, movement of mask104is performed using a step and repeat motion along the axis112. The detectors136are subjected to the light passing through the mask104for a finite amount of time to allow the detectors136to accumulate sufficient photo electrons to produce a low noise image from the object140. Thus, the pattern blocks116are commanded to hold a position above its corresponding detector for an appropriate length of time while the detector integrates the light. After this short dwell time the mask104is stepped along axis112so that each pattern block116moves to overlay the next detector in the row. After the mask104has moved sixteen steps (in this example) the motion is reversed and the mask104returns, step wise, to its starting position to repeat the cycle. In an exemplary embodiment, the total motion of the mask104is relatively small. For a detector136having a typical size of 15 microns and using a mask in which 16 mask steps are performed, the total linear mask motion is 240 microns. Two-axis motion may be used in alternative embodiments but would require a different spatial arrangement of the filter patterns116(e.g., Walsh-Hadamard filter pattern arrangement ofFIG. 2C).

With each step and dwell of the mask104, the pattern of light falling on a given detector136is spatially modulated by the pattern block116immediately above the detector136. The detector136thus reports out the corresponding Walsh-Hadamard filter pattern sequency component. The sequency of a Walsh-Hadamard function is a property analogous to the frequency of a sinusoid. A sequency ordered Walsh-Hadamard matrix is arranged so the number of sign changes in a row is in increasing order. For example, referring toFIG. 2, the matrix is sequency ordered where the successive rows have 0, 1, 2 and 3 sign changes. When the mask104is stepped to the next location, the detector136is masked by the next pattern block116in the sequence. The detector136then reports out the next sequency component. After sixteen steps, and sixteen different pattern blocks have covered the detector136, the detector136will have reported out the full sequency spectrum of the pattern of light falling on the detector136.

The following code fragments in C++ show the process of encoding and decoding a sampled function f[i] (i.e., creating a BW basis vector set, followed by expansion of f[i] in terms of these bases (i.e., applying a BW transform), to make a transformed function F[j]. Then, the process is inverted using a WH basis vector set (i.e. a WH transform) to recover a function, g[i], which matches the original f[i]: g[i]=f[i]. Recovery of the original light pattern involves performing a Walsh-Hadamard transform on the spectrum, which is performed, in one embodiment, according to the following code fragments and starting with the creation of the BW basis set from the WH basis set. Both basis sets are matrices with the basis vectors being the rows of their matrices. Note that the matrix is symmetrical. The WH matrix is also orthogonal, which means that multiplied by itself it produces the identity matrix:

double⁢⁢basis⁡[ROW]⁡[COL]={{⁢+1,+1,+1,+1,+1,+1,+1,+1,+1,+1,+1,+1,+1,+1,+1,+1},{+1,-1,+1,-1,+1,-1,+1,-1,+1,-1,+1,-1,+1,-1,+1,-1},{+1,+1,-1,-1,+1,+1,-1,-1,+1,+1,-1,-1,+1,+1,-1,-1},{+1,-1,-1,+1,+1,-1,-1,+1,+1,-1,-1,+1,+1,-1,-1,+1},{+1,+1,+1,+1,-1,-1,-1,-1,+1,+1,+1,+1,-1,-1,-1,-1},{+1,-1,+1,-1,-1,+1,-1,+1,+1,-1,+1,-1,-1,+1,-1,+1},{+1,+1,-1,-1,-1,-1,+1,+1,+1,+1,-1,-1,-1,-1,+1,+1},{+1,-1,-1,+1,-1,+1,+1,-1,+1,-1,-1,+1,-1,+1,+1,-1},{⁢+1,+1,+1,+1,+1,+1,+1,+1,-1,-1,-1,-1,-1,-1,-1,-1},{+1,-1,+1,-1,+1,-1,+1,-1,-1,+1,-1,+1,-1,+1,-1,+1),{+1,+1,-1,-1,+1,+1,-1,-1,-1,-1,+1,+1,-1,-1,+1,+1},{+1,-1,-1,+1,+1,-1,-1,+1,-1,+1,+1,-1,-1,+1,+1,-1},{+1,+1,+1,+1,-1,-1,-1,-1,-1,-1,-1,-1,+1,+1,+1,+1},{+1,-1,+1,-1,-1,+1,-1,+1,-1,+1,-1,+1,+1,-1,+1,-1},{+1,+1,-1,-1,-1,-1,+1,+1,-1,-1,+1,+1,+1,+1,-1,-1},{+1,-1,-1,+1,-1,+1,+1,-1,-1,+1,+1,-1,+1,-1,-1,+1},};
The following code fragment zeros out the negative entries:

Next, we transform f[i] to F[j] by matrix multiplication of f[i] by the BW matrix:

The return from F[j] to an auxiliary function g[i] involves, first, multiplication of F[j] by the WH matrix. Note that the result is rescaled according to the number of elements in the vector, 2/ROW:

//NOW, INVERT THE HADAMARD TRANSFORM TO GET g[ ] (note the rescaling):

The zero term, f[0], then needs adjustment. We make this adjustment by subtracting F[0]:

g[0]=g[0]−F[0];
This completes the recovery of g[i]=f[i] after f[i] has been expanded in the BW basis vector set.

FIG. 3is a schematic illustration of an apparatus300for scanning an object, according to another illustrative embodiment. In this embodiment, the mask is fixed in front of the detectors and the image is stepped across the detector array. The apparatus300includes a mask304that has a plurality of two-dimensional Walsh-Hadamard filter patterns316. The filter patterns316of the mask304control the passage of light through the mask304. The mask304is fixed with respect to the plurality of detectors336of the detector array332. The detectors336measure the light passing through the mask304from the second side324of the mask304to the first side328of the mask304.

The apparatus300also includes a lens assembly320located on the second side324of the mask304. The lens assembly320projects an image344of an object340through the mask304on to the plurality of light detectors336to generate a filtered image. The apparatus300also includes an actuator308coupled to an image displacing element314. In this embodiment, the actuator308is actuated to move along a single axis312parallel to the face348of the light detectors336. The actuator308is configured to controllably move the image displacing element314along the direction of axis312. The image displacing element314can be, for example, a mobile lens element, a steering mirror or a deformable prism. In this embodiment, the image displacing element314is a mobile lens element. The image displacing element314serves a dual function in this embodiment. The image displacing element314increase the pixel count and also stabilizes the image to compensate for camera shake. The actuator308movement can be controlled to account for shaking (undesirable movement) of, for example, the image displacing element314due to forces applied to the apparatus300by external disturbances.

The apparatus300also includes a processor350coupled to the actuator308and the detector array332. The processor350provides commands to the actuator308to controllably move the image displacing element314. The processor350also receives the outputs from the light detectors336of the detector array332. The processor350includes computer memory and stores the light detector336outputs in the computer memory for subsequent processing and/or retrieval.

The image displacing element314moves transversely (along axis312) to the propagating light. In so doing it displaces the image, along the direction of motion of the lens, with respect to the focal plane.FIG. 3shows only a single actuator with an implied single axis of motion. In some embodiments, the apparatus300may instead have two axes of actuation, one axis being aligned with the rows of the detector array and the other axis aligned with the columns. With two axis actuation, the motion would be a scan through sixteen positions in a four by four step pattern, corresponding toFIG. 2D. The four by four displacement pattern minimizes the total lens excursion and thereby minimizes optical aberrations that might result from the movement of the image displacing element314.

The image displacing element314is repetitively stepped in such a way that each portion of the image falls successively on a different detector336. Because each detector336has a different BW pattern block316, this repetitive stepping of the image will create a BW sequency analysis of the image for each of the sixteen basis vectors. In order to retrieve the sequency, so the subpixels of the image can be recovered, it is necessary to synchronously retrieve information from the different parts of the memory in which the set of stepped images is stored.

In normal operation, for each stepped image displacing element314position, an image is dissected, recorded and stored in memory in the processor350. Suppose, for example, that a first image is recorded with the image displacing element314in a first position; the lens is then stepped to a second position and another image is recorded and stored in a different portion of the memory. We consider a small patch of the image which is the size of a single detector. Because the second image is displaced from the first, this image patch will move to an adjacent detector and will therefore be spatially masked in a different way from the first case. This is because the image patch is now illuminating a detector which has a different BW mask pattern. In addition, this adjacent detector will report its output to a different memory location than the first detector. This process continues until the apparatus300has stepped the patch image through all sixteen detector336positions.

In order to reconstruct the fine pixel image of this detector-sized image patch, it is necessary to retrieve the patch's intensity measurements made from adjacent detectors as the image is stepped from each detector to an adjacent detector. This sequence of recorded intensities, retrieved from different parts of the computer memory associated with the processor, is then passed through a WH transformation and the desired fine pixel image for the selected image patch is produced.FIG. 4illustrates the technique. Four adjacent detectors336from the focal plane are shown. At time tna detector sized image patch illuminates detector n. The patch light is spatially modulated by a particular BW pattern. The output from this detector is converted from analog to digital form and is stored in a particular part of memory. This storage location, consisting of several bits of data storage, is indicated by a small black square. At time tn+1the image patch moves to illuminate detector n+1. The light of this patch passes through a different BW pattern. The output from this detector is converted to binary form and the resulting bits are stored in a different part of memory. Again the patch is stepped to detector n+2, and then n+3, and so on until all sixteen detectors, which are participating in recording this particular image patch, have been illuminated.

When the step-and-repeat process has been completed, the stored numbers are withdrawn from memory so as to provide sequency components n through n+3 (for this partial example). From the full set of sequency components (16 in this case) the high resolution image patch can be reconstructed. This reconstructed image patch will then be represented by sixteen micropixels which cover an area the size of a detector.

FIG. 5is a schematic illustration of an apparatus500for scanning an object, according to another illustrative embodiment. The apparatus500includes a mask504that has a plurality of two-dimensional Walsh-Hadamard filter patterns516. The filter patterns516of the mask504control the passage of light through the mask104. The mask504is fixed with respect to the plurality of detectors536of the detector array532. An actuator508is coupled to the detector array532to controllably move the detector array532along the direction of axis512. The detector array532measures light passing through the mask504.

The apparatus500also includes a processor550coupled to the actuator508and the detector array532. The processor550provides commands to the actuator508to controllably move the detector array532. The processor550also receives the outputs from the light detectors536of the detector array532. The processor550includes computer memory and stores the light detector536outputs in the computer memory for subsequent processing and/or retrieval. The apparatus also includes a lens assembly520located on the second side524of the mask504. The lens assembly504projects an image544of an object540through the mask504on to the plurality of light detectors536to generate a filtered image.

Movement of detector array532is performed using a step and repeat motion along the axis512. The detectors536are subjected to the light passing through the mask504for a finite amount of time to allow the detectors536to accumulate sufficient photo electrons to produce a low noise image from the image540. Thus, the pattern blocks516of the mask504are commanded to hold a position for an appropriate length of time while the detector536integrates the light. After this short dwell time the detector array532is stepped along axis512. After the detector array532has moved sixteen steps (in this example) the motion is reversed and the detector array532returns, step wise, to its starting position to repeat the cycle. Two-axis motion may be used in alternative embodiments but would require a different spatial arrangement of the filter patterns516.

FIG. 6is a flowchart600of a method for increasing pixel count delivered from a plurality of light detectors (e.g., light detectors136of the apparatus100ofFIG. 1) while scanning an object, according to an illustrative embodiment. The method includes positioning a mask604in front of a plurality of light detectors to control the passage of light through the mask filter patterns to the light detectors (e.g., mask patterns116of mask104and detectors136ofFIG. 1). The mask has a plurality of two dimensional Walsh-Hadamard filter patterns. The method also includes positioning a lens assembly608in front of the mask to project an image through the mask filter patterns on to the plurality of light detectors.

The method also includes moving612the mask, plurality of light detectors, or the image in a plane defined by a planar surface of the plurality of light detectors. The step of moving612can include moving one of the mask, plurality of light detectors, or the image along one or more axes. In some embodiments, the method includes moving the mask relative to the plurality of light detectors and the image. In other embodiments, the method includes moving the plurality of light detectors relative to the mask and image. In other embodiments, the method includes moving the image relative to the plurality of light detectors and the mask

The method also includes measuring the filtered image616projected through the mask with the plurality of light detectors. The method also includes storing a set of data620representing the measured filtered image using, for example, the processor150ofFIG. 1. The method also includes retrieving a representation of the projected image from the stored set of data by performing an inverse Walsh-Hadamard transform on the stored set of data, similarly as described previously herein.