Modulation method for the deformable mirror device (DMD)

Abstract: This is a method of digitally modulating a DMD system. It can comprise: receiving at least one data stream of data of N bits of resolution per pixel from an input device 30; inputing the data stream into N input shift registers 32; addressing at least N weighted DMDs 36 with the shift registers 32, with output of the DMDs 36 being combined to create one pixel of an image; and activating the DMDs 36 based on the data stream. Other methods and system are disclosed.

Description: (C) Copyright, *M* Texas Instruments Incorporated 1991. A portion of the disclosure of this patent document contains material which is subject to copyright and mask work protection. The copyright and mask work owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright and mask work rights whatsoever. This invention generally relates to analog scan convertors and more specifically to the use of the Deformable Mirror Device (DMD) to implement an analog scan convertor. Without limiting the scope of the invention, its background is described in connection with systems utilizing an IR sensor. Conventional systems which rely upon a linear IR sensor are being superseded by systems using multiline sensors. These sensors provide increased resolution as well as an associated increase in S/N ratio and a decrease in problems associated with aliasing. These sensors normally contain only the number of pixels needed for scanning the vertical scene dimension due to the complexity of fabrication. The scene data shifted out of these vertical sensors is naturally in the form of a vertical raster. However, the human eye is accustomed to viewing images rastered horizontally. Therefore, these sensors, in general, require a scan convertor in order to recreate the IR image in the visible domain for viewing by humans. A typical digital high dynamic range digital scan convertor to accomplish this task is expensive, bulky, heavy, and consumes a considerable amount of power. The DMD has several attractive features. The mirror elements have high reflectance over a wide optical bandwidth. Thus, they are capable of optical modulation from the UV to the IR. Pixel response times are typically 10-20 µs and the pixels may be operated in an analog or digital mode. The DMD can function either as an amplitude-dominant or phase-dominant modulator by a simple architectural modification of the mirror. The DMD is compatible with monolithic fabrication on a low-voltage, high density address circuit. Due to the low power dissipation of the DMD, high frame rates (typically 5 kHz and higher) are possible without adverse heating effects. There are numerous DMD pixel architectures, each designed for particular applications. They may be distinguished by their deformation mode, such as torsion or cantilever, by the pixel shape, and by the hinge support architecture. In PROCEEDINGS OF THE SID, vol. 24, no. 2, 1983, Los Angeles, USA, pages 199-205, L. Hornbeck: "128 x 128 Deformable Mirror Device", a DMD consisting of an X-Y array of mirror/air gap capacitors that can be electronically line-addressed by an underlying array of transistors is disclosed. The drains of the address transistors are connected to a serial-to-parallel converter. The gates are connected to a decoder. Lines of video are clocked continuously into S/P converter. As each line is entered it is parallel dumped onto a storage register to make room for entry of the next line of data. As the next line is clocked into the delay line, the drain lines are simultaneously charged to voltage levels corresponding to the contents of the storage register. The decoder then momentarily activates a gate, storing that line of video on the flouting source capacitors underlying each mirror/air gap capacitor element. This process continues line by line at a continuous input data rate. A similar deformable mirror device is disclosed in US-A-4 793 699. SUMMARY OF THE INVENTION A method of digitally modulating a DMD system and a digitally modulated DMD system according to the present invention are defined in independent Claims 1 and 14. Dependent Claims 2 to 13 and 15 to 27 concerne particular embodiments of the invention. According to one aspect of the present invention there is provided: A method of modulating a DMD system comprising; receiving at least one data stream of data from an input device; inputting said data stream into a serial register; and addressing one or more DMDs with said serial register, the output of the or each DMD being adapted to create one pixel of an image; Each of the activated DMDs may be turned off individually such that each of the activated DMDs is left on for a period of time equal to a line time multiplied by a weighting for that DMD, whereby the DMDs can be weighted by time. Weighting of the weighted DMDs may be based on the resolution per pixel such that a most significant bit is weighted 1/2, a next most significant bit is weighted 1/4, continuing weighting to a least significant bit which is weighted 1/2 N . Alternately, the weighting may be by a predetermined number of DMDs per bit, whereby the DMDs can be weighted by number of DMDs per bit or the weighting may be such that one or more of the DMDs reflect different maximum amounts of light, whereby the DMDs can be weighted by reflecting area and/or reflectance of an individual DMD. Preferably, the DMDs are imaged onto a solid state detector, the detector is a CCD, and the detector manipulates the image from the DMDs such that the image may be output in a different order from the detector than the image was input to the detector. Alternately, the DMD may be imaged onto the backside of a scan mirror and an image from the backside of the mirror may be optically projected to create an image nearly identical to the image received at the input device. The input device may be a computer. This may also be a method of modulating a DMD system which can comprise: receiving at least one data stream of data of N bits of resolution from an input device; inputing the data stream into N input shift registers; addressing 2 N DMDs with the shift registers, whereby the shift register for a most significant bit addresses 2 N-1 DMDs, the shift register for a next most significant bit addresses 2 N-2 DMDs, continuing this addressing scheme to a least significant bit whose shift register addresses 2 N-N DMDs and the DMDs combine to create one pixel of an image; and activating the DMDs individually based on the data stream. This is also a modulated DMD system which generally comprises: N shift registers to receive at least one data stream of data of N bits of resolution from an input device and DMDs addressed by the shift registers, whereby the DMDs combine to create one pixel of an image. Preferably, the shift registers and the DMDs are on a monolithic integrated circuit. BRIEF DESCRIPTION OF THE DRAWINGS Reference will now be made, by way of example, to the accompanying drawings, in which: FIG. 1 is a conventional IR imaging system; FIG. 2 is the electronic portion of an advanced IR imaging system; FIG. 3 is a view of a torsion beam DMD; FIG. 4 is a top level view of various torsion beam DMD hinge options; FIG. 5 is a first preferred embodiment of the present invention; FIG. 6 is a third preferred embodiment of the present invention using pulse width modulation; FIG. 7 is a block diagram of a portion of the system in FIG. 6. Corresponding numerals and symbols in the different figures refer to corresponding parts unless otherwise indicated. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS In a conventional IR system (FIG. 1) the input image 10 is focused with an appropriate optical system 24, into the plane of an IR detector 12 configured as a line of discrete pixels. Obviously, only one line of the image 10 can be detected at a given time. Consequently, one element of the imaging system is a mirror 14 pivoted around its central axis so that proper rotation of the mirror 14 can bring any desired line of the image 10 to focus on the detector 12. The signal from each of the discrete sensor elements in the detector 12 is connected to a discrete light emitting element (typically an LED) in a linear array 16. The visible light emission of this array 16 is reflected off the back surface 18 of the scan mirror 14, and an observer 20 viewing this back mirror surface 18 through an appropriate optical system 22 will see a recreation of the input IR scene 10 as the mirror 14 scans that scene 10 across the IR detector 12. Each individual connection path normally includes an amplifier with offset and linearity adjustments to compensate for pixel to pixel sensitivity and emissivity variations in both the detectors and emitters. Advanced IR sensors are usually composed of a number of rows of pixel elements rather than a single row. The addition of many more pixel elements on each detector results in a very large number of output connections. Such interconnections are often a source of yield loss in manufacture and a cause of failure in the field. Therefore, these advanced sensors often abandon parallel pixel outputs in favor of a serial shift register output for each row of pixels. Connecting these serial outputs to an array of LEDs would require a complex arrangement such as a monolithic LED substrate that would include serial to parallel convertors or a complex hybrid substrate that packaged multiple single line LED arrays on a superstrate with individual serial to parallel convertor chips. Cost effective LED arrays of this type are beyond the current state of the art, so a different type of display is needed. The typical solution, shown in FIG. 2, has been to digitize and store, in a digital video memory 70, the multiple parallel or serial outputs 72 of the multiline IR sensor 74. The video memory 70 is then read out in a serial fashion in order to drive a CRT display 76 or read out in a parallel fashion to drive a hybrid array of linear LED elements 78. The torsion beam DMD may be used for the embodiments described below. The torsion beam DMD pixel, as shown in FIG. 3 and FIG. 4, generally consists of a thick reflective beam 50 suspended over an air gap/spacer 52 and connected to rigid supports or support layer 54 by two thin torsion hinges 56 that are under tension. When an address electrode 58, underlying the beam 50, is energized, the torsion hinges 56 are twisted and the beam 50 rotates about the axis of the two hinges 56 coming to rest on an underlying landing electrode 60. The structure is generally formed on a substrate 62 of material such as silicon. FIG. 4 shows several hinge options which may be used in the torsion beam DMD. A solution to the advanced IR system display problem is presented in a first preferred embodiment, shown in FIG. 5. A DMD chip 28 may be fabricated such that the architecture of the chip is essentially the reverse of the IR sensor device 30 architecture. A number of analog serial shift registers 32 may be fabricated on the chip 28, or, if desired, may be external to the DMD chip. Their number and length match the number and length of the output shift registers on the IR sensor device 30. Each of these shift registers 32 feeds, preferably through serial to parallel convertors 34, the same number of DMD pixel elements 36 as fed the corresponding shift register on the IR device 30. The individual DMD pixels 36 may be arrayed (except perhaps for an overall scale factor) as the input IR device 30 pixels were arrayed. Thus, a visible (or other wavelength) light source reflected from the DMD will create an illumination pattern nearly identical to that incident on the sensor 30 in the IR spectrum. In a second preferred embodiment, the reproduced image may be projected onto and viewed in reflection from the backside 18 of the scan mirror 14, shown in FIG. 1. The operation of the conventional IR system may be recreated without the expense of either a digital scan convertor or a complex LED assembly. Intensity can be encoded in this reflected image in several ways. One method is analog modulation. If the outputs of the DMD's analog shift registers/serial to parallel convertors 32,34 are connected directly to the addressing electrodes beneath the pixels 36, then the deflection of each pixel 36 will be proportional to the analog signal and hence proportional to the input IR light intensity. This degree of DMD tilt is commonly used to achieve analog light modulation through the use of dark field optics. In the dark field system, the DMD may be aligned so that light reflected from a fully tilted DMD pixel falls completely within the aperture of the display optical system. The optical system may be designed with a limited aperture so that light reflected from an undeflected pixel falls completely outside the aperture. Obviously, intermediate positions reflect an intermediate amount of light into the optical aperture and hence create an intermediate level of brightness in the plane of the projected image. Another method to encode intensity is pulse width modulation. The accuracy of an image recreation based on analog modulation depends on a number of factors including the linearity and uniformity of the DMD pixel tilt angle and proper apodization of the projection optical system. Using pulse width modulation with the dark field optics described above eliminates both of these concerns. In a third preferred embodiment, shown in FIG. 6, instead of driving the DMD pixels 36 directly from the outputs of the shift registers/serial to parallel convertors 32,34, a ramp comparator amplifier 38(or any comparator type circuit that provides the same functionality) may be placed between each shift register/serial to parallel convertor 32,34 output and the corresponding DMD 36 address electrode. As shown in FIG. 7, at the end of each serial to parallel conversion three things happen: the analog outputs of the shift register/serial to parallel convertor 32,34 (FIG. 6) are placed on one input 40 of a comparator at each DMD pixel 36 location, a pixel driver amplifier 42 at each pixel 36 location may be switched on so as to drive each pixel 36 to its fully tilted position, and a voltage common to all of the other comparator inputs 44 begins to ramp from the lowest detectable shift register signal level to the highest detectable shift register signal level. The rate of this ramp is such that it reaches its maximum voltage at the completion of the line time. The comparator 38 outputs are connected to the pixel driver amplifier 42 inputs so that when each comparator 38 toggles, the associated pixel driver amplifier 42 output falls to zero and the associated DMD pixel 36 returns to its untilted state. Thus, the amount of time a given pixel spends tilted during a single line time (and hence the amount of light it provides to the displayed image during that line time) are directly proportional to the voltage delivered to that pixel 36 by the shift register/serial to parallel convertor 32,34. Either of these modulation schemes can be used to emulate the operation of a digital scan convertor in those instances where a separate display (i.e. one that does not rely on the scanning provided by the backside of an input scan mirror) is desired. As an example of this scan convertor functionality, the DMD may be imaged onto the top row of an area array CCD (or other solid state) detector. At the end of each line time the charge generated in the CCD may be shifted down one row in a systolic fashion. This sequence proceeds throughout the video frame time until the entire input image is stored in the CCD. At this time the CCD can be read out directly into the video of CRT or other display system. The DMD may be imaged onto the CCD in many different ways. Multiple DMDs and CCDs may be combined and their outputs manipulated electronically to accomplish complex optical processing tasks. Conversely, a linear DMD modulator can be used to create a display from imagery stored, for example, in CCD or in a digital frame memory. The serial analog data stream, which may be derived directly from the CCD or from the frame memory through a digital to analog convertor, is in put to the analog shift register that was originally described as being fed from the IR detector. A scanning mirror may then be driven in the same manner as a conventional IR system scanning mirror would be driven, and the image, projected line by line from the DMD is viewed in reflection from the mirror surface. In cases where digital data is easily available or where digital operation is deemed preferable, the DMD is easily utilized in a digital mode. This digital mode requires the fabrication of a unique DMD, similar in construction to those described above. For a digital data stream of N bits of resolution, a linear DMD with N input shift registers would be fabricated. Each shift register would represent a different digital resolution bit, from the most significant bit to the least significant bit. Each shift register would directly address its own line of DMD elements. Each line of elements would be activated at the beginning of each line time. The most significant bit line would be turned off half way through the line time, the next most significant, one quarter through the line time, the third, one eighth, and so on. In this way each pixel in each line of the image would be illuminated by N pixels, and each of those pixels would provide illumination for a time period proportional to its weight in the binary sum. Thus the total N bit binary light amplitude would be provided. A simpler implementation of this binary illumination weighting can be provided through spatial weighting as opposed to temporal weighting. As above, an N bit binary data stream drives N input shift registers. However, in this scheme each pixel along the shift register is spatially weighted. For example, in a system with seven bits of resolution the most significant bit shift register would drive 64 pixels at each DMD location. The next most significant bit 32 pixels. The next 16, and so on. This method requires no sub-line time temporal resolution, since a given percent illumination weight is provided by a percent of the reflective area utilization rather than by a percent of the available time utilization. These two methods of binary illumination weighting (temporal and spatial) may be combined. The result is higher resolution pixels due to the increased dynamic range of the combined methods. The DMD may be used in many different embodiments to provide a display device. The DMD can be configured, along with an appropriate optical system, to provide a scan convertor function with a small size, weight, cost, and power budget. Additionally, the resultant DMD system is capable of displaying any stored digital image or analog data stream, regardless of its source, for viewing. Several preferred embodiments have been described in detail hereinabove. It is to be understood that the scope of the invention also comprehends embodiments different from those described, yet within the scope of the claims. For example, the input device is described as an IR sensor, however, the input could come from many different sources, such as a television signal or a computer. Dark-field optics are discussed in relation to the use of this system, however, Schlieren optics or other appropriate optics systems may be used. Similarly, the DMD may be used to display the image directly or onto another detector for possible manipulation prior to display. Words of inclusion are to be interpreted as nonexhaustive in considering the scope of the invention. While this invention has been described with reference to illustrative embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of the invention, will be apparent to persons skilled in the art upon reference to the description. It is therefore intended that the appended claims encompass any such modifications or embodiments.

Claims: A method of digitally modulating a DMD system comprising: receiving at least one data stream from an input device (30); and addressing and activating one or more DMDs (36); characterized in that the method further comprises a) receiving at least one data stream of data of N bits of resolution per pixel from said input device (30); b) inputing said data stream into N input shift registers (32); c) addressing at least N binary weighted fixed size DMDs (36) with said shift registers (32), with output of said DMDs (36) being combined to create one pixel of an image; and d) activating said DMDs (36) based on said data stream. The method of claim 1, characterized in that the or each of said activated DMDs (36) is turned off individually such that each of said activated DMDs (36) is left on for a period of time equal to a line time multiplied by a weighting for that DMD (36). The method of claim 2, characterized in that the DMDs (36) are weighted by time. The method of claim 2 or claim 3, characterized in that weighting of said weighted DMDs (36) is based on said resolution per pixel such that a most significant bit is weighted 1/2, a next most significant bit is weighted 1/4, continuing weighting to a least significant bit which is weighted 1/2 N . The method of any of claims 2 to 4, characterized in that said weighting is by a predetermined number of DMDs (36) per bit, whereby the DMDs (36) can be weighted by number of DMDs (36) per bit. The method of any of claims 2 to 5, characterized in that said weighting is such that one or more of said DMDs (36) reflect different maximum amounts of light, whereby the DMDs (36) can be weighted by reflecting area and/or reflectance of an individual DMD (36). The method of any preceding claim, characterized in that a detector manipulates the image from said DMDs (36), whereby said image is output in a different order from said detector than said image was input to said detector. The method of any preceding claim, characterized in that addressing one or more DMDs (36) comprises addressing 2 N DMDs (36) with said shift registers (32), whereby the shift register (32) for a most significant bit addresses 2 N-1 DMDs (36), the shift register (32) for a next most significant bit addresses 2 N-2 DMDs, continuing this addressing scheme to a least significant bit whose shift register addresses 2 N-N DMDs. The method of any preceding claim, characterized in that said DMD (36) is imaged onto a solid state detector. The method of claim 9, characterized in that said detector is a CCD. The method of any preceding claim, characterized in that said DMD (36) is imaged onto the backside of a scan mirror (18). The method of claim 11, characterized in that an image from said backside of said mirror (18) is optically projected to create an image nearly identical to said image received at said input device (30). The method of any preceding claim, characterized in that said input device (30) is a computer. A digitally modulated DMD system comprising: means to receive at least one data stream of an input device (30); and DMDs (36) addressed by said receiving means; characterized by a) N shift registers (32) to receive at least one data stream of data of N bits of resolution from an input device (30); and b) N binary weighted fixed size DMDs (36) adressed by said shift registers (32); wherein said DMDs (36) combine to create one pixel of an image per data stream. The system of claim 14, characterized in that it further comprises a serial to parallel convertor (34) connected to said shift register (32). The system of claim 15, characterized in that it further comprises a comparator (38) connected to the output of the serial to parallel convertor (34) and to a ramp voltage. The system of claim 16, characterized in that the comparator (38) is a ramp comparator. The system of any of claims 14 to 17, characterized in that said DMDs (36) are imaged onto a solid state detector. The system of claim 18, characterized in that said detector is a CCD. The system of claim 18 or claim 19, characterized in that said detector manipulates the image from said DMDs (36), whereby said image is output in a different order from said detector than said image was input to said detector. The sytem of any of claims 14 to 20, characterized in that said DMD (36) is a torsion beam DMD. The system of any of claims 14 to 21, characterized in that said DMD (36) is imaged onto the backside of a scan mirror (18). The system of any of claims 14 to 22, characterized in that the input device (30) is an IR sensor. The system of any of claims 14 to 24, characterized in that the input device (30) is a computer. The system of any of claims 14 to 24, characterized in that said shift register (32) and said DMDs (36) are on a monolithic integrated circuit. The system of claim 25, characterized in that the monolithic integrated circuit includes the comparator (38). The system of any of claims 14 to 25, characterized in that the system is in the form of a scan converter.

Inventors: SAMPSELL JEFFREY B, SAMPSELL, JEFFREY B.
Applicants: TEXAS INSTRUMENTS INC, TEXAS INSTRUMENTS INCORPORATED
Application year: 1992
Main classification: G09G3
IPC codes: G02B26, H04N3, G09G3, H04N5