Abstract:
A projected image is overlaid on an actual view of an object, the image being identical to the actual view in size, shape and scale, and being perfectly registered with the actual view, e.g, as to position or location, and, accordingly being substantially parallax-free. The image may be of different “color” than the actual view, i.e., one may be a view or image of light in a visible wavelength region, band or bandwidth, and the other may be a view or image in a non-visible wavelength region, band or bandwidth, such as a view or image of infrared light or of x-ray radiation. The infrared view or image may be converted to a view or image in the visible bandwidth for viewing by a person. The image mentioned above may be a real image of an actual view of an object, scene or the like. Alternatively, the image may be an image other than of an actual view of an object, for example being a previously-obtained image along a same or similar point of view as the actual view of the object, scene, or the like. As another example, the image may be a computer-generated image based on previously-obtained data regarding the object, scene, or the like. The image may be an image obtained by a camera or some other means and then delivered, e.g., as by projection via a projector and conjugate optics, for superpositioning with respect to the real or actual view for viewing by a user.

Description:
This is a continuation-in-part of U.S. Application No. 08/845,520, filed Apr. 24, 1997, abandoned, which in turn claims the benefit of U.S. Provisional Application No. 60/016,183, filed Apr. 24,1996. 
     CROSS-REFERENCE To RELATED PATENT APPLICATIONS 
     Reference is made to copending U.S. patent application Ser. No. 08/295,383, filed Aug. 24, 1994, entitled “Optical system for a head mounted display using a retroreflector and method of displaying an image,” to copending U.S. patent application Ser. No. 08/383,466, filed Feb. 3, 1995, entitled “Head mounted display and viewing system using a remote retroreflector and method of displaying and viewing an image”, and to PCT patent application No. US95/07306, filed Jun. 7, 1995, entitled “Head mounted display and viewing system using a remote retro-reflector and method of displaying and viewing an image” (International Publication No. WO 96/06378, published Feb. 29, 1996), the entire disclosures of which hereby are incorporated by reference. 
     Reference is also made to pending U.S. Application Ser. No. 08/845,520, filed Apr. 24, 1997, which in turn claims the benefit of U.S. Provisional Application 60/016,183, filed Apr. 24, 1996—both of these are hereby incorporated by reference. 
    
    
     TECHNICAL FIELD 
     This invention relates generally, as indicated, to a projection display system with conjugate optics and image enhancement characteristics. 
     BACKGROUND 
     Several prior conjugate optics systems have been employed for displaying and viewing images. The systems may be head mounted in whole or in part or may be otherwise positioned relative to a viewer. Such systems use conjugate optics, for example, including a retroreflector which may be proximate or relatively remote from the viewer. 
     In conventional night vision goggles, which is an example of a device in which there is enhancement of an image which is otherwise difficult to view, electromagnetic energy input, such as that in the visible wavelength range (sometimes referred to as visible wavelength band or bandwidth) or in another wavelength range, such as infrared, is received by a detector and is enhanced, e.g., by converting the infrared radiation to visible light that can be seen by the user. There are a number of problems with such devices. One problem is the blinding light that is presented to the eyes of a viewer, say when a bright light, such as that from a flare, is provided as an input. Another problem is that such devices usually do not take advantage of the ambient light conditions which would otherwise allow the user to see an image even without enhancement. A further disadvantage with such systems is the relatively narrow field of view over which a user can see while wearing such device. 
     Accordingly, there is a strong need in the art to improve the enhancement techniques for viewed images. 
     There also is a strong need in the art to overcome the aforementioned problems previously encountered in night vision goggles and other vision enhancement devices. 
     SUMMARY 
     With the foregoing in mind, the present invention provides improvements in techniques to enhance images for viewing or the like. 
     In an embodiment of the invention a projected image is overlaid on an actual view of an object, the image being identical to the actual view in size, shape and scale, and being perfectly registered with the actual view, e.g; as to position or location, and, accordingly, being substantially parallax-free. The image may be of different “color” than the actual view, i.e., one may be a view or image of light in a visible wavelength region, band or bandwidth, and the other may be a view or image in a non-visible wavelength region, band or bandwidth, such as a view or image of infrared light or of x-ray radiation. The infrared view or image may be converted to a view or image in the visible bandwidth for viewing by a person. 
     The image mentioned above may be a real image of an actual view of an object, scene or the like. Alternatively, the image may be an image other than of an actual view of an object, for example being a previously-obtained image along a same or similar point of view as the actual view of the object, scene, or the like. As another example, the image may be a computer-generated image based on previously-obtained data regarding the object, scene, or the like. The image may be an image obtained by a camera or some other means and then delivered, e.g., as by projection via a projector and conjugate optics, for superpositioning with respect to the real or actual view for viewing by a user. 
     The present invention may use the various conjugate optics projection and display systems of the above-mentioned patent applications. Additionally, the invention involves the use of a camera type device, such as a charge coupled device (CCD), video camera, or the like, which is mounted in conjugate relation to the projector of a conjugate optics system of the type disclosed in the above-mentioned patent applications. The system allows the user to view the actual scene, say as by looking directly at a real life scene (not necessarily one that is generated by a projection or other display system). Meanwhile, the camera device can photograph the same scene and delivers the obtained image via a projector and conjugate optics display apparatus for viewing by the user. The image seen directly by the user and the image which is photographed and is projected via the conjugate optics are superimposed in precise registry for viewing by the user. Therefore, when there is adequate illumination to view, for example, the real life scene directly, the user will do so while peering at that scene. However, when there is inadequate illumination for direct visual viewing, the camera, projector and conjugate optics system presents the same view to the user. The user need not even know that the system of the invention is enhancing an image while the user is viewing the image; that is, the viewer may not even realize whether the scene being seen is that which the user is directly viewing or is one developed by the camera, projector and conjugate optics system. 
     One or more of these and other aspects, objects, features and advantages of the present invention are accomplished using the invention described and claimed below. 
     To the accomplishment of the foregoing and related ends, the invention, then, comprises the features hereinafter fully described and particularly pointed out in the claims. The following description and, the annexed drawings set forth in detail certain illustrative embodiments of the invention. These embodiments are indicative, however, of but a few of the various ways in which the principles of the invention may be employed. 
     Although the invention is shown and described with respect to certain preferred embodiments, it is obvious that equivalents and modifications will occur to others skilled in the art upon the reading and understanding of the specification. The present invention includes all such equivalents and modifications, and is limited only by the scope of the claims. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     In the annexed drawings: 
     FIG. 1 is a schematic illustration of a conjugate optics projection display with image enhancement in accordance with an embodiment of the invention; 
     FIG. 2 is a schematic view of the system of FIG. 1 used as a head mounted system for a viewer in a vehicle, such as an automotive vehicle, aircraft, watercraft, etc.; 
     FIG. 3 is a schematic illustration of a system in accordance with an embodiment of the invention used in an aircraft or other vehicle and able to receive image data from a remote source; and 
     FIG. 4 is a schematic illustration of an alternate embodiment of the system of the invention using a relatively remotely spaced retroreflector and beamsplitter. 
    
    
     DESCRIPTION 
     In the description below, reference is made to the drawings. 
     One example of the invention for enhancing a poorly lighted object or scene for a purpose similar to night vision devices, is shown in FIG.  1 . In FIG. 1 a viewer  10 , which is represented by the illustration of an eye:  10   e , is intending to view a scene or object  11 . The real life or real world scene  11  may be viewed by the viewer  10  by looking directly through a beamsplitter device  12 . The conjugate optics projection display with image enhancement  13  includes the beamsplitter  12 , a second beamsplitter  14 , a camera  15 , a projector  16 , the camera and projector being coupled together by a connection  17  representing wires, appropriate signal processing circuitry or the like, and a retroreflector  18 . 
     Consider first the direct (or real world) observation/view of the object  11 . The object  11  is observed by the viewer  10  through the beamsplitter  12 . That portion of the light traveling along the light path  20  (hereinafter reference numeral  20  may be used to represent the light path or the light in that light path; and similar convention will be used for other lights and light paths described below) from the object  11  transmitted by the beamsplitter  12  goes directly to the viewer&#39;s eye  10   e . The eye may or may not “see” this light, depending upon the wavelength of the light, e.g., is it in the visible wavelength region or is it in the infrared, ultraviolet or some other wavelength, and possibly also depending on its intensity, e.g., is the intensity so small that it-cannot be perceived by the eye  10   e.    
     That portion of the light  20  that is reflected by the beamsplitter  12  is directed upward along light path  21  to the second beamnsplitter  14 . A portion of the light  21  reaching the second beamsplitter  14  is reflected along light path  22  to the camera  15 . The remaining portion of the light  21  reaching the second beamsplitter is transmitted through to the conjugate optics projector  16  and can be considered lost. 
     The camera  15  may be a conventional video camera. It may be a CCD device. (Reference to camera, video camera and CCD may be used interchangeably herein.) In an embodiment the camera  15  is a CCD which is tuned to be sensitive to a desired bandwidth. In an embodiment that bandwidth is the infrared portion of the electromagnetic energy spectrum or a part of that infrared portion. 
     The image detected by the CCD  15  is electronically transmitted by the connection and/or additional signal processing and/or other circuitry  17  to the projector  16 . The projector  16  projects an image into the conjugate optics portion  30  of the system  13 . Such projected image is in the visible bandwidth and is of “adequate” brightness. Adequate brightness may be a brightness that is sufficient to be perceived by the viewer  10  under specified ambient light conditions, such as low light level; may be such that the image can be perceived under some other light conditions; may be such that the image will not interfere with the image of the object  11  that is directly viewed by the viewer  10  through the beamsplitter  12 , and so forth, as may be desired. 
     The light projected by the projector  16  is shown at  31 . A portion  32  of the projected light  31  is transmitted through both. the second beamsplitter  14  and the beamsplitter  12 , reaching the retroreflector  18 . The retroreflector  18  returns the light along a conjugate path as light  33  to the beamnsplitter  12 , where a portion  34  of the light is reflected to the user&#39;s eye  10   e . The drawings hereof are only schematic. Therefore, although the light paths  32  and  33  are shown at slightly spaced apart locations, it will be appreciated that those light paths are conjugate and, therefore are identical (but in opposite directions) between the retroreflector  18  and the beamsplitter  12 . 
     The camera  15  and the projector  16  are properly adjusted and aligned so that the projector image of the object  11  returned by the retroreflector  18  and reflected by the beamsplitter  12  to the viewer will be superimposed on the actual view of the object  11  seen by the viewer  10 . 
     The system  13  can be used to intensify an image simply by using the projector  16  to project an image that has the same wavelength characteristics as the wavelength characteristics of the actual object  11 , i.e., the same as the wavelength(s) of the light emanating from the object  11 . 
     Alternatively, the system  13  can be used to superimpose a projected image of one wavelength upon an actual view of a second wavelength. This would be an exemplary case for a night vision type of device or system. The system  13  has several advantages over traditional night vision devices. 
     One of those advantages is that in the traditional night vision system there is no direct optical link between the object and viewer; but such a direct link is possible using the system  13 . In a traditional night vision system a camera, such as a CCD, views the object  11  and electronically conveys the image to a direct view image display device. The viewer, such as viewer  10 , for example, would only see the image generated by the display; the viewer cannot directly view the object. Therefore, a failure in the conventional night viewing system may result in the viewer not seeing anything; or a bright flash may result in the viewer not seeing anything, either due to the blinding caused by the flash as amplified by the night vision system or by a shuttering of the night viewing system to prevent such blinding, but unfortunately reducing or eliminating the image for viewing. In contrast, in the present invention a direct optical view of the object  11  is provided through the beamsplitter  12 , while at the same time the projected image from the projector  16  is projected via the conjugate optics system to the viewer. This avoids the so-called “head-in-the-bucket” problem. 
     Another disadvantage of conventional night vision CCD devices, is that they are deliberately set to be extremely sensitive to low levels of light. consequently, if a high light level were to occur, the CCD would become saturated and “blinded”. When this happens, the electronic signal to the image display of such device provides no definitive image. If the CCD were to become severely oversaturated, it could be permanently damaged. To prevent the latter, a protective device, such as a high speed automatic shutter, sometimes is employed to sense high light levels and to shutter the light before it reaches the CCD as to do damage thereto. The result, though, is that no image signal is generated by the CCD, and the user would see nothing. However, in the present invention, a high illumination level of the object  11  may saturate the camera  15 , but at the same time that high illumination provides illumination of the object for the user  10  to view the object  11  directly. Also, a protective shutter for the camera may be employed without interfering with the direct view of the object. 
     The invention hereof may be used for a number of applications. One use is for night vision devices to enhance vision or to enable vision even at night when visible light is at a minimum. Another use is as a night driving aid to facilitate seeing while driving at night. Using the camera  15  of the invention to detect images that are beyond the range of the conventional headlights of an automobile, bus or truck, for example, those images may be projected by the projector  16  for viewing simultaneously with the driver&#39;s direct view of the road ahead able to be seen by the headlights. Safe driving can be enhanced because even though a driver may be “out-driving the headlights”, the actual view seen by the driver may be beyond the range of the headlights. Another use of the invention is in aviation allowing a pilot to land at or to fly carefully to an area that is in relative darkness, such as in a field in an emergency situation, at a small airport which does not have runway lights or at an airport where there may be a power outage preventing illuminating of the runway. These are but several of many uses for the invention. 
     In an exemplary use of the invention for night driving or flying, appropriate head gear  40 , such as a helmet, support straps, or other means may be worn by the viewer  10  to support the beamsplitters  12 ,  14 , CCD  15  and projector  16 . The retroreflector  18  may be mounted on the dashboard  41  of the vehicle  42 , such as an automotive vehicle, or on another surface of an aircraft. The viewer  10  may look through the beamsplitter  12  and out through the windscreen  43  of the vehicle  41  to see directly an image of the object  11 . The conjugate optics projection display with image enhancement  13 , meanwhile picks up an image of the object by the CCD  15  and projects the image by the projector  16  through the beamsplitters  14 ,  12  toward the retroreflector  18  for reflection via the beamsplitter  12  to the viewer as was described above. 
     The camera  15  may be of a type which will pick up a view that is as wide as the view expected to be seen by the eye  10   e  of the viewer  10 , and the conjugate optics system  13  of the invention is able to present a full view to the user  10 , e.g., in the manner described in the above-mentioned patent applications. Therefore, the user is not restricted to a narrow field of view as in prior night vision devices; rather a wide field of view may be seen by the viewer, which view may be essentially the same as the field of view that is directly seen. By mounting the system  13  on the head or with respect to the head of the viewer, the field of view seen directly by the viewer and the image which is photographed by the camera  15  and projected by the projector  16  to the viewer can be always superimposed even as the head of the viewer is moved. 
     Another way in which the invention may be used is to photograph the object  11  or the scene containing the object  11  at a time different than the time that the object is being observed by the viewer  10  in real time. The previously-obtained image may be played back to the viewer for viewing simultaneously with the real or actual view being observed through the beamsplitter  12 . Alternatively, the data provided to the projector  16  may be provided from a remote source, such as by radio signals, radar information, optical signals or the like. Consider the example illustrated in FIG. 3 hereof. 
     In FIG. 3 data is provided to the projector  16  to project an image for viewing by a pilot viewer  10  in an aircraft  50 . The data may be provided by a ground station  51 , such as one at an airport. The ground station  51  may include a radio  52  and an antenna  53  or some other device for transmitting the data to the viewer  10 ; and the aircraft  50  may include a radio  53  or other device for receiving the data. The data may represent the view that is expected to be seen by the pilot viewer  10  of the airport when on a landing approach even though a direct view of the airport is obscured by clouds. Various head tracking devices also may be used in conjunction with the system  13  used in the way illustrated in FIG. 3; the head tracking devices can provide a filtering of the data received from the ground station  51  or can provide information, say by the radio  53 , to the ground station to present appropriate data which would be expected to be seen by the pilot viewer  10  based on the direction that the pilot viewer is “looking” at a particular point in time. The data provided to the projector may take into consideration not only the direction that the pilot viewer is looking, but also it may take into consideration altitude, direction of approach to the airport, etc. Therefore, using the system  13  in combination with data provided from other than a camera that is simultaneously viewing the same object as the pilot, driver, etc. is or is trying to view, that individual can be presented with visual data that will facilitate and will enhance safe operation of the vehicle, whether automotive, aircraft, watercraft, spacecraft or other vehicle. The system  13  as used in an aircraft or other vehicle as is depicted in FIG. 3 also may employ the CCD  15  part of the system for the above-described function, such as to enhance the view in darkness, fog, or other vision obscuring condition. 
     In the conjugate optics projection system portion  13   a  of the system  13  the quality (resolution) of the projected image from the projector  16  depends on the resolution of the display of that projector and the resolution of the retroreflector  18 . Usually the resolution of the display governs the overall resolution of the projected image. However, if the angular resolution of the retroreflector is large due to larger corner cube features included therein or the locating of the retroreflector very close to the viewer&#39;s eye  10   e , the image resolution will be limited by the retroreflector  18 . In the embodiment of FIGS. 1-3, the beamsplitter  12  and the retroreflector  18  are relatively close to the viewer&#39;s eye  10   e , which could degrade the resolution or result in poor resolution of the projected image. 
     In FIG. 4, however, an alternative embodiment is illustrated in which a third beamsplitter  60  and the retroreflector  18 ′ are located relatively farther from the viewer&#39;s eye  10   e  than is the case in the embodiments of FIGS. 1-3. Primed reference numerals designate parts in the system of FIG. 4 that generally correspond to parts identified in the earlier figures by unprimed reference numerals. In the system  13 ′ the CCD  15 ′, projector  16 ′ and beamsplitters  12 ′ and  14 ′ are all located proximate to the viewer&#39;s eye  10   e , e.g., which would be the case for a head mounted system using head gear  40  mentioned above. Also, in the system  13 ′ of FIG. 4, the beamsplitter  60  and the retroreflector  18 ′ are a substantial distance away, e.g., at the aircraft windshield (windscreen) relative to the CCD  15 ′, projector  16 ′ and viewer  10 ′. 
     The arrangement of parts of the embodiment of system  13 ′ of FIG. 4 not only improves the angular resolution of the retroreflector  18 ′, but also permits the use of multiple remote retroreflector/beamsplitter combinations. For example, retroreflector  18 ′ and beamsplitter  60  combinations could be place at the side windows of an aircraft, thereby providing night vision capability in any direction that the pilot points his or her head. The direct view feature still is retained in this embodiment of the system  13 ′. 
     The beamsplitters shown in the various embodiments can be “tuned” for the application. Their transmission/reflection ratios can be adjusted for optimum performance. Further, the transmission/reflection ratios can be adjusted for specific wavelengths. For example, a beamsplitter can be tuned to be highly reflective in the infrared and highly transmissive in the visible bandwidth. 
     Also, narrow band filters may be used in various portions of the optical paths, in order to further enhance overall performance of the systems  13 ,  13 ′ and other equivalent embodiments of the invention. For example, with reference to the embodiment illustrated in FIG. 1, a color notch filter may be placed in the light path  20  between the object  11  and the beamsplitter  12 . The filter may be selected to filter out the wavelength(s) of light produced by the projector  16 . Thus the actual view of the object  11  seen by the viewer  10  has these wavelength(s) of light at least partially removed by the filter. However, the image projected by the projector  16  includes these wavelengths of light, thus “filling in” the combined actual view and view of the image seen by the viewer  10 . 
     With the foregoing in mind, then, an aspect of the invention relates to a device or a system for enhancing vision by imaging and projecting in a conjugate path an image separated either by spectral region or by computer generation from a direct view. 
     Another aspect is to use infrared, ultraviolet or other detection of object information and based on that information converting the same to image information in the visible range and projecting the information in superposition with a direct view of the object. 
     Another aspect relates to a system using a camera, such as a video camera, CCD or other camera, in a conjugate path with a projection device such that the electronic image from the camera falls at the same position as the object in space. 
     Other aspects include use of the above for infrared viewing, for night vision, for aircraft control, for input to a controlling individual of a vehicle, aircraft, or the like either by simultaneous projection of detected view while direct view is possible or by providing supplemental view from another source, such as a remote station, a prerecorded image, etc.