Abstract:
The visibility of an image projected onto a background of spatially and temporally varying brightness can be optimized by dynamically adjusting the brightness of the projected image. The brightness of the background is detected on a zone-by-zone basis and the brightnesses of the corresponding portions of the projected image are adjusted accordingly.

Description:
RELATED APPLICATION 
     This application is a continuation of U.S. patent application Ser. No. 08/994,289, filed Dec. 17, 1997, now allowed. 
    
    
     BACKGROUND OF THE INVENTION 
     See-through displays such as head-up instrument displays, head- or helmet-mounted displays, and related devices allow the user to simultaneously view the physical world and an electronically-generated and optically projected image of text, graphics, or other video material. Brightness variations in the physical scene often make it difficult to achieve proper brightness in the projected image so that both the physical scene and the projected image are properly visible throughout the scene. 
     Consider for instance the situation where a pilot&#39;s field of view encompasses sunlit regions outside of the cockpit as well as more dimly illuminated regions within the cockpit. If the projected image overlays all of these regions, a single global adjustment of the projected image brightness will be inadequate. If such a global adjustment were set so that the projected image is visible against the bright portions of the physical scene, the dimmer portions of the physical scene will be very difficult to see, if not masked altogether. If, on the other hand, the projected image brightness is adjusted to provide adequate brightness against the dimly illuminated regions, the projected image will not be visible against the brightly-illuminated portions of the physical scene. An optimal solution requires that the brightness of each display pixel or zone be adjusted on the basis of the brightness of the corresponding portion of the physical scene or background. Since the brightness of the pixels or zones of the physical scene can vary dynamically across the entire scene, the brightness of the corresponding portions of the projected image must be adjusted in real time to account for such changes. 
    
    
     BRIEF DESCRIPTION OF DRAWINGS 
     FIGS. 1 and 2 are diagrams of the optical paths of see-through displays; 
     FIG. 3 is a diagram of a combined image display and detector assembly having alternating emissive and photodetector devices; and 
     FIGS. 4 and 5 are schematic diagrams of circuits for controlling the brightness of a projected display. 
    
    
     DESCRIPTION OF THE INVENTION 
     The optical paths for two arrangements for a see-through display are illustrated in FIGS. 1 and 2. In both figures, the observer views the outside physical scene (or some other background image) through a combiner optics assembly  10  having first and second partially-mirrored surfaces  20  and  30 . The first surface  20  is oriented at right angles to the direction of light from the physical scene and, optionally, can be curved. The second surface  30  is flat and oriented at a  450  angle. The partially-mirrored surfaces  20  and  30  both partially reflect and partially pass light. The second surface  30  functions as a beam splitter, where light passes through the surface  30  on one axis  32  and is reflected by the surface on a reflective axis  34 . Light emanating from the physical scene (depicted by the solid line) passes through the first surface  20  and travels to the angled second surface  30 . The second surface  30  allows a portion or sample of this light to pass through and travel to the observer&#39;s eye, while the partial mirroring on the second surface  30  reflects the remaining portion of this light. 
     The light initially reflected by the second surface  30  travels upwardly (in the arrangement shown in the figures) to a fully-silvered mirror  40 , which deflects the light towards a relay optics assembly  50 . The relay optics assembly  50  passes the light to either a combined image detector and image display assembly  60  (FIG. 1) or a detector/display assembly  70  of discrete components (FIG.  2 ). The detector/display assembly  70  has a beam splitter  72 , an image display device  74 , and an image detector  76 . The mirror  40  could be eliminated if the relay optics assembly  50  were coincident with the path of reflection from the second surface  30 . 
     As shown by the solid line, a portion of the light emanating from the physical scene travels directly to the combined image detector and image display assembly  60  of FIG.  1 . Simultaneously, the combined assembly  60  projects light containing an image in the opposite direction (depicted by the dashed line), which travels back through the relay optics assembly  50 , after which it is deflected by the fully-silvered mirror  40  and again deflected by the second surface  30 , until it reaches the first surface  20 . There, the partial mirroring of the first surface  20  reflects a portion of the projected image back to the observer&#39;s eye, combining with light from the physical scene. 
     In FIG. 2, the beam splitter  72  deflects the light (from the physical scene, the solid line) towards the image detector  76 . Simultaneously, the image display device  74  generates a projected image (depicted by the dashed line) which reaches the observer in the same fashion as the arrangement of FIG.  1 . 
     The brightness of the projected image is controlled by the brightness of the physical scene. In the case of the arrangement of FIG. 1, the detector and image display device are integrated. Depending on the degree of control desired, individual detector elements can be provided for each emissive device or a detector element can be provided for a group of emissive devices proximate to the element on any desired ratio of detector elements to emissive devices, e.g., 1:2, 1:4, 1:16, etc. Alternatively, instead of pixels, the brightness can be sensed in quadrants, sectors, zones, or other convenient subdivisions of the display. 
     One arrangement for the combined image detector and image display assembly  60  of FIG. 1 is shown in FIG.  3 . The assembly has alternating emissive (or emitter) and photodetector devices, denoted by the letters “E” and “D,” respectively. Other deployments of emissive devices and photodetectors could be employed as will readily occur to those skilled in the art. The emissive devices can be electroluminescent, laser light emitting, or of another type as suits the application. Similarly, any suitable photodetector may be employed. 
     The brightness of the light received by a photodetector is used to control the desired brightness of an adjacent emitter. The partial schematic of FIG. 4 shows how the devices can be interconnected for a single pixel. For clarity, the video control circuitry and addressing circuitry are omitted. Conventional circuitry may be employed here as suits the particular application. 
     The detector D receives light, causing it to generate a voltage proportional to the light&#39;s brightness, which it provides to an input of an algorithm F. The video control circuitry (not shown) provides a nominal brightness command B for the pixel, appropriate to the image that will be projected, to another input of the algorithm F. If the relationship between the input brightness B and the necessary output brightness B′ is linear, then one might employ the following algorithm: 
     
       
         
           B′=B+kD 
         
       
     
     The constant k is adjusted to provide suitable performance. Other algorithms including non-linear relationships could be employed. 
     When the detector and the image display device are separated (FIG.  2 ), the output of the image detector  76  must be provided to the image display device  74 . If the switching is fast enough, the detector output can be sent directly to the image display device  74 , providing real time modification of the brightness level. 
     One arrangement for controlling the brightness of the image display device  74  is shown in the schematic diagram of FIG.  5 . The video brightness information (and perhaps other information related to the video image) is provided to a video brightness algorithm  100 . An address generator (not shown) provides raster scanning or other address information to both the image display device  74  and the image detector  76 . Depending on the capabilities of the components, the algorithm  100  can use brightness information from the image detector  76  directly on a pixel-by-pixel basis. Alternatively, the brightness information from the detector  76  can be stored on frame-by-frame basis in a RAM and then read out one frame behind. As noted above, the resolution of the brightness information can be obtained on a quadrant or other basis. In such a case, the switching demands may be considerably lessened. 
     If the image only occupies a relatively small portion of the display, the process of adjusting the brightness could be limited to a subset of the background, perhaps with some margin to create a border.