Patent Description:
Some nightvision systems function by receiving low levels of light reflected off of, or emitted from objects and providing that light to an image intensifier (sometimes referred to as I<NUM>). The image intensifier has a photocathode. When photons strike the photocathode, electrons are emitted into a vacuum tube, and directed towards a microchannel plate to amplify the electrons. The amplified electrons strike a phosphor screen. The phosphor screen is typically chosen such that it emits human visible light when the amplified electrons strike the phosphor screen. The phosphor screen light emission is coupled, typically through an inverting fiber-optic, to an eyepiece where the user can directly view the illuminated phosphor screen, thus allowing the user to see the objects.

Typical nightvision systems suffer from various lighting issues. For example, in some environments, there is not sufficient ambient lighting (such as light from the moon, stars, or other sources) to allow acceptable viewing of a scene using the nightvision system. Thus, often, external illuminators are used to add additional lighting. Typically, these illuminators emit infrared (IR) light that is not visible to a human user's naked eye, but is detectable by the nightvision system. However, these illuminators tend to emit a constant, set amount of light which can cause various issues in certain environments and scenarios. For example, a user moving from an outdoor environment to an indoor environment can result in a situation where the illuminator causes an unusually high amount of light to be captured by the nightvision system, which causes the scene in its entirety to be washed out. Alternatively, certain important portions of a scene may be washed out.

From <CIT> an enhanced vision system is known. The system includes a first optic subsystem and a transparent photodetector subsystem disposed within a common housing. The transparent photodetector subsystem receives the visible image exiting the first optic subsystem and converts a portion of the electromagnetic energy in the visible image to a signal communicated to image analysis circuitry.

From <CIT> a system compensating vignetting is known. The system includes an image capture system with a sensing efficiency that varies over a field of view of the image capture system to employ shaped illumination to compensate for the variation in the sensing efficiency. An illuminator is configured to illuminate the field of view of the image capture system with illumination shaped to have higher intensity where the sensing efficiency is lower, e.g., at the periphery of the field over view.

From <CIT> an interactive illumination for gesture and/or object recognition is known. The corresponding system is used for target illumination and mapping. The system includes a light source and an image sensor, where the light source is configured to: communicate with a processor, scan a target area within a field of view, receive direction from the processor regarding projecting light within the field of view on at least one target. The image sensor is configured to, communicate with the processor, receive reflected illumination from the target area within the field of view, generate data regarding the received reflected illumination; and send the data regarding the received reflected illumination to the processor.

One embodiment illustrated herein includes a nightvision system. The nightvision system includes an underlying device that is configured to provide output light in a first spectrum from input light received at the underlying device. A transparent optical device is optically coupled in an overlapping fashion to the underlying device. The transparent optical device is configured to transmit light in the first spectrum from the underlying device through the transparent optical device. The transparent optical device includes an active area of a semiconductor chip. The active area includes active elements configured to cause the underlying device to detect light from the underlying device and transparent regions formed in the active area which are transparent to the light in the first spectrum to allow light in the first spectrum to pass through from the underlying device to a user. The transparent regions are configured in size and shape to cause the transparent optical device to have a first transmission efficiency for light in the first spectrum. An image processor configured to process brightness maps produced using light detected by the first plurality of active elements is coupled to the transparent optical device. An illuminator is coupled to the image processor. The illuminator is configured to provide additional input light into the underlying device based on image processing performed by the image processor.

Embodiments illustrated herein are directed to using a transparent optical device in conjunction with nightvision equipment, such as an image intensifier (or other nightvision system). The transparent optical device includes light detection functionality that is able to detect brightness maps by detecting light produced by the nightvision equipment. Further, the transparent optical device is transparent to light produced by the nightvision equipment to allow light produced by the nightvision equipment to be transmitted to a user. Using the detected brightness map, embodiments can perform illumination functionality. In some embodiments, illumination functionality may be aided by using features such as object edge detection, object recognition, identifying regions of interest, etc. in a brightness map. In particular, automatic illumination functionality can be performed to improve contrast, and/or based on various items detected by the transparent optical device. Note that in some embodiments, the transparent optical device may further include display functionality for overlaying additional information on an image produced by the nightvision equipment.

Additional details are illustrated. Attention is now directed to <FIG>, where a specific example of a nightvision system is illustrated. In particular, <FIG> illustrates the PVS - <NUM> nightvision system <NUM>. In the example illustrated, the nightvision system <NUM> includes a housing <NUM>. As will be illustrated in more detail below in other figures, the housing <NUM> houses an image intensifier, a transparent optical device (see e.g., <FIG>, <FIG> and <FIG> and transparent optical device <NUM>), and various other components. The nightvision system <NUM> further includes an objective <NUM> which receives weak light reflected and/or generated in an environment. The objective <NUM> includes optics such as lenses, waveguides, and/or other optical components for receiving and transmitting light to an image intensifier, discussed in more detail below. The nightvision system <NUM> further includes an eyepiece <NUM>. The eyepiece <NUM> includes optics for focusing images created by the nightvision system <NUM>, including images created by an image intensifier and images created by a transparent optical device, into the eye of the user.

As discussed above, and with reference to <FIG>, modern ancillary functionality can be added to existing nightvision systems. <FIG> illustrates an image <NUM> including a heads-up display displayed on a nightvision image output from an intensifier tube. Some embodiments described herein are directed to implementing a heads-up display implemented by adding image overlay capabilities with a nightvision system, where the image overlay capabilities are added by using a transparent optical device including a display.

The heads-up display may display to the user, in or around the field-of-view of an environment, various pieces of information to create an augmented reality (AR) environment. Such information may include, for example, a navigational heading, the speed at which the user is moving, coordinates, communication messages (such as email, SMS, etc.), time of day or other timing information, vital signs for the user such as heart rate or respiration rate, indicators indicating whether an object being viewed by the nightvision system is friendly or adversarial, battery charge level for the nightvision system or other devices, weather conditions, contact information, audio information (such as volume, playlist information, artist, etc.), etc. In some embodiments, the heads-up display can superimpose thermal image data over intensified light images. In particular, a nightvision system <NUM> may include (or at least have access to data from) thermal cameras for detecting thermal characteristics in an environment. Data from thermal cameras can be used to control the transparent optical device <NUM> to display thermal imaging data, correlated with artifacts in intensified light images, to the user. For example, various colors can be used to represent various temperatures where the colors are output by the transparent optical device <NUM> to the user. The colors are overlaid on intensified light image artifacts to indicate temperature of objects in an environment. Note that the transparent optical device <NUM> (or other elements) includes photodetectors for detecting intensified light to determine brightness in a scent, the locations of various objects in the field of view, or other information. This information can be used for illumination functionality as described above and in more detail below, correlating thermal colors, target indicators, or other images output by the transparent optical device <NUM>.

Attention is now directed to <FIG>, <FIG> and <FIG>. <FIG>, <FIG> and <FIG> illustrate a block diagram, a cutaway and a block diagram respectively of embodiments of the invention. A nightvision system typically includes an objective <NUM> to focus input light <NUM> into an underlying device <NUM>. It should be appreciated that in various embodiments, the underlying device <NUM> may be one or more of a number of different types of nightvision devices, such as IR CCD cameras, CMOS cameras, image intensifiers, and the like. In the example illustrated in <FIG>, the underlying device <NUM> is an image intensifier. Input light <NUM> may be, for example, from ambient sources, such as light from heavenly bodies such as stars, the moon, or even faint light from the setting sun. Additionally, or alternatively, ambient sources could include light from buildings, automobiles, or other faint sources of light that cause reflection of light from an object being viewed in a nightvision environment into the objective. A second source of light may be light being emitted from an external source, such as the illuminator <NUM>, towards an object, reflected off the object, and into the objective. For example, the source may be an infrared source that is not viewable in the viewable spectrum for human observers. A third source of light may be light emitted by an object itself. For example, this may be related to infrared heat energy emitted by the object and directed into the objective. Nonetheless, the nightvision system is able to convert the light emitted from the source into a viewable image for the user.

The objective directs any input light <NUM> into the underlying device <NUM>. Note that the underlying device <NUM> may include functionality for amplifying light received from the objective to create a sufficiently strong image that can be viewed by the user. This may be accomplished using various technologies, in the image intensifier example of <FIG>, such as a photocathode <NUM>, a microchannel plate <NUM>, and a phosphor screen <NUM>. The photocathode <NUM> may be configured to generate photo electrons in response to incoming photons. Electrons from the photocathode <NUM> are emitted into the microchannel plate <NUM>. Electrons are multiplied in the microchannel plate <NUM>.

Electrons are emitted from the microchannel plate <NUM> to a phosphor screen <NUM> which glows as a result of electrons striking the phosphor screen <NUM>. This creates a monochrome image from the input light <NUM>.

A fiber-optic <NUM> carries this image as intensified light to the eyepiece (such as eyepiece <NUM> illustrated in <FIG>) of a nightvision system where it can be output to the user. This fiber-optic <NUM> can be twisted <NUM> degrees to undo the inversion caused by the system objective to allow for convenient direct viewing of the screen.

<FIG>, <FIG> and <FIG> further illustrate the transparent optical device <NUM>. The transparent optical device <NUM> allows intensified light to pass through the transparent optical device <NUM>, but also, in some embodiments, generates its own light, from LEDs or other light emitters, to transmit to a user. Creating a transparent optical device may be accomplished, for example, using the teachings of <CIT>, titled "Backside Etch Process For Transparent Silicon Oxide Technology".

The transparent optical device <NUM> is typically implemented behind the fiber-optic <NUM> (i.e., closer to the eyepiece than the fiber-optic <NUM>), but in other embodiments may be implemented in front of the fiber-optic <NUM>. The use of a fiber-optic within nightvision systems inverts and translates the focal plane allowing the transparent optical device <NUM> overlay to be presented on either side without impacting the ability for the eyepiece to focus on the image. However, certain manufacturing or mechanical constraints may incentivize placement of the transparent optical device <NUM> behind the fiber-optic including the difficulty in inserting electronics within the vacuum package. Placing the transparent optical device <NUM> external to the fiber-optic can be done to allow the transparent optical device <NUM> to be applied after the image intensifier tube has been manufactured and sealed, lowering production difficulties. As discussed above, the transparent optical device <NUM> may include functionality for displaying information to a user. Such information may include graphical content, including text, images, and the like. In some embodiments, the transparent optical device <NUM> may display in shaded monochrome. Alternatively, or additionally, the transparent optical device <NUM> may display in multiple colors. Alternatively, or additionally, the transparent optical device <NUM> may display in <NUM>-bit monochrome.

In the example illustrated in <FIG>, <FIG> and <FIG>, the transparent optical device <NUM> outputs display light <NUM> which can be sent to the eyepiece (such as the eyepiece <NUM> illustrated in <FIG>). As noted previously, the intensified light <NUM> is also provided to the eyepiece. Thus, an image such as that illustrated in <FIG> is presented to the user in the nightvision system.

As noted previously, the transparent optical device <NUM> is composed of active silicon elements, typically in a grid arrangement to implement various pixels. The active elements cause the device to have certain optical performance capabilities. Such capabilities may be one or more of abilities to output color output, output monochrome output, detect light, have a certain pixel density, have a certain pitch, etc. In particular, the transparent optical device <NUM> is a digital detector and potentially display having a certain pixel density for the detector and potentially a display. Often, each pixel is implemented on a single active island, although in other embodiments, an island may have multiple pixels, or even only a single sub-pixel element. Each pixel may have one or more transistors controlling one or more OLED emitters (or other light emitting devices). Pixels may additionally or alternatively include light detectors. This can be useful for detecting the intensified light from the phosphor screen <NUM> or other optical output when other underlying devices are used. This detected light can be used for illumination functionality and/or to characterize an image intensifier image. For example, the detected light can be used for recording scene events. Alternatively or additionally, the detected light can be used for improving placement of elements displayed on the heads-up display shown in <FIG>. For example, edge detection techniques may be used using the detected light, and images generated and displayed by the transparent optical device <NUM> can be keyed off of these detected edges. Edge detection techniques may be used using the detected light, and control of the illuminator <NUM> can be keyed off of these detected edges. Note that as will be discussed in more detail below in conjunction with the description of <FIG>, in some embodiments, multiple illuminators may be controlled. Indeed, in some embodiments the illuminators may be networked for mutual control to achieve certain desirable illumination characteristics.

In any case, the transparent optical device <NUM> is representative of a stacked device formed in a semiconductor chip that overlaps an underlying device <NUM>, in the case shown in <FIG>, the underlying device <NUM> is an image intensifier. The transparent optical device <NUM> is transparent to light in a first spectrum (according to some predefined transmission efficiency), which in this case is the visible spectrum of light output by the phosphor screen <NUM>. That is, the transparent optical device <NUM> is not fully transparent due to the blocking of the active devices, but transparency referred to herein refers to at least partial transparency according to some transmission efficiency. Note that overlapping as used herein means that elements are in the same optical path. This can be accomplished by having elements be in coaxial alignment when the optical path is straight. Alternatively, this can be accomplished by using various waveguides or other elements to align optical paths thus not requiring physical coaxial alignment.

A photodetector implemented in the transparent optical device absorbs a portion of the intensified light converting it to an electrical signal. For example, the photodetector can be a two-dimensional array of light detectors, such as photodiodes, that generates a charge current, or any other form of digital data level proportional to intensity of the intensified light as a function of position. Accordingly, the photodetector may generate a two-dimensional array of electrical charge that represents the intensified image. In some embodiments, this two-dimensional array of electrical charge can be periodically read from the photodetector (e.g., the detected signal can be read from the photodetector like in a charged coupled device (CCD) camera). In some embodiments, the two-dimensional array of electrical signals from the photodetector is processed (such as by the image processor <NUM>) and/or used locally, e.g., within the transparent optical device <NUM>, at the readout or pixel levels, to modulate in real time the amplitude of the display light <NUM> emitted from the transparent optical device <NUM>.

The transparent regions shown in the preceding figures can be created in a number of particular ways. In some embodiments, the transparent regions can be created by using the processes described in <CIT> titled "Backside Etch Process For Transparent Silicon Oxide Technology". Briefly, that application describes a process for creating transparent regions in otherwise opaque portions of semiconductor materials. For example, reference is now made to <FIG> which illustrates a transparent optical device <NUM> including active silicon islands (which may be native silicon islands) such as active silicon island <NUM>. In particular, active silicon islands include transistors such as transistor <NUM> which control detectors and potentially OLED emitters in an OLED stack <NUM>. In the example illustrated in <FIG>, each of the active silicon islands represents a pixel or sub-pixel of the transparent optical device <NUM>. Thus, an image can be detected by active elements in the active region. Additionally, in some embodiments, by illuminating various LEDs in the OLED stack <NUM> using the transistors in the active silicon islands, an image can be created and output to a user, such as by outputting display light such as the display light <NUM> illustrated in <FIG>.

As illustrated in <FIG>, intensified light is transmitted through the transparent optical device <NUM> to the eyepiece of the nightvision system, and then to the user. Note, however, that the intensified light is transmitted to the user through the transparent optical device <NUM>, meaning that the intensified light will be affected by characteristics of the transparent optical device <NUM>. In the example illustrated in <FIG>, light <NUM> represents the light output by the light emitting portions of the transparent optical device <NUM>. Light <NUM> represents intensified light from the phosphor screen <NUM> transmitted through the transparent optical device <NUM>. That is, light <NUM> may include or may be, in its entirety, light in the first spectrum.

Referring once again to <FIG>, light <NUM> represents a portion of light that is transmitted through transparent regions, illustrated by transparent region <NUM>, of the transparent optical device <NUM>, while light <NUM> represents a portion of light that is blocked by active portions of the transparent optical device <NUM>. Note that in some embodiments, the transparent region <NUM> is backfilled with a transparent back fill material.

However, transmission of light through the transparent optical device is nonetheless increased by removing portions of silicon that are not needed for implementing active electrical components or for supporting metal traces. For example, consider an example where dynamic pixel cells are used. In this particular example, assume that there are two sub pixels per pixel. Anode size for the sub pixels is <NUM> x <NUM>. Pixel area is <NUM> x <NUM>. Pixel pitch is <NUM> x <NUM>. In one example, this provides a resolution of <NUM> x <NUM>. In this particular transparent optical device, if non-active silicon islands (not shown) that are typically implemented are not removed, transparency of the transparent optical device is about <NUM>%. In contrast, transparency is about <NUM>% if the non-active silicon islands are removed such as in the structure illustrated in <FIG>. Thus, in this example, transparency of a transparent optical device is increased by more than <NUM>% by removing silicon and/or oxide trenches.

Note that various engineering trade-offs can be made to meet certain requirements. For example, increased transparency can be obtained by having a lower resolution and/or using fewer sub pixels as there is more space between pixels and/or sub pixels. If a higher resolution is needed, then that transparent optical device will have a lower transparency than an equivalently sized transparent optical device with a lower resolution. Thus, for example, a transparent optical device with a <NUM> pitch can obtain a transparency of <NUM>%, while a transparent optical device of <NUM> pitch can obtain a transparency of <NUM>%, while a transparent optical device having a <NUM> pitch will be about <NUM>% transparency when non-active silicon islands are removed from the transparent optical device in each of the illustrated examples. Thus, some embodiments may be able to create a transparent optical device with at least a <NUM> pitch with at least a transparency of <NUM>%, or a transparent optical device of at least a <NUM> pitch with at least a transparency of <NUM>%, or a transparent optical device having at least a <NUM> pitch with at least a <NUM>% transparency when silicon is removed between active silicon islands. The preceding illustrates one specific example related to a particular manufacturing process.

Pitch and transparency values may be specific to a given semiconductor manufacturing process-also known as the technology or process node-and will of course vary as the node changes. Typically designating the process's minimum feature size, the technology node will dictate the area of required active silicon for the display CMOS based on the transistor size. As the node minimum feature size decreases, whether it be through alternate foundries or improvements in technology, the same need for maximizing transparency applies. Indeed, the benefit to removing non-active silicon islands improves as the ratio of inactive- to active-silicon increases with smaller transistors.

The example numbers described herein are derived assuming a <NUM> technology/process node, although similar calculations can be performed for any specific technology size.

In the example illustrated in <FIG>, light <NUM> is input into the objective <NUM>, where it is transmitted to an underlying device <NUM>, in this case, an image intensifier. <FIG>, <FIG> and <FIG> further illustrate the transparent optical device <NUM>. As noted previously, the transparent optical device <NUM> includes light detectors that are able to detect light produced by the underlying device <NUM> to produce a brightness map <NUM>. The brightness map <NUM> indicates different light levels in different portions of a scene. In particular, analysis of the brightness map can show variations in brightness of a scene detected by the underlying device <NUM>. <FIG>, <FIG> and <FIG> further illustrate an image processor <NUM>. The image processor <NUM> is able to process brightness maps produced by the detectors in the transparent optical device <NUM>. In particular, the image processor <NUM> can analyze and identify the brightness variations in the brightness map.

The image processor <NUM> is further connected to an illuminator <NUM>. The illuminator <NUM> is configured to output light to provide additional input light (typically by reflecting that light off of objects in a scene) into the underlying device <NUM> based on the image processing. Typically, the illuminator <NUM> emits light in a wavelength that is not visible to a human user with the naked eye. For example, the illuminator <NUM> may be an infrared illuminator that is visible to the nightvision system <NUM>, but that is not visible to the human user with the naked eye. The illuminator <NUM> may be caused to provide additional light based on the processing. Alternatively, the illuminator may be caused to provide less light based on the processing. Alternatively, or additionally, the illuminator may be caused to provide a particular selectable pattern of light based on the processing. Alternatively, or additionally, the illuminator may be controlled as part of controlling a plurality of illuminators to achieve a particular illumination pattern, such as is illustrated in <FIG>.

In nightvision systems, the illuminator <NUM> may be implemented in a number of different fashions. For example, the illuminator <NUM> may be implemented as a device mounted to a head mounted nightvision goggle. Alternatively, the illuminator <NUM> may be mounted on a helmet proximate the nightvision system <NUM>. In some embodiments, both the nightvision system <NUM> and the illuminator <NUM> may be helmet mounted. Alternatively, in some embodiments, the illuminator may be mounted on an implement being used by the user of the nightvision system <NUM>. For example, the user may have a handheld flashlight having the illuminator <NUM> therein. Alternatively or additionally, the user may have another implement such as a test meter, electronic drill, electronic screw driver, weapon, etc., having a controllable illuminator mounted thereon.

In an alternative embodiment the illuminator <NUM> may be included in a vehicle, such as included in headlights. That is, a vehicle may include IR headlights that are controllable by an image processor <NUM> in the user nightvision system <NUM>.

In an alternative embodiment the illuminator <NUM> may be included in a range finder, such as a laser range finder device configured to identify distances of objects from the range finder (such as the ranging sensor <NUM>).

Various different mechanisms may be implemented to allow the image processor <NUM> to communication with illuminator <NUM>. For example, the image processor <NUM> may include, or be connected to wireless communication equipment where the illuminator <NUM> also includes corresponding wireless communication equipment, such as radios, antennas, amplifiers, etc. Alternatively, various wired connections may be implemented to communicate control information to the illuminator <NUM>. Note that in some embodiments, certain smart mounting equipment, such as smart picatinny rails or other mounting equipment may include communication equipment or other equipment to facilitate functionality of an illuminator <NUM> or nightvision system <NUM> mounted thereto.

Note that as illustrated in <FIG>, some embodiments may be implemented in a network of nightvision systems and illuminators. For example, <FIG> illustrates a plurality of illuminators <NUM>-<NUM> through <NUM>-<NUM>. Note that while a plurality of illuminators is illustrated, in other embodiments, a plurality of nightvision systems and corresponding transparent optical devices and image processors may be implemented. In some embodiments, the image processor <NUM> may be configured to control illuminators associated with other users and/or various static and/or mobile illuminators. Desired contrast can be achieved by controlling the plurality of illuminators <NUM>-<NUM>-<NUM>-<NUM> separately or in concert. This provides one method of enhancing illumination in certain locations, direction, power ranges, etc. Note that in some embodiments various image processors associated with various different nightvision systems can communicate as well. This allows for optimization techniques to be applied to ensure that one image processor controlling an illuminator does not interfere (above some predetermined threshold) with contrast and illumination operations being undertaken by a different image processor. Thus, using networked illuminators and/or networked image processors, embodiments can balance desired illumination changes with desired illuminator and/or contrast by other users.

Some embodiments may be implemented in a fashion to optimize dynamic range of an underlying device. In particular, devices tend to operate more consistently, more linearly, and/or with other dynamic range features in certain ranges of operation. This operating voltage can be selected in linear regions of operations and/or regions that provide the most dynamic range to allow for wider variations in exogenous light. The way an image intensifier power supply device adjusts operational voltages to result in ABC (auto brightness control), BSP (bright source protection), and auto gaiting has dynamic range ramifications. Illumination control is performed in concert with the intensifier power supply to optimize scene dynamic range. Thus, dynamic range optimization results from integration of illuminator control with image intensifier power supply control to apply optimal cathode, microchannel plate, and screen voltages.

Note that in some embodiments, the illuminator may be controllable to direct light in a particular direction and/or with a particular dispersion. This can be done, for example, to attempt to illuminate a specific object or specific region of interest. This control may be accomplished in one or more of a number of different fashions. For example, in some embodiments, digital micro mirror devices and/or digital light processing may be used to direct light to desired locations. Alternatively, other type of controllable mirror arrays, laser arrays, adjustable gratings, and/or the like may be used to direct light to desired locations.

Thus, embodiments include a nightvision system <NUM>. The nightvision system <NUM> includes an underlying device <NUM>. For example, an image intensifier is an example of an underlying device. Note that other underlying devices may be used in addition or alternatively. For example, some embodiments may include infrared CCD based or other low light level digital sensor system. The underlying device <NUM> is configured to provide output light in a first spectrum from input light received at the underlying device <NUM>. Thus, for example, the first spectrum may be light produced by the phosphor screen <NUM>.

The nightvision system <NUM> includes a transparent optical device, such as transparent optical device <NUM>, optically coupled in an overlapping fashion to the underlying device <NUM>. The transparent optical device <NUM> is configured to transmit light in the first spectrum from the underlying device <NUM> through the transparent optical device <NUM>. The transparent optical device <NUM> includes: an active area of a semiconductor chip.

The active area includes a plurality of active elements configured to cause the underlying device <NUM> to detect light from the underlying device <NUM>. For example, as described above, light detectors integrated into a photodetector may be used to detect light from the underlying device <NUM>.

The transparent optical device <NUM> further includes a plurality of transparent regions formed in the active area which are transparent to the light in the first spectrum to allow light in the first spectrum to pass through from the underlying device <NUM> to a user. The transparent regions are configured in size and shape to cause the transparent optical device <NUM> to have a particular transmission efficiency for light in the first spectrum.

The nightvision system further includes an image processor, such as image processor <NUM> coupled to the transparent optical device <NUM>. The image processor is configured to process brightness maps, such as brightness map <NUM>, produced using light detected by the plurality of active elements.

The nightvision system further includes an illuminator, such as illuminator <NUM>, coupled to the image processor <NUM>. The illuminator <NUM> is configured to output light into the underlying device <NUM> based on image processing performed by the image processor <NUM>.

As noted previously, the nightvision system may be implemented where the underlying device <NUM> comprises an image intensifier.

In some embodiments, the nightvision system may be implemented where the illuminator is configured to output light into the underlying device <NUM> based on coarse illumination factors and/or fine illumination factors. In some such examples, the fine illumination factors are based on the image processing performed by the image processor. While the coarse illumination factors are based on other inputs or analysis. As used herein coarse and fine are relative terms used with respect to each other. That is, coarse illumination is more coarse (or generally performed in larger increments) than fine illumination and fine illumination is more fine (or generally performed in smaller increments) than coarse illumination. In some embodiments, this can mean that a coarse illumination is performed first to attempt to achieve an initial level of illumination/contrast while fine illumination is thereafter performed to fine tune the illumination operation.

For example, in some embodiments, the nightvision system may include an orientation sensor <NUM> coupled to the illuminator <NUM>. In such examples, the coarse illumination factors are based on information received from the orientation sensor <NUM>. Such orientation sensors may include one or more of accelerometers, gyroscopes, magnetometers, internal measurement units, combinations thereof, and the like. Coarse illumination functionality is performed as a result of detecting orientation information. This could be a change in orientation (e.g., a change of a certain number of degrees in one or more axis), satisfaction of a condition of absolute orientation, or combinations thereof.

Note that this type of coarse illumination may be useful when large differences in illumination of a scene is required. For example, consider a case where a user desires high illumination when scanning a horizon, but then looks down at handheld items, the ground, or their feet, where the user desires less illumination. Thus, some embodiments may include functionality for performing a coarse illumination function to illuminate using higher powered illumination for distance viewing than up-close viewing as a result of orientation information obtained from an orientation sensor.

In some embodiments, the nightvision system may include a ranging sensor <NUM>. In some such examples, the coarse illumination factors are based on information received from the ranging sensor <NUM>. The ranging sensor <NUM> can include elements such as lasers, lidar, radar, sonar, combinations thereof, or other ranging equipment. Thus, coarse illumination functionality can be performed as a result of detecting ranging information to control illumination of the illuminator <NUM> dependent on a distance of objects of interest.

Note that while in typical embodiments coarse and fine illumination functionality can be performed, it should be appreciated that in other embodiments, any of the illumination factors and mechanisms may be used individually or in combination. Thus, in some embodiments, illumination functionality is performed based on using the orientation sensor <NUM> without using information from the image processor <NUM>. Alternatively, illumination functionality is performed based on using the ranging sensor <NUM> without using information from the image processor <NUM>. Alternatively, illumination functionality is performed based on using the image processor <NUM> without using information from the orientation sensor <NUM> or the ranging sensor <NUM>. Further, it should be appreciated that embodiments may use both the orientation sensor <NUM> and the ranging sensor <NUM> to perform coarse illumination functionality and then sequentially use information from the image processor <NUM> to perform fine illumination functionality.

The nightvision system may be implemented where the image processor performs edge detection. For example, such edge detection may be implemented using Sobel edge detection and/or Canny edge detection. In some such embodiments, the illuminator is configured to output light into the underlying device <NUM> based on the edge detection. Thus, for example, embodiments may be configured to provide illumination in an attempt to cause edges to have a particular characteristic as determined by the edge detection. For example, embodiments may be configured to cause the illuminator <NUM> to provide illumination to cause a certain contrast for a certain object in an environment. In some embodiments, overall scene contrast can be sacrificed to improve contrast on a particular object or region. Similar functionality may be performed for region detection and enhancing contrast for a particular region.

In some embodiments, the nightvision system may be implemented where the image processor performs object recognition. In some such embodiments, the illuminator is configured to output light into the underlying device <NUM> based on the object recognition. This can be implemented in a number of different fashions. For example, in some embodiments the nightvision system may be configured to perform illumination functionality for a largest object detected in a nightvision scene. Alternatively, embodiments may be configured to perform illumination functionality based on detection and classification of objects. For example, the image processor <NUM> may be able to identify that a certain object detected in a nightvision scene is likely a stop sign while a different object detected in the nightvision scene is likely a human. In this example, an algorithm may be implemented to cause the illuminator <NUM> to output light to enhance contrast for the object detected as a human while deprioritizing, and thus not enhancing, the object detected as a stop sign. Note that in some embodiments, the nightvision system <NUM> may include functionality for allowing user input to specify priorities when determining illumination functionality. For example, for specific missions, a user may wish to prioritize illumination functionality for objects classified as vehicles over objects classified as humans. Alternatively, a user may wish to prioritize illumination functionality for objects classified as animals over objects classified as humans. Thus, a user can specify priorities, and illumination functionality will be performed using the illuminator <NUM> on preferred objects recognized by the system.

Alternatively, or additionally, in some embodiments a user may be able to specify specific objects and not simply a classification of objects. For example, a user may be able to specify information to particularly identify a particular human based on height, weight, build, posture, or other characteristics. Thus, some embodiments may be able to specifically perform illumination functionality when the image processor <NUM> detects, within some acceptable probability, that a detected object can be classified as a specific object, such as a specific human. Similar functionality can be used for sub classifications of objects. For example, a user could specify illumination functionality should be prioritized for trucks whereas other vehicles such as cars, motorcycles, etc. are not prioritized as highly for illumination functionality.

The nightvision system <NUM> may be implemented where the image processor performs region of interest detection. In some such embodiments, the illuminator is configured to output light into the underlying device <NUM> based on the region of interest detection. These embodiments may be similar to the object detection embodiments above but focused on regions as opposed to specific objects.

The nightvision system <NUM> may be implemented where the nightvision system comprises an eye tracker <NUM> coupled to the image processor <NUM> to track a user's eye with respect to the processed brightness maps produced by the image processor. In some such embodiments, the illuminator <NUM> is configured to output light into the underlying device <NUM> based on tracking a user's eye with respect to the processed brightness maps produced by the image processor. In particular, embodiments can determine what objects a user is looking at in a nightvision scene using the eye tracker <NUM> and can perform illumination functionality for those objects based on that determination.

The nightvision system <NUM> may be implemented where the nightvision system <NUM> comprises an aiming laser <NUM> coupled to the image processor to facilitate aiming the nightvision system. In some such embodiments, the illuminator <NUM> is configured to output light into the underlying device <NUM> based on tracking the aiming laser <NUM> with respect to the processed brightness maps produced by the image processor <NUM>.

In some embodiments the nightvision system <NUM> may be implemented where the illuminator is configured to provide information to a user to indicate to the user manual illumination adjustments that can be made with respect to providing additional input light into the underlying device <NUM>. For example, as discussed previously, the transparent optical device <NUM>, may include not only detector functionality, but may also include display functionality. Thus, using the display functionality, information may be output to the user to indicate to the user that manual adjustments should be performed to cause certain objects, regions, etc. to have a certain contrast for the user. In a similar embodiment, the display functionality may be able to highlight, such as by pointing to and/or outlining an object for which contrast is enhanced. A user can then select other objects to have contrast enhanced. Selection of these other objects can be performed by using a user interface to select the object in the nightvision scene, which can be used to cause the illuminator <NUM> automatically adjust illumination to enhance contrast for the selected object. Alternatively, or additionally, the user can simply perform a manual override to manually adjust illuminator illumination on a particular object based on information presented to the user using the display functionality of the transparent optical device <NUM>.

Referring now to <FIG>, a method <NUM> is illustrated. The method <NUM> includes capturing a nightvision scene image using a transparent detector (act <NUM>). For example, the transparent optical device <NUM> may be used to capture a nightvision scene as described above. The method <NUM> includes processing a brightness map for illumination quality (act <NUM>). For example, the image processor <NUM> can determine illumination quality (e.g., contrast) of the captured nightvision scene. The method <NUM> may further include adjusting illumination based (act <NUM>) based on the processing of act <NUM>. Acts <NUM>-<NUM> can be repeated as needed to perform illumination functionality.

Claim 1:
A nightvision system comprising:
an underlying device (<NUM>), the underlying device (<NUM>) configured to provide output light in a first spectrum from input light received at the underlying device (<NUM>);
a transparent optical device (<NUM>) optically coupled in an overlapping fashion to the underlying device (<NUM>), wherein the transparent optical device (<NUM>) is configured to transmit light in the first spectrum from the underlying device (<NUM>) through the transparent optical device (<NUM>), the transparent optical device (<NUM>) comprising: an active area of a semiconductor chip, the active area comprising:
a first plurality of active elements configured to cause the underlying device (<NUM>) to detect light from the underlying device (<NUM>); and
a first plurality of transparent regions formed in the active area which are transparent to the light in the first spectrum to allow light in the first spectrum to pass through from the underlying device (<NUM>) to a user, the first plurality of transparent regions being configured in size and shape to cause the transparent optical device (<NUM>) to have a first transmission efficiency for light in the first spectrum;
the nightvision system further comprising an image processor (<NUM>) configured to process brightness maps, wherein the brightness maps indicate different light levels in different portions of a scene produced using light detected by the first plurality of active elements; and
an illuminator (<NUM>) coupled to the image processor (<NUM>), the illuminator (<NUM>) configured to provide additional input light into the underlying device (<NUM>) based on image processing performed by the image processor (<NUM>).