Patent Description:
Head worn displays (HWDs) may be worn by a user and provide a user with visual information. For virtual reality displays, all of the image on the display viewed by a user may be rendered. Augmented reality displays, on the other hand, combine light by optical means to present a user with virtual information superimposed on the real world.

Augmented reality HWDs typically compromise between cost, field of view (FOV) and size. Most traditional HWD application for augmented reality include increasing the size of conventional optics to the point they are overly large and cumbersome to the user in order to achieve a wide FOV.

<CIT> discloses a head worn display comprising transparent areas that can be controlled to overlay extra information.

<CIT> discloses a virtual display system for use with a heads down display (HDD) that can be easily integrated in the design of a cockpit without requiring extra display space.

In one aspect, embodiments of the inventive concepts disclosed herein are directed to a head worn display (HWD) including a head attachment region, an internal display and a controller. The internal display is viewable by a user and includes variable transparency areas. The internal display is arranged to display at least a portion of an external region external to the HWD. The controller is configured to control the variable transparency areas to block view of a region of the external region and to control the internal display to overlay information in a region of the internal display corresponding to the region of the external region which is blocked.

In a further aspect, embodiments of the inventive concepts disclosed herein are directed to a HWD including a head attachment region, an internal display viewable by a user of the HWD, one or more image detectors, and a controller. The one or more image detectors are arranged to detect an image of an external region external to the internal display. The controller is configured to control the internal display to display the detected image and to overlay information over the displayed detected image.

Embodiments of the inventive concepts disclosed herein regarding HWDs with variable transparency areas provide for functions such as enhancement, replacement, augmentation and occlusion in a display. The HWDs provide functionality for providing check lists at appropriate times, replacing the view of failed displays, providing visual information for emergency situations, and providing peripheral cue augmentation, for example. The embodiments and/or examples of the following description which are not covered by the appended claims are considered as not being part of the present invention.

HWDs may also be arranged in a mixed reality configuration where an internal display viewed by a user includes images of a region external to the internal display which is detected by image detectors. In the mixed reality configuration the internal display displays the detected image along with overlay information. Such an arrangement may obtain a wide FOV, such as nearly <NUM> degrees, for example. Further, real world imagery is relayed to the user via the image detectors.

<FIG> is a schematic illustrating a HWD <NUM> according to inventive concepts disclosed herein. The HWD <NUM> includes a head attachment region <NUM>, an internal display <NUM>, and a controller <NUM>. The HWD <NUM> may further include an eye tracker <NUM> and head tracker <NUM>.

The head attachment region <NUM> may be worn by a user, and may be a helmet, for example. Alternatively, the head attachment region <NUM> may include googles, for example. The internal display <NUM> is attached to the head attachment region <NUM>.

The HWD <NUM>, including the head attachment region <NUM>, internal display <NUM>, and controller <NUM>, is shown in <FIG> relative to a user eye <NUM>, and an external region <NUM>. The user eye <NUM> and the external region <NUM> are not part of the HWD <NUM>. The eye tracker <NUM> tracks the position and orientation of the user eye <NUM>. The head tracker <NUM> tracks the position and orientation of the user head.

<FIG> illustrates an arrangement of the internal display <NUM> from a side view relative to the user eye <NUM> and the external region <NUM>, while <FIG> illustrates an arrangement of the internal display <NUM> from a front view relative to the external region <NUM>.

The internal display <NUM> has a plurality of variable transparency areas <NUM>. For ease of illustration, <FIG> illustrates only nine variable transparency areas <NUM>. In general, the number of variable transparency areas <NUM> may be many more than nine. When the variable transparency areas <NUM> are transparent, at least a portion of the external region <NUM> may be seen on the internal display <NUM> by the user eye <NUM>. That is, the internal display <NUM> is arranged to display at least a portion of the external region <NUM> external to the HWD <NUM>.

As seen in <FIG>, the internal display <NUM> may include a variable transparency layer <NUM>, which includes the variable transparency areas <NUM>, and a combiner <NUM>. The combiner <NUM> combines an image of the external region <NUM> which is not blocked by the variable transparency layer <NUM> with an image provided via the controller <NUM>. The transparency layer <NUM> may be a electrochromic layer, for example, where the transparency of the variable transparency areas <NUM> are controlled by voltages applied via the controller <NUM>. An example of a variable transparency layer and combiner may be found, for example, in <CIT>.

Referring to <FIG>, the controller <NUM> may include a processor <NUM> and a memory <NUM>. The controller <NUM> controls the eye tracker <NUM> to determine the orientation and position of the user eye <NUM>, and possibly the head tracker <NUM> to determine the orientation and position of the user head. The controller <NUM> further controls the internal display <NUM> to vary the transparency of the variable transparency areas <NUM>. Still further the controller <NUM> controls the internal display <NUM> to display overlay information on the internal display <NUM>.

For example, the controller <NUM> may control the variable transparency areas <NUM> to block a view to the user eye of a region of the external region <NUM>, and to control the internal display <NUM> to overlay information in a region of the internal display <NUM> corresponding to the region of the external region <NUM> which is blocked. The overlay information may include text and/or symbols, for example.

<FIG> illustrates an example of an external region <NUM> according to the inventive concepts disclosed herein. The external region <NUM> may be a view within a cockpit of an aircraft, or within a drivers region of a vehicle, for example. The external region <NUM> may include a number of portions. For example, in the case that the external region <NUM> is a view within a cockpit of an aircraft, the portions may include window portions <NUM>, which provide a view of outside the cockpit, and cockpit instrument sections 154A, 154B and 154C corresponding to flight instruments. The cockpit instrument sections may include displays.

<FIG> illustrates an example of the internal display <NUM> illustrating overlay regions <NUM>, and a non-overlay region <NUM>. The overlay region <NUM> corresponds to regions of the external region <NUM> which are blocked from view via the variable transparency areas <NUM>. The non-overlay region <NUM> correspond to regions of the external region <NUM> which are not blocked from view via the variable transparency areas <NUM>, where those unblocked regions of the external region <NUM> are displayed on the internal display <NUM>. The controller <NUM> controls the internal display <NUM> to display overlay information at the overlay regions <NUM>.

<FIG> illustrates an example of the internal display <NUM> according to inventive concepts disclosed herein, where the overlay region <NUM> corresponds to a checklist. In this case the overlay information is information regarding the checklist, such as the items listed in the checklist. The overlay region <NUM> of the internal display <NUM> may be arranged to occlude a non-critical cockpit instrument section of the external region <NUM>, so as to not block important information. The non-critical cockpit instrument section which is blocked may be a flight plan, for example, when the overlay region <NUM> corresponds to a checklist with a startup sequence, for example.

The controller <NUM> may confirm that a user is looking at the checklist information using the eye tracker <NUM> and possibly the head tracker <NUM>. The eye tracker <NUM> tracks the user eye <NUM> orientation and position, and sends the user eye orientation and position data to the controller <NUM>. The controller <NUM> determines whether or not the user eye <NUM> is looking at the checklist information using the user eye orientation and position data, and using the position of the overlay region <NUM> corresponding to the checklist.

<FIG> illustrates an example of the internal display <NUM> according to inventive concepts disclosed herein, where the overlay region <NUM> corresponds to a display region of the external region <NUM> where the display region has failed. In this case the overlay information is virtual information regarding a replacement display region corresponding to the failed display region. The overlay region <NUM> of the internal display <NUM> may be arranged to occlude the failed display region of the external region <NUM>.

In the case of a failed display region of the external region <NUM>, a warning signal may be sent to the controller <NUM>. The warning signal is received by the controller <NUM>, and indicates that a display region of the external region has failed. The controller <NUM> controls the internal display <NUM> to overlay the replacement display region over the regions of the internal display <NUM> corresponding to the display region of the external region which has failed.

<FIG> illustrates an example of the internal display <NUM> according to inventive concepts disclosed herein, where the overlay region <NUM> corresponds to emergency information. In this case the overlay information is information regarding an emergency. The overlay region <NUM> of the internal display <NUM> may be arranged to occlude a non-critical cockpit instrument section of the external region <NUM>, so as to not block important information.

The controller <NUM> may receive an emergency signal indicating that an emergency has occurred. The controller <NUM> controls the internal display <NUM> to overlay emergency information indicating an emergency during the time that the emergency continues.

The controller <NUM> may confirm that the user is looking at the emergency information using the eye tracker <NUM>, and possibly the head tracker <NUM>. The eye tracker <NUM> tracks the user eye <NUM> orientation and position, and sends the user eye orientation and position data to the controller <NUM>. The controller <NUM> determines whether or not the user eye <NUM> is looking at the emergency information using the user eye orientation and position data, and using the position of the overlay region <NUM> corresponding to the emergency information. The controller <NUM> may electrochromically occlude or partially occlude all non-critical, nonemergency portions of the field of view and emphasize the critical portions.

<FIG> illustrates an example of the internal display <NUM> according to inventive concepts disclosed herein, where the overlay region <NUM> corresponds to direction information for a first overlay region 124a and to peripheral information for a second overlay region 124b. The first overlay region 124a for the direction information may be disposed in an inner region of the internal display <NUM>, for example. The second overlay region 124b for the peripheral information is disposed in a peripheral region of the internal display <NUM>. The direction information is information directing a user's attention to the second overlay region 124b. For example, if the second overlay region 124b is in the upper right hand corner of the internal display <NUM>, the direction information may be an arrow pointing to the second overlay region 124b. As another example, if the second overlay region 124b is in the upper right hand corner of the internal display <NUM>, the direction information may be text saying "LOOK IN UPPER RIGHT.

The controller <NUM> may confirm that a user is looking at the peripheral information using the eye tracker <NUM>, and possibly the head tracker <NUM>. The eye tracker <NUM> tracks the user eye <NUM> orientation and position, and sends the user eye orientation and position data to the controller <NUM>. The controller <NUM> determines whether or not the user eye <NUM> is looking at the peripheral information using the user eye orientation and position data, and using the position of the second overlay region 124b corresponding to the peripheral information.

According to inventive concepts disclosed herein, <FIG> is a schematic illustrating a HWD <NUM> according to inventive concepts disclosed herein. The HWD <NUM> includes a head attachment region <NUM>, an internal display <NUM>, a controller <NUM> and one or more image detectors <NUM>. The HWD may further include an eye tracker <NUM> and a head tracker <NUM>.

The HWD <NUM>, including the head attachment region <NUM>, internal display <NUM>, controller <NUM>, and one or more image detectors <NUM>, is shown in <FIG> relative to a user eye <NUM>, and an external region <NUM>. The user eye <NUM> and the external region <NUM> are not part of the HWD <NUM>. The eye tracker <NUM> tracks the position and orientation of the user eye <NUM>. The head tracker <NUM> tracks the position and orientation of the user head.

The one or more image detectors <NUM> are arranged to detect an image of the external region <NUM> external to the internal display <NUM>. The one or more image detectors <NUM> may be cameras, for example. In the case that the one or more image detectors <NUM> are a plurality of image detectors, the images from the one or more image detectors <NUM> may be combined to form a combined image.

The controller <NUM> may include a processor <NUM> and a memory <NUM>. The controller <NUM> controls the eye tracker <NUM> to determine the orientation and position of the user eye <NUM>. In this regard the eye tracker <NUM> may include a head tracker tracking the head orientation and position of the user. The controller <NUM> further controls the internal display <NUM> to display the detected image from the one or image detectors <NUM>, and further to display overlay information over the displayed detected image. The overlay information may include text and/or symbols, for example. The internal display <NUM> may be a virtual reality display, or may be an augmented reality display, for example.

The overlay information may include warning information, emergency information, checklist information, or replacement display information, in a similar fashion to the HWD <NUM> of <FIG>. The overlay information may include critical information. The eye tracker <NUM> may be arranged to track a position and orientation of the eye <NUM> of a user wearing the HWD. The head tracker <NUM> may be arranged to track a position and orientation of the head of a user wearing the HWD. The controller <NUM> may confirm that the user is viewing the critical information based on the tracked eye position and orientation of the user.

The controller <NUM> may be configured to provide foveated rendering. In this regard, the controller <NUM> may be configured to increase a resolution of a region of the internal display <NUM> viewed by an eye <NUM> of a user and to decrease a resolution of another region of the internal display not viewed by the user based on the tracked eye position of the user. In this case, the processing time of the controller may be reduced since the regions of the internal display not viewed are lower resolution and thus the rendering time for the lower resolution regions is reduced.

<FIG> illustrates an arrangement of the internal display <NUM> from a side view relative to the user eye <NUM>, at least one image detector <NUM>, and the external region <NUM>, while <FIG> illustrates an arrangement of the internal display <NUM> from a front view relative to the external region <NUM>.

<FIG> illustrates an example of an image <NUM> detected by the at least one image detector <NUM> of an external region <NUM> according to the inventive concepts disclosed herein. The image <NUM> of the external region <NUM> may be a view within a cockpit of an aircraft, or within a drivers region of a vehicle, for example. The image of the external region <NUM> may include a number of portions. For example, in the case that the image <NUM> of external region <NUM> is a view within a cockpit of an aircraft, the portions may include window portions <NUM>, which provide a view of outside the cockpit, and cockpit instrument sections 154A, 154B and 154C corresponding to flight instruments. The cockpit instrument sections may include displays. The external region <NUM> may include a region external to the cockpit, which would enclose the HWD <NUM>.

In the case that the internal display <NUM> is a virtual reality display, the display <NUM> may display only rendered images. In this case the internal display <NUM> displays the image from the at least one image detector <NUM>, and further the controller <NUM> controls the internal display <NUM> to display overlay information over the image from the at least one image detector <NUM>.

In some applications, such as within a cockpit of an aircraft where the internal display <NUM> is a virtual reality display and only rendered information is displayed on the internal display <NUM>, the at least one image detector <NUM> may be arranged to only image an external region separate from the cockpit instrument sections. In particular, the at least one image detector <NUM> may be arranged to image regions where the users hands may be, and regions beyond the cockpit, such as regions seen through the window portions <NUM>, to allow for navigation of the aircraft by a pilot. In this case, the image information of the instrument sections 154A, 154B and 154C within the cockpit are provided by the controller <NUM>, and are not based on images from the at least one image detector <NUM>.

In the case that the internal display <NUM> is an augmented reality display, the internal display <NUM> displays an image from the external region <NUM> as seen by the user eye <NUM>, the image from the at least one image detector <NUM>, and the display overlay information from the controller <NUM>. In this case the display <NUM> may be transparent to allow an image from the external region <NUM> to be seen by a user eye <NUM>.

In some application, such as within a cockpit of an aircraft where the internal display <NUM> is an augmented reality display and the internal display <NUM> displays an image from the external region <NUM> as seen by the user eye <NUM> through a transparent display, the at least one image detector <NUM> may be arranged to image an external region within the cockpit as well beyond the cockpit, such as another external region which may overlap the external region.

Embodiments of the inventive concepts disclosed herein regarding HWDs with variable transparency areas provide for functions such as enhancement, replacement, augmentation and occlusion in displays. The HWDs provide functionality for providing check lists at appropriate times, replacing the view of failed displays, providing visual information for emergency situations, and providing peripheral cue augmentation.

Claim 1:
A head worn display, HWD, (<NUM>) comprising:
a head attachment region (<NUM>);
an internal display (<NUM>) viewable by a user and including variable transparency areas (<NUM>),
the internal display (<NUM>) arranged to display at least a portion of an external region (<NUM>) external to the HWD (<NUM>);
and
a controller (<NUM>) configured to control the variable transparency areas (<NUM>) to block view of a region of the external region (<NUM>) and to control the internal display to overlay information in a region of the internal display corresponding to the region of the external region which is blocked;
characterized in that the controller (<NUM>) is configured to control the internal display (<NUM>) to display direction information in a first overlay region in an inner region of the internal display (<NUM>), and
to control the internal display to display peripheral information in a second overlay region in a peripheral region of the internal display (<NUM>),
wherein the direction information directs a user to view the peripheral information in the peripheral region of the internal display (<NUM>).