Patent Description:
Augmented reality devices also known as wearable display devices are designed to overlay virtual content on the real world. One challenge with incorporating the virtual content naturally with real-world content is incorporating the virtual content at an apparent depth that allows the virtual content to interact with real-world objects. Otherwise, the virtual content appears more as a two-dimensional display not truly integrated into the three-dimensional real-world environment. Unfortunately, augmented reality systems capable of displaying virtual content at varying depths tend to be too large or bulky for comfortable use or are only able to display virtual content at discrete distances from a user. Another challenge with displaying virtual content to a user is that certain types of displays may have a limited field of view incapable of providing a truly immersive virtual content. For these reasons, a small form factor device capable of accurately positioning virtual content at any desired distance across an immersive field of view would be desirable. Relevant prior art is found in <CIT>.

The inventive wearable display device is defined in claim <NUM>.

Representative applications of methods and apparatus according to the present application are described in this section. These examples are being provided solely to add context and aid in the understanding of the described embodiments. It will thus be apparent to one skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order to avoid unnecessarily obscuring the described embodiments. Other applications are possible, such that the following examples should not be taken as limiting.

Augmented Reality (AR) devices are configured to overlay virtual content on the real world. The virtual content can include information related to nearby real-world objects or people. In some instances, the virtual content would apply only to a general area and might not need to be associated with any viewable real-world objects. However, in many cases it is desirable to incorporate virtual content with real-world objects. For example, virtual content can include characters that interact with the user and/or objects in the real world. In order to carry this incorporation of virtual content out in a more realistic manner, the virtual content can be displayed in a manner that makes it appear to be positioned at a distance away from a user that corresponds to the real-world object(s) that that virtual content is interacting with. This co-location of virtual and real-world content can be helpful in improving user immersion. Unfortunately, many AR devices are only configured to display content at a single fixed distance from a user, which can affect how realistically the virtual content is incorporated into the real-world environment. This limitation can be more noticeable when virtual content is traveling directly towards or away from a user as apparent changes in depth can be limited to an object appearing larger or smaller. The ability to accurately portray depth information can also be beneficial in the display of Virtual Reality (VR) environments, where virtual content hides a user's view of real world objects.

One solution to establishing virtual content at variable distances from a user of an AR device is to incorporate tunable lenses into a transparent display system of the AR device. The tunable lenses can be configured to cooperate to alter an apparent position of virtual content with respect to a user. The tunable lenses or varifocal elements can take many forms, including e.g., liquid crystal lenses, tunable diffractive lenses or deformable mirror lenses. In general, any lens that could be configured to change shape or configuration to adjust incoming light in a way that changes the apparent depth of virtual content of an AR device could be applied. The tunable nature of the lenses or varifocal elements beneficially allows virtual content to appear to be positioned at almost any distance from the user of the AR device.

The tunable lenses can be positioned on forward and rear-facing surfaces of a transparent or translucent display system. A first tunable lens on the rear-facing or user-facing side of the display can be configured to alter the incoming light generated by the AR device in order to cause the incoming light to display virtual content that appears to be a desired distance from the AR device. A second tunable lens on the forward-facing or world-facing side of the display can be configured to cooperate with the first tunable lens by assuming a complementary configuration that cancels out at least some of the adjustments made by the first tunable lens. In this way, light reflecting off real-world objects and passing through both the first and second tunable lenses before arriving at a user's eyes is not substantially distorted by the first tunable lens.

According to the invention, the second tunable lens allows some changes made by the first tunable lens to be applied to the light arriving from the real-world objects. For example, the tunable lenses can be configured to apply near-sighted, far-sighted and/or astigmatism corrections for users benefitting from vision correction. These type of corrections could be applied equally to light associated with both virtual content and real-world objects. The correction could take the form of an offset between the first and second tunable lenses. In such a configuration, the second tunable lens would not be completely complementary to the first tunable lens since some of the first tunable lens changes would also be applied to a view of the real-world objects.

In some embodiments, the second tunable lens can be periodically used to distort the real world view instead of just cancelling out effects created by adjustments made by the first tunable lens. In this way, the combination of tunable lenses can provide for augmented virtuality, mediated reality and other types of experiences that manipulate the real as well as virtual content.

In some types of display devices the index of refraction of certain optical components can limit the ability of the display device to generate a field of view large enough to provide a user with an immersive augmented reality experience. One solution to this problem is to equip the display device with a tunable lens. The tunable lens can be used as an optical steering device by shaping the lenses to shift light emitted along the periphery of the device towards the eyes of a user. In this way, the effective viewing angle can be substantially increased by the tunable lens. According to the invention, a position at which light exits the display device can be sequentially shifted in a repeating scan pattern to produce a composite image. The optical steering device can be sequentially reshaped to optimize the optical steering device for each position in the scan pattern. For example, a first position in the scan pattern could be positioned on the far right side of the display device, while another position in the scan pattern could be near the bottom of the display device. By changing the optical steering device from shifting the light to the left in the first position to shifting the light upward in the second position the user can enjoy a wider field of view. By continuing to update the optical steering device in accordance with a current position of the scan pattern, portions of the light that would otherwise fall outside of a user's field of view become viewable.

These and other embodiments are discussed below with reference to <FIG>; however, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes only and should not be construed as limiting.

<FIG> shows an exemplary user wearing an augmented reality (AR) device <NUM>. AR device <NUM> can be configured to display virtual content that appears to be located in various locations across a room <NUM>. For example, virtual content <NUM> can be overlaid across a wall-mounted object <NUM>. Wall mounted object <NUM> can take the form of a picture or television mounted to a wall of room <NUM>. In this way, an appearance of wall-mounted object <NUM> can be altered by virtual content <NUM>. Similarly, AR device <NUM> could be configured to project virtual content <NUM> on couch <NUM> in a way that creates the impression that an object or personage is resting on the couch. However, in order to realistically portray the virtual content in relation to other objects in room <NUM> it is also important to establish the virtual content at a comparable distance from the user. A depth detection sensor can be used to characterize the distance of various objects from the user. Information retrieved by the depth sensor can then be used to establish a distance for virtual content associated with objects adjacent to the virtual content. This becomes more complex when the virtual objects change distances from AR device <NUM>. For example, virtual content <NUM> can take the form of a walking person taking a motion path that takes the person around table <NUM>. Data retrieved by the depth sensor of AR device <NUM> can be used to define a motion path that avoids table <NUM> as virtual content <NUM> moves from position <NUM>-<NUM> to position <NUM>-<NUM>. To accurately portray the position of virtual content <NUM> across its entire motion path, the perceived distance between AR device <NUM> and virtual content <NUM> should be constantly reduced.

<FIG> shows a display system <NUM> capable of displaying projected content at any distance. A projector <NUM> can display virtual content upon tunable lens <NUM>. Tunable lens <NUM> can then change its optical configuration to adjust a depth at which the projected content is displayed. Tunable lens <NUM> could leverage any of a number of technologies, including e.g., a liquid crystal lens. When tunable lens <NUM> is a liquid crystal lens, the lens can be configured to change its phase profile in accordance with an amount of voltage applied to the liquid crystal lens. While this configuration works well to adjust the depth of the virtual content, light arriving from real-world objects <NUM> and <NUM> would be undesirably distorted. For example, an apparent position of real-world objects <NUM> and <NUM> could be shifted closer or farther from the user as indicated by the two-headed arrows. For this reason, use of display system <NUM> with an AR device could be problematic because of the undesired distortion of light from real-world objects since one object of augmented reality is for the user to be able to maintain sight of a majority of the real world.

<FIG> shows a display system <NUM> capable of adjusting the apparent distance to virtual content without affecting the appearance of real-world content. This is accomplished by projecting virtual data between tunable lenses <NUM> and <NUM> with a waveguide <NUM> that redirects light from projector <NUM> between tunable lenses <NUM> and <NUM> and then through tunable lens <NUM> and towards the eye of a user. In this way, the light emitted by projector <NUM> can be adjusted by tunable lens <NUM>. Tunable lens <NUM> can be configured to adjust in a manner opposite to tunable lens <NUM>. The effect of this is that any light originating from real-world objects <NUM> or <NUM> can pass through depth display system <NUM> substantially unaffected. In this way, the virtual content from projector <NUM> can be the only content that undergoes a focus shift, resulting in a shift in apparent position limited to the virtual content emitted by the projector.

While tunable lens <NUM> can be configured to prevent any changes made by tunable lens <NUM> from being applied to the perception of real-world objects, in some embodiments, tunable lens <NUM> can be configured to cooperate with tunable lens <NUM> to, e.g., correct a user's vision. A vision correction could result in a multi-diopter change being applied by tunable lens <NUM> that could be equally applied to real-world objects <NUM> and <NUM> on account of tunable lens <NUM> not fully cancelling out the effects of tunable lens <NUM>. For example, tunable lens <NUM> could be reconfigured to apply a +<NUM> diopter adjustment. Tunable lens <NUM> could then apply no diopter adjustment at all so that both virtual content <NUM> and real-world objects undergo a +<NUM> diopter change, thereby allowing a user normally in need of a +<NUM> diopter vision correction to wear display system <NUM> without needing any additional vision correction. With such a vision correction scheme in place, movement of virtual content <NUM> could involve changing the diopter adjustment of tunable lens <NUM> to +<NUM> and the diopter adjustment of tunable lens <NUM> to -<NUM> in order to maintain a +<NUM> diopter offset for vision correction. Similarly, tunable lens <NUM> could be configured to apply an astigmatism adjustment that is not canceled out by tunable lens <NUM>.

The configuration shown in <FIG> can be operated in other ways that allow for the tunable lenses to apply different effects. In some embodiments, the tunable lenses can be configured to purposefully throw real-world objects out of focus to allow a user to focus on virtual content <NUM>-<NUM>. For example, it could be desirable for a software developer to, in a controlled gaming or entertainment environment, focus the user's attention on a message or even to enter into a more immersive virtual environment. By throwing real-world objects out of focus, the system would allow the system to mask out any distracting real-world stimulus without having to generate light to block the field of view across the entire display. In this way, the tunable optics can be used to shape the augmented reality experience.

<FIG> shows a top view of one specific configuration in which diffractive optics of a waveguide are arranged to guide three different colors of light emitted by a projector between tunable lenses and then towards a user. In some embodiments, waveguide <NUM> can include three discrete light paths <NUM>-<NUM>, <NUM>-<NUM> and <NUM>-<NUM> for different colors of light such as, e.g., red green and blue. Each of light paths <NUM> can utilize diffractive optics to direct light from a projector <NUM> between tunable lenses <NUM> and <NUM> and then out through tunable lens <NUM> towards the eye of a user. Waveguide <NUM> can be arranged in a way that causes the resulting virtual content to appear to be positioned at infinity when tunable lens <NUM> is not applying any changes to the light coming from waveguide <NUM>. In such a configuration, tunable lens <NUM> could be configured to decrease the apparent distance between the user and the virtual content being projected through the diffractive optics by varying amounts based on a desired position of the virtual content with respect to the user and other real-world objects. As depicted, the light from real-world objects remains substantially unaffected by tunable lenses <NUM> and <NUM> while the light passing through waveguide <NUM> is affected by tunable lens <NUM>.

<FIG> show a transparent display <NUM> of an AR device <NUM> displaying first virtual content <NUM> and second virtual content <NUM>. <FIG> depicts arrows that show how virtual content <NUM> and <NUM> move across transparent display <NUM> over a particular period of time. During this movement, virtual content <NUM> travels farther away from AR device <NUM> and virtual content <NUM> travels closer to AR device <NUM>. Because transparent display <NUM> includes tunable lenses for adjusting the apparent depth of the virtual content, an upper region <NUM> of transparent display <NUM> can be optically configured to display virtual content <NUM> moving farther away from AR device <NUM> and lower region <NUM> can be configured to display-virtual content <NUM> moving closer to AR device <NUM>. Transition region <NUM> can take the form of a region where the shape of the tunable lenses is gradually adjusted to accommodate the different optical configurations and prevent the appearance of a visual seam between upper and lower regions <NUM> and <NUM>. Transition region <NUM> can be larger or smaller depending on the amount of difference between regions <NUM> and <NUM>. While real-world object <NUM> is positioned within transition region <NUM>, it should be appreciated that when transparent display <NUM> includes two tunable lenses that cooperate to prevent distortion of real-world objects that even transition region <NUM> would have little to no effect on the appearance of real-world object <NUM>. For this reason, a processor of AR device <NUM> attempting to determine suitable areas of display <NUM> for upper region <NUM> and lower region <NUM> would only need to consider the path of motion for the virtual content when determining how to vary the optical configuration for independently moving virtual content.

<FIG> shows an exemplary embodiment where virtual content <NUM> is in motion and virtual content <NUM> remains stationary. In such a configuration motion region <NUM> can take up most of the viewable area of transparent display <NUM>, while stationary region <NUM> can take up a much smaller area that limited primarily to virtual content <NUM>. Furthermore, motion region <NUM> can alter an apparent distance between AR device <NUM> and virtual content <NUM>, while stationary region <NUM> can maintain the apparent distance to virtual content <NUM>. This narrow stationary region <NUM> can be even more convenient where head movement of the user is deemed unlikely or where the location of the virtual content within transparent display <NUM> is not governed by head movement of the user. For example, virtual content <NUM> could take the form of status information such as time of day, battery charge or navigation information. This type of information could be distracting if it were also to move with whatever other virtual content the user was interacting with. It should also be noted that again real-world content <NUM> remains unaffected by apparent depth changes affected by the tunable lenses of transparent display <NUM>.

<FIG> show side views of tunable lens <NUM> and how tunable lens <NUM> can be adjusted to accommodate different virtual content positions. <FIG> shows how tunable lens <NUM> can be substantially rectangular in shape and form a lens element <NUM> within the rectangular volume. Lens element <NUM> can be configured to reshape light emitted from a waveguide in order to establish virtual content at a desired distance from a user of an AR device. When tunable lens <NUM> takes the form of a liquid crystal lens, lens element <NUM> can change shape into lens element <NUM> in response to a voltage being applied to tunable lens <NUM>. The increased depth and curvature of lens element <NUM> can cause virtual content to appear closer the AR device than lens element <NUM>. In this way, tunable lens can be configured to change the apparent distance to virtual content viewed from an AR device.

<FIG> show how tunable lens <NUM> can be adjusted to accommodate motion of multiple independently moving objects represented by virtual content. In particular, <FIG> can show how tunable lens <NUM> would have to move to accommodate the virtual content motion depicted in <FIG>. <FIG> could correspond to the situation where virtual content <NUM> and <NUM> begin at the same distance from an AR device. <FIG> shows how tunable lens <NUM> transitions from forming lens element <NUM> to lens element <NUM>. The portion of lens element <NUM> corresponding to upper region <NUM> can have a thinner effective shape and smaller effective curvature to give the appearance of virtual content <NUM> moving farther away from the AR device while the portion of lens element <NUM> corresponding to lower region <NUM> can have a thicker effective shape and larger effective curvature to give the appearance of virtual content <NUM> moving closer to the AR device. Transition region <NUM> includes a gradient that smoothly changes the effective thickness of lens element <NUM> without creating a visible line affecting the real-world view through tunable lens <NUM>.

<FIG> shows a flow chart depicting a method for displaying virtual content at multiple depths using a small form factor AR device. At <NUM>, a depth sensor of an AR device characterizes real-world objects within a field of view of a user of the AR device by determining a distance between the user and the real-world objects. At <NUM>, a processor of the AR device is configured to determine a location or motion path for first virtual content relative to the characterized real-world objects. At <NUM>, an optical configuration of a first tunable lens of the AR device is configured for initial display of the first virtual content. At <NUM>, an optical configuration of a second tunable lens of the AR device is configured to prevent the first tunable lens from adversely affecting the view of real-world objects. This is accomplished by an optical configuration of the second tunable lens that cancels out at least a portion of the optical effects applied by the first tunable lens for the real-world objects. It should be noted that in some cases the second tunable lens can be complementary to the first tunable lens to cancel effects of the first tunable lens on the appearance of real-world objects. In some embodiments, certain vision enhancements can be applied by leaving some of the adjustments made by the first tunable lens unchanged. In some embodiments, where display of second virtual content is desired, the AR device can be configured to check to see whether the first and second virtual content should be the same distance from the user. At <NUM>, AR device can maintain the optical configuration by continuing to adjust the tunable lenses to track the position of the first and second virtual content.

At <NUM>, when first and second virtual content are at different distances from the user, the processor can be configured to apply different optical configurations to different regions of the AR device display using the tunable lenses. In this way, a user can be presented with virtual content at different distances from the user. In some embodiments, the second virtual content can be purposefully left out of focus when the user's attention is meant to be focused on the first virtual content. For example, focus can be transitioned to the second virtual content once interaction with the second virtual content is desired by the user or queued by a piece of software being executed by the AR device. In some embodiments, focus transitions between virtual content at different distance from the user can be queued by eye tracking sensors configured to determine whether the user is focusing on a particular virtual object. In other embodiments, a user can manually select virtual content for interaction at which point focus could be adjusted to properly depict the distance between the selected virtual object and the user. Imaging software can be used to apply a blurring effect to any virtual content projected by the AR device that is outside of the current depth plane to avoid any impression that all virtual content is the same distance from the user.

<FIG> show various embodiments configured to direct light from a display device into an eye of a user. <FIG> shows a top view of display device <NUM>, which includes a light projector <NUM> and a waveguide <NUM> configured to redirect light <NUM> emitted by projector <NUM> towards an eye <NUM> of a user. While waveguide <NUM> can be configured to emit imagery from thousands or even millions of locations, <FIG> shows light being emitted from five exemplary locations from which five output cones <NUM>-<NUM> through <NUM>-<NUM> are depicted. Each of output cones <NUM> represent light emitted from each location being spread across an angle <NUM>, which can be referred to as a service angle. As depicted, the limited size of each output cone prevents the light exiting waveguide <NUM> along output cones <NUM>-<NUM> and <NUM>-<NUM> from arriving at eye <NUM> of the user. In some embodiments, angle <NUM> is limited below a desired threshold by certain characteristics of the display technology such as for example the material refractive index of waveguide <NUM>. Additional details regarding light fields and waveguide-based display devices are provided in <CIT>, entitled "HIGH RESOLUTION HIGH FIELD OF VIEW DISPLAY".

<FIG> shows how display device <NUM> can incorporate one or more optical steering components, such as an optical steering device <NUM> configured to sequentially shift light <NUM> exiting waveguide <NUM> in different directions to expand the effective viewing angle <NUM> to angle <NUM>, as depicted. In this way, the user's eye <NUM> is able to view a wider field of view due to the larger effective angle <NUM> created by shifting output cones <NUM>-<NUM> - <NUM>-<NUM> in different directions. As depicted, the expanded effective angle <NUM> allows at least some of the light from output cones <NUM>-<NUM> and <NUM>-<NUM> to arrive at the user's eye <NUM>. In some embodiments, optical steering component <NUM> can take the form of one or more tunable prisms capable of assuming multiple different optical configurations (e.g. a liquid crystal lens having a reconfigurable phase profile). Each optical configuration can be configured to shift the direction of output cones <NUM> in a different direction. While <FIG> only shows light <NUM> being steered in two different directions, it should be appreciated that light <NUM> can be steered in many other different directions. Furthermore, it should be appreciated that in addition to a tunable prism other optical elements could be configured to redirect light towards the eyes of a user and that the exemplary prism embodiment should not be construed as limiting the scope of the disclosure.

Examples of other such optical elements (e.g., time-varying gratings) are described in further detail in <CIT>. In some examples, a polymer dispersed liquid crystal grating or other tunable grating may be implemented as optical steering components and used to steer output cones <NUM>-<NUM> - <NUM>-<NUM> by modifying an angle of TIR waveguided light, an angle at which light is outcoupled by an outcoupling optical element of the waveguide <NUM>, or a combination thereof. In some embodiments, one or more metasurfaces (e.g., made from metamaterials) may be implemented as optical steering components. Further information on metasurfaces and metamaterials that may be used as optical steering components in various embodiments of this disclosure can be found in <CIT>, <CIT>, and <CIT>. As such, it should be appreciated that optical steering components may be switchable or otherwise controllable to operate in a discrete number of different steering states, and that exemplary tunable optical steering devices should not be construed as limiting the scope of the disclosure.

<FIG> show exemplary scan patterns that can be generated by a suitable display device and help to expand the field of view of the display device. <FIG> shows a first scan pattern that includes four different image locations <NUM>, <NUM>, <NUM> and <NUM>. Each of the depicted image locations can represent an aggregate of the light emitted from the display device at a particular point in time. In some embodiments, light can be delivered to locations <NUM> - <NUM> in numerical order. An optical steering device can then be used to shift the light back towards the eye of a user in accordance with the active image location. For example, when image location <NUM> is active the optical steering device can be configured to shift light downwards towards an eye of a user. When a video source is being presented by the display device, portions of a video frame corresponding to each image location can be sequentially displayed at each of the four location for each frame of the video. For example, when the video source has a frame rate of <NUM>/<NUM> of a second, the corresponding portion of the video frame can be displayed at each location for <NUM>/<NUM> of a second. In this way, an expanded field of view can be achieved without a frame rate reduction. In this way, the resulting image created by the scan pattern maintains a fluid frame rate and also generates a composite image with a substantially higher spatial resolution than would be possible using a single stationary image location. For example, an image projector only capable of displaying <NUM> lines of vertical resolution could reproduce an image or video source with more than <NUM> lines of resolution using the aforementioned scan techniques. Additional details regarding scan patterns, tiling functionality, and tiled display configurations are provided in <CIT>.

<FIG> also shows how in some embodiments, portions of adjacent image location can overlap, as indicated by the hashed regions shown in <FIG>. This results in content within the image locations overlapping. The overlap can be used to further improve certain aspects of a composite image generated by the display device. For example, an increase in resolution within a central region <NUM> can be achieved by applying one or more super-resolution techniques. In particular, the portions of each image frame that overlap can be sub-sampled and slightly offset allowing an increase in pixel density in lieu of having pixels stacked atop one another. This produces a super-resolution effect in portions of the display with overlapping regions. For example, in embodiments where a display processor is capable of generating <NUM> resolution imagery (i.e. <NUM> lines of vertical resolution), the <NUM> resolution imagery could be used to achieve the super-resolution effect using an image source normally only capable of generating substantially lower resolutions by distributing the pixels within overlapped regions of the scan pattern. Furthermore, when a high frame rate video file is being displayed, each sequentially displayed frame can be associated with a different frame of the video. For example, when playing back a <NUM> frames per second video source, portions of the display within central region <NUM> could enjoy the full <NUM> frames per second frame rate, while non-overlapped regions would only be updated at a rate of <NUM> frames per second. Overlapped regions near central region <NUM> could be refreshed at a rate of <NUM> or <NUM> frames per second depending on the number of overlapped locations in a particular region.

<FIG> shows a second scan pattern with a large central region <NUM>. This second scan pattern results in a ninth of the total image being overlapped by each of image locations <NUM> - <NUM>. In this way, a resolution or frame rate within central region <NUM> can be substantially greater than in the non-overlapped regions. The depicted scan pattern can achieve a resolution or frame rate increase of up to four times, which corresponds generally to the number of overlapping frames. This type of scan pattern can be particularly beneficial when content of interest is located in the central region of the display. In some embodiments, the scan pattern can be changed to create increasing amounts of overlap in situations where less virtual content is being presented in peripheral regions of the display.

<FIG> shows how imagery positioned in each of image locations <NUM> - <NUM> cooperatively generates a composite image <NUM> that as depicted takes the form of a desk lamp. In addition to creating a composite image <NUM> that is larger than any single one of image locations <NUM> - <NUM>, the sequential display of imagery at each of image locations <NUM> - <NUM> allows optical properties of the display to be changed in accordance with which of image locations <NUM> - <NUM> is currently active during a given scan. For example, when imagery is being displayed at image location <NUM>, the optical properties could be adjusted to shift light representing the base of lamp <NUM> up towards the user's eye. In some embodiments, the location of the scan pattern can be positioned in a location of the display that places a virtual image such as lamp <NUM> within a central region of the scan pattern. This allows central and/or important features of the image to be displayed in a larger number of the image locations.

<FIG> show how the first scan pattern depicted in <FIG> can be shifted around within display region <NUM>. <FIG> shows the first scan pattern in the upper left corner of display region <NUM> at a time t<NUM>. <FIG> shows the first scan pattern shifted towards the upper right corner of display region <NUM> at a time t<NUM>. In some embodiments, the display device can include an eye gaze tracker. Sensor data provided by the eye gaze tracker can be utilized to shift the scan pattern to a location within display region <NUM> corresponding to a user's current focus point. In some embodiments, this sensor data can help keep central region <NUM> in a location that covers a user's foveal vision (i.e. that portion of a user's vision with the highest acuity). By continually adjusting the scan pattern in this manner a user's impression of immersion can be improved. This method can be particularly effective when prominent content frequently shifts away from a central portion of display region <NUM>. Exemplary systems and techniques for performing foveal tracking, rendering foveated virtual content, and displaying foveated virtual content to a user are described in further detail in <CIT>, entitled "HIGH RESOLUTION HIGH FIELD OF VIEW DISPLAY".

<FIG> shows the first scan pattern shifted again towards a lower portion of display region <NUM>. <FIG> also shows how image locations <NUM> - <NUM> can change in accordance with the position of the first scan pattern and/or to better represent content being provided to a user. For example, an area across which the first scan pattern extends could be reduced in order to better represent a virtual image that is much smaller than the standard scan pattern size. In some embodiments, changing the scan pattern can help to optimize overlapping regions of the scan pattern for a particular representation of virtual content.

10A - 10E show various phase profiles for an optical steering device similar to optical steering device <NUM>. In particular, <FIG> shows a front view of an exemplary optical steering device <NUM> in a first optical configuration <NUM> for shifting light vertically. <FIG> shows a cross-sectional side view of optical steering device <NUM> in accordance with section line A-A. Optical steering device <NUM> can take the form of a liquid crystal lens capable of changing its phase profile in accordance with an amount of voltage applied to the liquid crystal lens. More specifically, optical steering device <NUM> may include two conducting layers with structures capable of producing an electric field responsive to application of voltage thereto. In this way, optical steering device <NUM> can shift between multiple different optical configurations. The first optical configuration <NUM> can have a phase profile with multiple refractive indexes within optical steering device <NUM>. In some examples, the local refractive index of the optical steering device <NUM> may be tailored to meet a prism function or another desired optical function. In at least some of these examples, the optical steering device <NUM> may exhibit a relatively large phase gradient (e.g., ~π rad/µm). In particular, the refractive index can vary in accordance with a saw-tooth profile, as depicted, each of the teeth can be configured to receive a portion of light <NUM> and emit light <NUM> in a different direction. The size and/or spacing of the teeth can be adjusted to reduce or increase the change in the angle of light passing through optical steering device <NUM>. For example, the angle of each wedge could be gradually reduced as the pattern of wedges approaches a central region of the display. This first optical configuration <NUM> could be used to shift frames of a scan pattern located in a lower central region of the display.

<FIG> shows a front view of optical steering device <NUM> in a second optical configuration <NUM>. <FIG> shows a cross-sectional top view of optical steering device <NUM> in the second optical configuration in accordance with section line B-B. In the second optical configuration, optical steering device <NUM> is configured to shift light <NUM> laterally. As depicted, optical steering device <NUM> receives light <NUM> and outputs laterally shifted light <NUM>. While both the first and second depicted optical configurations shift the direction of the light they do so only vertically or horizontally. It should be appreciated that in some embodiments, two optical steering devices can be layered atop one another to shift the received light both vertically and horizontally.

10E shows how by orienting a series of tooth-shaped ridges diagonally across optical steering device <NUM>, light can be shifted both vertically and horizontally by a single optical steering device <NUM>. 10E also shows how the depicted diagonal configuration can accomplish the shift in light that would otherwise utilize one-dimensional optical steering devices <NUM> and <NUM>. The depicted configuration could be assumed by optical steering device <NUM> during frames of a scan pattern in which light exits the waveguide through a lower right region of a display region. It should be noted that while multiple optical configurations have been depicted, optical steering device <NUM> can be rearranged in many other configurations that have not been depicted.

<FIG> show how an optical steering device can include lenses stacked atop one another to shift incoming light both vertically and horizontally. <FIG> shows a perspective view of the phase shift of optical steering device <NUM>, which includes horizontal shift lens <NUM> and vertical shift lens <NUM>. The two lenses can be stacked atop one another in order to redirect light incident to the lenses. This optical configuration allows incoming light to be shifted both vertically and horizontally. <FIG> shows an optical steering device <NUM> having a reduced thickness achieved by using an array of multiple lenses. In some embodiments, a liquid crystal lens can be used to form an optical configuration equivalent to multiple horizontal shift lenses <NUM> and vertical shift lenses <NUM>, as depicted in <FIG>.

<FIG> show a cross-sectional side view and top view respectively of a liquid crystal lens <NUM> having a Fresnel lens configuration. The Fresnel lens configuration can take the form of an optical steering device in order to magnify or de-magnify select virtual content. In particular, <FIG> depicts a Fresnel lens configuration configured to both change the magnification of and laterally shift light passing through liquid crystal lens <NUM>. This type of configuration could be incorporated into any of the optical steering devices described herein. In some embodiments, a user could request magnification of a particular region of the screen. In response, the optical steering device could be configured to form a Fresnel lens configuration over the particular region in order to magnify the content without having to change or update the light generating a particular image stream being viewed by the user.

<FIG> show different ways in which optical steering devices can be incorporated into augmented reality display devices. In some embodiments, the optical steering devices can take the form of liquid crystal lenses capable of selectively changing refractive index in order to change the direction, perspective and/or magnification of light incident thereon. <FIG> shows a cross-sectional top view of display device <NUM>. Optical steering devices <NUM> and <NUM> are positioned on opposing sides of waveguide <NUM> of display device <NUM>. Waveguide <NUM> can include one or more discrete pathways for carrying different colors of light to an eye <NUM> of a user of display device <NUM>. Optical steering devices <NUM> and <NUM> can have a substantially complementary configuration that allows for light <NUM> reflected off real-world objects to pass through both optical steering devices to reach eye <NUM> in a substantially undistorted manner. Light <NUM> carried by waveguide <NUM> can then be configured to visualize virtual content by undergoing beam steering in accordance with one or more scan patterns without adversely distorting light <NUM>. The scan patterns and sequential beam steering described previously, increase the effective field of view of display device <NUM>. <FIG> shows a wearable display device <NUM> according to the invention and how optical steering devices <NUM> and <NUM> can be incorporated with varifocal lenses <NUM> so that both dynamic focus shift and field of view expansion can be applied to light <NUM>.

<FIG> show display devices configured to receive multiple image streams. <FIG> shows a top cross-sectional view of display device <NUM> includes both waveguide <NUM> and waveguide <NUM>. Waveguide <NUM> can be configured to carry light <NUM>, which forms a wide field of view, low-resolution stream of images. As depicted, light <NUM> does not undergo any scanning pattern prior to reaching eye <NUM>. Waveguide <NUM> can be configured to carry light <NUM>, which forms a narrow field of view, high-resolution stream of images. Light <NUM> entering waveguide <NUM> can be configured to exit display device <NUM> in a region where sensors indicate the user's eyes are focused. In this way, dynamic foveation of the narrow field of view, high-resolution stream of images can be achieved resulting in a user being given the impression that all of display device <NUM> is emitting high-resolution imagery. Light <NUM> can be projected across the surface of display device and optical steering device <NUM> can dynamically steer light <NUM> in a scan pattern that increases the effective field of view for the user. The resulting enlargement of the stream of images generated by light <NUM> due to the scan pattern can help a region of a user's field of view capable of discerning the high-resolution to be fully covered by light <NUM> even when the user is focusing on content near the edge of a display region of the display device. In this way, significant savings in hardware costs and processing power can be achieved because display <NUM> need not display high-resolution imagery across a user's entire field of view.

<FIG> shows a front view of display device <NUM> , as well as representations of imagery as perceived by the user as light <NUM> and <NUM> is projected onto the retina of the user's eye <NUM>. <FIG> shows a portion of waveguide <NUM> capable of emitting light and a region of waveguide <NUM> emitting light <NUM>. Light <NUM> is depicted producing a narrow field of view image stream that shifts in a scan pattern. Light <NUM> provides a wide field of view that remains stationary. In some embodiments, the area across which light <NUM> extends represents the maximum viewable area across which light is viewable without an optical steering device. As depicted, a portion of light <NUM> can extend outside the region covered by light <NUM> on account of light <NUM> benefiting from optical steering device <NUM>, which shifts light <NUM> towards eye <NUM>.

<FIG> shows display device <NUM>, which adds optical steering devices <NUM> and <NUM> to the configuration depicted in <FIG>. In this embodiment, light <NUM>, which generates narrow field of view, high-resolution imagery, can be directed through waveguide <NUM> and light <NUM>, which generates wide field of view, low-resolution imagery, can be directed through waveguide <NUM>. Optical steering device <NUM> is configured to independently steer light <NUM>. Optical steering device <NUM> is configured to prevent light <NUM> and light <NUM> from being distorted by the steering of optical steering device <NUM>. Optical steering device <NUM> can maintain a phase profile substantially complementary to the phase profile of optical steering device <NUM>. In this way, virtual content generated by light <NUM> and <NUM> can extend across expanded fields of view, thereby further improving the immersive experience for a user of display device <NUM>. It should be noted that in some embodiments, the functionality of optical steering devices <NUM> and <NUM> can be combined into a single optical steering device capable of assuming a phase profile that both shifts light <NUM> in a desired scan pattern and preemptively compensates for any interference being generated by optical steering device <NUM>.

In some embodiments, light <NUM> and <NUM> generated by one or more projectors of display device <NUM> can be configured to display streams of imagery at substantially the same spatial resolution. Optical steering devices <NUM> and <NUM> can then be configured to act independently to apply scan patterns to light <NUM> and <NUM> in order to maximize an effective field of view of display device <NUM>. The separation between waveguides <NUM> and <NUM> can be configured to generate different apparent distances between the user and virtual content generated by light <NUM> and <NUM>. In this way, depth perception distances can be adjusted without a set of varifocal lenses, as shown in <FIG>. It should be noted that in some embodiments, waveguides at different distances and varifocal lenses can be used in combination to concurrently show virtual content at multiple different apparent distances from eye <NUM>. The varifocal lenses could then be used to change the apparent distance between eye <NUM> and virtual content as previously described.

<FIG> shows a front view of display device <NUM>, as well as representations of imagery as perceived by the user as light <NUM> and <NUM> is projected onto the retina of the user's eye <NUM>. In particular, <FIG> shows how light <NUM> can have a narrow field of view that shifts in a first scan pattern and how light <NUM> can provide a wide field of view that shifts in a second scan pattern different than the first scan pattern. In some embodiments, the scan patterns can have the same size and/or resolution. Such a configuration could have the benefit of varying the apparent distance between the user and the virtual content.

The various aspects, embodiments, implementations or features of the described embodiments can be used separately or in any combination. Various aspects of the described embodiments can be implemented by software, hardware or a combination of hardware and software. The described embodiments can also be embodied as computer readable code on a computer readable medium for controlling manufacturing operations or as computer readable code on a computer readable medium for controlling a manufacturing line. The computer readable medium is any data storage device that can store data, which can thereafter be read by a computer system. Examples of the computer readable medium include read-only memory, random-access memory, CD-ROMs, HDDs, DVDs, magnetic tape, and optical data storage devices. The computer readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Claim 1:
A wearable display device (<NUM>, <NUM>), comprising:
a first tunable lens (<NUM>, <NUM>, <NUM>, <NUM>);
a second tunable lens (<NUM>, <NUM>, <NUM>, <NUM>);
a first optical steering device (<NUM>);
a second optical steering device (<NUM>);
a waveguide (<NUM>, <NUM>, <NUM>) positioned between the first tunable lens (<NUM>, <NUM>, <NUM>, <NUM>) and the second tunable lens (<NUM>, <NUM>, <NUM>, <NUM>) and between the first optical steering device (<NUM>) and the second optical steering device (<NUM>), wherein said first and second optical steering devices have a complimentary configuration, the waveguide (<NUM>, <NUM>, <NUM>) being configured to direct light representing virtual content through the first tunable lens (<NUM>, <NUM>, <NUM>, <NUM>) and the first optical steering device (<NUM>) and towards a user of the wearable display device (<NUM>, <NUM>);
a light source (<NUM>, <NUM>, <NUM>, <NUM>) configured to emit the light representing the virtual content in accordance with a scan pattern, wherein the scan pattern results in the light being emitted through different regions of the first optical steering device (<NUM>); and
a processor configured to:
direct the first tunable lens (<NUM>, <NUM>, <NUM>, <NUM>) to change shape to alter an apparent distance between the virtual content and the user of the wearable display device, and
direct the second tunable lens (<NUM>, <NUM>, <NUM>, <NUM>) to change shape to maintain apparent distances between real world objects and the user of the wearable display device,
the processor being further configured to shape the first optical steering device (<NUM>) to shift at least a portion of the light received from the light source and exiting the waveguide through one or more peripheral regions of the first optical steering device (<NUM>) towards a central region (<NUM>), so as to shift light emitted along the periphery of the device towards an eye of the user.