Patent Description:
Virtual and augmented reality systems generally include displays that project light into the eyes of a user. Unfortunately, these systems are not designed to project content along the outer periphery of a user's field of view or beyond a small central portion of the user's field of view, due to output-angle limitations of available display technologies. This can reduce the level of immersion felt by a user of these systems that might otherwise be possible when content is delivered from angles extending all the way to an outer periphery of a user's field of view. For this reasons, mechanisms for stimulating the outer periphery of a user's field of view are desirable. Known wearable display devices are disclosed in documents <CIT>, <CIT> and <CIT>.

Further features of the invention are defined in the dependent claims.

This disclosure describes a wearable device configured to present immersive virtual, augmented and mixed reality content to a user. In an embodiment, a head-mounted display with a wraparound display assembly is provided that is configured to display content to most or all of a user's field of view. The display assembly can be configured to display content in far-peripheral regions of the user's field of view differently than content upon which a user can focus. For example, spatial or angular resolution, color resolution, refresh rate and intensity (i.e. brightness) can be adjusted to save resources and/or to bring attention to virtual content positioned within a far-peripheral region. In some embodiments, these changes can save processing resources without detracting from the user's overall experience.

This disclosure describes a head-mounted display assembly that includes the following: a first display; a second display at least partially surrounding the first display; and an attachment member configured to couple the first and second displays to the head of a user. The second display has a larger curvature than the first display.

A wearable display device is disclosed and includes the following: a frame including an attachment member configured to secure the display device to the head of a user; and a display assembly coupled to the frame, the display assembly comprising: a main display, and a peripheral display arranged along a periphery of the main display.

A display of a head-mounted display device is disclosed. The display includes the following: a first region having a first resolution; a second region at least partially surrounding the first region and having a second resolution substantially lower than the first resolution; and a transition region between the first region and the second region having a variable resolution that is lower on the side of the transition region adjacent to the first region than the side of the transition region adjacent the second region.

Numerous benefits are achieved by way of the present invention over conventional techniques. For example, embodiments of the present invention provide a superior immersive experience over head-mounted displays not targeting the far-peripheral regions of a user's field of view. Furthermore, a lower cost peripheral display can be used to cover the far-peripheral regions since the human eye is less capable of discerning high-resolution spatial and color imagery in peripheral regions of the user's field of view. For this reason, the present invention allows for a more immersive experience without adding substantially to the overall cost of the head-mounted display.

In addition, parts of the wearable frame that would by default simply act as obstructions, can now be surfaces for light display and modulation. These previously obstructing structures can be made aesthetically pleasing or interactive. These previously obstructing structures can also be made 'invisible' to the viewer by matching the displayed content to the scene behind the structure/wearable.

These and other embodiments of the invention along with many of its advantages and features are described in more detail in conjunction with the text below and attached figures.

Representative applications of methods and apparatus according to the present application are described in this section. These examples are being provided solely to add context and aid in the understanding of the described embodiments. It will thus be apparent to one skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order to avoid unnecessarily obscuring the described embodiments. Other applications are possible, such that the following examples should not be taken as limiting.

Head-mounted display devices or wearable display devices can be configured to provide an immersive user experience by projecting virtual content directly into the eyes of a user. Unfortunately, the displays associated with these types of devices do not generally extend to cover the user's entire field of view. While the user's ability to focus on objects is limited to between about <NUM> and <NUM> degrees off-axis, most user's eyes are capable of detecting content and particularly fast movement past <NUM> degrees off-axis in some directions. For this reason, to create a truly immersive experience, a display needs to be designed to cover the outer periphery of the user's vision.

One solution to this problem is to incorporate a peripheral display for displaying content to a peripheral region of a user's field of view that falls outside of a user's field of regard. The field of regard is made up of the portion of the user's field of view upon which a user can directly focus. Because the peripheral display shows content outside of the user's field of regard, the need to seamlessly blend or transition content from the peripheral display to a main display is minimal. Furthermore, since the visual acuity of a user is substantially reduced in the peripheral region, the peripheral display can run in reduced acuity modes that save power and/or processing power. For example, the peripheral display can display content at a lower spatial or angular resolution, a lower color resolution, a different intensity and/or a lower refresh rate. In some embodiments, portions of the display may not be capable of displaying high spatial, angular and/or color resolution imagery due to, e.g. reduced pixel densities. In addition to allowing the wearable device to operate at lower power levels, these lower acuity display modes allow the hardware costs associated with the peripheral display to be substantially lower on account of the peripheral display not needing to have the ability to generate high resolution imagery at high refresh rates. In some embodiments, the peripheral display can take the form of a transparent OLED (organic light emitting diode) display. The transparent OLED can include an array of pixels distributed across a transparent and flexible substrate. In some embodiments, the substrate can be formed from a blend of polymers. In other embodiments, the peripheral display can also take the form of a pico-projector projecting content onto internal and/or external surface of the wearable display device.

Another solution involves using a customized display that covers the user's entire field of view. The customized display can be designed to display content with spatial and color resolutions that decrease towards the periphery of the display. In some embodiments, the resolution can fall off gradually towards the periphery of the display. In some embodiments, the resolution change can be based on a current position of a user's eyes. For example, if an eye-tracking sensor determines the user's eyes are focused towards one side of the display, the opposite side of the display can be configured to display a commensurately lower resolution.

These and other embodiments are discussed below with reference to <FIG>; however, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes only and should not be construed as limiting.

<FIG> shows a wearable display device <NUM> that includes high-resolution main displays <NUM> and a lower resolution peripheral display <NUM> that surrounds main displays <NUM>. In some embodiments, peripheral display <NUM> can be arranged to conform to an interior-facing surface of temple arms <NUM>. It should be noted that a size of main displays <NUM> can be adjusted to coincide with an average field of regard for a user wearing wearable display device <NUM>. Selecting a display technology for peripheral display <NUM> that includes a flexible substrate material capable of bending and flexing with temple arms <NUM> can allow peripheral display <NUM> to conform with at least a portion of the interior facing surface of temple arms <NUM>. In some embodiments, the flexible substrate material can be flexible enough to accommodate temple arms <NUM> folding against the displays for storage.

<FIG> shows a wearable display device <NUM> having a single main display <NUM> that covers the field of regard for both eyes of a user operating wearable display device <NUM>. In some embodiments, main display <NUM> can utilize a different display technology than peripheral display <NUM>. For example, main display <NUM> could take the form of a light field display device, which can include one or more waveguides configured to project light fields onto the user's retina. The output of a light field display is an angular representation of content and can be configured to project varied angular resolutions. <CIT>, <CIT>, and/or <CIT>, all provide detailed examples of light field display devices capable of user as a main display. Peripheral display <NUM> could take the form of a screen-based display device, which can include a "screen" on which content is displayed (e.g., LCD, OLED, projector + projection screen, CRT, etc.). The output of this type of device is a spatial representation of content as presented on a screen. Main display <NUM> and peripheral display <NUM> can be coupled to the ears of a user by temples <NUM>.

In some embodiments, both the main display(s) and peripheral display(s) can be transparent, allowing the outside world to be viewable in areas where digital content is not being actively displayed. <FIG> shows a wearable display device <NUM> having two separate main displays <NUM> and two separate peripheral displays <NUM>. Main displays <NUM> can be configured to cooperatively cover the field of regard of the eyes of a user, while the peripheral displays <NUM> can cooperatively cover any portion of the field of view not covered by main displays <NUM>. Temples <NUM> represent attachment members suitable for engaging the ears of a user and bridge <NUM> joins the two separate main displays <NUM> together.

<FIG> show cross-sectional views of various configurations of wearable display device <NUM>. <FIG> shows how a front portion of wearable display device <NUM> can take the form of a peripheral display device <NUM>. In this way, peripheral display device <NUM> can act as a protective cover for main displays <NUM> and <NUM>. Main displays <NUM> and <NUM> are depicted including multiple different layers that represent different waveguides for directing different wavelengths of light to a user. In some embodiments, the main displays <NUM> and <NUM> can be adhered or otherwise attached to a surface of the peripheral display <NUM>. For example, such a surface of the peripheral display <NUM> can be a contiguous sheet or piece of material that extends beyond the perimeter of the main display <NUM> and the main display <NUM> so as to provide peripheral display functionality. Peripheral display <NUM> and main displays <NUM> and <NUM> can be transparent so that a user is able to perceive the outside world in addition to any virtual content generated by peripheral displays <NUM> and main displays <NUM> and <NUM>. In some embodiments, the portions of peripheral display device <NUM> that overlap main displays <NUM> and <NUM> can be configured not to display content so that preventing the displays from displaying the same content. In some embodiments, peripheral display <NUM> can be configured to display content on startup while main displays <NUM> and <NUM> go through warm up cycles. Subsequent to initialization of main displays <NUM> and <NUM>, the portions of peripheral display <NUM> that overlap main displays <NUM> and <NUM> could be disabled. In some embodiments, peripheral display <NUM> could take over for main displays <NUM> and <NUM> when interactive or high-resolution content is not being actively displayed. For example, if a user enters a configuration menu where displayed content is limited to text or simple menu structures, allowing one or more portions of peripheral display <NUM> that overlap main displays <NUM> and <NUM> to run in lieu of main displays <NUM> and <NUM> could help save power and reduce heat generation in embodiments where main displays <NUM> and <NUM> are more power-hungry and/or generate more heat than peripheral display <NUM>. For example, peripheral display <NUM> could take the form of a flexible, transparent OLED display capable of consuming less power than main displays <NUM> and <NUM> when the main displays are driven by relatively high-energy consuming light projectors.

In some implementations, some or all portions of the peripheral display <NUM> can be operated to present content in tandem with the main displays <NUM> and <NUM> for further user experience enhancement. For example, portions of the peripheral display <NUM> that are attached to or otherwise overlapping the main displays <NUM> and <NUM> could present a flash of white light while the main displays <NUM> and <NUM> present virtual content resembling fire/flames so as to simulate an explosion for a user engaged in a mixed reality gameplay experience. In another example, portions of the peripheral display that are attached to or otherwise overlapping the main displays <NUM> and <NUM> could present text and/or serve to highlight real world objects within a user's field of view. Moreover, by utilizing portions of the peripheral display <NUM> that are attached to or otherwise overlapping the main displays <NUM> and <NUM> as well as portions of the peripheral display <NUM> that are not attached to the main displays <NUM> and <NUM> (e.g., regions of the peripheral display <NUM> between and/or surrounding the outer perimeters of main displays <NUM> and <NUM>), the boundaries between the two types of display devices may be appear smoother to users. In some examples, some or all of the functionality of portions of the peripheral display <NUM> that are attached to or otherwise overlapping the main displays <NUM> and <NUM>, as described herein with reference to <FIG>, may also extend to portions of peripheral displays overlapping one or more main displays (relative to a user's field of view) as described in further detail below with reference to <FIG> and <FIG>.

<FIG> shows wearable display device <NUM> with an alternative configuration in which main displays can be positioned forward of peripheral display <NUM>. In such a configuration, main displays <NUM> and <NUM> can include a protective cover layer <NUM> that protects main displays <NUM> and <NUM> from damage. Wearable display device <NUM> can be operated in a similar manner to wearable display device <NUM>, allowing the peripheral display <NUM> to takeover operation from main displays <NUM> and <NUM> in certain situations.

Similarly, in some embodiments and as shown in <FIG>, the peripheral display could extend through a central portion of main displays <NUM> and <NUM>. In some embodiments, peripheral display <NUM> can act as a spacer to accentuate a distance between a first portion of the display and a second portion of the display. This distance can help light emitted from a portion of the main displays on an exterior facing surface of the peripheral display appear to originate from a farther away than the portion positioned along an interior facing surface of the peripheral display.

<FIG> shows a front view of wearable display device <NUM>. The front view of wearable display device <NUM> demonstrates how peripheral display <NUM> is able to border and surround both of main displays <NUM> and <NUM>. In some embodiments, a periphery of main displays <NUM> and <NUM> can have reduced spatial, angular and/or color resolution in order to blend with lower resolution data being displayed on peripheral display <NUM>. <CIT> and <CIT> both entitled "High Resolution High Field of View Display", to which this application claims priority, describe various ways in which the resolution of a projection-based display system can be configured with a varying angular resolution.

<FIG> show side views of various wearable display devices <NUM>, <NUM> and <NUM>. Wearable display device <NUM> includes a visor component <NUM>, which provides a rigid substrate to which main display <NUM> and peripheral display <NUM> can be coupled. While visor component can be optically neutral, it can also be configured to create a slight magnification or reduction of objects within the field of view of the visor. In some embodiments, visor could include a polarizing layer and/or tinted layer, which could be helpful during outside use. Peripheral display <NUM> can extend from an edge of visor component <NUM> to a periphery of main display <NUM>. The displays can be affixed to one another in many ways. For example, peripheral display <NUM> can be adhesively coupled to main display <NUM>. In some embodiments, an optically transparent frame can be positioned between visor component <NUM> and peripheral display <NUM> to help maintain a shape of peripheral display <NUM>. <FIG> shows how a peripheral display <NUM> can be adhered to an interior-facing surface of visor component <NUM>. In this way, visor component <NUM> can be configured to set a shape and position of peripheral display <NUM>. <FIG> shows wearable display device <NUM> and how peripheral display <NUM> can be adhered to a peripheral portion of visor component <NUM>. In some embodiments, peripheral display <NUM> can be affixed to a recessed region defined by visor component <NUM>. In this way, peripheral display <NUM> need only be sized to fill a portion of a user's field of view extending outside of main display <NUM>.

<FIG> shows a cross-sectional side view of wearable display device <NUM> and how main display <NUM> can be surrounded by a transparent curved optical element <NUM> that supports a periphery of peripheral display <NUM>. In some embodiments, an index of refraction of curved optical element <NUM> can be tuned to minimize distortion of light emitted by peripheral display <NUM>. In some embodiments, the transparent curved optical element <NUM> can take the form of a transparent frame that used to support and position various other components associated with wearable display device <NUM>. For example, in some embodiments, waveguides configured to transmit light into main display <NUM> can extend through an opening or channel defined by transparent curved optical element <NUM>.

<FIG> show how peripheral displays can be wrapped around the edges of a main display device and utilize various optics to direct light emitted from the peripheral displays toward reflectors that reorient the light back into the eyes of a user of the wearable display device. <FIG> shows freeform optic <NUM> surrounding main display <NUM>. Freeform optic <NUM> can include an at least partially reflective surface <NUM> configured to redirect light <NUM> emitted by peripheral display <NUM> back toward a user's eye. In this way, freeform optic <NUM> is able to expand an effective size of the active display of wearable display device <NUM> without the need for extending a peripheral display out to an extreme end of the device. An alternative embodiment is depicted by wearable display device <NUM>, which can instead include a prism <NUM> having a triangular cross-section arranged along the periphery of main display <NUM>. Prism <NUM> can redirect light <NUM> emitted by peripheral display <NUM> that wraps around the edges of main display <NUM>.

<FIG> shows a visual field diagram depicting the outer perimeter of an exemplary monocular field of view <NUM> for a human eye in two-dimensional angular space. As shown in <FIG>, temporal-nasal and inferior-superior axes of the visual field diagram serve to define the two-dimensional angular space within which the outer perimeter of the monocular field of view <NUM> is mapped. In this way, the visual field diagram of <FIG> may be seen as being equivalent or similar to a "Goldmann" visual field map or plot for a human eye. As indicated by the depicted arrangement of the temporal-nasal and inferior-superior axes, the visual field diagram shown in <FIG> represents a visual field diagram for the left eye of a human. While field of view can vary slightly from person to person, the depicted field of view is close to what many humans are capable of viewing with their left eye. It follows that a visual field diagram depicting the outer perimeter of an exemplary monocular field of view of the right eye might resemble something of a version of the visual field diagram of <FIG> in which the temporal nasal axis and the outer perimeter of the monocular field of view <NUM> have been mirrored about the inferior-superior axis. The visual field diagram of <FIG> further depicts the outer perimeter of an exemplary field of regard <NUM> for the human eye, which represents a portion of the monocular field of view <NUM> in angular space within which the person can fixate. In addition, the visual field diagram of <FIG> also depicts the outer perimeter of an exemplary foveal field <NUM> for the human eye, which represents a portion of the monocular field of view <NUM> in angular space in direct view of the fovea of the human eye at a given point in time. As depicted, a person's foveal field <NUM> can move anywhere within field of regard <NUM>. Portions of the monocular field of view <NUM> outside of foveal field <NUM> in angular space can be referred herein as the peripheral region of the person's field of view. Because of the ability of human eyes to distinguish a high level of detail outside of the foveal field <NUM> is quite limited, displaying reduced resolution imagery outside of the foveal field <NUM> is unlikely to be noticed and can allow for substantial savings on power expenditure for processing components responsible for generating content for the display.

<FIG> shows an exemplary wearable display device <NUM> configured to provide virtual content across an area suitable for covering the field of view of a user as depicted in <FIG>. Wearable display device <NUM> includes main displays <NUM> supported by frame <NUM>. Frame <NUM> can be attached to the head of a user using an attachment member taking the form of temple arms <NUM>. In some embodiments, the image quality displayed by wearable display device <NUM> can be gradually reduced in either or both of main displays <NUM> and peripheral display <NUM> so that areas near and within the field of regard have a higher quality (e.g. higher spatial and/or color resolution) than areas near the edge of main display <NUM>. In some embodiments, the periphery of main displays <NUM> can be configured to match a quality or imagery characteristic of peripheral display <NUM>. In some embodiments, the reduction in image quality can be accomplished by changing the spatial resolution, color bit depth and/or refresh rate of main display <NUM>. For example, the color bit depth could be reduced from <NUM> bits to <NUM> or <NUM> bits to reduce both the requisite processing power and peripheral display complexity. In some embodiments, the color bit depth can be reduced so that only grayscale or black and white content is displayed.

<FIG> shows field of view <NUM> and field of regard <NUM> overlaid upon one of main displays <NUM>. <FIG> shows how main display <NUM> can cover field of regard <NUM> and in cooperation with peripheral display <NUM> cover a majority of field of view <NUM> for a user of wearable display device <NUM>. While main display <NUM> is shown covering all of field of regard <NUM>, the periphery of main display <NUM> can be configured to optimize system resources by reducing the resolution of any portion of main display <NUM> not actively covering field of regard <NUM>. In some embodiments, sensors associated with wearable display device <NUM> can be configured to identify the position of the wearable display with the eyes of a user of the wearable display in order to identify regions of main display <NUM> not presenting content within field of regard <NUM>. Since eye position can vary due to the shape of a head of a user of wearable display device <NUM>, an oversized main display <NUM> can be helpful in allowing main display <NUM> to cover the full field of regard for a broad cross-section of users. In some embodiments, a registration mechanism can also help to ensure proper eye-display positioning. For example, the registration mechanism can take the form of adjustable nosepieces and temples that can be used to accommodate differing facial features by confirming a user's field of regard is covered by main display <NUM> and the user's peripheral field of view is substantially covered by peripheral display <NUM>. To help in achieving this alignment, peripheral display <NUM> can have an asymmetric shape configured to conform with a shape of a user's peripheral field of view <NUM>, as depicted. In some embodiments, a user's ability to observe real-world content surrounding wearable display device within <NUM> can be obstructed by components supporting the operation of the wearable display device. Peripheral display <NUM> can be configured to overlay content on those portions of the peripheral display that overlay the obstructing components. In some embodiments, real-world content can be displayed along the interior facing surface of temples <NUM> utilizing imagery obtained from world cameras arranged along the exterior-facing surface of temples <NUM>.

Referring now to <FIG>, an exemplary embodiment of an AR system configured to provide virtual content to a user will now be described. In some embodiments, the AR system of <FIG> may represent a system to which the wearable display device <NUM> of <FIG> belongs. The AR system of <FIG> uses stacked light-guiding optical element assemblies <NUM> and generally includes an image generating processor <NUM>, a light source <NUM>, a controller <NUM>, a spatial light modulator ("SLM") <NUM>, an injection optical system <NUM>, and at least one set of stacked eyepiece layers or light guiding optical elements ("LOEs"; e.g., a planar waveguide) <NUM> that functions as a multiple plane focus system. The system may also include an eye-tracking subsystem <NUM>. It should be appreciated that other embodiments may have multiple sets of stacked LOEs <NUM>, but the following disclosure will focus on the exemplary embodiment of <FIG>.

The image generating processor <NUM> is configured to generate virtual content to be displayed to the user. The image generating processor may convert an image or video associated with the virtual content to a format that can be projected to the user in <NUM>-D. For example, in generating <NUM>-D content, the virtual content may need to be formatted such that portions of a particular image are displayed at a particular depth plane while others are displayed at other depth planes. In one embodiment, all of the image may be generated at a particular depth plane. In another embodiment, the image generating processor may be programmed to provide slightly different images to the right and left eyes such that when viewed together, the virtual content appears coherent and comfortable to the user's eyes.

The image generating processor <NUM> may further include a memory <NUM>, a GPU <NUM>, a CPU <NUM>, and other circuitry for image generation and processing. The image generating processor <NUM> may be programmed with the desired virtual content to be presented to the user of the AR system of <FIG>. It should be appreciated that in some embodiments, the image generating processor <NUM> may be housed in the wearable AR system. In other embodiments, the image generating processor <NUM> and other circuitry may be housed in a belt pack that is coupled to the wearable optics. The image generating processor <NUM> is operatively coupled to the light source <NUM> which projects the light associated with the desired virtual content and one or more spatial light modulators (described below).

The light source <NUM> is compact and has high resolution. The light source <NUM> includes a plurality of spatially separated sub-light sources <NUM> that are operatively coupled to a controller <NUM> (described below). For instance, the light source <NUM> may include color specific LEDs and lasers disposed in various geometric configurations. Alternatively, the light source <NUM> may include LEDs or lasers of like color, each one linked to a specific region of the field of view of the display. In another embodiment, the light source <NUM> may comprise a broad-area emitter such as an incandescent or fluorescent lamp with a mask overlay for segmentation of emission areas and positions. Although the sub-light sources <NUM> are directly connected to the AR system of <FIG> in <FIG>, the sub-light sources <NUM> may be connected to system via optical fibers (not shown), as long as the distal ends of the optical fibers (away from the sub-light sources <NUM>) are spatially separated from each other. The system may also include condenser (not shown) configured to collimate the light from the light source <NUM>.

The SLM <NUM> may be reflective (e.g., a DLP DMD, a MEMS mirror system, an LCOS, or an FLCOS), transmissive (e.g., an LCD) or emissive (e.g. an FSD or an OLED) in various exemplary embodiments. The type of spatial light modulator (e.g., speed, size, etc.) can be selected to improve the creation of the <NUM>-D perception. While DLP DMDs operating at higher refresh rates may be easily incorporated into stationary AR systems, wearable AR systems typically use DLPs of smaller size and power. The power of the DLP changes how <NUM>-D depth planes/focal planes are created. The image generating processor <NUM> is operatively coupled to the SLM <NUM>, which encodes the light from the light source <NUM> with the desired virtual content. Light from the light source <NUM> may be encoded with the image information when it reflects off of, emits from, or passes through the SLM <NUM>.

Referring back to <FIG>, the AR system also includes an injection optical system <NUM> configured to direct the light from the light source <NUM> (i.e., the plurality of spatially separated sub-light sources <NUM>) and the SLM <NUM> to the LOE assembly <NUM>. The injection optical system <NUM> may include one or more lenses that are configured to direct the light into the LOE assembly <NUM>. The injection optical system <NUM> is configured to form spatially separated and distinct pupils (at respective focal points of the beams exiting from the injection optical system <NUM>) adjacent the LOEs <NUM> corresponding to spatially separated and distinct beams from the sub-light sources <NUM> of the light source <NUM>. The injection optical system <NUM> is configured such that the pupils are spatially displaced from each other. In some embodiments, the injection optical system <NUM> is configured to spatially displace the beams in the X and Y directions only. In such embodiments, the pupils are formed in one X, Y plane. In other embodiments, the injection optical system <NUM> is configured to spatially displace the beams in the X, Y and Z directions.

Spatial separation of light beams forms distinct beams and pupils, which allows placement of in-coupling gratings in distinct beam paths, so that each in-coupling grating is mostly addressed (e.g., intersected or impinged) by only one distinct beam (or group of beams). This, in turn, facilitates entry of the spatially separated light beams into respective LOEs <NUM> of the LOE assembly <NUM>, while minimizing entry of other light beams from other sub-light sources <NUM> of the plurality (i.e., cross-talk). A light beam from a particular sub-light source <NUM> enters a respective LOE <NUM> through an in-coupling grating (not shown) thereon. The in-coupling gratings of respective LOEs <NUM> are configured to interact with the spatially separated light beams from the plurality of sub-light sources <NUM> such that each spatially separated light beam only intersects with the in-coupling grating of one LOE <NUM>. Therefore, each spatially separated light beam mainly enters one LOE <NUM>. Accordingly, image data encoded on light beams from each of the sub-light sources <NUM> by the SLM <NUM> can be effectively propagated along a single LOE <NUM> for delivery to an eye of a user.

Each LOE <NUM> is then configured to project an image or sub-image that appears to originate from a desired depth plane or FOV angular position onto a user's retina. The respective pluralities of LOEs <NUM> and sub-light sources <NUM> can therefore selectively project images (synchronously encoded by the SLM <NUM> under the control of controller <NUM>) that appear to originate from various depth planes or positions in space. By sequentially projecting images using each of the respective pluralities of LOEs <NUM> and sub-light sources <NUM> at a sufficiently high frame rate (e.g., <NUM> for six depth planes at an effective full-volume frame rate of <NUM>), the system of <FIG> can generate a <NUM>-D image of virtual objects at various depth planes that appear to exist simultaneously in the <NUM>-D image.

The controller <NUM> is in communication with and operatively coupled to the image generating processor <NUM>, the light source <NUM> (sub-light sources <NUM>) and the SLM <NUM> to coordinate the synchronous display of images by instructing the SLM <NUM> to encode the light beams from the sub-light sources <NUM> with appropriate image information from the image generating processor <NUM>.

The AR system also includes an optional eye-tracking subsystem <NUM> that is configured to track the user's eyes and determine the user's focus. In one embodiment, only a subset of sub-light sources <NUM> may be activated, based on input from the eye-tracking subsystem, to illuminate a subset of LOEs <NUM>, as will be discussed below. Based on input from the eye-tracking subsystem <NUM>, one or more sub-light sources <NUM> corresponding to a particular LOE <NUM> may be activated such that the image is generated at a desired depth plane that coincides with the user's focus/accommodation. For example, if the user's eyes are parallel to each other, the AR system of <FIG> may activate the sub-light sources <NUM> corresponding to the LOE <NUM> that is configured to deliver collimated light to the user's eyes, such that the image appears to originate from optical infinity. In another example, if the eye-tracking sub-system <NUM> determines that the user's focus is at <NUM> meter away, the sub-light sources <NUM> corresponding to the LOE <NUM> that is configured to focus approximately within that range may be activated instead. It should be appreciated that, in this particular embodiment, only one group of sub-light sources <NUM> is activated at any given time, while the other sub-light sources <NUM> are deactivated to conserve power.

<FIG> illustrates schematically the light paths in an exemplary viewing optics assembly (VOA) that may be used to present a digital or virtual image to a viewer, according to an embodiment of the present invention. In some embodiments, the VOA could be incorporated in a system similar to wearable display device <NUM> as depicted in <FIG>. The VOA includes a projector <NUM> and an eyepiece <NUM> that may be worn around a viewer's eye. The eyepiece <NUM> may, for example, may correspond to LOEs <NUM> as described above with reference to <FIG>. In some embodiments, the projector <NUM> may include a group of red LEDs, a group of green LEDs, and a group of blue LEDs. For example, the projector <NUM> may include two red LEDs, two green LEDs, and two blue LEDs according to an embodiment. In some examples, the projector <NUM> and components thereof as depicted in <FIG> (e.g., LED light source, reflective collimator, LCoS SLM, and projector relay) may represent or provide the functionality of one or more of light source <NUM>, sub-light sources <NUM>, SLM <NUM>, and injection optical system <NUM>, as described above with reference to <FIG>. The eyepiece <NUM> may include one or more eyepiece layers, each of which may represent one of LOEs <NUM> as described above with reference to <FIG>. Each eyepiece layer of the eyepiece <NUM> may be configured to project an image or sub-image that appears to originate from a respective desired depth plane or FOV angular position onto the retina of a viewer's eye.

In one embodiment, the eyepiece <NUM> includes three eyepiece layers, one eyepiece layer for each of the three primary colors, red, green, and blue. For example, in this embodiment, each eyepiece layer of the eyepiece <NUM> may be configured to deliver collimated light to the eye that appears to originate from the optical infinity depth plane (<NUM> diopters). In another embodiment, the eyepiece <NUM> may include six eyepiece layers, i.e., one set of eyepiece layers for each of the three primary colors configured for forming a virtual image at one depth plane, and another set of eyepiece layers for each of the three primary colors configured for forming a virtual image at another depth plane. For example, in this embodiment, each eyepiece layer in one set of eyepiece layers of the eyepiece <NUM> may be configured to deliver collimated light to the eye that appears to originate from the optical infinity depth plane (<NUM> diopters), while each eyepiece layer in another set of eyepiece layers of the eyepiece <NUM> may be configured to deliver collimated light to the eye that appears to originate from a distance of <NUM> meters (<NUM> diopter). In other embodiments, the eyepiece <NUM> may include three or more eyepiece layers for each of the three primary colors for three or more different depth planes. For instance, in such embodiments, yet another set of eyepiece layers may each be configured to deliver collimated light that appears to originate from a distance of <NUM> meter (<NUM> diopter).

Each eyepiece layer comprises a planar waveguide and may include an incoupling grating <NUM>, an orthogonal pupil expander (OPE) region <NUM>, and an exit pupil expander (EPE) region <NUM>. More details about incoupling grating, orthogonal pupil expansion, and exit pupil expansion are described in <CIT> and <CIT>. Still referring to <FIG>, the projector <NUM> projects image light onto the incoupling grating <NUM> in an eyepiece layer <NUM>. The incoupling grating <NUM> couples the image light from the projector <NUM> into the waveguide propagating in a direction toward the OPE region <NUM>. The waveguide propagates the image light in the horizontal direction by total internal reflection (TIR). The OPE region <NUM> of the eyepiece layer <NUM> also includes a diffractive element that couples and redirects a portion of the image light propagating in the waveguide toward the EPE region <NUM>. More specifically, collimated light propagates horizontally (i.e., relative to view of <FIG>) along the waveguide by TIR, and in doing so repeatedly intersects with the diffractive element of the OPE region <NUM>. In some examples, the diffractive element of the OPE region <NUM> has a relatively low diffraction efficiency. This causes a fraction (e.g., <NUM>%) of the light to be diffracted vertically downward toward the EPE region <NUM> at each point of intersection with the diffractive element of the OPE region <NUM>, and a fraction of the light to continue on its original trajectory horizontally along the waveguide via TIR. In this way, at each point of intersection with the diffractive element of the OPE region <NUM>, additional light is diffracted downward toward the EPE region <NUM>. By dividing the incoming light into multiple outcoupled sets, the exit pupil of the light is expanded horizontally by the diffractive element of the OPE region <NUM>. The expanded light coupled out of the OPE region <NUM> enters the EPE region <NUM>.

The EPE region <NUM> of the eyepiece layer <NUM> also includes a diffractive element that couples and redirects a portion of the image light propagating in the waveguide toward a viewer's eye. Light entering the EPE region <NUM> propagates vertically (i.e., relative to view of <FIG>) along the waveguide by TIR. At each point of intersection between the propagating light and the diffractive element of the EPE region <NUM>, a fraction of the light is diffracted toward the adjacent face of the waveguide allowing the light to escape the TIR, emerge from the face of the waveguide, and propagate toward the viewer's eye. In this fashion, an image projected by projector <NUM> may be viewed by the viewer's eye. In some embodiments, the diffractive element of the EPE region <NUM> may be designed or configured to have a phase profile that is a summation of a linear diffraction grating and a radially symmetric diffractive lens. The radially symmetric lens aspect of the diffractive element of the EPE region <NUM> additionally imparts a focus level to the diffracted light, both shaping the light wavefront (e.g., imparting a curvature) of the individual beam as well as steering the beam at an angle that matches the designed focus level. Each beam of light outcoupled by the diffractive element of the EPE region <NUM> may extend geometrically to a respective focus point positioned in front of the viewer, and may be imparted with a convex wavefront profile with a center of radius at the respective focus point to produce an image or virtual object at a given focal plane.

Descriptions of such a viewing optics assembly and other similar set-ups are further provided in <CIT>, <CIT>, and <CIT>. It follows that, in some embodiments, the exemplary VOA may include and/or take on the form of one or more components described in any of the patent applications mentioned above with reference to <FIG>.

<FIG> shows how peripheral display <NUM> can conform to the contours of the face of a user <NUM>. In some embodiments, peripheral display <NUM> can have a greater curvature than main display <NUM> so that peripheral display <NUM> can contact the face of user <NUM> without requiring substantial curvature of the higher resolution main display <NUM>. Contact between peripheral display <NUM> and the face of the user effectively allows peripheral display <NUM> to project content <NUM> alongside any external light <NUM> reaching an eye <NUM> of user <NUM> from above or below main display <NUM>. In some embodiments, peripheral display <NUM> can be configured to deform in order to conform to the face of user <NUM>. Furthermore, main display <NUM> can also undergo deformation to accommodate certain contours of a face of user <NUM>. In some embodiments, a mechanical coupling between peripheral display <NUM> and main display <NUM> can be configured to accommodate rotation of peripheral display <NUM> with respect to main display <NUM>. For example, a flexible or elastomeric coupling accommodating the rotation can couple main display <NUM> to peripheral display <NUM>. An interior-facing surface of peripheral display <NUM> can include a pad or sealing element for increasing the comfort of user <NUM> while wearing wearable display device <NUM>. In other embodiments, peripheral display <NUM> can extend in a more vertical direction than depicted from main display <NUM> so as not to contact the face of user <NUM> while the user is wearing wearable display device <NUM>.

<FIG> shows how a radius of curvature R<NUM> for main display <NUM> is substantially greater than radius of curvature R<NUM> and radius of curvature R<NUM>. Since curvature is inversely proportional to radius of curvature, main display <NUM> has a much smaller curvature than peripheral display <NUM>. <FIG> also illustrates how radius of curvature R<NUM> can be different than radius of curvature R<NUM>. Differences in curvature can be changed even more when peripheral display <NUM> bends and flexes to accommodate the shape of the face of user <NUM>.

<FIG> shows a top view of wearable device <NUM> worn on a user's head. As depicted, wearable device <NUM> can include a visor <NUM> having a primary viewing port corresponding to a surface to upon which main displays <NUM> are mounted. Visor <NUM> can also include walls extending from the viewing port toward a user's face on top, bottom and lateral sides of the viewing port. In some embodiments, the walls can protrude from the viewing port at a substantially orthogonal angle. Peripheral display <NUM> can then be adhered to an interior or exterior facing surface of the walls so that imagery can be overlaid upon light entering through any one of the walls. In some embodiments, peripheral display <NUM> can also cover portions of the primary viewing port that are not covered by main displays <NUM>. It should be noted that while wearable device <NUM> is not depicted extending all the way to a user's head that in some embodiments, the walls of visor <NUM> can be configured to come into full contact with a user's face, allowing most if not all portions of the user's peripheral vision to be covered by peripheral display <NUM>.

<FIG> shows how a peripheral display <NUM> can be incorporated into wearable device <NUM> in a more limited manner. Peripheral display <NUM> can be embodied by two flexible displays extending from a portion of each temple <NUM> to an interior-facing surface of visor <NUM>. The flexible nature of peripheral display <NUM> can then accommodate folding of temples <NUM> into visor <NUM>. In this way, a lateral periphery of a user's peripheral vision can be covered without reducing the stowability of wearable device <NUM>. In some embodiments, peripheral display <NUM> can also extend within portions of visor <NUM>. For example, portions of visor <NUM> not covered by main displays <NUM> could be covered by additional portions of peripheral display <NUM>. In some embodiments, peripheral display <NUM> can be single display extending from one of temples <NUM> to the other temple <NUM>.

<FIG> shows an interior-facing surface of a wearable or head-mounted display device <NUM>. Wearable display device <NUM> includes a frame or visor <NUM> pivotally coupled with temple arm <NUM> by hinges <NUM>. As depicted, frame <NUM> supports main display <NUM> and provides a surface to which peripheral display <NUM> can be affixed. Peripheral display <NUM> is shaded for emphasis. In particular, <FIG> shows how peripheral display <NUM> can display virtual content <NUM> as it repeatedly enters and exits a position in space that causes peripheral display <NUM> to render a representation of virtual content <NUM>.

<FIG> shows a flowchart describing how peripheral display <NUM> represents virtual content <NUM> travelling along a path. The dashed line indicates the path of virtual content <NUM>. Because virtual content <NUM> follows a path through three-dimensional space, wearable display device <NUM> will not always show virtual content <NUM>. Segment <NUM> represents a portion of the path that occurs outside of the field of view of the head-mounted display. Segment <NUM> is that portion of the path that corresponds to virtual content <NUM> first being located in a position in which peripheral display <NUM> is responsible for displaying virtual content <NUM>. In some embodiments, peripheral display <NUM> can be configured to show virtual content <NUM> at higher intensity levels and/or refresh rates in order to help a user become aware of virtual content <NUM> more quickly. For example, because peripheral vision is typically more effective at tracking fast moving objects a higher refresh rate could help a user to identify objects being represented by peripheral display <NUM>. In some embodiments, peripheral display <NUM> could at least initially depict virtual content <NUM> at segment <NUM> as a bright blob of color or a flashing light in order to help guide a user's attention to the incoming content. In some embodiments, a peripheral portion of peripheral display <NUM> could be illuminated in a predetermined manner in order to alert a user that a particular event has occurred. For example, a quickly flashing light could indicate an incoming augmented reality object is imminently entering a user's field of view while a slowly pulsing blue orb could indicate receipt of a text or in-game message.

At segment <NUM>, as virtual content <NUM> approaches more closely to main display <NUM>, a clear view of the outside world can be blocked by frame <NUM>, when frame <NUM> is optically opaque. In some embodiments, the portion of peripheral display <NUM> positioned in front of frame <NUM> can be configured to display real-world content gathered by a camera mounted to the wearable device to present a user with a view effectively unobstructed by frame <NUM>. In this way, the real-world content can be mixed with virtual content <NUM> to create a virtual representation of virtual and real-world content. In some embodiments, the real-world content reproduced by peripheral display <NUM> can be based at least in part upon a measured intensity and color of ambient light present in the surrounding environment. Such an implementation can create a greater feeling of unrestricted vision and immersion without the need to incorporate a video feed from an additional camera. Any fine detail missing from constructing the view in this manner could go largely unnoticed on account of a user not being able to focus directly on that portion of the user's field of view. It should be noted that the step of overlaying real-world imagery atop mask frame <NUM> is an optional operation and in some embodiments, it could be more desirable to either not show any content at all during segment <NUM> to accentuate the presence of frame <NUM> or to just show virtual content as it travels across segment <NUM>. Once virtual content reaches segment <NUM>, peripheral display <NUM> could begin displaying virtual content <NUM> in greater detail as a person's ability to perceive higher resolution increases.

At segment <NUM>, main display <NUM> takes over display of virtual content <NUM>. Because peripheral display <NUM> and main display <NUM> are in abutting contact, virtual content <NUM> can stay continuously in view as it transitions from peripheral display <NUM> to main display <NUM>. At segment <NUM>, peripheral display <NUM> resumes display of virtual content <NUM> and blends virtual content <NUM> with background imagery that masks frame <NUM> from a user's view. It should be noted that as with segment <NUM><NUM>, the display of background real-world imagery can be an optional step. At segment <NUM> peripheral display <NUM> creates a representation of only virtual content <NUM> and at segment <NUM> peripheral display <NUM> ceases displaying virtual content <NUM>.

<FIG> shows a perspective view of an exemplary wearable display device <NUM> without a peripheral display. Wearable display device <NUM> includes main displays <NUM>. Each of main displays <NUM> can include an eye tracking sensor <NUM> configured to track the movement of the eyes of a user of wearable display device <NUM>. In some embodiments, the resolution of imagery depicted by main displays <NUM> can be adjusted to account for movement of the eyes of the user as determined by eye tracking sensors <NUM>. For example, the resolution can vary across the surface of main displays <NUM> so that processing power can be devoted to providing high resolution in only those areas being focused on by the eyes of a user. The other areas can be rendered in lower resolution. Wearable display device <NUM> also includes projector assemblies <NUM>, which are integrated into temple arms <NUM>. Projector assemblies <NUM> can include projectors that shine light through diffractive optics that is then reflected into the eyes of a user through main displays <NUM>. Wearable display device <NUM> can also include camera assemblies <NUM>. Each of camera assemblies <NUM> can include a number of camera modules <NUM> for observing and characterizing the environment surrounding wearable display device <NUM>. Characterization of the environment can be important for numerous reasons including for example for incorporating virtual content with real life objects in the environment. For example, being able to identify items such as chairs using the camera modules could allow a virtual character to sit on one of the real world chairs instead of having to generate a virtual chair or give the appearance of being seated in the air. In some embodiments, wearable display device <NUM> can include one or more camera modules <NUM> with depth detection sensors for synchronizing the depth of virtual content displayed by main displays <NUM>. As with projector assemblies <NUM>, camera assemblies <NUM> can be incorporated with temple arms <NUM>.

<FIG> shows how peripheral display <NUM> can be incorporated into wearable display device <NUM>. As depicted, peripheral display <NUM> can be arranged along the periphery of each of main displays <NUM>. Peripheral display <NUM> can also extend between main displays <NUM> to prevent any coverage gap above bridge <NUM>. In some embodiments, temple regions <NUM> of peripheral display <NUM> can extend farther away from main displays <NUM> than the rest of peripheral display <NUM>. Temple regions <NUM> can be configured to display content to obscure projector assemblies <NUM> and camera assemblies <NUM> from a user's peripheral field of view. This can help a user feel more immersed in the surrounding virtual and/or real-world content.

<FIG> shows a wearable display device <NUM> that includes two displays <NUM> joined by a bridge <NUM>. In particular, <FIG> shows how displays <NUM> can have two different regions configured to display content in different ways. High acuity regions <NUM> can transition to low acuity regions <NUM> in transition regions <NUM> as indicated by the protruding star patterns. The change in acuity can be accomplished in many different ways. In some embodiments, the low acuity region can have the same number of pixels as the high acuity region and simply display content at a lower resolution. For example, four pixels in low acuity region <NUM> could display the same value so that low acuity regions <NUM> have a spatial resolution four times lower than the spatial resolution of the high acuity regions <NUM>. In other embodiments, the spacing between pixels in low acuity regions <NUM> could be greater than in high acuity regions <NUM>. In some embodiments, the pixels in low acuity regions <NUM> could be larger than those in high acuity regions <NUM> due to the additional space provided by the greater pixel spacing. Transition region <NUM> could also have pixels that were spaced gradually farther apart to create a more even transition between regions <NUM> and <NUM>. It should be noted that high acuity regions <NUM> and low acuity regions <NUM> can have many different variations not limited to differences in spatial recognition. For example, low acuity regions <NUM> could display fewer colors, refresh at different rates and even display virtual content at different levels of intensity (i.e. brightness) than high acuity regions <NUM>.

<FIG> shows a wearable display device <NUM> having displays <NUM> with multiple regions <NUM>, <NUM> and <NUM>. Regions <NUM> can be designed to correspond to the capability of the human eye to distinguish color and spatial resolution. Since the center of the eye has the highest concentration of cones, which have the best capability to distinguish detail and color, region <NUM> can be configured to emit the highest resolution and truest color reproduction. Region <NUM> can be configured to display virtual content at a relatively lower spatial and/or color resolution. In some embodiments, region <NUM> can be arranged along a border of a field of regard of a user of wearable display device <NUM>. For this reason, differences between region <NUM> and region <NUM> can be implemented over a transition zone between regions <NUM> and <NUM>, such that the change in resolution is not obvious or distracting to a user of wearable display device <NUM>. Similarly, region <NUM> can cover the portion of a user's field of view corresponding to the far peripheral field of view. Region <NUM> can be configured to display virtual content at even lower resolutions than region <NUM>. For example, region <NUM> can be configured to display virtual content in gray scale.

<FIG> shows a display <NUM> similar to display <NUM> and <NUM>. A distribution of pixels <NUM> can vary across display <NUM>. In particular, pixels <NUM> are shown having a lower density in a peripheral region <NUM> and a higher density in a central region <NUM>. By setting display <NUM> up in this manner, the spatial resolution of any imagery displayed by display <NUM> can be gradually reduced as virtual content moves from central region <NUM> into peripheral region <NUM> of display <NUM>.

<FIG> and <FIG> describe in detail a display technology that can be used with main displays, such as main displays <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> and <NUM>. In some embodiments, a peripheral display can also utilize this type of display technology. The displays can incorporate eye-tracking apparatus or not for further optimizing the position in which high and low resolution imagery are being displayed.

In <FIG>, a viewer's eye <NUM> is oriented in a first manner with respect to an eyepiece <NUM>, such that the viewer may be able to see the eyepiece <NUM> in a relatively straightforward direction. The orientation of the viewer's eye <NUM> in <FIG> may, for instance, be the same as or similar to the orientation of the viewer's eye <NUM> as described above with reference to <FIG>, and may be determined by the AR system using one or more of the sensing components and/or techniques described herein. As such, in the stage depicted in <FIG>, the AR system may employ head-tracked and fovea-tracked render perspectives at relative positions and orientations. The FOV of the fovea-tracked render perspective employed by the AR system may, for instance, encompass virtual object <NUM>, but may not encompass either of virtual objects <NUM> and <NUM>. It follows that, in <FIG>, the AR system may render virtual object <NUM> as it would be captured from the perspective of the fovea-tracked virtual camera in high definition, and may render virtual objects <NUM> and <NUM> as they would be captured from the perspective of the head-tracked virtual camera in lower definition. In addition, the AR system may project light representing such renderings of virtual objects <NUM>, <NUM>, and <NUM> through the eyepiece <NUM> and onto the retina of the viewer's eye <NUM>. In some embodiments, the AR system may also render virtual object <NUM> as it would be captured from the perspective of the head-tracked virtual camera in lower definition.

<FIG> also illustrates an exemplary light field 630A that is outcoupled by the eyepiece <NUM> and projected onto the retina of the viewer's eye <NUM>. The light field 630A may include various angular light components representative of one or more of the abovementioned renderings of virtual objects <NUM>, <NUM>, and <NUM>. For example, angular light components of the light field 630A that are representative of the virtual object <NUM> as it would be captured from the perspective of the head-tracked virtual camera may include those which are to be projected onto the retina of the viewer's eye <NUM> at angles ranging from -α to -β angular units relative to the viewer's eye <NUM>, and angular light components of the light field 630A that are representative of the virtual object <NUM> as it would be captured from the perspective of the head-tracked virtual camera may include those which are to be projected onto the retina of the viewer's eye <NUM> at angles ranging from ε to ζ angular units relative to the viewer's eye <NUM>. Similarly, angular light components of the light field 630A that are representative of the virtual object <NUM> as it would be captured from the perspective of the fovea-tracked virtual camera may include those which are to be projected onto the fovea of the viewer's eye <NUM> at angles ranging from -γ to δ angular units relative to the viewer's eye <NUM>. As such, components of the light field 630A that are representative of virtual object <NUM> (i.e., components to be projected at angles ranging from -γ to δ angular units relative to the viewer's eye <NUM>) may be more densely distributed in angular space than components of the light field 630A that are representative of virtual object <NUM> or <NUM> (i.e., components to be projected at angles ranging from -α to -β or ε to ζ angular units relative to the viewer's eye <NUM>). In this way, the resolution at which the virtual object <NUM> may be rendered and presented to the viewer may be higher than the resolution at which virtual object <NUM> or <NUM> may be rendered and presented to the viewer.

In <FIG>, the viewer's eye <NUM> is oriented in a second manner with respect to the eyepiece <NUM> different from the first manner in which the viewer's eye <NUM> is oriented with respect to the eyepiece <NUM> in <FIG>. The orientation of the viewer's eye <NUM> in <FIG> may be determined by the AR system using one or more of the sensing components and/or techniques described herein. As such, in the stage depicted in <FIG>, the AR system may employ head-tracked and fovea-tracked render perspectives at relative positions and orientations similar to those of the head-tracked and fovea-tracked render perspectives. In the particular example of <FIG>, the FOV of the fovea-tracked render perspective employed by the AR system may, for instance, encompass virtual object <NUM>, but may not encompass either of virtual objects <NUM> and <NUM>. It follows that, in <FIG>, the AR system may render virtual object <NUM> as it would be captured from the perspective of the fovea-tracked virtual camera in high definition, and may render virtual objects <NUM> and <NUM> as they would be captured from the perspective of the head-tracked virtual camera in lower definition. In addition, the AR system may project light representing such renderings of virtual objects <NUM>, <NUM>, and <NUM> through the eyepiece <NUM> and onto the retina of the viewer's eye <NUM>. In some embodiments, the AR system may also render virtual object <NUM> as it would be captured from the perspective of the head-tracked virtual camera in lower definition.

<FIG> also illustrates an exemplary light field 630B that is outcoupled by the eyepiece <NUM> and projected onto the retina of the viewer's eye <NUM>. The light field 630B may include various angular light components representative of one or more of the abovementioned renderings of virtual objects <NUM>, <NUM>, and <NUM>. For example, angular light components of the light field 630B that are representative of the virtual object <NUM> as it would be captured from the perspective of the head-tracked virtual camera may include those which are to be projected onto the retina of the viewer's eye <NUM> at angles ranging from -α to -β angular units relative to the viewer's eye <NUM>, and angular light components of the light field 630B that are representative of the virtual object <NUM> as it would be captured from the perspective of the head-tracked virtual camera may include those which are to be projected onto the retina of the viewer's eye <NUM> at angles ranging from -γ to δ angular units relative to the viewer's eye <NUM>. Similarly, angular light components of the light field 630B that are representative of the virtual object <NUM> as it would be captured from the perspective of the fovea-tracked virtual camera may include those which are to be projected onto the fovea of the viewer's eye <NUM> at angles ranging from ε to ζ angular units relative to the viewer's eye <NUM>. As such, components of the light field 630B that are representative of virtual object <NUM> (i.e., components to be projected at angles ranging from ε to ζ angular units relative to the viewer's eye <NUM>) may be more densely distributed in angular space than components of the light field 630A that are representative of virtual object <NUM> or <NUM> (i.e., components to be projected at angles ranging from -α to -β or -γ to δ angular units relative to the viewer's eye <NUM>). In this way, the resolution at which the virtual object <NUM> may be rendered and presented to the viewer may be higher than the resolution at which virtual object <NUM> or <NUM> may be rendered and presented to the viewer. Indeed, from the stage of <FIG> to the stage of <FIG>, the AR system described herein with reference thereto has effectively reoriented the perspective from which virtual content may be viewed in high resolution in accordance with the change in gaze of the viewer's eye <NUM> between stages.

<FIG> illustrate schematically a display system <NUM> according to some other embodiments of the present invention. The display system <NUM> includes an image source <NUM>, a beam splitter <NUM>, a first optical lens <NUM>, a second optical lens <NUM>, a third optical lens <NUM>, a fourth optical lens <NUM>, a fifth optical lens <NUM>, a sixth optical lens <NUM>, a scanning mirror <NUM>, a polarizer <NUM> and a switching polarization rotator <NUM>. These components allow the projector to input light into a display from multiple image sources to help produce a composite image at the display that contains imagery with varying resolutions.

More specifically, <FIG> illustrate a display system <NUM> in each of three different stages. In each of the three stages, the image source <NUM>, which can be coupled to a temple of a wearable display device, can output a range of angular light field components representative of virtual content as would be captured from the perspective of a head-tracked virtual camera and a range of angular light field components representative of virtual content as would be captured from the perspective of a fovea-tracked virtual camera. The two sets of angular light field components may, for instance, be time-division multiplexed, polarization-division multiplexed, wavelength-division multiplexed, or the like. As such, the angular light field components associated with the head-tracked virtual camera can be diverted upward by the polarization beam splitter <NUM> along a first optical path through the first and second optical lenses <NUM> and <NUM>, and the angular light field components associated with the fovea-tracked virtual camera can pass through the polarization beam splitter <NUM> along a second optical path through third and fourth optical lenses <NUM> and <NUM> toward the scanning mirror <NUM> and reflected upward through fifth and sixth optical lenses <NUM> and <NUM>.

The virtual content represented by the angular light field components associated with the head-tracked virtual camera may be rendered upstream from the image source <NUM> at a relatively low resolution, while the virtual content represented by the angular light field components associated with the fovea-tracked virtual camera may be rendered upstream from the image source <NUM> at a relatively high resolution. And, as shown in <FIG>, the display system <NUM> may be configured to output the angular light field components associated with the head-tracked render perspective and the angular light field components associated with the fovea-tracked render perspective as high FOV and low FOV light fields, respectively. In each of <FIG>, the light field components that propagate along the first optical path are output by the display system <NUM> as a relatively wide cone of light <NUM>.

In the stage depicted in <FIG>, the scanning mirror <NUM> is in a first position. As such, it can be seen that the light field components that pass through the polarization beam splitter <NUM> and propagate along the second optical path are output by the display system <NUM> as a relatively narrow cone of light 854A spanning a substantially central region of angular space. Within the context of the examples described above with reference to <FIG>, the display system <NUM> could, for instance, place the scanning mirror <NUM> in the first position shown in <FIG> when the user's eye is oriented in a manner similar to that of the viewer's eye <NUM> in <FIG>. In this way, the light components 854A may represent virtual content in a relatively centralized region of render space, such as virtual object <NUM>. Further to the examples of <FIG>, the relatively wide cone of light <NUM> may, for instance, include virtual content in off-centered regions of render space, such as virtual objects <NUM> and <NUM>. In some examples, the relatively wide cone of light <NUM> may further include light components that represent the same virtual content as is represented by the light components 854A, but in lower resolution.

In the stage depicted in <FIG>, the scanning mirror <NUM> is in a second position different from the first position. As such, it can be seen that the light field components that pass through the polarization beam splitter <NUM> and propagate along the second optical path are output by the display system <NUM> as a relatively narrow cone of light 854B spanning one substantially off-centered region of angular space. Within the context of the examples described above with reference to <FIG>, the display system <NUM> could, for instance, place the scanning mirror <NUM> in the second position shown in <FIG> when the user's eye is oriented in a manner similar to that of the viewer's eye <NUM> while the viewer is looking at virtual object <NUM>. In this way, the light components 854B may represent virtual content in one relatively off-centered region of render space, such as virtual object <NUM>. Further to the examples of <FIG>, the relatively wide cone of light <NUM> may, for instance, include virtual content in the other off-centered region of render space, such as virtual object <NUM>, as well as virtual content in the centralized region of render space, such as virtual object <NUM>. In some examples, the relatively wide cone of light <NUM> may further include light components that represent the same virtual content as is represented by the light components 854B, but in lower resolution.

In the stage depicted in <FIG>, the scanning mirror <NUM> is in a third position different from the first and second positions. As such, it can be seen that the light field components that pass through the polarization beam splitter <NUM> and propagate along the second optical path are output by the display system <NUM> as a relatively narrow cone of light 854C spanning another, different substantially off-centered region of angular space. Within the context of the examples described above with reference to <FIG>, the di splay system <NUM> could, for instance, place the scanning mirror <NUM> in the second position shown in <FIG> when the user's eye is oriented in a manner similar to that of the viewer's eye <NUM> in <FIG>. In this way, the light components 854C may represent virtual content in the other relatively off-centered region of render space, such as virtual object <NUM>. Further to the examples of <FIG>, the relatively wide cone of light <NUM> may, for instance, include virtual content in the off-centered region of render space described above with reference to <FIG>, such as virtual object <NUM>, as well as virtual content in the centralized region of render space, such as virtual object <NUM>. In some examples, the relatively wide cone of light <NUM> may further include light components that represent the same virtual content as is represented by the light components 854C, but in lower resolution.

The various aspects, embodiments, implementations or features of the described embodiments can be used separately or in any combination. Various aspects of the described embodiments can be implemented by software, hardware or a combination of hardware and software. The described embodiments can also be embodied as computer readable code on a computer readable medium for controlling manufacturing operations or as computer readable code on a computer readable medium for controlling a manufacturing line. The computer readable medium is any data storage device that can store data, which can thereafter be read by a computer system. Examples of the computer readable medium include read-only memory, random-access memory, CD-ROMs, HDDs, DVDs, magnetic tape, and optical data storage devices. The computer readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Claim 1:
A wearable display device, comprising:
a frame (<NUM>) including an attachment member configured to secure the wearable display device to a head of a user;
a projector (<NUM>) coupled to the frame (<NUM>); and
a display assembly coupled to the frame (<NUM>), the display assembly comprising:
a main display (<NUM>) comprising one or more optical waveguides configured to project an angular representation of virtual content onto an eye of the user, and
a peripheral display (<NUM>) arranged along a periphery of the main display (<NUM>) so as to cover a different portion of a field of view of the eye of the user than the main display, the peripheral display (<NUM>) configured to generate a spatial representation of the virtual content, and wherein the peripheral display (<NUM>) is in abutting contact with the one or more optical waveguides of the main display (<NUM>).