Patent Description:
<CIT> and <CIT> disclose multiple depth/focal plane optical systems comprising: a light modifying device configured to generate a modified pixel beam; an eyepiece configured to output the modified pixel beam; and a processing module configured to perform operations comprising: determining a virtual distance of a virtual depth plane from the optical system at which the virtual object is to be displayed; comparing the virtual distance to at least one distance threshold; and based on comparing the virtual distance to the at least one distance threshold, causing the light modifying device to modify the collimated pixel beam to generate the modified pixel beam.

Despite the progress made in these display technologies, there is a need in the art for improved methods, systems, and devices related to augmented reality systems, particularly, display systems.

The invention is directed to an optical system according to claim <NUM>.

The invention is also directed to a method of operating an optical system, according to claim <NUM>.

Numerous benefits are achieved by way of the present disclosure over conventional techniques. For example, embodiments enable a single focal plane system to have several of the same benefits as a two-focal plane system, such as reduced VAC in both the near-field and far-field virtual depth planes. Additionally, since the pixel beam can be modified prior to injection into the eyepiece, embodiments are compatible with existing eyepieces that employ pupil-expansion combiner eyepiece technology. Embodiments also eliminate the need for clipping planes that are often employed for near field depth planes, thereby reducing the inconvenience to users due to virtual content disappearing. Other benefits of the present disclosure will be readily apparent to those skilled in the art.

The accompanying drawings, which are included to provide a further understanding of the disclosure, are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the detailed description serve to explain the principles of the disclosure. No attempt is made to show structural details of the disclosure in more detail than may be necessary for a fundamental understanding of the disclosure and various ways in which it may be practiced.

In the appended figures, similar components and/or features may have the same numerical reference label. Further, various components of the same type may be distinguished by following the reference label with a letter or by following the reference label with a dash followed by a second numerical reference label that distinguishes among the similar components and/or features. If only the first numerical reference label is used in the specification, the description is applicable to any one of the similar components and/or features having the same first numerical reference label, irrespective of the suffix.

Mixed-reality (MR) and augmented reality (AR) wearable displays are capable of presenting virtual content to a user over a wide depth range. For many displays, a user may experience varying levels of accommodation-vergence conflict (VAC) at different depths, which occurs when the user's brain receives mismatching cues between the distance of a virtual object from the user's eyes and the focusing distance required for the eyes to focus on that virtual object. VAC leads to visual fatigue, headache, nausea, and eyestrain, and remains a significant source of discomfort for users. Accordingly, to maintain user comfort, modern MR and AR wearable displays may consider a VAC budget allowance when delivering virtual content over a depth range, which may result in a depth range that is significantly reduced.

Various approaches to mitigate VAC have been implemented. One approach includes adding a second depth plane and a vari-focal switch based on eye-tracking to the optical system. Another approach is to add a vari-focal element with the ability to sweep eyepiece focal planes across a broad range. These approaches come with increased volume in the form of additional eyepiece layers and/or through integration of liquid-fillable tunable lens pairs straddling the eyepiece, as well as increased complexity due to complex illumination schemes.

Some embodiments of the present invention provide an optical system with a delimited zone, within which a limited amount of VAC is tolerated by a user, and outside of which an expanded depth of field can be switched on to disrupt human visual system accommodation cues. In some embodiments, the delimited zone can be defined based on a single or multiple fixed focal plane(s) or a single or multiple variable focus plane(s). Virtual content having an associated virtual depth plane that lies within the delimited zone may be projected to the user in a normal manner, whereas virtual content outside the delimited zone is modified by a light modifying device so as to reduce the reliability of the accommodation cues.

According to the invention, the light modifying device may cause the collimated light generated by a projector to become converging when entering the eyepiece. This causes the virtual image light (i.e., light associated with a virtual image) that is outcoupled from the leaky-grating of the eyepiece to also be converging. However, the chief ray of each beamlet does not change direction, resulting in a virtual image with vergence cues but very weak accommodation cues. Such a virtual image can disrupt the vergence-accommodation response in areas of the depth of field where VAC would exceed the threshold tolerance. Thus, embodiments disclosed herein can extend the depth of field of the optical system, since the user's eye may not be able to focus on pixels at the virtual depth plane. Additionally or alternatively, the light modifying device may reduce the diameter of each collimated pixel beam generated by the projector. This can cause the light that is outcoupled from the leaky-grating of the eyepiece to likewise have pixel beams with reduced diameters, thereby disrupting the accommodation cues associated with the outcoupled light.

In some instances, optical see-through (OST) AR devices can improve virtual content being presented to a user by applying optical power to the virtual image light using one or more lens assemblies arranged within an optical stack. Embodiments of the present invention are compatible with existing systems that utilize lens assemblies to vary the virtual depth plane of the virtual object.

<FIG> illustrates an AR scene <NUM> as viewed through a wearable AR device, according to some embodiments. AR scene <NUM> is depicted wherein a user of an AR technology sees a real-world park-like setting <NUM> featuring various real-world objects <NUM> such as people, trees, buildings in the background, and a real-world concrete platform <NUM>. In addition to these items, the user of the AR technology also perceives that they "see" various virtual objects <NUM> such as a robot statue <NUM>-<NUM> standing upon the real-world concrete platform <NUM>, and a cartoon-like avatar character <NUM>-<NUM> flying by, which seems to be a personification of a bumble bee, even though these elements (character <NUM>-<NUM> and statue <NUM>-<NUM>) do not exist in the real world. Due to the extreme complexity of the human visual perception and nervous system, it is challenging to produce a virtual reality (VR) or AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements.

<FIG> illustrates an AR device 200A having a single fixed focal plane, according to some examples. During operation, a projector <NUM> of AR device 200A may project virtual image light <NUM> (i.e., light associated with virtual content) onto an eyepiece <NUM>-<NUM>, which may cause a light field (i.e., an angular representation of virtual content) to be projected onto a retina of a user in a manner such that the user perceives the corresponding virtual content as being positioned at some location within the user's environment. For example, virtual image light <NUM> outcoupled by eyepiece <NUM>-<NUM> may cause the user to perceive character <NUM>-<NUM> as being positioned at a first virtual depth plane <NUM>-<NUM> and statue <NUM>-<NUM> as being positioned at a second virtual depth plane <NUM>-<NUM>. The user perceives the virtual content along with world light <NUM> corresponding to one or more world objects <NUM>, such as platform <NUM>.

In some examples, AR device 200A includes a first lens assembly <NUM>-<NUM> positioned on the user side of eyepiece <NUM>-<NUM> (the side of eyepiece <NUM>-<NUM> closest to the eye of the user) and a second lens assembly <NUM>-<NUM> positioned on the world side of eyepiece <NUM>-<NUM>. Each of lens assemblies <NUM>-<NUM>, <NUM>-<NUM> may be configured to apply optical power to the light passing therethrough.

<FIG> illustrates an AR device 200B having two fixed focal planes, according to some examples. During operation, projector <NUM> may project virtual image light <NUM> onto first eyepiece <NUM>-<NUM> and a second eyepiece <NUM>-<NUM>, which may cause a light field to be projected onto a retina of a user in a manner such that the user perceives the corresponding virtual content as being positioned at some location within an environment of the user. For example, virtual image light <NUM> outcoupled by first eyepiece <NUM>-<NUM> may cause the user to perceive character <NUM>-<NUM> as being positioned at a first virtual depth plane <NUM>-<NUM> and virtual image light <NUM> outcoupled by second eyepiece <NUM>-<NUM> may cause the user to perceive statue <NUM>-<NUM> as being positioned at a second virtual depth plane <NUM>-<NUM>.

<FIG> illustrates the relationship between VAC and the distance of the virtual depth plane for each of AR devices 200A, 200B described in reference to <FIG> and <FIG>, respectively. For AR device 200B, the two-focal plane system provides switchable focal planes at <NUM> diopters (<NUM> meters) and <NUM> diopters (<NUM> meters), with a switch point at <NUM> diopters (<NUM> meters), a near content limit (clipping plane) at <NUM> diopters (<NUM> meters), and an ability to provide imagery never exceeding <NUM> diopter VAC between that plane and infinity. For AR device 200A, the single fixed focal plane system has a focal plane location at <NUM> diopters (<NUM> meters) and a near content limit of <NUM> diopters (<NUM> meters) and a far content limit of <NUM> diopters (<NUM> meters), assuming a maximum allowable VAC of <NUM> diopter. Such a configuration would have a usable range of <NUM>-<NUM> meters with content falling outside of that range requiring some solution to mitigate exceeding the VAC limit.

<FIG> illustrates a schematic view of an example wearable AR device <NUM>, according to some examples of the present invention. AR device <NUM> may include a left eyepiece 402A and a left lens assembly 405A arranged in a side-by-side configuration and a right eyepiece 402B and a right lens assembly 405B also arranged in a side-by-side configuration. In some examples, AR device <NUM> includes one or more sensors including, but not limited to: a left front-facing world camera 406A attached directly to or near left eyepiece 402A, a right front-facing world camera 406B attached directly to or near right eyepiece 402B, a left side-facing world camera 406C attached directly to or near left eyepiece 402A, and a right side-facing world camera 406D attached directly to or near right eyepiece 402B. In some examples, AR device <NUM> includes one or more image projection devices such as a left projector 414A optically linked to left eyepiece 402A and a right projector 414B optically linked to right eyepiece 402B.

Some or all of the components of AR device <NUM> may be head mounted such that projected images may be viewed by a user. In one particular implementation, all of the components of AR device <NUM> shown in <FIG> are mounted onto a single device (e.g., a single headset) wearable by a user. In another implementation, one or more components of a processing module <NUM> are physically separate from and communicatively coupled to the other components of AR device <NUM> by one or more wired and/or wireless connections. For example, processing module <NUM> may include a local module <NUM> on the head mounted portion of AR device <NUM> and a remote module <NUM> physically separate from and communicatively linked to local module <NUM>. Remote module <NUM> may be mounted in a variety of configurations, such as fixedly attached to a frame, fixedly attached to a helmet or hat worn by a user, embedded in headphones, or otherwise removably attached to a user (e.g., in a backpack-style configuration, in a belt-coupling style configuration, etc.).

Processing module <NUM> may include a processor and an associated digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to assist in the processing, caching, and storage of data. The data may include data captured from sensors (which may be, e.g., operatively coupled to AR device <NUM>) or otherwise attached to a user, such as cameras <NUM>, an ambient light sensor, eye trackers, microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, and/or gyros. For example, processing module <NUM> may receive image(s) <NUM> from cameras <NUM>. Specifically, processing module <NUM> may receive left front image(s) 420A from left front-facing world camera 406A, right front image(s) 420B from right front-facing world camera 406B, left side image(s) 420C from left side-facing world camera 406C, and right side image(s) 420D from right side-facing world camera 406D. In some examples, image(s) <NUM> may include a single image, a pair of images, a video comprising a stream of images, a video comprising a stream of paired images, and the like. Image(s) <NUM> may be periodically generated and sent to processing module <NUM> while AR device <NUM> is powered on, or may be generated in response to an instruction sent by processing module <NUM> to one or more of the cameras. As another example, processing module <NUM> may receive ambient light information from an ambient light sensor. As another example, processing module <NUM> may receive gaze information from one or more eye trackers. As another example, processing module <NUM> may receive image information (e.g., image brightness values) from one or both of projectors <NUM>.

Cameras 406A, 406B may be positioned to capture images that substantially overlap within the field of view of a user's left and right eyes, respectively. Accordingly, placement of cameras <NUM> may be near a user's eyes but not so near as to obscure the user's field of view. Alternatively or additionally, cameras 406A, 406B may be positioned so as to align with the incoupling locations of virtual image light 422A, 422B, respectively. Cameras 406C, 406D may be positioned to capture images to the side of a user, e.g., in a user's peripheral vision or outside the user's peripheral vision. Image(s) 420C, 420D captured using cameras 406C, 406D need not necessarily overlap with image(s) 420A, 420B captured using cameras 406A, 406B.

Eyepieces 402A, 402B may comprise transparent or semi-transparent waveguides configured to direct and outcouple light generated by projectors 414A, 414B, respectively. Specifically, processing module <NUM> may cause left projector 414A to output left virtual image light 422A onto left eyepiece 402A, and may cause right projector 414B to output right virtual image light 422B onto right eyepiece 402B. In some examples, each of eyepieces 402A, 402B may comprise a plurality of waveguides corresponding to different colors. In some examples, lens assemblies 405A, 405B may be coupled to and/or integrated with eyepieces 402A, 402B. For example, lens assemblies 405A, 405B may be incorporated into a multi-layer eyepiece and may form one or more layers that make up one of eyepieces 402A, 402B.

In some examples, AR device <NUM> includes one or more light modifying devices 404A, 404B for modifying virtual image light 422A, 422B. Specifically, a left light modifying device 404A may be positioned in an optical path between left projector 414A and left eyepiece 402A so as to modify left virtual image light 422A prior to being outputted onto left eyepiece 402A, and a right light modifying device 404B may be positioned in an optical path between right projector 414B and right eyepiece 402B so as to modify right virtual image light 422B prior to being outputted onto right eyepiece 402B. In some examples, light modifying devices 404A, 404B may be integrated with projectors 414A, 414B. In some examples, light modifying devices 404A, 404B may be integrated with eyepieces 402A, 402B.

In some examples, projectors 414A, 414B may include a micro-electromechical system (MEMS) spatial light modulator (SLM) scanning device. In such examples, light modifying devices 404A, 404B may employ a varifocal mirror or lens that can be used in the laser beams prior to the scanning mirrors. If a relay optical system is used, one of the optical elements within the relay optics could be vari-focal and be switched to provide converging pixel rays to the ICG formed on the eyepieces. If a standard projection system is used with a pixel-based SLM (such as a liquid crystal on silicon (LCOS)), the SLM itself could be translated in the z-axis (perpendicular to the array), such that the projection lens produces a finite external focal plane (and thus convergent pixel rays). In some examples, a vari-focal lens could be incorporated between the projection/relay lens of the microdisplay and the ICG of the eyepiece itself, converting the output collimated pixel rays into convergent states.

<FIG> illustrates an example function of a viewing optics assembly <NUM> of an AR device and the resulting user visual percept of the system's output. Viewing optics assembly <NUM> includes a projector <NUM> and an eyepiece <NUM>. Projector <NUM> generates a collimated pixel beam <NUM> that is carried onto an eyepiece <NUM> at an input coupling grating (ICG) <NUM> formed on eyepiece <NUM>. After being diffracted by ICG <NUM>, collimated pixel beam <NUM> propagates in eyepiece <NUM> until an output grating formed on eyepiece <NUM> diffracts the light toward the user.

A leaky-grating light-guide, pupil-expanding eyepiece with no programmed optical power produces a virtual image at infinity. The percept is produced by multiple output "beamlets" (emitted replicants of the input pixel wavefronts) collected through the pupil and imaged onto the retina of the user's eye. In this case, when the user's eye is focused at infinity, a sharp image of the pixel is formed on the retina. When the eye is focused at another plane (for example at <NUM> meters from the user) a blurry image of the pixel is formed on the retina.

<FIG> illustrates an example function of a viewing optics assembly <NUM> of an AR device and the resulting user visual percept of the system's output. Viewing optics assembly <NUM> includes a projector <NUM> that generates a collimated pixel beam <NUM> that is carried onto an eyepiece <NUM> at an ICG <NUM> formed on eyepiece <NUM>. After being diffracted by ICG <NUM>, collimated pixel beam <NUM> propagates in eyepiece <NUM> until an output grating formed on eyepiece <NUM> diffracts the light toward the user.

Viewing optics assembly <NUM> includes a -<NUM> diopters lens assembly <NUM> that modulates the wavefronts of the emitted beamlets, diverging them with respect to each other and diverging each ray independently, so as to both focus pixel light and converge beamlets at <NUM> meters from the user's eye. Lens assembly <NUM> shifts the chief rays of the emerging beamlets and diverges the collimated output to a single pixel focus position at the focal length of the lens. In this case, when the user's eye is focused at <NUM> meters, a sharp image of the pixel is formed on the retina. When the eye focuses at infinity, that image is blurred.

In the example illustrated in <FIG>, the depth of focus of the image is determined by several factors, including the beamlet packing density (determined by the beam diameter, the eyepiece substrate thickness, along with several other factors), the size of the user's pupil, the optical quality of the lens assembly <NUM>, and the inherent depth of field of the user's eye. Each of these factors may be considered to determine an acceptable VAC budget figure for the system. In some examples, <NUM> diopters can be used as the VAC budget figure, although this value can be higher or lower in practice.

<FIG> illustrates an example function of a viewing optics assembly <NUM> of an AR device and the resulting user visual percept of the system's output, according to the invention. Viewing optics assembly <NUM> includes a projector <NUM> that generates a collimated pixel beam <NUM> that is modified by a light modifying device <NUM> to produce a modified pixel beam <NUM> having a converging wavefront. Modified pixel beam <NUM> is carried onto an eyepiece <NUM> at an ICG <NUM> formed on eyepiece <NUM>. After being diffracted by ICG <NUM>, modified pixel beam <NUM> propagates in eyepiece <NUM> until an output grating formed on eyepiece <NUM> diffracts the light toward the user.

In the example illustrated in <FIG>, modifying the wavefronts of the imaged pixels introduces optical power to the projection system, transforming an infinity-focused system into a system that produces a finite image position in front of the projector. In such a configuration, a single pixel produces a converging (curved) wavefront at the pupil plane of the projector. When a converging pixel ray enters the eyepiece, the exiting beamlets maintain this convergence, however, the chief ray of each beamlet does not change direction. In this case, when the user's eye is focused either at <NUM> meters or at infinity, a blurred image of the pixel is formed on the retina. Additionally, the perceived pixel when the user's eye is focused at <NUM> meters may be different from the perceived pixel when the user's eye is focused at infinity, as depicted by different types of blur in <FIG>.

<FIG> illustrates an example function of a viewing optics assembly <NUM> of an AR device and the resulting user visual percept of the system's output, according to the invention. Viewing optics assembly <NUM> includes a projector <NUM> that generates a collimated pixel beam <NUM> that is modified by a light modifying device <NUM> to produce a modified pixel beam <NUM> having a converging wavefront. Modified pixel beam <NUM> is carried onto an eyepiece <NUM> at an ICG <NUM> formed on eyepiece <NUM>. After being diffracted by ICG <NUM>, modified pixel beam <NUM> propagates in eyepiece <NUM> until an output grating formed on eyepiece <NUM> diffracts the light toward the user. Viewing optics assembly <NUM> further includes a -<NUM> diopters lens assembly <NUM> positioned between eyepiece <NUM> and the user's eye that modulates the wavefronts of the emitted beamlets.

In the example illustrated in <FIG>, lens assembly <NUM> collimates each beamlet output while simultaneously re-directing the chief ray of each beamlet to pivot around a point at the focal plane of the lens. As a result, when the user's eye is focused at <NUM> meters, a blurred image of the pixel is formed on the retina. When the user's eye is focused at infinity, a percept comprising a repeated structure of blurred images is produced. The user's eye is unable to bring the blurred image into focus, thereby disrupting the user's physiological vergence-accommodation cues and reducing the uncomfortable effects of vergence-accommodation conflict. This percept having a repeated structure allows virtual content to exist on planes outside of the VAC threshold. As a result, the depth of field of the optical system can extend beyond the VAC threshold without discomfort, since the user's eye will not be able to focus on pixels at the virtual depth plane.

<FIG> illustrates an example function of a viewing optics assembly <NUM> of an AR device and the resulting user visual percept of the system's output, according to the invention. Viewing optics assembly <NUM> includes a projector <NUM> that generates a collimated pixel beam <NUM> that is modified by a light modifying device <NUM> such as a spatial light modulator (SLM), relay optics, polarizers, beam splitters, lenses or a combination thereof, to produce a modified pixel beam <NUM> having a reduced diameter. Modified pixel beam <NUM> is carried onto an eyepiece <NUM> at an ICG <NUM> formed on eyepiece <NUM>. After being diffracted by ICG <NUM>, modified pixel beam <NUM> propagates in eyepiece <NUM> until an output grating formed on eyepiece <NUM> diffracts the light toward the user. Viewing optics assembly <NUM> further includes lens assemblies <NUM> including a -<NUM> diopter component positioned between eyepiece <NUM> and the user's eye and a +<NUM> diopter component positioned on the world side of eyepiece <NUM>.

In the example illustrated in <FIG>, vergence-accommodation cues are disrupted and the depth of field of the system is extended by modulating the diameter of the laser beam, rather than through divergence/convergence of the image light. This may be performed by light modifying device <NUM> prior to injecting the light into ICG <NUM>. In this case, the percept is driven by the inability of the lens assembly between eyepiece <NUM> and the user's eye to provide a small focal spot due to the reduced size of the pixel beam.

<FIG> illustrate an example light modifying device for reducing the diameter of the collimated pixel beam, according to some embodiments of the present invention. By varying the position of a second lens <NUM> relative to a first lens <NUM> and a third lens <NUM>, the diameter of the input collimated pixel beam can be expanded, reduced, or left unmodified. In reference to <FIG>, second lens <NUM> is adjusted to be positioned closer to first lens <NUM> than to third lens <NUM> (e.g., adjacent to first lens <NUM>), causing the diameter of the collimated pixel beam to become expanded upon exiting the light modifying device. In reference to <FIG>, second lens <NUM> is adjusted to be positioned at a midpoint between first lens <NUM> and third lens <NUM>, causing the diameter of the collimated pixel beam to be left unmodified upon exiting the light modifying device. In reference to <FIG>, second lens <NUM> is adjusted to be positioned closer to third lens <NUM> than to first lens <NUM> (e.g., adjacent to third lens <NUM>), causing the diameter of the collimated pixel beam to become reduced upon exiting the light modifying device.

In some embodiments, the light modifying device illustrated in <FIG> is used to dynamically change the diameter of a MEMs laser beam. In some instances, the light modifying device may be positioned prior to the MEMs mirror(s) so as to modify the laser beam prior to entering the MEMs mirror(s).

<FIG> illustrates an example control scheme for a light modifying device and the corresponding user visual percept of the system's output, according to some embodiments of the present invention. In some embodiments, a VAC delimited zone <NUM> is defined based on a desired VAC limit, such as <NUM> diopter. VAC delimited zone <NUM> may include a lower distance threshold <NUM>, below which the VAC experienced by a user exceeds the VAC limit, and an upper distance threshold <NUM>, above which the VAC experienced by a user exceeds the VAC limit.

Under the control scheme, when it is determined that the distance of the virtual depth plane (from the AR device or user) is less than lower distance threshold <NUM>, the light modifying device is caused to modify the wavefront of the collimated pixel beam. When it is determined that the distance of the virtual depth plane is greater than lower distance threshold <NUM> and less than upper distance threshold <NUM> (i.e., is within VAC delimited zone <NUM>), the light modifying device is caused to not modify the collimated pixel beam and to output the collimated pixel beam without modification. When it is determined that the distance of the virtual depth plane is greater than upper distance threshold <NUM>, the light modifying device is caused to modify the wavefront of the collimated pixel beam.

The control scheme may optionally implement gradual modifications to the collimated pixel beam at or near the distance thresholds. For example, the light modifying device may impart partial modifications to the collimated pixel beam for virtual distances just before a distance threshold, greater modifications at the distance threshold, and full modifications well past the distance threshold. As one example, for an upper distance threshold of <NUM> meters, a control scheme may be implemented in which the collimated pixel beam is converged at <NUM>% for a virtual distance of <NUM> meters, <NUM>% for a virtual distance of <NUM> meters, <NUM>% for a virtual distance of <NUM> meters, <NUM>% for a virtual distance of <NUM> meters, and <NUM>% for a virtual distance of <NUM> meters. In the same or a different example, for a lower distance threshold of <NUM> meters, a control scheme may be implemented in which the collimated pixel beam is converged at <NUM>% for a virtual distance of <NUM> meters, <NUM>% for a virtual distance of <NUM> meters, <NUM>% for a virtual distance of <NUM> meters, <NUM>% for a virtual distance of <NUM> meters, and <NUM>% for a virtual distance of <NUM> meters. Control schemes with longer or shorter transition bands than the above examples may be implemented. One of ordinary skill in the art will see various variations, alternatives, and modifications.

<FIG> illustrates an example method for defining a VAC delimited zone <NUM>, according to some embodiments of the present invention. First, the VAC experienced by a user is plotted as a function of the distance of the virtual depth plane from the AR device (alternatively referred to as the "VAC plot"). In some embodiments, the VAC plot is determined based on the focal plane design of the AR device. For the VAC plot illustrated in <FIG>, a <NUM> meters focal plane is utilized. Next, the VAC limit is plotted alongside the VAC experienced by the user. Next, intersection points <NUM>, <NUM> between the two plots are identified and the corresponding distances are used as lower and upper distance thresholds of VAC zone <NUM>, respectively.

<FIG> illustrates various examples of VAC delimited zones that may be defined based on VAC plots for various single focal plane systems. As the focal plane of the AR device increases, both the lower distance threshold and the upper distance threshold of the VAC delimited zone increase, presenting a trade-off between near-field versus far-field performance. Additional depth planes can be added to the system to increase the VAC delimited zone.

<FIG> illustrates an example method <NUM> of operating an optical system (e.g., AR device <NUM>), according to some embodiments of the present invention. One or more steps of method <NUM> may be performed in a different order than the illustrated embodiment, and one or more steps of method <NUM> may be omitted during performance of method <NUM>. Furthermore, two or more steps of method <NUM> may be performed simultaneously or concurrently with each other.

At step <NUM>, a VAC delimited zone (e.g., VAC delimited zones <NUM>, <NUM>) is defined. In some embodiments, the VAC delimited zone is defined based on the number of focal planes of the optical device and/or their corresponding focal plane locations. For example, the VAC associated with a single focal plane system with a focal plane location at <NUM> diopters can be estimated and used to determine the VAC delimited zone, which may be significantly smaller than the VAC delimited zone determined using the VAC associated with a multiple focal plane system, such as, for example, a two-focal plane system with focal plane locations at <NUM> diopters and <NUM> diopters. In some embodiments, the VAC delimited zone is additionally (or alternatively) defined based on a VAC limit, which may be specified by a user or may be predetermined for the system. In some embodiments, the VAC delimited zone is defined by finding the intersection point(s) (e.g., intersection points <NUM>, <NUM>) between the VAC associated with the optical system and the VAC limit, as described at least in reference to <FIG>, <FIG>, and <FIG>.

In some embodiments, the VAC delimited zone is defined as a function of distance from the optical system, where distances inside the VAC delimited zone correspond to virtual depth planes at which virtual content causes a user to experience VAC less than the VAC limit, and distances outside the VAC delimited zone correspond to virtual depth planes at which virtual content causes a user to experience VAC greater than the VAC limit. In some embodiments, the VAC delimited zone includes at least one distance threshold. For example, the VAC delimited zone may include a lower distance threshold (e.g., lower distance threshold <NUM>) and/or an upper distance threshold (e.g., upper distance threshold <NUM>), the lower distance threshold being less than the upper distance threshold.

At step <NUM>, a virtual distance of a virtual depth plane (e.g., virtual depth planes <NUM>) from the optical system at which a virtual object (e.g., virtual objects <NUM>) is to be displayed is determined. The virtual distance may be expressed in meters, diopters, or some other unit that indicates physical displacement. In some embodiments, the virtual distance is determined by a processing module (e.g., processing module <NUM>). In some embodiments, the virtual distance is determined prior to, during, or after the collimated pixel beam associated with the virtual object is generated by the optical system.

At step <NUM>, the virtual distance is compared to the lower distance threshold and/or the upper distance threshold. In some embodiments, it is determined whether the virtual distance is less than the lower distance threshold, greater than the lower distance threshold and less than the upper distance threshold, or greater than the upper distance threshold. For example, in some embodiments, step <NUM> may include determining whether the virtual distance is less than the lower distance threshold. As another example, in some embodiments, step <NUM> may include determining whether the virtual distance is greater than the upper distance threshold. As another example, in some embodiments, step <NUM> may include determining whether the virtual distance is less than the lower distance threshold or greater than the upper distance threshold. In some embodiments, step <NUM> is equivalent to determining whether the virtual distance is outside the VAC delimited zone.

At step <NUM>, a collimated pixel beam (e.g., collimated pixel beams <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) associated with the virtual object is generated by the optical system. In some embodiments, the collimated pixel beam is generated by a projector (e.g., projectors <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) of the optical system. The collimated pixel beam may contain color, brightness, and size information for displaying the virtual object. For example, the collimated pixel beam may include light from a single LED color source (e.g., red) or from multiple LED color sources (e.g., red, green, and blue).

At step <NUM>, the collimated pixel beam is modified to generate a modified pixel beam (e.g., modified pixel beams <NUM>, <NUM>, <NUM>). In some embodiments, the collimated pixel beam is modified by a light modifying device (e.g., light modifying devices <NUM>, <NUM>, <NUM>, <NUM>) of the optical system. In some embodiments, whether or not step <NUM> is performed may depend on the comparison performed in step <NUM>. For example, in some embodiments, step <NUM> is performed only when it is determined that the virtual distance is outside the VAC delimited zone. For example, step <NUM> may only be performed in response to determining that the virtual distance is less than the lower distance threshold or in response to determining that the virtual distance is greater than the upper distance threshold. In some embodiments, the light modifying device is integrated with the projector. In some embodiments, the light modifying device is separate from the projector.

In some embodiments, step <NUM> includes step <NUM> and/or step <NUM>. At step <NUM>, the collimated pixel beam is converged. In some embodiments, the collimated pixel beam is converged by the light modifying device. At step <NUM>, a diameter of the collimated pixel beam is reduced. In some embodiments, the diameter of the collimated pixel beam is reduced by the light modifying device.

At step <NUM>, the modified pixel beam is injected into an eyepiece (e.g., eyepieces <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) of the optical system. In some embodiments, the modified pixel beam is injected into an ICG (e.g., ICGs <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) formed on the eyepiece.

At step <NUM>, the modified pixel beam is outputted from the eyepiece of the optical system. In some embodiments, the modified pixel beam is outputted from a leaky-grating formed on the eyepiece. In some embodiments, the modified pixel beam is outputted from the eyepiece toward a user's eye.

<FIG> illustrates a simplified computer system <NUM> according to an example described herein. Computer system <NUM> as illustrated in <FIG> may be incorporated into devices described herein. <FIG> provides a schematic illustration of one example of computer system <NUM> that can perform some or all of the steps of the methods provided by various examples. It should be noted that <FIG> is meant only to provide a generalized illustration of various components, any or all of which may be utilized as appropriate. <FIG>, therefore, broadly illustrates how individual system elements may be implemented in a relatively separated or relatively more integrated manner.

Computer system <NUM> is shown comprising hardware elements that can be electrically coupled via a bus <NUM>, or may otherwise be in communication, as appropriate. The hardware elements may include one or more processors <NUM>, including without limitation one or more general-purpose processors and/or one or more special-purpose processors such as digital signal processing chips, graphics acceleration processors, and/or the like; one or more input devices <NUM>, which can include without limitation a mouse, a keyboard, a camera, and/or the like; and one or more output devices <NUM>, which can include without limitation a display device, a printer, and/or the like.

Computer system <NUM> may further include and/or be in communication with one or more non-transitory storage devices <NUM>, which can comprise, without limitation, local and/or network accessible storage, and/or can include, without limitation, a disk drive, a drive array, an optical storage device, a solid-state storage device, such as a random access memory ("RAM"), and/or a read-only memory ("ROM"), which can be programmable, flash-updateable, and/or the like. Such storage devices may be configured to implement any appropriate data stores, including without limitation, various file systems, database structures, and/or the like.

Computer system <NUM> might also include a communications subsystem <NUM>, which can include without limitation a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and/or a chipset such as a Bluetooth™ device, an <NUM> device, a WiFi device, a WiMax device, cellular communication facilities, etc., and/or the like. The communications subsystem <NUM> may include one or more input and/or output communication interfaces to permit data to be exchanged with a network such as the network described below to name one example, other computer systems, television, and/or any other devices described herein. Depending on the desired functionality and/or other implementation concerns, a portable electronic device or similar device may communicate image and/or other information via the communications subsystem <NUM>. In other examples, a portable electronic device, e.g. the first electronic device, may be incorporated into computer system <NUM>, e.g., an electronic device as an input device <NUM>. In some examples, computer system <NUM> will further comprise a working memory <NUM>, which can include a RAM or ROM device, as described above.

Computer system <NUM> also can include software elements, shown as being currently located within the working memory <NUM>, including an operating system <NUM>, device drivers, executable libraries, and/or other code, such as one or more application programs <NUM>, which may comprise computer programs provided by various examples, and/or may be designed to implement methods, and/or configure systems, provided by other examples, as described herein. Merely by way of example, one or more procedures described with respect to the methods discussed above, might be implemented as code and/or instructions executable by a computer and/or a processor within a computer; in an aspect, then, such code and/or instructions can be used to configure and/or adapt a general purpose computer or other device to perform one or more operations in accordance with the described methods.

A set of these instructions and/or code may be stored on a non-transitory computer-readable storage medium, such as the storage device(s) <NUM> described above. In some cases, the storage medium might be incorporated within a computer system, such as computer system <NUM>. In other examples, the storage medium might be separate from a computer system e.g., a removable medium, such as a compact disc, and/or provided in an installation package, such that the storage medium can be used to program, configure, and/or adapt a general purpose computer with the instructions/code stored thereon. These instructions might take the form of executable code, which is executable by computer system <NUM> and/or might take the form of source and/or installable code, which, upon compilation and/or installation on computer system <NUM> e.g., using any of a variety of generally available compilers, installation programs, compression/decompression utilities, etc., then takes the form of executable code.

For example, customized hardware might also be used, and/or particular elements might be implemented in hardware, software including portable software, such as applets, etc., or both.

As mentioned above, in one aspect, some embodiments may employ a computer system such as computer system <NUM> to perform methods in accordance with various embodiments of the technology. According to a set of embodiments, some or all of the procedures of such methods are performed by computer system <NUM> in response to processor <NUM> executing one or more sequences of one or more instructions, which might be incorporated into the operating system <NUM> and/or other code, such as an application program <NUM>, contained in the working memory <NUM>. Such instructions may be read into the working memory <NUM> from another computer-readable medium, such as one or more of the storage device(s) <NUM>. Merely by way of example, execution of the sequences of instructions contained in the working memory <NUM> might cause the processor(s) <NUM> to perform one or more procedures of the methods described herein. Additionally or alternatively, portions of the methods described herein may be executed through specialized hardware.

The terms "machine-readable medium" and "computer-readable medium," as used herein, refer to any medium that participates in providing data that causes a machine to operate in a specific fashion. In an embodiment implemented using computer system <NUM>, various computer-readable media might be involved in providing instructions/code to processor(s) <NUM> for execution and/or might be used to store and/or carry such instructions/code. In many implementations, a computer-readable medium is a physical and/or tangible storage medium. Such a medium may take the form of a non-volatile media or volatile media. Non-volatile media include, for example, optical and/or magnetic disks, such as the storage device(s) <NUM>. Volatile media include, without limitation, dynamic memory, such as the working memory <NUM>.

Common forms of physical and/or tangible computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read instructions and/or code.

Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor(s) <NUM> for execution. Merely by way of example, the instructions may initially be carried on a magnetic disk and/or optical disc of a remote computer. A remote computer might load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and/or executed by computer system <NUM>.

Specific details are given in the description to provide a thorough understanding of exemplary configurations including implementations. However, configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail in order to avoid obscuring the configurations. This description provides example configurations only, and does not limit the scope, applicability, or configurations of the claims. Rather, the preceding description of the configurations will provide those skilled in the art with an enabling description for implementing described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the disclosure.

Also, configurations may be described as a process which is depicted as a schematic flowchart or block diagram. Although each may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may have additional steps not included in the figure. Furthermore, examples of the methods may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored in a non-transitory computer-readable medium such as a storage medium. Processors may perform the described tasks.

Having described several example configurations, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, wherein other rules may take precedence over or otherwise modify the application of the technology. Also, a number of steps may be undertaken before, during, or after the above elements are considered. Accordingly, the above description does not bind the scope of the claims.

Also, the words "comprise", "comprising", "contains", "containing", "include", "including", and "includes", when used in this specification and in the following claims, are intended to specify the presence of stated features, integers, components, or steps, but they do not preclude the presence or addition of one or more other features, integers, components, steps, acts, or groups.

Claim 1:
An optical system comprising:
a projector (<NUM>) configured to generate a collimated pixel beam associated with a virtual object;
a light modifying device (<NUM>) configured to modify the collimated pixel beam to generate a modified pixel beam, wherein modifying the collimated pixel beam includes selectively converging the collimated pixel beam and/or reducing a diameter of the collimated pixel beam, depending on the operational requirements;
an eyepiece (<NUM>) configured to output the modified pixel beam; and
a processing module configured to perform operations comprising:
determining (<NUM>) a virtual distance of a virtual depth plane from the optical system at which the virtual object is to be displayed;
comparing (<NUM>) the virtual distance to at least one distance threshold; and
based on comparing the virtual distance to the at least one distance threshold, causing the light modifying device to modify (<NUM>) the collimated pixel beam to generate the modified pixel beam;
wherein the optical system is a single focal plane system.