Patent Publication Number: US-11662585-B2

Title: Virtual/augmented reality system having reverse angle diffraction grating

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     The present application is a continuation of U.S. patent application Ser. No. 16/391,577, filed Apr. 23, 2019 entitled “VIRTUAL/AUGMENTED REALITY SYSTEM HAVING REVERSE ANGLE DIFFRACTION GRATING,”, which claims priority to U.S. patent application Ser. No. 15/287,637, filed Oct. 6, 2016 entitled “VIRTUAL/AUGMENTED REALITY SYSTEM HAVING REVERSE ANGLE DIFFRACTION GRATING,”, which claims priority to U.S. Provisional Application Ser. No. 62/238,052, filed on Oct. 6, 2015 entitled “VIRTUAL/AUGMENTED REALITY SYSTEM HAVING REVERSE ANGLE DIFFRACTION GRATING,”. The content of the aforementioned patent applications are hereby expressly incorporated by reference in their entirety for all purposes as though set forth in full. 
    
    
     FIELD OF THE INVENTION 
     The present invention generally relates to systems and methods configured to facilitate interactive virtual or augmented reality environments for one or more users. 
     BACKGROUND 
     Modern computing and display technologies have facilitated the development of systems for so-called “virtual reality” or “augmented reality” experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner where they seem to be, or may be perceived as, real. A virtual reality (VR) scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input, whereas an augmented reality (AR) scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the end user. 
     For example, referring to  FIG.  1   , an augmented reality scene  4  is depicted wherein a user of an AR technology sees a real-world park-like setting  6  featuring people, trees, buildings in the background, and a concrete platform  8 . In addition to these items, the end user of the AR technology also perceives that he “sees” a robot statue  10  standing upon the real-world platform  8 , and a cartoon-like avatar character  12  flying by which seems to be a personification of a bumble bee, even though these elements  10 ,  12  do not exist in the real world. As it turns out, the human visual perception system is very complex, and producing a VR or AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. 
     VR and AR systems typically employ head-worn displays (or helmet-mounted displays, or smart glasses) that are at least loosely coupled to a user&#39;s head, and thus move when the end user&#39;s head moves. If the end user&#39;s head motions are detected by the display system, the data being displayed can be updated to take the change in head pose (i.e., the orientation and/or location of user&#39;s head) into account. 
     As an example, if a user wearing a head-worn display views a virtual representation of a three-dimensional (3D) object on the display and walks around the area where the 3D object appears, that 3D object can be re-rendered for each viewpoint, giving the end user the perception that he or she is walking around an object that occupies real space. If the head-worn display is used to present multiple objects within a virtual space (for instance, a rich virtual world), measurements of head pose can be used to re-render the scene to match the end user&#39;s dynamically changing head location and orientation and provide an increased sense of immersion in the virtual space. 
     Head-worn displays that enable AR (i.e., the concurrent viewing of real and virtual elements) can have several different types of configurations. In one such configuration, often referred to as a “video see-through” display, a camera captures elements of a real scene, a computing system superimposes virtual elements onto the captured real scene, and a non-transparent display presents the composite image to the eyes. Another configuration is often referred to as an “optical see-through” display, in which the end user can see through transparent (or semi-transparent) elements in the display system to view directly the light from real objects in the environment. The transparent element, often referred to as a “combiner,” superimposes light from the display over the end user&#39;s view of the real world. 
     VR and AR systems typically employ a display system having a projection subsystem and a display surface positioned in front of the end user&#39;s field of view and on which the projection subsystem sequentially projects image frames. In true three-dimensional systems, the depth of the display surface can be controlled at frame rates or sub-frame rates. The projection subsystem may include one or more optical fibers into which light from one or more light sources emit light of different colors in defined patterns, and a scanning device that scans the optical fiber(s) in a predetermined pattern to create the image frames that sequentially displayed to the end user. 
     In one embodiment, the display system includes one or more planar waveguides that are generally parallel to the field of view of the user, and into which light from the optical fiber(s) is injected. One or more linear diffraction gratings are embedded within the waveguide(s) to change the angle of incident light propagating along the waveguide(s). By changing the angle of light beyond the threshold of total internal reflection (TIR), the light escapes from one or more lateral faces of the waveguide(s). The linear diffraction grating(s) have a low diffraction efficiency, so only a fraction of the light energy is directed out of the waveguide(s), each time the light encounters the linear diffraction grating(s). By outcoupling the light at multiple locations along the grating(s), the exit pupil of the display system is effectively increased. The display system may further comprise one or more collimation elements that collimate light coming from the optical fiber(s), and one or more optical coupling elements that optically couple the collimated light to, or from, an edge of the waveguide(s). 
     In a typical optical fiber scanning display system, each optical fiber acts as a vibrating cantilever that sweeps through relatively large deflections from a fulcrum in order to scan the light in accordance with a designed scan pattern. However, due to the large deflections of the collimated light, the size of the optical coupling element(s) must be relatively large, thereby increasing the size of the display system. This size of the optical coupling element(s) becomes more problematic in the case of a stacked waveguide architecture, which requires the optical element(s) associated with the waveguides that are more distance from the scanning optical fiber(s) to be larger to accommodate the larger span of the scanned collimated light. 
     For example, with reference to  FIG.  2   , one embodiment of a display system  20  comprises one or more light sources  22  that generate image data that is encoded in the form of light that is spatially and/or temporally varying, an optical fiber  24  optically coupled to the light source(s)  22 , and a collimation element  26  that collimates the light exiting the distal end of the optical fiber  24 . The display system  20  further comprises a piezoelectric element  28  to or in which the optical fiber  24  is mounted as a fixed-free flexible cantilever, and drive electronics  30  electrically coupled to the piezoelectric element  22  to activate electrically stimulate the piezoelectric element  28 , thereby causing the distal end of the optical fiber  24  to vibrate in a pre-determined scan pattern that creates deflections  32  about a fulcrum  34 . 
     The display system  20  includes a waveguide apparatus  38  that includes a plurality of planar waveguides  40   a - 40   e  that are generally parallel to the field-of-view of the end user, and one or more diffractive optical elements (DOEs)  42   a - 42   e  associated with each of the planar waveguides  40 . Light originating from the optical fiber  24  propagates along selected ones of the planar waveguides  40  and intersects with the corresponding DOEs  42 , causing a portion of the light to exit the face of the waveguide apparatus  38  towards the eyes of the end user that is focused at one or more viewing distances depending on the selected planar waveguide(s)  40 . 
     The display system  20  further comprises optical coupling elements in the form of diffractive optical elements (DOEs)  44   a - 44   e  that are integrated within the ends of the respective planar waveguides  40   a - 40   e  and that reflect the collimate light into selected ones of the planar waveguides  40 . As can be seen, as the distance between each DOE  44  and the end of the optical fiber  24  increases, the length of the respective DOE  44  must increase in order to accommodate the increasing linear span of the deflection angle of the optical fiber  24 . This necessarily adds size and complexity to the waveguide apparatus  38  due to the largest DOE  44 , and in this case, the DOE  44   e.    
     As another example, with reference to  FIG.  3   , another embodiment of a display system  50  is similar to the display system  10  of  FIG.  2   , with the exception that the display system  50  comprises an optical coupling element in the form of an optical distribution waveguide  52  that have DOEs  54   a - 54   e  that reflect the collimate light into selected ones of the planar waveguides  40 . The width of the distribution waveguide  52  must be large enough to accommodate the maximum linear span of the deflection angle of the optical fiber  24 , thereby necessarily adding size and complexity to the waveguide apparatus  38 . 
     There, thus, is a need to reduce the size of optical coupling element(s) used to couple light from one or more optical fibers into one or more planar waveguides in a virtual reality or augmented reality environment. 
     SUMMARY 
     Embodiments of the present invention are directed to devices, systems and methods for facilitating virtual reality and/or augmented reality interaction for one or more users. 
     In accordance with the present inventions, a display subsystem for a virtual image generation system for use by an end user is provided. The virtual image generation system may, e.g., comprise memory storing a three-dimensional scene, and a control subsystem configured for rendering a plurality of synthetic image frames of the three-dimensional scene, in which case, the display subsystem may be configured for sequentially displaying the plurality of image frames to the end user. 
     The display subsystem comprises a planar waveguide apparatus, an optical fiber, and at least one light source configured for emitting light from a distal end of the optical fiber. In one embodiment, the planar waveguide apparatus is configured for being positioned in front of the eyes of the end user. The planar waveguide apparatus may have a partially transparent display surface configured for being positioned in the field of view between the eyes of the end user and an ambient environment. In one embodiment, the display subsystem may further comprise a frame structure configured for being worn by the end user, in which case, the frame structure may carry the planar waveguide apparatus. 
     The display subsystem further comprises a mechanical drive assembly to which the optical fiber is mounted as a fixed-free flexible cantilever. The drive assembly is configured for displacing a distal end of the optical fiber about a fulcrum in accordance with a scan pattern, such that the outputted/emitted light diverges from a longitudinal axis coincident with the fulcrum. In one embodiment, the mechanical drive assembly comprises a piezoelectric element to which the optical fiber is mounted, and drive electronics configured for conveying electrical signals to the piezoelectric element, thereby causing the optical fiber to vibrate in accordance with the scan pattern. The display subsystem may optionally further comprise a collimation element configured for collimating light from the optical fiber. 
     The display subsystem further comprises an optical modulation apparatus configured for converging the light from the optical fiber towards the longitudinal axis. In one embodiment, the optical modulation apparatus is configured for converging the light on a focal point on the longitudinal axis. The focal point may be, e.g., located within the optical waveguide input apparatus, such as the center of the optical waveguide input apparatus along the longitudinal axis. The optical modulation apparatus may, e.g., comprise at least one diffraction grating. Each of the diffraction grating(s) may have a diffraction pattern that matches the geometry of the scan pattern. For example, if the scan pattern is a spiral scan pattern, the diffraction pattern may be a spiral diffraction pattern. In another embodiment, the optical modulation apparatus comprises two orthogonal diffraction gratings in series, such that one of the diffraction gratings diffracts light along a first axis, and another one of the diffraction gratings diffracts light along a second axis orthogonal to the first axis. 
     The display subsystem further comprises an optical waveguide input apparatus configured for directing the light from the optical modulation apparatus down the planar waveguide apparatus, such that the planar waveguide apparatus displays one or more image frames to the end user. In one embodiment, the planar waveguide apparatus comprises a plurality of planar waveguides configured for respectively displaying the image frame(s) at different focal points to the end user, in which case, the optical waveguide input apparatus may be configured for directing the light down selected ones of the plurality of planar waveguides. In another embodiment, the optical waveguide input apparatus comprises a plurality of diffractive optical elements respectively extending parallel along the planar waveguides, and respectively directing the light from the optical modulation apparatus down the planar waveguides. In still another embodiment, the optical waveguide input apparatus comprises a distribution waveguide extending perpendicularly to the planar waveguides, and the distribution waveguide comprises a plurality of diffractive optical elements that respectively direct the light from the optical modulation apparatus down the planar waveguides. 
     Additional and other objects, features, and advantages of the invention are described in the detail description, figures and claims. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The drawings illustrate the design and utility of preferred embodiments of the present invention, in which similar elements are referred to by common reference numerals. In order to better appreciate how the above-recited and other advantages and objects of the present inventions are obtained, a more particular description of the present inventions briefly described above will be rendered by reference to specific embodiments thereof, which are illustrated in the accompanying drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which: 
         FIG.  1    is a picture of a three-dimensional augmented reality scene that can be displayed to an end user by a prior art augmented reality generation device; 
         FIG.  2    is a plan view of one embodiment of a prior art display system that can be used in an augmented reality generation device; 
         FIG.  3    is a plan view of another embodiment of a prior art display system that can be used in an augmented reality generation device; 
         FIG.  4    is a block diagram of a virtual image generation system constructed in accordance with one embodiment of the present inventions; 
         FIG.  5    is a plan view of an exemplary frame generated by the virtual image generation system of  FIG.  4   . 
         FIG.  6    is a plan view of one scanning pattern that can be used to generate a frame; 
         FIG.  7    is a plan view of another scanning pattern that can be used to generate a frame; 
         FIG.  8    is a plan view of still another scanning pattern that can be used to generate a frame; 
         FIG.  9    is a plan view of yet another scanning pattern that can be used to generate a frame; 
         FIG.  10   a    is a plan view of one technique that can be used to wear the virtual image generation system of  FIG.  4   ; 
         FIG.  10   b    is a plan view of another technique that can be used to wear the virtual image generation system of  FIG.  4   ; 
         FIG.  10   c    is a plan view of still another technique that can be used to wear the virtual image generation system of  FIG.  4   ; 
         FIG.  10   d    is a plan view of yet another technique that can be used to wear the virtual image generation system of  FIG.  4   ; 
         FIG.  11    is a plan view of one embodiment of a display subsystem for use in the virtual image generation system of  FIG.  4   ; 
         FIG.  12    is one embodiment of a primary planar waveguide for use in the display subsystem of  FIG.  11   ; 
         FIG.  13   a    is perspective view of one embodiment of a display subsystem for use in the virtual image generation system of  FIG.  4   ; 
         FIG.  13   b    is a perspective view of the display subsystem of  FIG.  13   a   , particularly showing light rays extending from one focal point; 
         FIG.  13   c    is a perspective view of the display subsystem of  FIG.  13   a   , particularly showing light rays extending from another focal point; 
         FIG.  14    is a plan view of another embodiment of a display subsystem for use in the virtual image generation system of  FIG.  4   ; 
         FIG.  15    is a plan view of one embodiment of a planar waveguide apparatus for use in the display subsystem of  FIG.  13   ; 
         FIG.  16    is a plan view of another embodiment of a planar waveguide apparatus for use in the display subsystem of  FIG.  13   ; 
         FIG.  17    is a profile view of the planar waveguide apparatus of  FIG.  16   ; 
         FIG.  18   a    is a plan view of one embodiment of an optical coupling subsystem and scanning device that can be used in the display subsystem of  FIG.  13   , particularly showing the convergence of light beams onto a focal point in the center of the optical waveguide input apparatus; 
         FIG.  18   b    is a plan view of the optical coupling subsystem and scanning device of  FIG.  18   a   , particularly showing the convergence of light beams onto a focal point at the edge of the optical waveguide input apparatus; 
         FIG.  19    is a plan view of one embodiment of an optical waveguide input apparatus for use in the optical coupling subsystem of  FIG.  13     a;    
         FIG.  20    is a plan view of another embodiment of an optical waveguide input apparatus for use in the optical coupling subsystem of  FIG.  13     a;    
         FIG.  21    is a plan view of a spiral diffraction pattern that can be used in an optical modulation apparatus in the optical coupling subsystem of  FIG.  18     a;    
         FIG.  22   a    is one embodiment of an optical modulation apparatus that can be used in the optical coupling subsystem of  FIG.  18   a   ; and 
         FIG.  22   b    is another embodiment of an optical modulation apparatus that can be used in the optical coupling subsystem of  FIG.  18     a.    
     
    
    
     DETAILED DESCRIPTION 
     The description that follows relates to display systems and methods to be used in virtual reality and/or augmented reality systems. However, it is to be understood that the while the invention lends itself well to applications in virtual or augmented reality systems, the invention, in its broadest aspects, may not be so limited. 
     Referring to  FIG.  4   , one embodiment of a virtual image generation system  100  constructed in accordance with present inventions will now be described. The virtual image generation system  100  may be operated as an augmented reality subsystem, providing images of virtual objects intermixed with physical objects in a field of view of an end user  50 . There are two fundamental approaches when operating the virtual image generation system  100 . A first approach employs one or more imagers (e.g., cameras) to capture images of the ambient environment. The virtual image generation system  100  inter-mixes the virtual images into the data representing the images of the ambient environment. A second approach employs one or more at least partially transparent surfaces through which the ambient environment can be seen and on to which the virtual image generation system  100  produces images of virtual objects. 
     The virtual image generation system  100 , and the various techniques taught herein, may be employed in applications other than augmented reality and virtual reality subsystems. For example, various techniques may be applied to any projection or display subsystem, or may be applied to pico projectors where movement may be made by an end user&#39;s hand rather than the head. Thus, while often described herein in terms of an augmented reality subsystem or virtual reality subsystem, the teachings should not be limited to such subsystems of such uses. 
     At least for augmented reality applications, it may be desirable to spatially position various virtual objects relative to respective physical objects in a field of view of the end user  50 . Virtual objects, also referred to herein as virtual tags or tag or call outs, may take any of a large variety of forms, basically any variety of data, information, concept, or logical construct capable of being represented as an image. Non-limiting examples of virtual objects may include: a virtual text object, a virtual numeric object, a virtual alphanumeric object, a virtual tag object, a virtual field object, a virtual chart object, a virtual map object, a virtual instrumentation object, or a virtual visual representation of a physical object. 
     To this end, the virtual image generation system  100  comprises a frame structure  102  worn by an end user  50 , a display subsystem  104  carried by the frame structure  102 , such that the display subsystem  104  is positioned in front of the eyes  52  of the end user  50 , and a speaker  106  carried by the frame structure  102 , such that the speaker  106  is positioned adjacent the ear canal of the end user  50  (optionally, another speaker (not shown) is positioned adjacent the other ear canal of the end user  50  to provide for stereo/shapeable sound control). The display subsystem  104  is designed to present the eyes  52  of the end user  50  with photo-based radiation patterns that can be comfortably perceived as augmentations to physical reality, with high-levels of image quality and three-dimensional perception, as well as being capable of presenting two-dimensional content. The display subsystem  104  presents a sequence of synthetic image frames at high frequency that provides the perception of a single coherent scene. 
     The display subsystem  104  comprises a projection subsystem  108  and a partially transparent display screen  110  on which the projection subsystem  108  projects images. The display screen  110  is positioned in the end user&#39;s  50  field of view between the eyes  52  of the end user  50  and an ambient environment. 
     In the illustrated embodiment, the projection subsystem  108  takes the form of an optical fiber scan-based projection device, and the display screen  110  takes the form of a waveguide-based display into which the scanned light from the projection subsystem  108  is injected to produce, e.g., images at single optical viewing distance closer than infinity (e.g., arm&#39;s length), images at multiple, discrete optical viewing distances or focal planes, and/or image layers stacked at multiple viewing distances or focal planes to represent volumetric 3D objects. These layers in the light field may be stacked closely enough together to appear continuous to the human visual subsystem (i.e., one layer is within the cone of confusion of an adjacent layer). Additionally or alternatively, picture elements may be blended across two or more layers to increase perceived continuity of transition between layers in the light field, even if those layers are more sparsely stacked (i.e., one layer is outside the cone of confusion of an adjacent layer). The display subsystem  104  may be monocular or binocular. 
     Thus, the display subsystem  104  generates a series of synthetic image frames of pixel information that present an image of one or more virtual objects to the user. For example, referring to  FIG.  5   , a synthetic image frame  118  is schematically illustrated with cells  120   a - 120   m  divided into horizontal rows or lines  122   a - 122   n . Each cell  120  of the frame  118  may specify values for each of a plurality of colors for the respective pixel to which the cell  120  corresponds and/or intensities. For instance, the frame  118  may specify one or more values for red  124   a , one or more values for green  124   b , and one or more values for blue  124   c  for each pixel. The values  124  may be specified as binary representations for each of the colors, for instance, a respective 4-bit number for each color. Each cell  120  of the frame  118  may additionally include a value  124   d  that specifies an amplitude. 
     The frame  118  may include one or more fields, collectively  126 . The frame  118  may consist of a single field. Alternatively, the frame  118  may comprise two, or even more fields  126   a - 126   b . The pixel information for a complete first field  126   a  of the frame  118  may be specified before the pixel information for the complete second field  126   b , for example occurring before the pixel information for the second field  126   b  in an array, an ordered list or other data structure (e.g., record, linked list). A third or even a fourth field may follow the second field  126   b , assuming a presentation subsystem is configured to handle more than two fields  126   a - 126   b.    
     Referring now to  FIG.  6   , the frame  118  is generated using a raster scan pattern  128 . In the raster scan pattern  128 , pixels  130  (only one called out) are sequentially presented. The raster scan pattern  128  typically presents pixels from left to right (indicated by arrows  132   a ,  132   b , then from top to bottom (indicated by arrow  134 ). Thus, the presentation may start at the upper right corner and traverse left across a first line  136   a  until the end of the line is reached. The raster scan pattern  128  typically then starts from the left in a next line down. The presentation may be temporarily blacked out or blanked when returning from the end of one line to the start of the next line. This process repeats line-by-line until the bottom line  136   n  is completed, for example at the bottom right most pixel. With the frame  118  being complete, a new frame is started, again returning the right of the top most line of the next frame. Again, the presentation may be blanked while returning from the bottom left to the top right to present the next frame. 
     Many implementations of raster scanning employ what is termed as an interlaced scan pattern. In interlaced raster scan patterns, lines from the first and the second fields  126   a ,  126   b  are interlaced. For example, when presenting lines of the first field  126   a , the pixel information for the first field  126   a  may be used for the odd numbered lines only, while the pixel information for the second field  126   b  may be used for the even numbered lines only. Thus, all of the lines of the first field  126   a  of the frame  118  ( FIG.  5   ) are typically presented before the lines of the second field  126   b . The first field  126   a  may be presented using the pixel information of the first field  126   a  to sequentially present line 1, line 3, line 5, etc. Then the second field  126   b  of the frame  118  ( FIG.  5   ) may be presented following the first field  126   a , by using the pixel information of the second field  126   b  to sequentially present line 2, line 4, line 6, etc. 
     Referring to  FIG.  7   , a spiral scan pattern  140  may be used instead of the raster scan pattern  128  to generate the frame  118 . The spiral scan pattern  140  may consist of a single spiral scan line  142 , which may include one or more complete angular cycles (e.g., 360 degrees) which may be denominated as coils or loops. As with the raster scan pattern  128  illustrated in  FIG.  6   , the pixel information in the spiral scan pattern  140  is used to specify the color and/or intensity of each sequential pixel, as the angle increments. An amplitude or radial value  146  specifies a radial dimension from a starting point  148  of the spiral scan line  142 . 
     Referring to  FIG.  8   , a Lissajous scan pattern  150  may alternatively be used to generate the frame  118 . The Lissajous scan pattern  150  may consist of a single Lissajous scan line  152 , which may include one or more complete angular cycles (e.g., 360 degrees), which may be denominated as coils or loops. Alternatively, the Lissajous scan pattern  150  may include two or more Lissajous scan lines  152 , each phase shifted with respect to one another to nest the Lissajous scan lines  152 . The pixel information is used to specify the color and/or intensity of each sequential pixel, as the angle increments. An amplitude or radial value specifies a radial dimension  154  from a starting point  156  of the Lissajous scan line  152 . 
     Referring to  FIG.  9   , a multi-field spiral scan pattern  158  may alternatively be used to generate the frame  118 . The multi-field spiral scan pattern  158  includes two or more distinct spiral scan lines, collectively  160 , and in specifically, four spiral scan lines  160   a - 160   d . The pixel information for each spiral scan line  160  may be specified by a respective field of a frame. Advantageously, multiple spiral scan lines  160  may be nested simply by shifting a phase between each successive ones of the spiral scan lines  160 . The phase difference between spiral scan lines  160  should be a function of the total number of spiral scan lines  160  that will be employed. For example, four spiral scan lines  160   a - 160   d  may be separated by a 90 degree phase shift. An exemplary embodiment may operate at a 100 Hz refresh rate with 10 distinct spiral scan lines (i.e., subspirals). Similar to the embodiment of  FIG.  7   , one or more amplitude or radial values specify a radial dimension  162  from a starting point  164  of the spiral scan lines  160 . 
     Further details describing display subsystems are provided in U.S. Provisional Patent Application Ser. No. 61/801,219, entitled “Display Subsystem and Method”, and U.S. Provisional Patent Application Ser. No. 61/845,907, entitled “Planar Waveguide Apparatus With Diffraction Element(s) and Subsystem Employing Same”, which are expressly incorporated herein by reference. 
     Referring back to  FIG.  4   , the virtual image generation system  100  further comprises one or more sensors (not shown) mounted to the frame structure  102  for detecting the position and movement of the head  54  of the end user  50  and/or the eye position and inter-ocular distance of the end user  50 . Such sensor(s) may include image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, and/or gyros). 
     For example, in one embodiment, the virtual image generation system  100  comprises a head worn transducer subsystem  126  that includes one or more inertial transducers to capture inertial measures indicative of movement of the head  54  of the end user  50 . Such may be used to sense, measure, or collect information about the head movements of the end user  50 . For instance, such may be used to detect measurement movements, speeds, acceleration, and/or positions of the head  54  of the end user  50 . 
     The virtual image generation system  100  further comprises one or more forward facing cameras  128 , which may be used to capture information about the environment in which the end user  50  is located. The forward facing camera(s)  128  may be used to capture information indicative of distance and orientation of the end user  50  with respect to that environment and specific objects in that environment. When head worn, the forward facing camera(s)  128  is particularly suited to capture information indicative of distance and orientation of the head  54  of the end user  50  with respect to the environment in which the end user  50  is located and specific objects in that environment. The forward facing camera(s)  128  may, for example, be employed to detect head movement, speed, and/or acceleration of head movements. The forward facing camera(s)  128  may, for example, be employed to detect or infer a center of attention of the end user  50 , for example, based at least in part on an orientation of the head  54  of the end user  50 . Orientation may be detected in any direction (e.g., up/down, left, right with respect to the reference frame of the end user  50 ). 
     The virtual image generation system  100  further comprises a pair of rearward facing cameras  129  to track movement, blinking, and depth of focus of the eyes  52  of the end user  50 . Such eye tracking information may, for example, be discerned by projecting light at the end user&#39;s eyes, and detecting the return or reflection of at least some of that projected light. Further details discussing eye tracking devices are provided in U.S. Patent Application Ser. No. 61/801,219 (Attorney Docket No. ML-30006-US), entitled “Display Subsystem and Method,” U.S. Patent Application Ser. No. 62/005,834, entitled “Methods and Subsystem for Creating Focal Planes in Virtual and Augmented Reality,” and U.S. Patent Application Ser. No. 61/776,771, entitled “Subsystem and Method for Augmented and Virtual Reality,” which are expressly incorporated herein by reference. 
     The virtual image generation system  100  further comprises a user orientation detection module  130 . The patient orientation module  130  detects the instantaneous position of the head  54  of the end user  50  and may predict the position of the head  54  of the end user  50  based on position data received from the sensor(s). Significantly, detecting the instantaneous position of the head  54  of the end user  50  facilitates determination of the specific actual object that the end user  50  is looking at, thereby providing an indication of the specific textual message to be generated for that actual object and further providing an indication of the textual region in which the textual message is to be streamed. The patient orientation module  130  also tracks the eyes  52  of the end user  50  based on the tracking data received from the sensor(s). 
     The virtual image generation system  100  further comprises a control subsystem that may take any of a large variety of forms. The control subsystem includes a number of controllers, for instance one or more microcontrollers, microprocessors or central processing units (CPUs), digital signal processors, graphics processing units (GPUs), other integrated circuit controllers, such as application specific integrated circuits (ASICs), programmable gate arrays (PGAs), for instance field PGAs (FPGAs), and/or programmable logic controllers (PLUs). 
     In the illustrated embodiment, the virtual image generation system  100  comprises a central processing unit (CPU)  132 , a graphics processing unit (GPU)  134 , and one or more frame buffers  136 . The CPU  132  controls overall operation, while the GPU  134  renders frames (i.e., translating a three-dimensional scene into a two-dimensional image) from three-dimensional data stored in the remote data repository  150  and stores these frames in the frame buffer(s)  136 . While not illustrated, one or more additional integrated circuits may control the reading into and/or reading out of frames from the frame buffer(s)  136  and operation of the scanning device of the display subsystem  104 . Reading into and/or out of the frame buffer(s)  146  may employ dynamic addressing, for instance, where frames are over-rendered. The virtual image generation system  100  further comprises a read only memory (ROM)  138  and a random access memory (RAM)  140 . The virtual image generation system  100  further comprises a three-dimensional data base  142  from which the GPU  134  can access three-dimensional data of one or more scenes for rendering frames. 
     The various processing components of the virtual image generation system  100  may be physically contained in a distributed subsystem. For example, as illustrated in  FIGS.  10   a - 10   d   , the virtual image generation system  100  comprises a local processing and data module  144  operatively coupled, such as by a wired lead or wireless connectivity  146 , to the display subsystem  104  and sensors. The local processing and data module  144  may be mounted in a variety of configurations, such as fixedly attached to the frame structure  102  ( FIG.  10   a   ), fixedly attached to a helmet or hat  56  ( FIG.  10   b   ), embedded in headphones, removably attached to the torso  58  of the end user  50  ( FIG.  10   c   ), or removably attached to the hip  60  of the end user  50  in a belt-coupling style configuration ( FIG.  10   d   ). The virtual image generation system  100  further comprises a remote processing module  148  and remote data repository  150  operatively coupled, such as by a wired lead or wireless connectivity  150 ,  152 , to the local processing and data module  144 , such that these remote modules  148 ,  150  are operatively coupled to each other and available as resources to the local processing and data module  144 . 
     The local processing and data module  144  may comprise a power-efficient processor or controller, as well as digital memory, such as flash memory, both of which may be utilized to assist in the processing, caching, and storage of data captured from the sensors and/or acquired and/or processed using the remote processing module  148  and/or remote data repository  150 , possibly for passage to the display subsystem  104  after such processing or retrieval. The remote processing module  148  may comprise one or more relatively powerful processors or controllers configured to analyze and process data and/or image information. The remote data repository  150  may comprise a relatively large-scale digital data storage facility, which may be available through the internet or other networking configuration in a “cloud” resource configuration. In one embodiment, all data is stored and all computation is performed in the local processing and data module  144 , allowing fully autonomous use from any remote modules. 
     The couplings  146 ,  152 ,  154  between the various components described above may include one or more wired interfaces or ports for providing wires or optical communications, or one or more wireless interfaces or ports, such as via RF, microwave, and IR for providing wireless communications. In some implementations, all communications may be wired, while in other implementations all communications may be wireless. In still further implementations, the choice of wired and wireless communications may be different from that illustrated in  FIGS.  10   a - 10   d   . Thus, the particular choice of wired or wireless communications should not be considered limiting. 
     In the illustrated embodiment, the patient orientation module  130  is contained in the local processing and data module  144 , while CPU  132  and GPU  134  are contained in the remote processing module  148 , although in alternative embodiments, the CPU  132 , GPU  124 , or portions thereof may be contained in the local processing and data module  144 . The 3D database  142  can be associated with the remote data repository  150 . 
     Referring now to  FIGS.  11  and  12   , the display screen  110  comprises a primary waveguide apparatus  200 . The primary waveguide apparatus  200  includes one or more primary planar waveguides  202  (only one shown in  FIGS.  11  and  12   ), and one or more diffractive optical elements (DOEs)  204  (only one shown in  FIGS.  11  and  12   ) associated with each of at least some of the primary waveguides  202 . As best illustrated in  FIG.  12   , each primary waveguide  202  has a first end  206   a  and a second end  206   b , the second end  206   b  opposed to the first end  206   a  along a length  208  of the primary waveguide(s)  202 . Each of the primary waveguide(s)  202  has a first face  210   a  and a second face  210   b , at least the first and the second faces  210   a ,  210   b  (collectively  210 ) forming an at least partially internally reflective optical path (illustrated by arrow  212   a  and broken line arrow  212   b , collectively  212 ) along at least a portion of the length  208  of the primary waveguide(s)  202 . The primary waveguide(s)  202  may take a variety of forms that provide for substantially total internal reflection (TIR) for light striking the faces  210  at less than a defined critical angle. Each of the primary waveguide(s)  202  may, for example, take the form of a pane or plane of glass, fused silica, acrylic, or polycarbonate. 
     The DOEs  204  (illustrated in  FIGS.  11  and  12    by dash-dot double lines) may take a large variety of forms which interrupt the TIR optical path  212 , providing a plurality of optical paths (illustrated by arrows  214   a  and broken line arrows  214   b , collectively  214 ) between an interior  216  and an exterior  218  of the primary waveguide  202  extending along at least a portion of the length  206  of the primary waveguide  202 . The DOEs  204  may allow positioning of apparent objects and focus plane for apparent objects. Such may be achieved on a frame-by-frame, subframe-by-subframe, or even pixel-by-pixel basis. 
     As illustrated in  FIG.  12   , the light propagates along the primary waveguide(s)  202  with at least some reflections or “bounces” resulting from the TIR propagation. It is noted that some implementations may employ one or more reflectors in the internal optical path, for instance thin-films, dielectric coatings, metalized coatings, etc., which may facilitate reflection. Light propagates along the length  208  of the primary waveguide(s)  202 , and intersects with one or more DOEs  204  at various positions along the length  208 . The DOE(s)  204  may be incorporated within the primary waveguides  202  or abutting or adjacent one or more of the faces  210  of the primary waveguide(s)  202 . The DOE(s)  204  accomplishes at least two functions. The DOE(s)  204  shift an angle of the light, causing a portion of the light to escape TIR, and emerge from the interior  216  to the exterior  218  via one or more faces  210  of the primary waveguide(s)  202 . The DOE(s)  204  focus the out-coupled light at one or more viewing distances. Thus, someone looking through a face  210   a  of the primary waveguides  202  can see digital imagery at one or more viewing distances. 
     Referring to  FIGS.  13   a - 13   c   , the display screen  110  comprises a distribution waveguide apparatus  222  to relay light along a first axis (vertical or Y-axis in  FIG.  13   a   ), and expand the light&#39;s effective exit pupil along the first axis (e.g., Y-axis). The distribution waveguide apparatus  222 , may, for example include one or more distribution planar waveguides  224  (only one shown) and a DOE  226  (illustrated by double dash-dot line) associated with each of the distribution planar waveguides  224 . The distribution planar waveguide  224  may be similar or identical in at least some respects to the primary waveguide  202 , having a different orientation therefrom. Likewise, the DOE  226  may be similar or identical in at least some respects to the DOE  204 . For example, the distribution planar waveguide  220  and/or DOE  226  may be comprised of the same materials as the primary waveguide  202  and/or DOE  204 , respectively. 
     The relayed and exit-pupil expanded light is optically coupled from the distribution waveguide apparatus  222  into the primary waveguide  202 . The primary waveguide  202  relays light along a second axis, preferably orthogonal to first axis, (e.g., horizontal or X-axis  FIG.  13   a   ). Notably, the second axis can be a non-orthogonal axis to the first axis. The primary waveguide  202  expands the light&#39;s effective exit pupil along that second axis (e.g. X-axis). In particular, the distribution planar waveguide  224  can relay and expand light along the vertical or Y-axis, and pass that light to the primary waveguide  202 , which relays and expands light along the horizontal or X-axis. 
     The display screen  110  may generate an image at a single focus plane that is capable of being positioned closer than optical infinity. Collimated light propagates vertically, as shown in  FIG.  13   b    along the distribution planar waveguide  224  by total internal reflection, and in doing so repeatedly intersects with the DOE  226 . The DOE  226  preferably has a low diffraction efficiency (e.g., less than 50%). This causes a fraction (e.g., 10%) of the light to be diffracted toward an edge of the larger primary planar waveguide  202  at each point of intersection with the DOE  226 , and a fraction of the light to continue on its original trajectory down the length of the distribution planar waveguide  224  via TIR. At each point of intersection with the DOE  226 , additional light is diffracted toward the entrance of the primary waveguide  202 . By dividing the incoming light into multiple outcoupled sets, the exit pupil of the light is expanded vertically by the DOE  226  in the distribution planar waveguide  224 . This vertically expanded light coupled out of distribution planar waveguide  224  enters the edge of the primary waveguide  202 . 
     Light entering the primary waveguide  202  propagates horizontally (as shown in  FIG.  13   b   ) along the primary waveguide  202  via TIR. The light intersects with the DOE  204  at multiple points as it propagates horizontally along at least a portion of the length of the primary waveguide  202  via TIR. The DOE  204  may advantageously be designed or configured to have a phase profile that is a summation of a linear diffraction grating and a radially symmetric diffractive lens. The DOE  204  may advantageously have a low diffraction efficiency. At each point of intersection between the propagating light and the DOE  204 , a fraction of the light is diffracted toward the adjacent face of the primary waveguide  202  allowing the light to escape the TIR, and emerge from the face of the primary waveguide  202 . The radially symmetric lens aspect of the DOE  204  additionally imparts a focus level to the diffracted light, both shaping the light wavefront (e.g., imparting a curvature) of the individual beam as well as steering the beam at an angle that matches the designed focus level. As illustrated in  FIG.  13   b   , four beams  228   a - 228   d  extend geometrically to a focal point  228 , and each beam is advantageously imparted with a convex wavefront profile with a center of radius at the focal point  228  to produce an image or virtual object  230   a  at a given focal plane. 
     With reference to  FIG.  13   c   , the display screen  110  may generate a multi-focal volumetric display, image or light field. A first set of four beams  228   a - 228   d  extends geometrically to a focal point  230   a , and each beam  228   a - 228   d  is advantageously imparted with a convex wavefront profile with a center of radius at the focal point  230   a  to produce another portion of the image or virtual object  232   a  at a respective focal plane. A second set of four beams  228   e - 228   h  extends geometrically to a focal point  230   b , and each beam  228   e - 228   h  is advantageously imparted with a convex wavefront profile with a center of radius at focal point  230   b  to produce another portion of the image or virtual object  232   b  at a respective focal plane. 
     In the embodiments of the display subsystem  104  illustrated in  FIGS.  11 - 13   , a single projection subsystem  108  is used to provide image data to the display screen  110 . In contrast to the display system illustrated in  FIGS.  11 - 13   , the display subsystem  104  may comprise a plurality of projection subsystems  108   a - 108   e  (only five shown, collectively  108 ) to provide respective image data to the display screen  110 , as illustrated in  FIG.  14   . The projection subsystems  108  are generally arrayed or arranged along are disposed along an edge  234  of the display screen  110 . There may, for example, be a one to one (1:1) ratio or correlation between the number of planar waveguides  202  and the number of projection subsystems  108 . 
     The display subsystem  104  can enable the use of a single primary planar waveguide  202 . The multiple projection subsystems  108  can be disposed, for example, in a linear array along the edge  234  of a primary planar waveguide  202  that is closest to a temple of the end user&#39;s head. Each projection subsystem  108  injects modulated light encoding sub-image data into the primary planar waveguide  202  from a different respective position, thus generating different pathways of light. These different pathways can cause the light to be coupled out of the primary planar waveguide  202  by a multiplicity of DOEs at different angles, focus levels, and/or yielding different fill patterns at the exit pupil. Different fill patterns at the exit pupil can be beneficially used to create a light field display. Each layer in the stack or in a set of layers (e.g., 3 layers) in the stack may be employed to generate a respective color (e.g., red, blue, green). Thus, for example, a first set of three adjacent layers may be employed to respectively produce red, blue and green light at a first focal depth. A second set of three adjacent layers may be employed to respectively produce red, blue and green light at a second focal depth. Multiple sets may be employed to generate a full 3D or 4D color image field with various focal depths. 
     Referring now to  FIG.  15   , each planar waveguide  202  may include a plurality of DOEs  204   a - 204   d  (four illustrated, each as a double dash-dot line, collectively  204 ). The DOEs  204  are stacked, arrayed, or arranged along an axis  236  that is generally parallel to the field-of-view of the display screen  110 . While illustrated as all being in the interior, in some implementations one, more or even all of the DOEs  204  may be on an exterior of the primary waveguide  202 . 
     In some implementations, each DOE  204  may be capable of being independently switched ON and OFF. That is, each DOE  204  can be made active, such that the respective DOE  204  diffracts a significant fraction of light that intersects with the respective DOE  204 , or it can be rendered inactive such that the respective DOE  204  either does not diffract light intersecting with the respective DOE  204  at all, or only diffracts an insignificant fraction of light. “Significant” in this context means enough light to be perceived by the human visual system when coupled out of the primary waveguide  202 , and “insignificant” means not enough light to be perceived by the human visual system, or a low enough level to be ignored by a viewer. 
     The switchable DOEs  204  may be switched on one at a time, such that only one DOE  204  in the primary planar waveguide  202  is actively diffracting the light in the primary planar waveguide  202 , to emerge from one or more faces  210  of the primary planar waveguide  202  in a perceptible amount. Alternatively, two or more DOEs  204  may be switched ON simultaneously, such that their diffractive effects are combined. 
     Each DOE  204  in the set of DOEs can have a different phase map. For example, each DOE  204  can have a respective phase map such that each DOE  204 , when switched ON, directs light to a different position in X, Y, or Z. The DOEs  204  may, for example, vary from one another in their linear grating aspect and/or their radially symmetric diffractive lens aspect. If the DOEs  204  vary from one another in their diffractive lens aspect, different DOEs  204  (or combinations of DOEs  204 ) will produce sub-images at different optical viewing distances—i.e., different focus distances. If the DOEs  204  vary from one another in their linear grating aspect, different DOEs  204  will produce sub-images that are shifted laterally relative to one another. Such lateral shifts can be beneficially used to create a foveated display, to steer a display image with non-homogenous resolution or other non-homogenous display parameters (e.g., luminance, peak wavelength, polarization, etc.) to different lateral positions, to increase the size of the scanned image, to produce a variation in the characteristics of the exit pupil, and/or to generate a light field display. Lateral shifts may be advantageously employed to preform tiling or realize a tiling effect in generated images. 
     For example, a first DOE  204  in the set, when switched ON, may produce an image at an optical viewing distance of 1 meter (e.g., focal point  230   b  in  FIG.  13   c   ) for a viewer looking into the primary or emission face  210   a  of the primary waveguide  202 . A second DOE  204  in the set, when switched ON, may produce an image at an optical viewing distance of 1.25 meters (e.g., focal point  230   a  in  FIG.  13   b   ) for a viewer looking into the primary or emission face  210   a  of the primary waveguide  202 . By switching exemplary DOEs  204  ON and OFF in rapid temporal sequence (e.g., on a frame-by-frame basis, a sub-frame basis, a line-by-line basis, a sub-line basis, pixel-by-pixel basis, or sub-pixel-by-sub-pixel basis) and synchronously modulating the image data being injected into the primary waveguide  202  by the projection subsystem  108 , a composite multi-focal volumetric image is formed that is perceived to be a single scene to the viewer. By rendering different objects or portions of objects to sub-images relayed to the eye of the viewer (at location  232   b  in  FIG.  13   c   ) by the different DOEs  204 , virtual objects or images are placed at different optical viewing distances, or a virtual object or image can be represented as a 3D volume that extends through multiple planes of focus. 
     Referring now to  FIG.  16   , the display screen  110  may comprise a plurality of planar waveguides  202   a - 202   d  (four shown, collectively  202 ). The primary waveguides  202   a - 200   d  are stacked, arrayed, or arranged along an axis  236  that is generally parallel to the field-of-view of the display screen  110 . Each of the primary waveguides  202  includes at least one DOE  204  (illustrated by dash-dot double line, only one called out in  FIG.  16   ). While illustrated as all being in the interior, in some implementations one, more or even all of the DOEs  204  may be on an exterior of the primary waveguides  202 . Additionally or alternatively, while illustrated with a single linear array of DOEs  204  per planar waveguide  202 , one or more of the primary waveguides  202  may include two or more stacked, arrayed or arranged DOEs  204 , similar to the implementation described with respect to  FIG.  15   . 
     Each of the primary waveguides  202  may function analogously to the operation of the DOEs  204  in the embodiment of  FIG.  15   . That is, the DOEs  204  of the respective planar waveguides  202  may each have a respective phase map, the phase maps of the various DOEs  204  being different from one another. While dynamic switching (e.g., ON/OFF) of the DOEs  204  was employed in the embodiment of  FIG.  15   , such can be avoided in the embodiment of  FIG.  16   . Instead of, or in additional to dynamic switching, the display system  110  may selectively route light to the primary waveguides  202  based on the respective phase maps. Thus, rather than turning ON a specific DOE  204  having a desired phase map, the display system  110  may route light to a specific planar waveguide  202  that has or is associated with a DOE  204  with the desired phase mapping. Again, this may be in lieu of, or in addition to, dynamic switching of the DOEs  204 . 
     In one example, the projection subsystems may be selectively operated to selectively route light to the primary waveguides  202  based on the respective phase maps. In another example, each DOE  204  may be capable of being independently switched ON and OFF, similar to as explained with reference to switching DOEs  204  ON and OFF in the embodiment of  FIG.  15   . The DOEs  204  may be switched ON and OFF to selectively route light to the primary waveguides  202  based on the respective phase maps. 
     As illustrated in  FIG.  16   , light rays outwardly emanate from two of the primary waveguides  202   a ,  202   d . For sake of illustration, a first planar waveguide  202   a  produces a plane or flat wavefront (illustrated by flat lines  238  about rays  240 , only one instance of each called out for sake of drawing clarity) at an infinite focal distance. In contrast, another one of the primary waveguides  202   d  produces a convex wavefront (illustrated by arc  242  about rays  244 , only one instance of each called out for sake of drawing clarity) at a defined focal distance less than infinite (e.g., 1 meter). As illustrated in  FIG.  17   , the primary waveguides  202  may laterally shift the appearance and/or optical viewing distances—i.e., different focus distances of a virtual object  246   a - 246   c  with respect to an exit pupil  248 . 
     Referring back to  FIGS.  11 - 13   , the projection subsystem  108  includes one or more light sources  250  that produces the light (e.g., emits light of different colors in defined patterns), a scanning device  252  that scans the light in a predetermined scan pattern (e.g., such as those described above with respect to  FIGS.  5 - 9   ) in response to control signals, and an optical coupling subsystem  254  that couples the light from the scanning device  252  into the display screen  110 . 
     The light source(s)  250  may take any of a large variety of forms, for instance, a set of RGB lasers (e.g., laser diodes capable of outputting red, green, and blue light) operable to respectively produce red, green, and blue coherent collimated light according to defined pixel patterns specified in respective frames of pixel information or data. Laser light provides high color saturation and is highly energy efficient. 
     The scanning device  252  comprises one or more optical fibers  256  (e.g., single mode optical fiber), each of which has a proximal end  256   a  into which light is received from the light source(s)  250  and a distal end  256   b  from which light is provided to the partially transparent display screen  110 . The scanning device  252  further comprises a mechanical drive assembly  258  to which the optical fiber  256  is mounted. The drive assembly  258  is configured for displacing the distal end  256   b  of the optical fiber  256  about a fulcrum  260  in accordance with a scan pattern, such that the outputted light diverges from a longitudinal axis  262  coincident with the fulcrum  260 . It should be appreciated that although the display subsystem  104  has been described as being implemented with a scanning fiber technology, it should be appreciated that the display subsystem  104  may be based on any display technology, e.g., liquid crystal displays (LCD), digital light processing (DLP) displays, etc. 
     The drive assembly  208  comprises a piezoelectric element  264  to which the optical fiber  256  is mounted, and drive electronics  266  configured for conveying electrical signals to the piezoelectric element  264 , thereby causing the distal end  256   b  of the optical fiber  256  to vibrate in accordance with the scan pattern. Thus, operation of the light source(s)  250  and drive electronics  266  are coordinated in a manner that generates image data that is encoded in the form of light that is spatially and/or temporally varying. 
     In the illustrated embodiment, the piezoelectric element  264  takes the form of a hollow tube, in which case, the distal end  256   b  of the optical fiber  256  is threaded or received through the piezoelectric tube  264 . The distal end  256   b  of the optical fiber  256  protrudes from the piezoelectric tube  264  as a fixed-free flexible cantilever  268  (shown in  FIGS.  18   a  and  18   b   ). The piezoelectric tube  264  is associated with four quadrant electrodes (not illustrated). The electrodes may, for example, be plated on the outside, outer surface or outer periphery or diameter of the piezoelectric tube  264 . A core electrode (not illustrated) is also located in a core, center, inner periphery or inner diameter of the tube  264 . 
     The drive electronics  266  are electrical coupled via wires  270  to drive opposing pairs of electrodes (not shown) to bend the piezoelectric tube  264  in two axes independently. The protruding distal tip of the optical fiber  256  has mechanical modes of resonance. The frequencies of resonance depend upon a diameter, length, and material properties of the optical fiber  256 . By vibrating the piezoelectric tube  264  near a first mode of mechanical resonance of the fiber cantilever  268 , the fiber cantilever  268  is caused to vibrate, and can sweep through large deflections about the fulcrum  260 . By stimulating resonant vibration in two axes, the tip of the fiber cantilever  268  is scanned biaxially in an area filling 2D scan. By modulating an intensity of the light source(s)  250  in synchrony with the scan of the fiber cantilever  268 , light emerging from the fiber cantilever  268  forms an image. Descriptions of such a set up are provided in U.S. patent application Ser. No. 13/915,530, International Patent Application Ser. No. PCT/US2013/045267, and U.S. Provisional Patent Application Ser. No. 61/658,355, all of which are incorporated by reference herein in their entireties. 
     As briefly discussed above, the optical coupling subsystem  254  optically couples light from the scanning device  252  to the waveguide apparatus  102 . The optical coupling subsystem  254  includes an optical waveguide input apparatus  272 , for instance, one or more reflective surfaces, diffraction gratings, mirrors, dichroic mirrors, or prisms to optically couple light into the end of the waveguide apparatus  102 . The optical coupling subsystem  254  additionally or alternatively includes a collimation element  274  that collimates light from the optical fiber  256 . 
     As briefly discussed above, the light emitted from the scanning device  252  initially diverges from the longitudinal axis  262  as the distal end  256   b  of the optical fiber  256  is vibrated about the fulcrum  260 . At each position of the optical fiber  256 , the light initially fans out from the distal end  256   b , and is collimated to a narrow light ray by the collimation element  274 . Without modification, a relatively large optical waveguide input apparatus  272  will be needed to accommodate the relatively large deflections in the distal end  256   b  of the optical fiber  256 . 
     To this end, the optical coupling subsystem  254  comprises an optical modulation apparatus  276  configured for converging the light from the collimation element  274  towards the longitudinal axis  262 , and in the illustrated embodiment, converging the light on a focal point  278  in the center of the optical waveguide input apparatus  272 , as shown in  FIG.  18   a   . Notably, focusing the light at the center of the optical waveguide input apparatus  272  allows the size of the optical waveguide input apparatus  272  to be minimized. That is, focusing the light at the center of the optical waveguide input apparatus  272  minimizes the worst-case divergent span of the swept light path at the edges of the optical waveguide input apparatus  272 . For example, if the light is focused on the front edge of the optical waveguide input apparatus  272 , as illustrated in  FIG.  18   b   , the optical waveguide input apparatus  272  must be made larger to accommodate the larger divergent span of the swept light path at the rear edge of the optical waveguide input apparatus  272 . 
     Referring now to  FIG.  19   , the interaction between the input optical modulation apparatus  276  and one embodiment of an optical waveguide input apparatus  272  will be described. In this case, the optical waveguide input apparatus  272  takes the form of a distribution waveguide apparatus  222  with associated distribution waveguide  224  and DOEs  226   a - 226   e  described with respect to  FIGS.  13   a - 13   c   . As shown in  FIG.  19   , the optical modulation apparatus  276  converges the light from the collimation element  274  on a focal point  278  located at the DOE  226   c , thereby minimizing the size of the DOEs  226 . As a result, the overall width of the distribution waveguide  224  is minimized. The light is then selectively conveyed down one or more of the primary waveguides  202   a - 202   e.    
     Referring now to  FIG.  20   , the DOEs  226  can alternatively be incorporated directly into the primary waveguide apparatus  200 . In this case, the DOEs  226  respectively extend parallel along the primary planar waveguides  202   a - 202   e  of the waveguide apparatus  200 , such that the DOEs  226  respectively direct the light from the optical modulation apparatus  276  down the primary waveguides  202 . As shown in  FIG.  20   , the optical modulation apparatus  276  converges the light from the collimation element  274  on a focal point  278  located the DOE  226   c , thereby minimizing the size of the DOEs  226 . Since the focal point  278  is at the center of the input optical modulation apparatus  276 , the lengths of the DOEs  226   a  and  226   e  can be made equal in order to minimize the worst-case DOE  226 . As a result, the overall size of the waveguide apparatus  200  is minimized. 
     In one embodiment, the optical modulation apparatus  276  comprises one or more diffraction gratings, each of which can be characterized as an optical component with a periodic structure on the order of the light wavelength that splits and diffracts light into several beams travelling in different directions. The diffraction gratings can either be transmissive or reflective, and be composed of, e.g., surface nano-ridges, nano-patterns, slits, etc. that may be photolithographically printed on a substrate. In an alternative embodiment, the optical modulation apparatus  276  may comprise one or more lenses. 
     In the illustrated embodiment, the optical modulation apparatus  276  has a diffraction pattern that matches the geometry of the scan pattern, such that the collimation of the light is preserved at a target resolution. For example, if a spiral scan pattern is used, the diffraction pattern may have a pattern of diffraction elements  280 , as illustrated in  FIG.  21   . If a single diffraction grating is used, each diffraction element may diffract light rays  282   a  and  282   b  inwardly in two dimensions (e.g., in the case of  FIG.  22   a   , in the x- and y-directions, such that each diffraction element  280   a  and  280   b  (only two shown) diffracts the light towards a single focal point  278  at the origin of the x-y coordinate system). Alternatively, two orthogonal diffraction gratings  276   a  and  276   b  in series can be used, such that the diffraction elements  278   a  and  278   b  of one diffraction grating  276   a  diffracts the respective light rays  280   a  and  280   b  along one axis of the x-y coordinate system (e.g., in the case of  FIG.  22   b   , the x-direction), and the diffraction elements of the other diffraction grating  276   b  diffracts the light rays  280   a  and  280   b  along the other axis of the x-y coordinate system (e.g., in the case of  FIG.  22   b   , the y-direction). 
     Although particular embodiments of the present inventions have been shown and described, it will be understood that it is not intended to limit the present inventions to the preferred embodiments, and it will be obvious to those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the present inventions. Thus, the present inventions are intended to cover alternatives, modifications, and equivalents, which may be included within the spirit and scope of the present inventions as defined by the claims.