Control system for navigation in virtual reality environment

In a control system for navigating in a virtual reality environment, a user may select a virtual feature in the virtual environment, and set an anchor point on the selected feature. The user may then move, or adjust position, relative to the feature, and/or move and/or scale the feature in the virtual environment, maintaining the portions of the feature at the set anchor point within the user's field of view of the virtual environment.

FIELD

This application relates, generally, to movement and scaling in an augmented reality and/or a virtual reality environment.

BACKGROUND

An augmented reality (AR) and/or a virtual reality (VR) system may generate a three-dimensional (3D) immersive virtual environment. A user may experience this 3D immersive virtual environment through interaction with various electronic devices, such as, for example, a helmet or other head mounted device including a display, glasses or goggles that a user looks through when viewing a display device, gloves fitted with sensors, external handheld devices that include sensors, and other such electronic devices. Once immersed in the 3D virtual environment, the user may move through the virtual environment and move to other areas of the virtual environment, through physical movement and/or manipulation of an electronic device to interact with the virtual environment and personalize interaction with the virtual environment.

SUMMARY

In one aspect, a method may include generating a virtual environment, detecting a first input at a user interface of a controller in communication with a head mounted display device, setting an anchor point on a selected virtual feature in the virtual environment in response to the first input, detecting a second input, and, in response to the second input, defining an area of the feature surrounding the anchor point, and adjusting at least one of a position or a scale of the virtual feature in the virtual environment, while maintaining a portion of the virtual feature within the defined area within a user field of view of the virtual environment

In another aspect, a system may include a computing device configured to generate an immersive virtual environment, the computing device a memory storing executable instructions, and a processor configured to execute the instructions. The processor may cause the computing device to generate a virtual environment, detect a first input at a user interface of a controller in communication with a head mounted display device, set an anchor point on a selected virtual feature in the virtual environment in response to the first input, detect a second input, and, in response to the second input, define an area of the feature surrounding the anchor point, and adjust at least one of a position or a scale of the virtual feature in the virtual environment, while maintaining a portion of the virtual feature within the defined area within a user field of view of the virtual environment.

DETAILED DESCRIPTION

A user immersed in a 3D virtual environment wearing, for example, a head mounted display (HMD) device may explore the 3D virtual environment and interact with the 3D virtual environment through various different types of inputs. These inputs may include, for example, physical interaction including, for example, manipulation of an electronic device separate from the HMD, manipulation of the HMD itself, and/or through hand/arm gestures, head movement and/or head and/or eye directional gaze and the like. A user may implement one or more of these different types of interactions to execute a particular action in the virtual environment, such as, for example, moving through the virtual environment or moving the virtual environment relative to the user, moving, or transitioning, or teleporting, from a first area of the virtual environment to a second area of the virtual environment, adjusting a perspective through which the virtual environment is experienced, and the like.

A system and method, in accordance with implementations described herein, may facilitate manipulation of features in the virtual environment, and may facilitate movement, or navigation, through the virtual environment, and may allow the user to view and experience the virtual environment from different perspectives and scales. A system and method, in accordance with implementations described herein, may also provide the user with a substantially seamless virtual movement experience in virtual reality, while avoiding the motion sickness and disorientation sometimes associated with a disconnect between the dynamic visual movement experienced in virtual reality and the lack of actual, physical motion corresponding to the dynamic visual movement.

In the example implementation shown inFIG. 1, a user wearing an HMD100is holding a handheld electronic device102. The handheld electronic device102may be, for example, a controller, a gyromouse, a smartphone, a joystick, or another portable controller(s) that may be paired with, and communicate with, the HMD100for interaction in the immersive virtual environment generated by the HMD100. Hereinafter, simply for ease of discussion and illustration, the handheld electronic device102will be referred to as a controller102. The controller102may be operably coupled with, or paired with the HMD100via, for example, a wired connection, or a wireless connection such as, for example, a WiFi or Bluetooth connection. This pairing, or operable coupling, of the controller102and the HMD100may provide for communication between the controller102and the HMD100and the exchange of data between the controller102and the HMD100. This may allow the controller102to function as a controller in communication with the HMD100for interacting in the immersive virtual environment generated by the HMD100. That is, the controller102may be manipulated in a plurality of different manners. Manipulation of the controller102may be translated into a corresponding selection, or movement, or other type of interaction, in the immersive virtual environment generated by the HMD100. This may include, for example, an interaction with, manipulation of, or adjustment of a virtual object, a change in scale or perspective with respect to the virtual environment, a movement (e.g., navigation, teleportation, transport) of the user from a current location in the virtual environment to a selected destination or feature in the virtual environment, and other such interactions. In some implementations, as discussed above, the user may interact with virtual features in the virtual environment through physical gestures, such as hand gestures, detected by a system equipped to detect and track the user's hands, without relying on the use of a separate controller.

FIGS. 2A and 2Bare perspective views of an example HMD, such as, for example, the HMD100worn by the user inFIG. 1, andFIG. 2Cillustrates an example controller, such as, for example, the controller102shown inFIG. 1.

The controller102may include a housing103in which internal components of the device102are received, and a user interface104on an outside of the housing103, accessible to the user. The user interface104may include a plurality of different types of manipulation devices, including, for example, a touch sensitive surface106configured to receive user touch inputs, manipulation devices105including buttons, knobs, joysticks, toggles, slides and other such manipulation devices. In some implementations, the controller102may also include a light source108configured to selectively emit light, for example, a beam or ray, through a port in the housing103, for example, in response to a user input received at the user interface104.

The HMD100may include a housing110coupled to a frame120, with an audio output device130including, for example, speakers mounted in headphones, also be coupled to the frame120. InFIG. 2B, a front portion110aof the housing110is rotated away from a base portion110bof the housing110so that some of the components received in the housing110are visible. A display140may be mounted on an interior facing side of the front portion110aof the housing110. Lenses150may be mounted in the housing110, between the user's eyes and the display140when the front portion110ais in the closed position against the base portion110bof the housing110. In some implementations, the HMD100may include a sensing system160including various sensors such as, for example, audio sensor(s), image/light sensor(s), positional sensors (e.g., inertial measurement unit including gyroscope and accelerometer), and the like. The HMD100may also include a control system170including a processor190and various control system devices to facilitate operation of the HMD100.

In some implementations, the HMD100may include a camera180to capture still and moving images. The images captured by the camera180may be used to help track a physical position of the user and/or the controller102in the real world, and/or may be displayed to the user on the display140in a pass through mode, allowing the user to temporarily leave the virtual environment and return to the physical environment without removing the HMD100or otherwise changing the configuration of the HMD100to move the housing110out of the line of sight of the user.

In some implementations, the HMD100may include a gaze tracking device165to detect and track an eye gaze of the user. The gaze tracking device165may include, for example, an image sensor165A, or multiple image sensors165A, to capture images of the user's eyes, for example, a particular portion of the user's eyes, such as, for example, the pupil, to detect, and track direction and movement of, the user's gaze. In some implementations, the HMD100may be configured so that the detected gaze is processed as a user input to be translated into a corresponding interaction in the immersive virtual experience.

A block diagram of a system providing for manipulation and control of navigation in an augmented and/or virtual reality environment is shown inFIG. 3. The system may include a first electronic device300in communication with a second electronic device302. The first electronic device300may be, for example an HMD as described above with respect toFIGS. 1, 2A and 2B, generating an immersive virtual environment, and the second electronic device302may be, for example, a controller as described above with respect toFIGS. 1 and 2C, that is in communication with the first electronic device300to facilitate user interaction with the immersive virtual environment generated by the first electronic device300.

The first electronic device300may include a sensing system360and a control system370, which may be similar to the sensing system160and the control system170, respectively, shown inFIGS. 2A and 2B. The sensing system360may include one or more different types of sensors, including, for example, a light sensor, an audio sensor, an image sensor, a distance/proximity sensor, a positional sensor (e.g., an inertial measurement unit including a gyroscope and accelerometer) and/or other sensors and/or different combination(s) of sensors, including, for example, an image sensor positioned to detect and track the user's eye gaze, such as the gaze tracking device165shown inFIG. 2B. The control system370may include, for example, a power/pause control device, audio and video control devices, an optical control device, a transition control device, and/or other such devices and/or different combination(s) of devices. The sensing system360and/or the control system370may include more, or fewer, devices, depending on a particular implementation. The elements included in the sensing system360and/or the control system370may have a different physical arrangement (e.g., different physical location) within, for example, an HMD other than the HMD100shown inFIGS. 2A and 2B. The first electronic device300may also include a processor390in communication with the sensing system360and the control system370, a memory380, and a communication module350providing for communication between the first electronic device300and another, external device, such as, for example, the second electronic device302.

The second electronic device302may include a communication module306providing for communication between the second electronic device302and another, external device, such as, for example, the first electronic device300. In addition to providing for the exchange of data between the first electronic device300and the second electronic device302, the communication module306may also be configured to emit a ray or beam as described above. The second electronic device302may include a sensing system304including an image sensor and an audio sensor, such as is included in, for example, a camera and microphone, an inertial measurement unit, a touch sensor such as is included in a touch sensitive surface of a controller, or smartphone, and other such sensors and/or different combination(s) of sensors. A processor309may be in communication with the sensing system304and a control unit305of the second electronic device302, the control unit305having access to a memory308and controlling overall operation of the second electronic device302.

As noted above, a controller, such as, for example, the controller102described above, may be manipulated by a user, sometimes in combination with the functionality of the HMD100described above, for interaction and navigation in the virtual environment. An example implementation of this is shown inFIGS. 4A-4E.

A user, wearing an HMD100that generates a virtual environment to be experienced by the user, may operate a controller102to navigate and manipulate virtual objects and virtual features in the virtual environment400. As shown inFIG. 4A, a fixed, set controller reference point410, which may remain fixed relative to the controller102as the controller102moves, may be associated with the controller102. Hereinafter, simply for ease of discussion and illustration, the set controller reference point410will be illustrated in the drawings as a set of virtual crosshairs, with the set controller reference point410at the intersection of the pair of virtual crosshairs. The set controller reference point410/virtual crosshairs410may remain in a fixed position with respect to the controller102, moving together with the controller102as the user moves the controller102relative to virtual objects and features in the virtual environment400. In the example shown inFIG. 4A, the virtual crosshairs410include a visual representation, or icon, that remains in a fixed position relative to the controller102, and essentially moves together with the controller102. The virtual crosshairs410will be represented in this manner hereinafter, simply for ease of discussion and illustration. However, in some implementations, the virtual crosshairs410may represent a known position in space (known by the controller102and/or the HMD100) relative to the controller102, but not be rendered in the form of a visual representation, or icon, to the user, so that the user's view of and access to the virtual objects and features in the virtual environment400are not obscured by a visual representation of the virtual crosshairs410.

FIG. 4Aillustrates the start of a user's virtual interaction with a virtual feature450, with the virtual feature450shown as it is viewed by the user at the start of the virtual interaction. As shown inFIG. 4A, at the start of a user's interaction with the virtual feature450in the virtual environment400, for example, at a time t=0, the user may define an anchor point420associated with the virtual feature450. InFIG. 4A, The anchor point420may identify, for example, a portion of the virtual feature450which the user wishes to remain within his field of view as the user moves and/or changes position and/or perspective relative to the virtual objects and features in the virtual environment400. At the start of the user's interaction with the virtual feature450, or time t=0, the anchor point420may be defined by a virtual target ray415extending from a set user reference point100A, through the virtual crosshairs410, and intersecting with the selected portion of the virtual feature450as defined by the direction of the virtual target ray415which may be controlled by the user moving the reference point100A and/or the virtual crosshairs410.

The user reference point100A may be defined at a set location on the HMD100, for example, a position at or near the user's eyes, a position corresponding to a location between the user's eyes, or other set position. This set user reference point100A may remain set, or constant, on or relative to the HMD100, as the user, and in particular, the user's head, moves and changes position/orientation in the virtual environment400. Similarly, the location/position/orientation of the virtual crosshairs410relative to the controller102may remain set, or at a constant position on or relative to the controller102, as the user, and in particular, the user's arm/hand holding the device102, change location/position/orientation in the virtual environment. The location and position/orientation of the HMD100and location and position/orientation of the controller102relative to the HMD100may be known and tracked, substantially in real time, by the system, and the constant location of the set user reference point100A and the constant location of the virtual crosshairs410relative to the controller102may also be known and tracked, substantially in real time.

In the example shown inFIG. 4A, the virtual target ray415, extending from the set user reference point100A, through the virtual crosshairs410to the selected anchor point420, is represented by a dotted line. However, in some implementations, the virtual target ray415is not necessarily rendered as a visual representation or object to be viewed by the user, so that the user's view of and access to the virtual objects and features in the virtual environment are not obscured by the visual representation of the virtual target ray420. In some implementations, a visual indicator, for example, a dot, may be rendered at the point on the virtual feature450identified by the virtual target ray415to be the anchor point420. An example of the user's view, or perspective, from the user's virtual position relative to the virtual feature450shown inFIG. 4A, as the user sets the anchor point420, is shown inFIG. 4B. The arrangement shown inFIG. 4Amay represent the start of the user's interaction with the virtual feature450, or time t=0, by, for example, manipulating a device on the user interface104of the controller102, such as, for example, depressing a button or trigger.

Depression of the button or trigger (or actuation of other manipulation device on the user interface104of the controller102) may be used to identify and set the anchor point420, as shown inFIG. 4A. After setting the anchor point420, user movement, for example, head and hand/arm movement, and corresponding movement of the virtual target ray415extending through the virtual crosshairs410to the anchor point420(due to movement of the set user reference point100A on the user's head and movement of the controller102held in the user's hand) may cause the feature450to be translated and scaled in the virtual environment400. For example, once the anchor point420is set as described above, a downward movement of the virtual target ray415as shown inFIG. 4C(from the position shown inFIG. 4A, while the button or trigger remains depressed) may cause a downward translation of the anchor point420, resulting in a change in scale of the virtual environment400(as the user's feet remain on the virtual ground, and the virtual feature450remains on the virtual ground), and in particular, a change in scale of the virtual feature450in the virtual environment400anchored by the anchor point420, and the user's view or perspective relative to the virtual feature450, as shown inFIG. 4D.

Similarly, once the anchor point420is set as described above, an upward movement of the virtual target ray415as shown inFIG. 4E(for example, from the position shown inFIG. 4A, while the button or trigger remains depressed) may cause an upward translation of the anchor point420, resulting in a change in scale of the virtual environment400, and in particular, a change in scale, or perspective of the user relative to the virtual feature450in the virtual environment400anchored by the anchor point420, and the user's view or perspective relative to the feature, as shown inFIG. 4F.

Adjusting scale in a virtual environment may be a change (an increase or a decrease) in the user's size relative to the virtual features in the virtual environment, or a corresponding change in the user's perspective relative to the virtual features in the virtual environment (or, may be considered a change, i.e., an increase or a decrease, in a size of the virtual features in the virtual environment relative to the user). For example, the user may choose to scale up so that, from the user's perspective, the user experiences the virtual environment as though his size has increased relative to the virtual features in the virtual environment (and/or the virtual features appear to have decreased in size/scale). Similarly, the user may choose to scale down, so that, from the user's perspective, the user experiences the virtual environment as though his size has decreased relative to the virtual features in the virtual environment (and/or the virtual features appear to have increased in size/scale). This type of scaling in the virtual environment may be considered a virtual adjustment in size/scaling of the user, in particular the user's perspective relative to the virtual features in the virtual environment, or a virtual adjustment in size/scaling of the virtual features in the virtual environment relative to the user. Hereinafter, simply for ease of discussion, scaling will be considered to include a virtual adjustment of the user's size/scale relative to the virtual features and/or a virtual adjustment of the size/scale of the virtual features in the virtual environment. The movement from the position shown inFIG. 4A(illustrated in dotted lines, as a ghosted image inFIG. 4C) to the position shown inFIG. 4C, and/or from the position shown inFIG. 4A(illustrated in dotted lines, as a ghosted image shown inFIG. 4E) to the position shown inFIG. 4E, illustrate exemplary upward and downward movements, and corresponding translation of the virtual feature450in the virtual environment400and/or scaling of the user relative to the virtual environment400(or scaling of the virtual environment400relative to the user). However, movements in other directions (while the button or trigger remains depressed) may also cause the anchor point420to remain anchored to the identified portion of the selected virtual feature450through the corresponding movement. For example, a movement to the right may generate an effect of the virtual feature450moving around, or orbiting around, the user. Similarly, a movement of the user's arm closer to the user may generate an effect of zooming in closer to the virtual feature450.

In response to certain movements, the system may rely on certain fixed parameters associated with the set user reference point, the position of the virtual crosshairs410relative to the controller102and a set virtual reference point400A, the set virtual reference point400A remaining substantially stationary and unchanging throughout the various movements and/or translation of features and/or scaling as described above. Hereinafter, the set virtual reference point400A will be a reference plane corresponding to the virtual ground, or virtual floor, in the virtual environment400. That is, as illustrated in the example shown inFIG. 5, in response to an example downward movement (from the arm/hand/controller position shown in dotted lines to the arm/hand/controller position shown in solid lines), the system may rely on certain fixed parameters to determine whether it is the user's intention to scale the feature450to the scaled feature450A, or to the scaled feature450B, or to the scaled feature450C, which all fall along the virtual target ray415. This will be explained in more detail with reference toFIGS. 6A-6F.

InFIG. 6A, at the start of an interaction with the virtual feature450in the virtual environment400(for example, at time t=0), the user may be positioned facing the virtual feature450on the virtual reference plane400A, which, hereinafter, will be considered to be the virtual ground400A. In the example shown inFIG. 6A, at time t=0, the user holds the controller102with his arm extended, so that the virtual target ray415extends from the user set reference point100A through the virtual crosshairs410to the portion of the selected feature450to set the anchor point420. In some implementations, a size, or dimension(s) (i.e., height and width) of the virtual crosshairs410may be constant, and may be set, for example, for a particular application, or by the user, or by the system, and the like. The set dimensions of the virtual crosshairs410may define a first disk d10at time t=0, the first disk d10having a constant radius r1defined by the set dimensions of the virtual crosshairs410. An example of what may be viewed by the user shown inFIG. 4A, at the start of the user's interaction with the virtual feature450, or time t=0, by, for example, manipulating a device on the user interface104of the controller102, such as, for example, depressing a button or trigger, is illustrated inFIG. 6B.

The first disk d10may define a cone440, the cone440having an origin at the set user reference point100A, and extending tangentially to the first disk d10towards a plane surrounding the anchor point420. At time t=0, an angle α0may be defined as the half-angle of the cone440defined by the set user reference point100A and the first disk d10. A second disk d20having a radius r2may be defined as the cross section through the cone440at a distance corresponding to the anchor point420, i.e., at a virtual plane, substantially perpendicular to the virtual target ray415, corresponding to/including the anchor point420.

Once set as shown inFIG. 6A, the virtual radius r2, relative to the virtual environment400, may remain fixed, or constant with respect to the virtual environment400. That is, once set, the portion of the virtual feature450captured within the confines of the second disk d20having the constant virtual radius r2relative to the virtual environment400, as shown inFIG. 6B, may remain the same, while the virtual feature450, including the portion of the virtual feature450captured within the second disk d20, is translated and/or scaled in response to user movement of the anchor point420. This will be described in more detail with respect toFIGS. 6C-6F.

FIG. 6Cillustrates a subsequent point in time, t>0, illustrating a downward movement of the virtual target ray415, corresponding to a downward movement of the anchor point420and a scaling down of the virtual feature450. An example of what may be viewed by the user shown inFIG. 4C, after movement from the position shown inFIG. 6Ato the position shown inFIG. 6C(to be described in more detail below), is illustrated inFIG. 6D.FIG. 6Eillustrates a subsequent point in time, t>0, illustrating an inward movement of the first disk d1tcloser to the set user reference point100A, corresponding to a closing in movement of the anchor point420, and decreasing a perceived virtual distance between the user and the virtual feature450. An example of what may be viewed by the user shown inFIG. 4E, after movement from the position shown inFIG. 6Ato the position shown inFIG. 6E(to be described in more detail below), is illustrated inFIG. 6F.

In each subsequent point in time t>0 (i.e., t1, t2, . . . tN), after setting the anchor point420as described above with respect toFIG. 6A, the angle at may be calculated as the half angle of the cone440defined by the set user reference point100A and the first disk d1t. At each subsequent point in time t>0, the radius r1of the first disk d1tremains constant/fixed (as the dimensions of the virtual crosshairs410remain constant) relative to the user. At each subsequent point in time t>0, the virtual radius r2of the second disk d2tremains constant/fixed relative to the virtual environment400, as the anchor point420moves and remains on the virtual target ray415. The virtual reference plane400A, or virtual ground400A, also remains fixed, or constant, and aligned with the real world ground. However, a distance between the set user reference point100A and the first disk d1tmay change as the user's arm/hand holding the controller102moves. This may cause the half angle at to also change at subsequent points in time t>0 (i.e., t1, t2, . . . tN) in response to the user's movement of the anchor point420as described above. This recalculation of the half angle at as the anchor point420is moved in this manner, while the radius r1of the first disk d1tremains constant relative to the user and the virtual radius r2of the second disk d2tremains constant relative to the virtual environment400, the anchor point420remains on the virtual target ray415, and the virtual reference plane400A remains aligned with the real world ground, allow the virtual environment to be translated and/or scaled as shown.

In the example shown inFIGS. 6A-6E, the first disk d10/dt0, representing an area surrounding the set controller reference point410, is defined as a disk based on the virtual crosshairs having an intersection point at the set controller reference point410. This example geometry, in which the area surrounding the set controller reference point410has a circular disk shape, and the area surrounding the anchor point420, and the area surrounding the anchor point420has a circular disk shape, defined by the cone, may facilitate the calculation and tracking/adjustment of the half angle α as the user and/or the virtual features are scaled in response to movement of the controller, and may generate reduced processing workload. However, the circular shaped disk is shown simply for ease of discussion and illustration, and the area surrounding the set controller reference point410may have other closed curve shapes. Similarly, in the example shown inFIGS. 6A-6E, the cone440is defined by geometry originating at the set user reference point100A and extending tangentially beyond the first disk d10. However, this geometry may also vary, based on the geometry of the area defined surrounding the set user reference point410. Similarly, in the example shown inFIGS. 6A-6E, the area surrounding the anchor point420is defined as a circular shaped disk. However, the shape of this area surrounding the anchor point420will also be defined by the closed curve shape of the area surrounding the set controller reference point and the extension of that geometry to an intersection with a plane of the virtual feature450.

As shown inFIG. 7A, in some implementations, one or more virtual photo spheres570(for example, first, second and third virtual photo spheres570A,570B,570C) may be available to the user for selection in the virtual environment. Each photo sphere may provide a 360-degree panoramic experience of, for example, a particular feature, location and the like, in the virtual environment. To move to one of the virtual photospheres570, the user may, for example, direct the virtual beam500generated by the handheld electronic device102as described above toward a selected one of the virtual photo spheres570, such as, for example, the third virtual photo sphere570C, to move into a 360-degree panoramic experience of an interior of the virtual feature550, as shown inFIG. 7A. Upon further manipulation of the handheld electronic device102, for example, release of a button directing the virtual beam550to the selected virtual photo sphere570C, the user may be moved, or teleported, or transported, to the selected virtual photo sphere570C, as shown inFIG. 7B.

When moving, or transporting, or teleporting to the selected virtual photo sphere570, the user may also choose to adjust in scale relative to the features in the virtual environment as discussed in detail above. In particular, when moving to the selected virtual photo sphere570, the user may choose to increase is size/scale, or decrease in size/scale, relative to the virtual elements included in the 360-degree panoramic experience provided by the selected virtual photo sphere570in the manner described above in detail.

Once within the virtual photo sphere570, the user may move within the virtual photo sphere570. For example, the user may turn to view a different portion of the 360-degree panoramic experience provided by the virtual photo sphere570, and/or the user may walk from a virtual position C to a virtual position D within the virtual photo sphere570, as shown inFIG. 8B, to approach, or get closer to, a virtual element included in the 360-degree panoramic experience provided by the virtual photo sphere570. In some implementations, the user may, for example, walk within the virtual photo sphere570to what may be considered an edge of the 360-panoramic virtual display within the virtual photo sphere570, with the virtual elements displayed in the 360-degree panoramic experience of the virtual photo sphere570displayed in front of the user appearing larger as the user walks in the direction of, or approaches, the virtual elements. Similarly, if the user were to turn around, for example, turn 180 degrees after arriving at the virtual position D, the virtual elements that were once behind the user may appear smaller, as the user has walked away from the virtual elements displayed in that portion of the virtual photo sphere570.

A method700of navigating and/or scaling in an augmented and/or a virtual reality environment, in accordance with implementations described herein, is shown inFIG. 8. Upon detection of a selection of a virtual feature (block710), it may be determined whether or not a request to set a virtual anchor point has been received (block720). The request to set a virtual anchor point may be, for example, input by a user as described above in detail with respect toFIGS. 4A and 6A. If a request to set a virtual anchor point has been received, the virtual anchor point may be set (block730). If the virtual anchor point is set and a request to adjust a user's position, or view, or perspective, or scale, relative to the virtual environment (or a request to scale the virtual environment relative to the user) is received (block750), the system may adjust the position and/or view and/or perspective and/or scale in accordance with the received request (block760), as described above in detail with respect toFIGS. 4B-4F and 6B-6F. If a virtual anchor point has not been set but a request to move or otherwise adjust the user's position relative to the selected virtual feature is received (block740), the system may carry out the requested adjustment in accordance with the received request (block760). Upon completion of the requested virtual adjustment (block770), the process may be repeated until the virtual experience is terminated (block780).

FIG. 9shows an example of a generic computer device800and a generic mobile computer device850, which may be used with the techniques described here. Computing device800is intended to represent various forms of digital computers, such as laptops, desktops, tablets, workstations, personal digital assistants, televisions, servers, blade servers, mainframes, and other appropriate computing devices. Computing device850is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed in this document.

Computing device800includes a processor802, memory804, a storage device806, a high-speed interface808connecting to memory804and high-speed expansion ports810, and a low speed interface812connecting to low speed bus814and storage device806. The processor802can be a semiconductor-based processor. The memory804can be a semiconductor-based memory. Each of the components802,804,806,808,810, and812, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor802can process instructions for execution within the computing device800, including instructions stored in the memory804or on the storage device806to display graphical information for a GUI on an external input/output device, such as display816coupled to high speed interface808. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices800may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).

The memory804stores information within the computing device800. In one implementation, the memory804is a volatile memory unit or units. In another implementation, the memory804is a non-volatile memory unit or units. The memory804may also be another form of computer-readable medium, such as a magnetic or optical disk.

The high speed controller808manages bandwidth-intensive operations for the computing device800, while the low speed controller812manages lower bandwidth-intensive operations. Such allocation of functions is exemplary only. In one implementation, the high-speed controller808is coupled to memory804, display816(e.g., through a graphics processor or accelerator), and to high-speed expansion ports810, which may accept various expansion cards (not shown). In the implementation, low-speed controller812is coupled to storage device806and low-speed expansion port814. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.

The computing device800may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server820, or multiple times in a group of such servers. It may also be implemented as part of a rack server system824. In addition, it may be implemented in a personal computer such as a laptop computer822. Alternatively, components from computing device800may be combined with other components in a mobile device (not shown), such as device850. Each of such devices may contain one or more of computing device800,850, and an entire system may be made up of multiple computing devices800,850communicating with each other.

Computing device850includes a processor852, memory864, an input/output device such as a display854, a communication interface866, and a transceiver868, among other components. The device850may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of the components850,852,864,854,866, and868, are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.

The processor852can execute instructions within the computing device850, including instructions stored in the memory864. The processor may be implemented as a chipset of chips that include separate and multiple analog and digital processors. The processor may provide, for example, for coordination of the other components of the device850, such as control of user interfaces, applications run by device850, and wireless communication by device850.

Processor852may communicate with a user through control interface858and display interface856coupled to a display854. The display854may be, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. The display interface856may comprise appropriate circuitry for driving the display854to present graphical and other information to a user. The control interface858may receive commands from a user and convert them for submission to the processor852. In addition, an external interface862may be provide in communication with processor852, so as to enable near area communication of device850with other devices. External interface862may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.

Device850may also communicate audibly using audio codec860, which may receive spoken information from a user and convert it to usable digital information. Audio codec860may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of device850. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on device850.

The computing device850may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a cellular telephone880. It may also be implemented as part of a smart phone882, personal digital assistant, or other similar mobile device.

Implementations of the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. Implementations may implemented as a computer program product, i.e., a computer program tangibly embodied in an information carrier, e.g., in a machine-readable storage device (computer-readable medium), for processing by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers. Thus, a computer-readable storage medium can be configured to store instructions that when executed cause a processor (e.g., a processor at a host device, a processor at a client device) to perform a process.