Methods and apparatus for interacting with a distant object within a virtual reality environment

In at least one general aspect, a method can include determining a position of a user in a virtual reality (VR) environment, determining a position of object in the VR environment, defining a vector between the position of the user and the position of the object, and defining an elongated collision volume along the vector. The elongated collision volume may have a medial portion around at least a portion of the object, a proximal portion disposed proximal to the object, and a distal portion disposed proximal to the object. The proximal portion may have a cross-sectional area less than a cross-sectional area of the distal portion.

TECHNICAL FIELD

This disclosure relates generally to virtual reality (VR), and, more particularly, to methods and apparatus for interacting with a distant object within a VR environment.

BACKGROUND

In virtual-reality (VR) environments, a user can extend beyond his/her physical limits when interacting (e.g., grabbing, reaching, touching, etc.) with an object. In this case, it may generally be difficult for the user to determine a distance of the object while attempting to interact with the object.

SUMMARY

In one general aspect, a method can include determining a position of a user in a virtual reality (VR) environment, determining a position of object in the VR environment, defining a vector between the position of the user and the position of the object, and defining an elongated collision volume along the vector. The elongated collision volume may have a medial portion around at least a portion of the object, a proximal portion disposed proximal to the object, and a distal portion disposed distal to the object. The proximal portion may have a cross-sectional area less than a cross-sectional area of the distal portion.

Implementations may include one or more of the following features. For example, the method may include determining whether the position of the object is outside of a threshold reach of the user, and defining the elongated collision volume in response to when the object is outside of the threshold reach of the user.

In some implementations, when the position of the user is a first position of the user, the vector being a first vector, the method may include detecting a change in position of the user to a second position of the user, defining a second vector between the second position of the user and the position of the object, and aligning the elongated collision volume along the second vector.

In some implementations, when the position of the object is a first position of the object, the vector being a first vector, the method may include detecting a change in position of the object to a second position of the object, defining a second vector between the second position of the object and the position of the user, and aligning the elongated collision volume along the second vector.

In some implementations, the elongated collision volume may have a trapezoidal shape. The elongated collision volume may have a trapezoidal shape along a first side and a trapezoidal shape along a second side. In some implementations, at least one of the distal portion of the elongated collision volume or the proximal portion of the elongated volume may have a volume greater than a volume of the medial portion of the elongated collision volume. In some implementations, the distal portion of the elongated collision volume may have a volume greater than a volume of the proximal portion of the elongated volume.

In another general aspect, a method can include determining a position of a user in a virtual reality (VR) environment, identifying a collision volume associated with an object, and modifying the collision volume along a direction between the object and the position of the user to define a modified collision volume.

Implementations may include one or more of the following features. For example, the method may include elongating the collision volume associated with the object at least one of toward or away from the position of the user. The modified collision volume of the object may rotate in response to a change in position of the user.

In some implementations, the method may include determining whether the object is outside of a threshold reach of the user. The modified collision volume may be identified when the object is outside of the threshold reach of the user.

In another general aspect, an apparatus may include a sensor configured to determine a position of a user within a virtual reality (VR) environment, and a processor programmed to determine the position of the user in the VR environment, identify a collision volume associated with an object, and modify the collision volume along a direction between the object and the position of the user to defined a modified collision volume.

DETAILED DESCRIPTION

VR content creators may generate objects that are out of reach of a user (also can be an avatar) (e.g., out of reach of an arm of a user, out of reach of a user without modifying the reach of the user). Further, users prefer not to have to walk to every object in a VR environment to be able to reach an object (e.g., reach object using their arm). When a virtual object is outside of the user's reach, the user may have difficulty in judging the distance while interacting with the virtual object. In other words, depth perception may not be correct at longer distances, so users have difficulty in accurately determining how far to reach to interact with the virtual object at a distance in the VR environment. Therefore, a user may under-reach (e.g., may not quite reach the object) or over-reach (e.g., may reach beyond) when interacting with the virtual object. Methods and apparatus that address at least these problems are disclosed herein. The object can be any type of virtual object (e.g., a tool, a graphical object, a user, etc.).

Disclosed example methods and apparatus may identify a collision volume associated with an object. As illustrated in exemplary embodiments, a collision volume may be an area (invisible) at (e.g., around, surrounding) or near the object in the VR environment. The collision volume can be associated with the object to provide the object with more volume and/or with a simpler shape (e.g., a box) with which a user can interact. The additional volume may be easier to calculate, process, etc. when a user is interacting with the object in VR than the actual shape of the object (which can be complex). For example, the collision volume can be relatively larger than the object, and thus, allows the user to easily interact (e.g., grab, touch, contact, hold, reach, etc.) with the object (by interacting with the collision volume). Hence, due to the larger size of the collision volume associated with the object, the user need not be as precise when interacting with the object.

Disclosed example methods and apparatus may further modify the collision volume of the object to define a modified collision volume. The modified collision volume may be defined by elongating the collision volume associated with an object (e.g., an original size of a collision volume around the object) toward and/or away from a position of the user. This modified collision volume provides the object with an extended collision volume (e.g., with even more depth), which permits the user to more easily (and quickly) interact with the object at a distance. Hence, due to a volume (e.g., a size) of the modified collision volume, the user need not be as precise when interacting with the object at a distance.

The modified collusion volume may further move (e.g., rotate) in response to a change in position of the user. When the position of the user moves in relation to the virtual object, the modified collusion volume also moves along a direction (e.g., a line between a location (e.g., a vertex of a center) of the virtual object and a location (e.g., a center of the user)) of a line of sight of the user. This dynamic movement of the modified collusion volume in relation to the user provides a collusion volume that is elongated (e.g., always elongated) toward or away from the user (e.g., along a line between the user and the object). Further, the modified collusion volume provides the user an ability to interact with the virtual objects outside of the reach of the user (e.g., outside of arm's reach), thus, providing an enhanced VR experience for the user and the content creator. In this way, a user can naturally navigate substantially, most, generally, etc. all of a VR environment, while interacting with an object that is within reach and/or out of reach.

FIG. 1Aillustrates a third person view of a user50interacting with an object10in a virtual reality (VR) environment, in accordance with an example embodiment.FIG. 1Aillustrates a third person perspective view of the user50wearing a head mounted display (HMD)9through which a virtual reality environment can be displayed to the user50. The objects illustrated inFIG. 1Aare objects seen by the user50within the virtual reality environment through the HMD9, but are illustrated as seen by the third person view for ease of description. For example, the object10shown inFIG. 1A, is seen by the user50within the by the user50through the HMD9, but in reality will not be seen by a third person. The other figures within this detailed description are similarly illustrated from a third person perspective for ease of description. In some implementations, the examples described within the detailed description can be applied to multiple objects and/or multiple users.

The object10may be at a distance from the user50(between position14of the user50and a position12of the object10). That is, the object10is beyond a threshold distance Q (also can be referred to as a reach threshold) of a reach of the user50(e.g., outside of the user's normal reach). As shown inFIG. 1A, the threshold distance Q is at an end of (e.g., approximately at an end of) a reach of the user50. In some implementations, the threshold distance Q can be approximately from the position14of the user to an end of a reach of a hand of the user50(or avatar of the user50). For example, the threshold distance Q may be greater than 1 meter, as measured from the user50.

The object10has the position12(e.g., a central position) which may be aligned with the position14(e.g., a central position) of the user50. A vector16(also can be a line and can be referred to as a line) is defined which is between the position14of the user50and the position12of the object10.

As illustrated in an example embodiment, the user50may use (e.g., employ) an item70(e.g., a pole, a stick, etc.) to interact with the object10. As shown inFIG. 1A, the item70allows the user50to interact with the object10outside of the threshold distance Q of the user50. In other words, the item70is used by the user50to interact with the object10outside of the typical reach (e.g., threshold distance Q) of the user50. Although not shown, in some implementations, in the VR environment, the user's50arm may be extended (e.g., stretched) to have a reach beyond on the threshold distance Q (e.g., similar to the reach of the item70).

In order to interact (e.g., grab, touch, contact, hold, reach, etc.) with the object10, a collision volume30is defined at or near the object10to assist the user50in interacting with the object10. The collision volume30may be a volume at (e.g., around, surrounding) or near the object10to provide the object10with more depth. The collision volume30is shown herein for discussion purposes, but in the virtual environment will not be visible to the user50. The collision volume30may be generally larger than the object10, and thus, allows the user50to easily interact with the object10. In some implementations, because the collision volume30is larger than the object10, the user50may interact with the collision volume30without interacting with (e.g., intersecting or coming virtually in contact with) the object10.

In some implementations, the collision volume30may have similar shape as the object10. For example, if the object10is a vase, the collision volume30may have a similar shape of a vase, yet the collision volume30will be larger than the object10(e.g., vase). In some implementations, the collision volume30may be a graphical representation of the object10. For example, the graphical representation may have a shape of at least one of a cube, a rectangular-cube, a sphere, a tetrahedron, or a polyhedron. In some implementations, the graphical representation of the collision volume30should be larger than the object10.

As noted above, in some implementations, the collision volume30may surround (or encompass) the object10. For example, the collision volume30may entirely surround the collision volume30so that a wall of the collision volume30does not intersect any portion of the object10. In some implementations, a portion (e.g., a central portion) of the collision volume30may intersect (e.g., co-axially aligned in all 3D coordinates) with a portion (e.g., a central portion) of the object10. In some implementations, the collision volume30may not surround the object10. For example, the collision volume30may have a wall that intersects a portion of the object10. In some implementations, the object10(or a portion thereof) may be disposed outside of a wall of the collision volume30. In some implementations, the collision volume30may be off-center from a central portion of the object10. For example, a center of the collision volume30may be in front of the object10, back of the object10, or at one side (left-side, right-side, top-side or bottom-side) of a center of the object10(e.g., as seen from the user's line of sight.

As shown inFIG. 1A, in an exemplary embodiment, the collision volume30may be a modified collision volume. Specifically, the collision volume30may be modified from a collision volume associated with (e.g., surrounding) the object10, when the object10is within a reach of the user50(e.g., within the threshold distance Q). In some implementations, the collision volume30may be modified from a typical collision volume associated with the object10. An example of a typical collision volume30is illustrated, as an example, inFIG. 1B.

An exemplary method to modify the collision volume30is to elongate the collision volume30towards and/or away from the position14of the user50. In an exemplary embodiment as illustrated, the elongated collision volume30may have a shape of a rectangular-cube. Hence, the elongated collision volume30permits the user10to easily interact with the object10due to the elongated collision volume30being larger (e.g., longer) in relation to the position of the user50. More specifically, the elongated collision volume30may resolve under-reaching or over-reaching the object10when the user50attempts to interact with the object10. Further, this allows the user50to interact with the elongated collision volume30without actually interacting (e.g., touching, grabbing, contacting, etc.) with the object10.

The elongated collision volume30may have a proximal portion32and a distal portion34. The proximal portion32may be closest to the position of the user50and the distal portion34may be farthest to the position of the user50. In some implementations, the user50may likely interact with the proximal portion32of the elongated collision volume since it is the closest portion to the position of the user50. The elongated collision volume30may also have a medial portion36(e.g., center of the elongated collision volume) between the proximal portion32and the distal portion34. The medial portion36may be around (e.g., co-axially aligned in all 3D coordinates) with the position12of the object10so that the position12of the object10is disposed within the medial portion36of the collision volume30. In some implementations, the medial portion36may not surround the position12of the object10.

In some implementations, an interface between the medial portion36of the collision volume30and the proximal portion32of the collision volume30can be disposed between the position12of the object10and the position14of the user50. In some implementations, an interface between the medial portion36of the collision volume30and the proximal portion32of the collision volume30can be disposed outside of (e.g., outside of a wall of, on a frontside of) the object10.

In some implementations, the position12of the object10can be disposed between the position14of the user50and an interface between the medial portion36and distal portion34of the collision volume30. In some implementations, an interface between the medial portion36of the collision volume30and the distal portion34of the collision volume30can be disposed outside of (e.g., outside of a wall of, on a backside of) the object10.

In some implementations, the proximal portion32of the collision volume30may have a volume smaller than a volume of the medial portion36of the of the collision volume30. In some implementations, the distal portion34of the collision volume30may have a volume greater than a volume of the medial portion36of the of the collision volume30. More details related to such implementations (e.g., some trapezoidal-shaped implementations) are described in more detail below.

In some implementations, the proximal portion32and the distal portion34of the collision volume30may each have a volume smaller than a volume of the medial portion36of the of the collision volume30. In some implementations, the proximal portion32and the distal portion34of the collision volume30may each have a volume greater than a volume of the medial portion36of the of the collision volume30. In some implementations, the proximal portion32and the distal portion34of the collision volume30may each have a volume that is the same as a volume of the medial portion36of the of the collision volume30.

In some implementations, the proximal portion32of the collision volume30may have a length (along the vector16) smaller or longer than a length (along the vector16) of the medial portion36of the of the collision volume30. In some implementations, the distal portion34of the collision volume30may have a length (along the vector16) smaller or longer than a length (along the vector16) of the medial portion36of the of the collision volume30. In some implementations, the proximal portion32and/or the distal portion34of the collision volume30may have a length (along the vector16) equal to a length (along the vector16) of the medial portion36of the of the collision volume30.

In some implementations, the proximal portion32and/or the distal portion34may be modified (e.g., elongated) along a same direction (e.g., parallel) of the vector16. In some implementations, the elongated collision volume30may be modified (e.g., elongated) in relation to the vector16such that the proximal portion32and/or the distal portion34may extend along a direction non parallel to (e.g., perpendicular to) the vector16. For example, the proximal portion32and/or the distal portion34may extend 90 degrees in relation to the vector16. In some implementations, the proximal portion32and/or the distal portion34may extend at an angle to the vector16. For example, the proximal portion32and the distal portion34may extend at ±30 degrees in relation to the vector16.

In some implementations, the elongated collision volume30may have a shape of a trapezoid. In other words, a volume at the distal portion34of the elongated collision volume30may be greater than a volume at the proximal portion32of the elongated collision volume30. In some implementations, this shape can ensure that the user50interacts with the object10during an over-reaching action of the user50, due to having a greater volume at the distal portion34. Further, depth perception for an object(s) at a distance may not be as accurate to the user50, so therefore, the elongated collision volume30assists the user50to interact with the object10.

In some implementations, the elongated collision volume30(or a portion thereof) (e.g., the medial portion36, the distal portion34, the proximal portion32) may have other shapes, for example, but not limited to, a tetrahedron, a polyhedron, an octahedron, a cone, and an ellipsoid.

FIG. 1Billustrates another third person view of an example implementation of a system for interacting with an object10, in accordance with an example embodiment.FIG. 1Bis similar toFIG. 1Aexcept that the object10is within the threshold distance Q.

Referring toFIG. 1B, the object10(or a portion thereof (e.g., a wall of the object10)) may be at or within the user's50normal arm reach (e.g., within the threshold distance Q). For example, the threshold distance Q to interact (e.g., reach) the object10may be 1 meter or less. If the user's reach is within 1 meter, the user50may be interacting with the object10. In order to assist the user50in interacting with the object10, a collision volume30′ may be provided at or near the object10. The exemplary collusion volume30′ as illustrated inFIG. 1Bis a non-modified (e.g., non-elongated) version of the collision volume30ofFIG. 1A.

In some implementations, the collision volume30′ may be larger and may have the same shape as the object10. For example, the collision volume30′ has a shape of a cube which is the same shape as the object10, in this case. In some implementations, the collision volume30′ may not have the same shape as the object10. The collision volume30′ may be a graphical representation (e.g., a sphere, a tetrahedron, a polyhedron, etc.) of the object10, which is larger than the object10. In some implementations, this larger collision volume30′ identified near the object10permits the user50to interact with the object10more easily (and quicker). In some implementations, the collision volume30′ may have the same volume as the medial portion36of the collision volume30shown in, for example,FIG. 1A.

In some implementations, the collision volume30′ may surround the object10. For example, the collision volume30′ (e.g., a center portion of the collision volume) may surround (e.g., aligned in all 3D coordinates) with a center portion of the object10.

In some implementations, the collision volume30′ may not surround the center portion of the object10. In other words, the collision volume30′ may be off-center from the center portion of the object10. For example, a portion of the collision volume30′ may be in front of the object10, back of the object10, or at one side (left-side, right-side, top-side or bottom-side) of the object10(e.g., as seen from the user's line of sight).

To accomplish an effective and realistic presentation of interaction of object(s) in the VR environment, the present systems and methods use collision volume to enable to user to interact with the object(s) more easily. For example, consider if the user50is on a boat and the user50interacts with an object (e.g., fish) that is within the threshold distance (e.g., less than 1 meter), the user50can reach their arm into the water to interact (e.g., touch, hold, grab, etc.) with the fish. In such an example, the user50may actually interact with a collision volume30that is at or near the fish. The collision volume30can be generally larger than the fish to provide an invisible volume that may be easily interacted with by the user50.

In another example, when the same user50interacts with another object (e.g., another boat) that is out of the threshold distance (e.g., greater than 1 meter), the user's50reach cannot interact with the boat. In this example, the boat that is at a distance (outside the threshold distance) will have a collision volume30that may be automatically modified to assist the user50to interact with the boat. The modified collision volume may be elongated towards and/or away a position of the user50to create even a larger invisible area at or near the boat. The enlarged modified collision volume allows the user50to interact with the boat which may be outside of the threshold distance.

In some implementations, the user50may be able to interact with one or more objects. For example, the user50may interact with a first and/or second object(s) that is/are within the threshold distance (within arm's reach) and a third and/or fourth object(s) that is/are outside of the threshold distance (outside arm's reach). Each object can have a collision volume and/or a modified collision volume.

FIG. 2is a block diagram illustrating a collision volume module144in accordance with an example embodiment. The collision volume module144can generate, modify, and/or execute the collision volume associated with VR objects and/or scenes operated in a VR system100, as shown inFIG. 3. The collision volume module144may include a reach module151, an object position module153, a user position module155, a direction module157, and a collision volume definition module159.

The reach module151may be configured to determine whether a reach of the user (e.g., user50) is within a reach threshold152(e.g., a threshold distance). For example, if the user's reach is within 1 meter, the user may interact with a collision volume (e.g., collision volume30) associated with an object (e.g., object10). If the user's reach is outside the reach threshold (e.g., greater than 1 meter), the collision volume definition module159can be configured to modify the collision volume (e.g., elongates) associated with the object such that the user may interact with the modified collision volume.

In some implementations, the reach module151can be configured to determine whether the object is within the reach of the user based on a position of the object determined using the object position module153and a position of the user determined using the user position module155. A distance between the position of the user and the position of the object can be compared by the reach module151with the reach threshold152to determine whether the object is within reach of the user. For example, if the distance is greater than the reach threshold152, the reach module151can determine that the object is outside of the reach of the user.

In some implementations, the object position module153may be configured to determine the position of the object in relation to the user. In some implementations, the user position module155may be configured to determine the position of the user in relation to the object.

In some implementations, the direction module157may be configured to determine the direction of the user with respect to the object, and vice versa. In some implementations, the direction module157may be configured to determine the direction of the user with respect to the object, and vice versa, based on the respective positions of the user and object. In some implementations, the direction module157can be configured to determine a vector (e.g., a line) between the user and the object.

In some implementations, the collision volume definition module159can be configured to define a collision volume associated with the object10. In some implementations, the collision volume definition module159may be configured to define the collision volume based on a size and/or shape of the object. In some implementations, the collision volume definition module159may identify the shape of the collision volume as the same shape as the object. In some implementations, the collision volume definition module159may identify the shape of the collision volume as a graphical representation of the object. For example, the graphical representation may have a shape of at least one of a cube, a rectangular-cube, a sphere, a tetrahedron, or a polyhedron. In some implementations, the size of the collision volume is generally larger than the object.

In some implementations, the collision volume definition module159may be configured to define a modified collision volume. The modified collision volume may be elongated towards and/or away from a position of the user by the collision volume definition module159. For example, the modified collision volume may have a trapezoidal shape, including a proximal portion and a distal portion. The proximal portion may have a cross-sectional area less than a cross-sectional area of the distal portion.

FIG. 3illustrates a diagram (which includes a third person view of a user) of an example virtual reality (VR) system100for creating and interacting with a three-dimensional (3D) VR environment102in accordance with this disclosure. More details related toFIG. 3are described below.

FIGS. 4A and 4Bare schematic diagrams of a user50in relation to a collision volume30(e.g., a modified collision volume) when a user50moves in relation to an object10, in accordance with an example embodiment.FIG. 4Aillustrates the user50in a first position in relation to the object10, defining a first vector16. The first vector16may be aligned between a position B1of the user and a position A1of the object10.FIG. 4Billustrates the user50in a second position B2(moved from position B1) in relation to the object10, defining a second vector16′. The second vector16′ may be aligned between a position of the user B2and a position A1of the object10.

As shown inFIG. 4B, when the user50moves position from B1to B2, the modified collision volume30associated with the object10also moves (e.g., rotates) such that the second vector16′ is aligned between the position of the user B2and the position A1of the object10. The collision volume30is moved to be aligned with the second vector16′. The movement of the modified collision volume30is dynamic, which means that the modified collision volume30moves when the position of the user50changes. As illustrated in this exemplary implementation, the second vector may be, for example, 30 degrees in relation to the first vector. In some other implementations, the second vector may be defined between 0 and 360 degrees in relation to the first vector.

AlthoughFIGS. 4A and 4Bare illustrated in a two-dimensional (2D) example, the principles described herein can be applied in three-dimensions with movement along X,Y, and Z axis. In some implementations, a yaw, pitch, and/or roll of the object10and/or the user50may change. In such implementations, the collision volume30can remain aligned along the vectors16,16′ even with such changes.

FIGS. 5A and 5Bare schematic diagrams of a user50in relation to a modified collision volume30when an object10moves (from A1to A2) in relation to the user50, in accordance with another example embodiment.FIG. 5Aillustrates the object10in a first position in relation to the user50, defining a first vector16. The first vector16may be aligned between a position B1of the user and a position A1of the object10.FIG. 5Billustrates the object10in a second position A2in relation to the user50, defining a second vector16″. The second vector16″ may be aligned between a position B1of the user and a position A2of the object10.

As shown inFIG. 5B, when the object10moves position from A1to A2, the modified collision volume30associated with the object10moves such that the second vector16″ is aligned between the second position A2of the object10and the position of the user50(e.g., the position of the user B1and the position A2of the object10are aligned). The collision volume30is moved to be aligned with the second vector16″. The movement of the modified collision volume30is dynamic, which means that the modified collision volume30moves when the position of the object10moves.

AlthoughFIGS. 5A and 5Bare illustrated in a two-dimensional (2D) example, the principles described herein can be applied in three-dimensions with movement along X,Y, and Z axis. In some implementations, a yaw, pitch, and/or roll of the object10and/or the user50may change. In such implementations, the collision volume30can remain aligned along the vectors16,16″ even with such changes.

FIGS. 6A and 6Bare schematic diagrams of a user50in relation to multiple modified collision volumes30(e.g.,30A through30C) when the user moves in relation to objects10(e.g.,10A through10C), in accordance with another example embodiment. The objects10are illustrated as being aligned along a plane approximately the same distance from the user50for simplicity in description. In some implementations, the object10may be different sizes, different distances from the user50, and/or so forth.

FIG. 6Aillustrates the user50in a first position D1in relation to multiple objects10(in this case three objects), defining first vectors18,18′,18″, respectively. The vectors18,18′,18″ may be aligned between a position D1of the user50and respective positions C1, C2, C3of the objects10.FIG. 6Billustrates the user50in a second position D2in relation to the multiple objects10, defining vectors19,19′,19″, respectively. The vectors19,19′,19″ may be aligned between a position of the user D2and positions C1, C2, C3, respectively, of the multiple objects10.

In these implementations, the vectors18,18′, and18″ are non-parallel with one another, and even with movement of the position of the user50, the vectors19,19′, and19″ are non-parallel with one another. Accordingly, the collision volumes30A through30C are also aligned non-parallel with one another. The trapezoidal shapes of the collision volumes30A through30C reduces (e.g., prevents, minimizes) overlap between the collision volumes30A through30C.

As shown inFIG. 6B, when the user50moves position from D1to D2, the modified collision volumes30associated with the respective objects10also move such that modified collision volumes30are aligned with the vectors19,19′,19″, which are aligned between the position of the user D2and the positions C1, C2, C3, respectively of the objects10. The movement of the modified collision volumes30are dynamic, which means that each of the modified collision volumes30move when the position of the user50

FIG. 6Cis a schematic diagram of the user50in relation to multiple, overlapping modified collision volumes30, in accordance with another example embodiment.

As shown inFIG. 6C, when the user50moves position from D2to D3, the modified collision volumes30associated with the objects10move such that the respective vectors20all align along a same vector (e.g., between a position of the user D3and positions C1, C2, C3of the objects10). This also causes the modified collision volumes30to overlap. In some implementations, this overlapping configuration can permit the user50to interact with multiple objects10simultaneously.

In some implementations, because the modified collision volumes30are overlapping to resolve this issue, the system can be configured to employ an algorithm to determine an order of selecting the object. In one implementation, the system may select the object that is closest to the user. In this case, the system may determine that the user50is targeting interaction with the object10containing position C1, which is closest to the user50. In some implementations, the system may be configured to select the object with a position (e.g., a center position) that is closest to a selection point (e.g., a hand, a selection device) of the user50. For example, if the user50attempts to reach for the farthest object (e.g., object with position C3) and positions a virtual selection device (e.g., a hand, a wand) relatively close to the farthest object (and closer to a position of the farthest object than the other objects), the system may be configured to select the farthest object10for interaction.

In some implementations, the positions C1, C2, C3of the objects10may not be on a same vector as the position of the user50. For example, while position C2of the object10may be on the same vector as the position of the user50, position C1of the object10may be above the vector of the position of the user50and position C3of the object10may be below the vector of the position of the user50, while the respective collision volumes are overlapping. In this case, as similarly discussed above, the system may select the modified collision volume which is closest to the user50or may select the modified collision volume whose position is closest to a selection point of the user50.

Although exemplary implementations described herein illustrate the user50interacting with three objects10, the user50may interact with more or less than three objects. Further, exemplary implementations illustrate the collision volume30of a trapezoidal shape; however other shapes may be employed (e.g., a cube, a rectangular-cube, a sphere, a tetrahedron, a polyhedron, etc.),

AlthoughFIGS. 6A through 6Care illustrated in a two-dimensional (2D) example, the principles described herein can be applied in three-dimensions with movement along X,Y, and Z axis. In some implementations, a yaw, pitch, and/or roll of the object10and/or the user50may change. In such implementations, the collision volumes30A through30C can remain aligned along the various vectors18even with such changes.

FIG. 7illustrates an example method700that may be performed to determine user's interaction with an object within a VR environment as disclosed herein. The example method700ofFIG. 7begins with determining a position of a user (block710), determining a position of an object (block720), defining a line (e.g., a vector) between the position of the user and the position of the object (block730), and defining a modified (e.g., elongated) collision volume along the line (block740).

At block710, the position of the user is determined in relation to the object. At block720, the position of the object is determined in relation to the user.

When the user or the object moves, a line is defined between a position of the object is aligned to a position of the user

At block740, if the object is outside of a threshold reach of the user, the collision volume (e.g., elongates) associated with the object is modified such that the user may interact with the modified collision volume. The modified collision volume may be elongated along the line where the elongated collision volume has a medial portion around at least a portion of the object. The elongated collision volume may have a proximal portion disposed proximal to the object and a distal portion disposed distal to the object. In some implementations, the proximal portion may have a cross-sectional area that is less than a cross-sectional area of the distal portion. If the object is within the threshold reach of the user, the collision volume may not be modified.

The example method700ofFIG. 7, or other methods disclosed herein, may, for example, be implemented as machine-readable instructions carried out by one or more processors to control or operate the example display assemblies disclosed herein. A processor, a controller and/or any other suitable processing device may be used, configured and/or programmed to execute and/or carry out the example methods disclosed herein. For instance, the example method700ofFIG. 7, or other methods disclosed herein may be embodied in program code and/or machine-readable instructions stored on a tangible and/or non-transitory computer-readable medium accessible by a processor, a computer and/or other machine having a processor, such as that discussed below in connection withFIG. 3. Machine-readable instructions can include, for example, instructions that cause a processor, a computer and/or a machine having a processor to perform one or more particular processes. Many other methods of implementing the example method700ofFIG. 7, or other methods disclosed herein may be employed. For example, the order of execution may be changed, and/or one or more of the blocks and/or interactions described may be changed, eliminated, sub-divided, or combined. Additionally, any of the entire example method700ofFIG. 7, or other methods disclosed herein may be carried out sequentially and/or carried out in parallel by, for example, separate processing threads, processors, devices, discrete logic, circuits, etc.

FIG. 3illustrates a diagram of an example virtual reality (VR) system100for creating and interacting with a three-dimensional (3D) VR environment102in accordance with the teachings of this disclosure is shown. In general, the system100may provide the VR environment102, and VR content that enable a person (e.g., a user, a wearer, etc.) to access, view, use and/or interact with the VR environment102. The VR system100can provide the user with options for accessing the content, applications, virtual objects, real objects, and VR controls. The example VR system100ofFIG. 1may include a user108wearing a head-mounted display (HMD)110, and having a handheld controller122. The example VR environment102shown inFIG. 1is a representation of the images being displayed for the user inside their HMD110.

The example VR system100may include any number of computing and/or electronic devices that can exchange data over a network125. The devices may represent clients or servers, and can communicate via the network125or any other additional and/or alternative network(s). Example client devices include, but are not limited to, the HMD110, the handheld controller112, a mobile device131(e.g., a smartphone, a personal digital assistant, a portable media player, etc.), a laptop or netbook132, a desktop computer133, an electronic tablet (not shown), a camera (not shown), a gaming device (not shown), and any other electronic or computing devices that can communicate using the network125or other network(s) with other computing or electronic devices or systems, or that may be used to access VR content or operate within a VR environment. The devices110,112, and131-133may represent client or server devices. The devices110,112, and131-133can execute a client operating system, and one or more client applications that can access, render, provide, or display VR content on a display device included in or in conjunction with each respective device110,112and131-133.

The example VR system100may include any number of VR content systems140storing content and/or VR software modules in the form of VR applications (e.g., collision volume module144as shown inFIG. 3) that can generate, modify, and/or execute VR objects and scenes. In some examples, devices110,112and131-133and the VR content system140include one or more processors and one or more memory devices, which can execute a client operating system and one or more client applications. In some implementations, the HMD110, the controller112, the other devices131-133or the VR content system140may be implemented by example computing devices (e.g., P00and/or P50ofFIG. 8).

The VR applications (e.g., collision volume module144) can be configured to execute on any or all of devices110,112and131-133. The HMD110can be connected to devices131-133to access VR content on VR content system140, for example. Device131-133can be connected (wired or wirelessly) to HMD110, which can provide VR content for display. A user's VR system can be HMD110alone, or a combination of device131-133and HMD110.

In some implementations, the VR system100may include a motion sensing device and, under some circumstance, the user may be able to control a graphical representation in the VR environment102(and/or the user in the VR environment) in conjunction with the motion sensing device or the controller112.

In some implementations, the VR system100may include a haptic based system or device. Such haptic based systems may provide the user with physical stimuli, such as mechanical feedback, corresponding at least in part to interactions between the graphical representation of the user, the VR environment102, and/or objects depicted in the VR environment102.

For example, the user may, within the VR environment102, interact with an object (e.g., a virtual object). The object can be, or can include, any of a wide variety of objects at least some of which represent or correspond to real world objects. Such an interaction may involve a graphical representation of the user (or a portion thereof) interacting with a graphical representation of the object within the VR environment102, for example.

In some implementations, the term computer-readable medium can include any type of tangible or non-transitory computer-readable medium and to expressly exclude propagating signals. Example computer-readable medium include, but are not limited to, a volatile and/or non-volatile memory, a volatile and/or non-volatile memory device, a compact disc (CD), a digital versatile disc (DVD), a read-only memory (ROM), a random-access memory (RAM), a programmable ROM (PROM), an electronically-programmable ROM (EPROM), an electronically-erasable PROM (EEPROM), an optical storage disk, an optical storage device, a magnetic storage disk, a magnetic storage device, a cache, and/or any other storage media in which information is stored for any duration (e.g., for extended time periods, permanently, brief instances, for temporarily buffering, and/or for caching of the information) and that can be accessed by a processor, a computer and/or other machine having a processor.

Referring toFIG. 9, an example of a generic computer device P00and a generic mobile computer device P50, which may be used with the techniques described here. The computing devices P50may be used to implement any of the devices disclosed herein including, but not limited to, HMD110, controller, devices126-128and130, HMD215and controller220. Computing device P00is intended to represent various forms of digital computers, such as laptops, desktops, tablets, workstations, personal digital assistants, televisions, servers, blade servers, mainframes, and other appropriate computing devices. Computing device P50is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed in this document.

Computing device P00includes a processor P02, memory P04, a storage device P06, a high-speed interface P08connecting to memory P04and high-speed expansion ports P10, and a low speed interface P12connecting to low speed bus P14and storage device P06. The processor P02can be a semiconductor-based processor. The memory P04can be a semiconductor-based memory. Each of the components P02, P04, P06, P08, P10, and P12, are interconnected using various busses, connections, memories, caches, etc. and may be mounted on a common motherboard or in other manners as appropriate. The processor P02can process instructions for execution within the computing device P00, including instructions stored in the memory P04or on the storage device P06to light-emitting portion graphical information for a GUI on an external input/output device, such as light-emitting portion P16coupled to high speed interface P08. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices P00may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).

The memory P04stores information within the computing device P00. In one implementation, the memory P04is a volatile memory unit or units. In another implementation, the memory P04is a non-volatile memory unit or units. The memory P04may also be another form of computer-readable medium, such as a magnetic or optical disk.

The high speed controller P08manages bandwidth-intensive operations for the computing device P00, while the low speed controller P12manages lower bandwidth-intensive operations. Such allocation of functions is exemplary only. In one implementation, the high-speed controller P08is coupled to memory P04, light-emitting portion P16(e.g., through a graphics processor or accelerator), and to high-speed expansion ports P10, which may accept various expansion cards (not shown). In the implementation, low-speed controller P12is coupled to storage device P06and low-speed expansion port P14. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wi-Fi) may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.

The computing device P00may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server P20, or multiple times in a group of such servers. It may also be implemented as part of a rack server system P24. In addition, it may be implemented in a personal computer such as a laptop computer P22. Alternatively, components from computing device P00may be combined with other components in a mobile device (not shown), such as device P50. Each of such devices may contain one or more of computing device P00, P50, and an entire system may be made up of multiple computing devices P00, P50communicating with each other.

Computing device P50includes a processor P52, memory P64, an input/output device such as a light-emitting portion P54, a communication interface P66, and a transceiver P68, among other components. The device P50may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of the components P50, P52, P64, P54, P66, and P68, are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.

The processor P52can execute instructions within the computing device P50, including instructions stored in the memory P64. The processor may be implemented as a chipset of chips that include separate and multiple analog and digital processors. The processor may provide, for example, for coordination of the other components of the device P50, such as control of user interfaces, applications run by device P50, and wireless communication by device P50.

Processor P52may communicate with a user through control interface P58and light-emitting portion interface P56coupled to a light-emitting portion P54. The light-emitting portion P54may be, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Light-emitting portion) or an OLED (Organic Light-emitting Diode) light-emitting portion, or other appropriate light-emitting portion technology. The light-emitting portion interface P56may comprise appropriate circuitry for driving the light-emitting portion P54to present graphical and other information to a user. The control interface P58may receive commands from a user and convert them for submission to the processor P52. In addition, an external interface P62may be provided in communication with processor P52, so as to enable near area communication of device P50with other devices. External interface P62may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.

The memory P64stores information within the computing device P50. The memory P64can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. Expansion memory P74may also be provided and connected to device P50through expansion interface P72, which may include, for example, a SIMM (Single Inline Memory Module) card interface. Such expansion memory P74may provide extra storage space for device P50, or may also store applications or other information for device P50. Specifically, expansion memory P74may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, expansion memory P74may be provide as a security module for device P50, and may be programmed with instructions that permit secure use of device P50. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory, as discussed below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as the memory P64, expansion memory P74, or memory on processor P5that may be received, for example, over transceiver P68or external interface P62.

Device P50may communicate wirelessly through communication interface P66, which may include digital signal processing circuitry where necessary. Communication interface P66may provide for communications under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, through radio-frequency transceiver P68. In addition, short-range communication may occur, such as using a Bluetooth, Wi-Fi, or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module P70may provide additional navigation- and location-related wireless data to device P50, which may be used as appropriate by applications running on device P50.

Device P50may also communicate audibly using audio codec P60, which may receive spoken information from a user and convert it to usable digital information. Audio codec P60may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of device P50. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on device P50.

The computing device P50may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a cellular telephone P80. It may also be implemented as part of a smart phone P82, personal digital assistant, or other similar mobile device.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a light-emitting portion device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal light-emitting portion) monitor) for light-emitting portioning information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

Terms such as, but not limited to, approximately, substantially, generally, etc. are used herein to indicate that a precise value or range thereof is not required and need not be specified. As used herein, the terms discussed above will have ready and instant meaning to one of ordinary skill in the art.

Moreover, use of terms such as up, down, top, bottom, side, end, front, back, etc. herein are used with reference to a currently considered or illustrated orientation. If they are considered with respect to another orientation, it should be understood that such terms must be correspondingly modified.

Further, in this specification and the appended claims, the singular forms “a,” “an” and “the” do not exclude the plural reference unless the context clearly dictates otherwise. Moreover, conjunctions such as “and,” “or,” and “and/or” are inclusive unless the context clearly dictates otherwise. For example, “A and/or B” includes A alone, B alone, and A with B.

Additionally, connecting lines and connectors shown in the various figures presented are intended to represent exemplary functional relationships and/or physical or logical couplings between the various elements. It should be noted that many alternative and/or additional functional relationships, physical connections or logical connections may be present. Moreover, no item or component is essential to the practice of this disclosure unless the element is specifically described as “essential” or “critical”. Additionally, the figures and/or drawings are not drawn to scale, but rather are drawn for clarity of illustration and description.

Although certain example methods, apparatuses and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. It is to be understood that terminology employed herein is for the purpose of describing particular aspects, and is not intended to be limiting. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.