Spatial position calculation system for objects in virtual reality or augmented reality environment

A user control apparatus has a laser emitter that emits a laser beam in a real-world environment. Further, the user control apparatus has an optical element that receives the laser beam and generates a plurality of laser beams such that a starting point and a plurality of endpoints, each corresponding to one of the plurality of laser beams, form a laser frustum. In addition, the user control apparatus has an image capture device that captures an image of a shape of the laser frustum based on a reflection of the plurality of laser beams from an object in the real-world environment so that a spatial position of the object in the real-world environment is determined for an augmented reality or virtual reality user experience.

BACKGROUND

This disclosure generally relates to the field of user experiences. More particularly, the disclosure relates to a virtual reality (“VR”) and/or an augmented reality (“AR”) environment.

2. General Background

VR and AR configurations typically involve a determination of user control and consumption devices (e.g., a remote control) with respect to three-dimensional (“3-D”) locations of objects in the real world. Such configurations thereby allow for an augmented user experience. Yet, multiple sensors and transmitters are often used in current configurations to perform such analyses. The multitude of sensors and transmitters often leads to a cumbersome and expensive user experience. As a result, current configurations do not provide an optimal VR and/or AR user experience.

SUMMARY

In one aspect, a user control apparatus has a laser emitter that emits a laser beam in a real-world environment. Further, the user control apparatus has an optical element that receives the laser beam and generates a plurality of laser beams such that a starting point and a plurality of endpoints, each corresponding to one of the plurality of laser beams, form a laser frustum. In addition, the user control apparatus has an image capture device that captures an image of a shape of the laser frustum based on a reflection of the plurality of laser beams from an object in the real-world environment so that a spatial position of the object in the real-world environment is determined for an augmented reality or virtual reality user experience.

In another aspect, a system has a storage device that stores one or more dimensions of a real-world environment. Further, the system has a receiver that receives an image of a shape of a laser frustum based on a plurality of laser beams being emitted toward, and reflected from, an object in the real-world environment. In addition, the system has a processor that determines a spatial position of the object in the real-world environment based on the one or more dimensions of the real-world environment and the shape of the laser frustum. The spatial position of the object is determined for an augmented reality or virtual reality user experience.

In another aspect, a computer program product comprises a non-transitory computer readable storage device having a computer readable program stored thereon. The computer readable program when executed on a computer causes the computer to perform the functionality of the system described herein. In yet another aspect, a process performs the functionality of the system described herein.

DETAILED DESCRIPTION

A configuration for calculating the real-world spatial position of an object in a VR and/or AR environment is provided. The configuration includes a user control device that emits one or more lasers for determining the position of the user control device within a real-world environment.

FIG. 1illustrates a spatial position calculation system100that calculates the spatial position of an object in a real-world environment. The spatial position calculation system100comprises a processor102, a memory106, e.g., random access memory (“RAM”) and/or read only memory (“ROM”), a data storage device108, a laser emitter104, and an image capture device109. An example of the laser emitter104is a laser diode that emits light that may then be projected through an optical element (e.g., a lens) or rotated to generate a plurality of laser beams. The image capture device109(e.g., a camera) may capture one or more images of the plurality of laser beams and then provide those one or more captured images to the data storage device108. For instance, the image capture device109may capture images of the plurality of laser beams as they are reflected from an object in a real-world environment.

In addition, the data storage device108may store spatial position code110that, when executed by the processor102, determines the position of one or more objects in a real-world environment. Further, the processor102may execute VR/AR rendering code112to render a VR/AR environment based on the real-world spatial position of the one or more objects determined by the processor102when executing the spatial position code110.

In one aspect, the data storage device108loads the spatial position code110and the VR/AR rendering code112from a computer readable storage device, e.g., a magnetic or optical drive, diskette, or non-volatile memory, DVD, CD-ROM, etc. In another aspect, the data storage device108is the computer readable storage device. As such, the spatial position code110, the VR/AR rendering code112, and associated data structures of the present disclosure may be stored on a computer readable storage device.

The spatial position calculation system100improves the functioning of a computing device by reducing the processing time that a VR/AR system uses to determine the real-world spatial position of an object. Instead of performing multiple computations from various sensors and transmitters, the spatial position calculation system100reduces computational processing time via the laser emitter104and the image capture device109.

In one aspect, the components of the spatial position calculation system100are integrated within one device (e.g., a remote control, headset, etc.). For example, light is emitted, images are captured, and spatial positions are calculated in one integrated device. In another aspect, the components may communicate with each other remotely through a network connection. For example, the processor102may be stored on a remote server that communicates with a VR/AR headset that has an integrated laser emitter104and image capture device109.

FIG. 2Aillustrates an example of a user control device200with the integrated componentry ofFIG. 1. For illustrative purposes, the user control device200is illustrated as a smartphone that may be used to provide inputs to a VR/AR user experience via a display screen201. Other types of devices (e.g., remote control, headset, glasses, watch, smart wearable device, etc.) may be used instead.

The user control device200includes the laser emitter104and the image capture device109illustrated inFIG. 1. The laser emitter104(e.g., a light emitting diode (“LED”)) may emit one or more lasers through an optical element202(e.g., lens, beam splitter, etc.) to generate a plurality of laser beams204at one or more fixed angles. For example, the laser emitter104in conjunction with the optical element202may generate four laser beams204that emanate from the user control device200to form a laser frustum203(i.e., a structure having a particular shape). For example, the laser frustum203may have a pyramidal shape that emanates from the user control device200at fixed angles to form a square base if the endpoints of the four laser beams204are connected. Various fixed angles, quantities of lasers, and shapes for the laser frustum203may be used to perform spatial positioning with the user control device200. Further, various other forms of laser beam generation other than via the optical element202(e.g., rotation of the laser emitter104without the optical element202) may be implemented.

In addition, the image capture device109may capture the positions of the endpoints of the four laser beams204as reflected from an object in the real-world environment. The captured image data may then be analyzed by the processor102illustrated inFIG. 1to determine the spatial positioning of the object in the real-world environment so that the position may be used for an AR and/or VR user experience.

FIG. 2Billustrates a magnified view of the user control device200illustrated inFIG. 2A. The laser emitter104emits a single laser beam205that passes through the optical element202. Further, the optical element202splits the single laser beam205into the plurality of laser beams204. In one aspect, the laser emitter104and the optical element202are two distinct components that are positioned on a surface of the user control device200. In another aspect, the laser emitter104and the optical element202are encapsulated in a single structure that is positioned on a surface of the user control device200.

In addition, the image capture device109is illustrated as a distinct component from the laser emitter104and the optical element202. In another aspect, the image capture device109, the laser emitter104, and the optical element202are encapsulated in a single structure that is positioned on a surface of the user control device200.

Even though the top portion of the user control device200is illustrated as the position for which the various components ofFIGS. 2A and 2Bare located, other portions may be used instead. For instance, the sides, bottom, etc. may be used.

FIG. 2Cillustrates an AR/VR configuration210that implements the user control device200illustrated inFIGS. 2A and 2B. For example, the user control device200may be a hand controller that receives one or more inputs from a user of the AR/VR configuration210based on an AR/VR experience provided by an AR/VR headset211. In other words, the hand controller200may communicate by transmitting and/or receiving data to and/or from, respectively, the AR/VR headset211.

Further, the laser emitter104, optical element202, and image capture device109may be situated on the top, or other portion, of the illustrated hand controller200. The hand controller200may then send the captured image data, as described with respect toFIGS. 2A and 2B, to a computer213(e.g., laptop, personal computer, tablet device, smartphone, smartwatch, gaming console, etc.) for image analysis. Based on the captured image data, the computer213then determines, with the processor102illustrated inFIG. 1, the location of the hand controller200with respect to a real-world environment for corresponding use in the AR/VR experience. The computer213may be in local communication with the hand controller200(e.g., via a wireless network or cable connection) or in remote communication with the hand controller200(e.g., client-server communication through a network).

In an alternative aspect, the processor102may be integrated within the AR/VR headset211so that the hand controller200sends the captured image data to the AR/VR headset211for image analysis by the AR/VR headset211. In yet another aspect, the processor102may be integrated within the illustrated hand controller200or other form of user control device.

FIGS. 3A, 4A, 5A, and 6Aillustrate a user301using the user control device200illustrated inFIGS. 2A and 2Bto configure an AR/VR environment based on the spatial positioning of objects in a real-world environment302. Initially, the user301may perform a calibration configuration where the user301moves the user control device200around the real-world environment302, e.g. a physical room, so that the processor102(FIG. 1) may determine the dimensions of the real-world environment302. Alternatively, the dimensions of the real-world environment302may be known to the processor102prior to the AR/VR user experience without a calibration being performed (e.g., data input of the room dimensions, previously determined dimensions, etc.).

FIG. 3Aillustrates the user301positioned in a first portion of the real-world environment302whereby the user301is directly in front of, and in close proximity to, a wall in the real-world environment302. The endpoints303-306of the laser beams204reflect off of the wall in an evenly spaced manner. The image capture device109captures an image of the spacing of the endpoints204to the processor102. Based on the predetermined dimensions of the real-world environment302and the spacing of the endpoints303-306, the processor102(FIG. 1) calculates the spatial positioning of the user control device200with respect to the wall.

FIG. 3Billustrates an example of the image of the endpoints303-306reflected when the user301is in close proximity to the wall inFIG. 3A. In one aspect, the image may be captured within a predefined area310(e.g., predetermined or calibrated dimensions of the wall or a portion of the wall). Accordingly, the spacing of the endpoints303-306may be measured not just from each other, but also with respect to the perimeter of the predefined area310. For instance, the endpoint303may have an equal distance, or at least no offset, with respect to the left side of the predefined area310as the distance of the endpoint306with respect to the right side of the predefined area310.

FIG. 4Aillustrates the user301positioned in a second portion of the real-world environment302whereby the user301is directly in front of (but at a farther distance than that ofFIG. 3A) a wall in the real-world environment302.FIG. 4Billustrates an example of the image of the endpoints303-306reflected when the user301is at a farther distance to the wall illustrated inFIG. 4A. As a result, the endpoints303-306are more spaced apart than inFIG. 3B; such spacing allows the processor102to determine that the distance of the user control device200with respect to the wall, has increased.

FIG. 5Aillustrates the user301positioned in a third portion of the real-world environment302whereby the user301is off to an angle with respect to the wall.FIG. 5Billustrates an example of the image of the endpoints303-306reflected when the user301is off to the angle with respect to the wall illustrated inFIG. 5A. As a result, the orientation, in addition to the spacing, of the endpoints303-306may be different than that of the first portion (FIG. 3A) or the second portion (FIG. 4A) of the wall.

For instance, each of the laser beams204intersects the wall inFIGS. 3A and 4Aat the same distance from the user control device200. In contrast, with respect toFIG. 5A, the laser beam204corresponding to endpoint304intersects the wall at a closer proximity to the user control device200than the laser beam204corresponding to the laser beam305. As a result, the image captured by the image capture device109illustrates the endpoint304being further from the left portion of the predefined area310than the endpoint305is from the right portion of the predefined area310. Further, the vertical spacing between the endpoints303and304is less than the vertical spacing between the endpoints305and306since the laser beams204corresponding to the endpoints303and304travel a shorter distance as they intersect the wall302sooner than the laser beams204corresponding to the endpoints305and306.

FIG. 6Aillustrates the user301tilting the user control device200illustrated inFIG. 5A.FIG. 6Billustrates an example of the image of the endpoints303-306reflected from the wall illustrated inFIG. 6Aas result of the tilt.

FIG. 7Illustrates a process700that may be utilized by the spatial position calculation system100(FIG. 1) to determine a spatial position of an object in a real-world environment302(FIG. 3A). At a process block701, the process700stores, at a storage device, one or more dimensions of the real-world environment302. Further, at a process block702, the process700receives, at a receiver, an image of a shape of a laser frustum203(FIG. 2A) based on a plurality of laser beams204being emitted toward, and reflected from, an object in the real-world environment302. In addition, at a process block703, the process700determines, with the processor102(FIG. 1), a spatial position of the object in the real-world environment302based on the one or more dimensions of the real-world environment302and the shape of the laser frustum203. The spatial position of the object is determined for an AR or VR user experience.

The processes described herein may be implemented in a specialized processor. Such a processor will execute instructions, either at the assembly, compiled or machine-level, to perform the processes. Those instructions can be written by one of ordinary skill in the art following the description of the figures corresponding to the processes and stored or transmitted on a computer readable medium. The instructions may also be created using source code or any other known computer-aided design tool. A computer readable medium may be any medium, e.g., non-transitory computer readable storage device, capable of carrying those instructions and include a CD-ROM, DVD, magnetic or other optical disc, tape, silicon memory (e.g., removable, non-removable, volatile or non-volatile), packetized or non-packetized data through wireline or wireless transmissions locally or remotely through a network. A computer is herein intended to include any device that has a specialized, general, multi-purpose, or single purpose processor as described above. For example, a computer may be a desktop computer, laptop, smartphone, tablet device, set top box, etc.

It is understood that the apparatuses, systems, computer program products, and processes described herein may also be applied in other types of apparatuses, systems, computer program products, and processes. Those skilled in the art will appreciate that the various adaptations and modifications of the aspects of the apparatuses, systems, computer program products, and processes described herein may be configured without departing from the scope and spirit of the present apparatuses, systems, computer program products, and processes. Therefore, it is to be understood that, within the scope of the appended claims, the present apparatuses, systems, computer program products, and processes may be practiced other than as specifically described herein.