Systems and methods for using a gyroscope to change the resistance of moving a virtual weapon

This disclosure relates to systems and methods for using a gyroscope to change the resistance of moving a virtual weapon to simulate a feel of moving the virtual weapon. A feel of moving a virtual weapon may be simulated by determining a position and/or an orientation of a virtual blade of the virtual weapon, determining controls for a gyroscope based on the position and/or the orientation of the virtual blade, and operating the gyroscope in accordance with the controls for the gyroscope to provide resistance to changing the orientation of the hilt, wherein the resistance to changing the orientation of the hilt simulates the feel of moving the virtual weapon.

FIELD

This disclosure relates to systems and methods for using a gyroscope to change the resistance of moving a virtual weapon to simulate a feel of moving the virtual weapon.

BACKGROUND

Toy weapons that include a hilt and a blade are known. Such toys do not change the resistance of moving the toy to simulate a feel of moving a virtual weapon.

SUMMARY

This disclosure relates to using a gyroscope to change the resistance of moving a virtual weapon to simulate a feel of moving the virtual weapon. A feel of moving a virtual weapon may be simulated by determining a position and/or an orientation of a virtual blade of the virtual weapon, determining an overlay image comprising the virtual blade, where the virtual blade is placed within the overlay image according to the position and/or the orientation of the virtual blade, displaying the overlay image so that the virtual blade appears to be attached to the hilt, determining controls for a gyroscope based on the position and/or the orientation of the virtual blade, and operating the gyroscope in accordance with the controls for the gyroscope to provide resistance to changing the orientation of the hilt, wherein the resistance to changing the orientation of the hilt simulates the feel of moving the virtual weapon.

A system configured to simulate a feel of moving a virtual weapon may include a display, an image sensor, a hilt, a motion and orientation sensor, a gyroscope, one or more processors, and/or other components. In some implementations, the system may include a speaker. In some implementations, the system may include a haptic generator. In some implementations, the image sensor and the one or more processors may be carried on the display, and the field of view of the image sensor may be a function of the position and the orientation of the display.

The display may be configured to display an overlay image. In some implementations, the display may include one or more of a head-mounted display, an optical head-mounted display, a see-through display, an optical see-through display, a video see-through display, a visor, eyeglasses, sunglasses, a smartphone, a tablet, a mobile device, a projector, and/or other displays.

The image sensor may be configured to generate visual output signals conveying visual information within a field of view of the image sensor. The image sensor may include one or more of a charge-coupled device sensor, an active pixel sensor, a complementary metal-oxide semiconductor sensor, an N-type metal-oxide-semiconductor sensor, and/or other image sensors.

The motion and orientation sensor may be carried by the hilt. The motion and orientation sensor may be configured to generate motion and/or orientation output signals conveying motion and/or orientation information of the hilt. In some implementations, the motion and orientation sensor may include an inertial measurement unit and/or other motion and orientation sensors.

The gyroscope may be carried by the hilt. In some implementations, the gyroscope may include a first gyroscope carried in a first portion of the hilt and a second gyroscope carried in a second portion of the hilt. The first portion and the second portion may be located in different ends of the hilt.

The one or more processors may be configured to determine a position and/or an orientation of a virtual blade of the virtual weapon based on the visual output signals and/or the motion and/or orientation output signals. When the hilt is within the field of view of the image sensor, the one or more processors may be configured to determine a position and/or an orientation of the virtual blade based on the visual output signals.

In some implementations, the one or more processors may be configured to determine the position and/or the orientation of the virtual blade based on a position and/or an orientation of a landmark. The landmark may be carried by the hilt. The landmark may indicate a reference point for the hilt that facilitates determination of a position and/or an orientation of the virtual blade. In some implementations, the landmark may include a light emitting diode and/or other landmarks.

When the hilt is not within the field of view of the image sensor, the one or more processors may be configured to determine the position and/or the orientation of the virtual blade based on the motion and/or orientation output signals. In some implementations, the one or more processors may be configured to determine the position and/or the orientation of the virtual blade based on the visual output signals and based on the motion and/or orientation output signals.

In some implementations, the one or more processors may be configured to select the virtual blade based on a user input received through an input device. An input device may include a key entry device, a touch entry device, an imaging device, a sound device, and/or other input devices. In some implementations, the one or more processors may be configured to select the virtual blade based on a landmark.

The one or more processors may be configured to determine an overlay image. The overlay image may comprise a virtual blade of a virtual weapon. The virtual blade may be placed within the overlay image according to the position and/or the orientation of the virtual blade.

The one or more processors may be configured to effectuate displaying of the overlay image on the display. The displaying may be effectuated so that the virtual blade appears to be attached to the hilt.

The one or more processors may be configured to determine controls for the gyroscope based on the position and/or the orientation of the virtual blade. The controls for the gyroscope may determine one or more of speed and/or direction of rotation of the gyroscope around one or more axis at a time, over a period of time, at a location, or over a range of locations.

In some implementations, the one or more processors may be configured to determine changes in the position and/or the orientation of the virtual blade. The one or more processors may be configured to determine the controls for the gyroscope based on the changes in the position and/or the orientation of the virtual blade.

In some implementations, the one or more processors may be configured to detect an object based on the visual output signals. The object may be a physical object or a virtual object. The one or more processors may be configured to determine when the virtual blade touches the object. The one or more processors may be configured to determine the controls for the gyroscope based on the virtual blade touching the object.

In some implementations, the one or more processors may be configured to change the type, the shape and/or the size of the virtual blade. The one or more processors may be configured to determine the controls for the gyroscope based on the changed type, the changed shape and/or the changed size of the virtual blade

The one or more processors may be configured to effectuate operation of the gyroscope in accordance with the controls for the gyroscope. The gyroscope may be operated to provide resistance to changing the orientation of the hilt. The resistance to changing the orientation of the hilt may simulate the feel of moving the virtual weapon.

In some implementations, the one or more processors may be configured to effectuate operation of the speaker. The operation of the speaker may be effectuated in response to changes in the position and/or the orientation of the virtual blade based one or more of the visual output signals and/or the motion and/or orientation output signals.

In some implementations, the one or more processors may be configured to effectuate operation of the haptic generator. The operation of the haptic generator may be effectuated in response to changes in the position and/or the orientation of the virtual blade based one or more of the visual output signals and/or the motion and/or orientation output signals.

DETAILED DESCRIPTION

FIG. 1illustrates a system10configured to simulate a feel of moving a virtual weapon. System10may include one or more of processor11, display12, image sensor13, motion and orientation sensor14, gyroscope15, electronic storage16, bus17, a hilt, and/or other components. The hilt may be configured to carry (e.g., attach to, support, hold, and/or otherwise carry) one or more components of system10. To simulate a feel of moving a virtual weapon, a position and/or an orientation of a virtual blade of the virtual weapon may be determined. The virtual blade may be selected and placed within an overlay image according to the position and/or the orientation of the virtual blade. The overlay image may be displayed so that the virtual blade appears to be attached to the hilt. Controls for gyroscope15may be determined based on the position and/or the orientation of the virtual blade. Gyroscope15may be operated in accordance with the controls for gyroscope15to provide resistance to changing the orientation of hilt. The resistance to changing the orientation of hilt may simulate the feel of moving the virtual weapon. In some implementations, system10may include a speaker. In some implementations, system10may include a haptic generator.

Display12may be configured to display an overlay image. In some implementations, display12may include one or more of a head-mounted display, an optical head-mounted display, a see-through display, an optical see-through display, a video see-through display, a visor, eyeglasses, sunglasses, a smartphone, a tablet, a mobile device, a projector, and/or other displays. In some implementations, processor11and image sensor13may be carried on display12, and the field of view of image sensor13may be a function of the position and the orientation of display12.

Image sensor13may be configured to generate visual output signals conveying visual information within the field of view of image sensor13. Visual information may include one or more of an image, a video, and/or other visual information. When a hilt is within the field of view of image sensor13, visual information may include one or more of an image, a video, and/or other visual information regarding the hilt. Image sensor13may include one or more of a charge-coupled device sensor, an active pixel sensor, a complementary metal-oxide semiconductor sensor, an N-type metal-oxide-semiconductor sensor, and/or other image sensors.

Motion and orientation sensor14may be carried by the hilt. Motion and orientation sensor14may be configured to generate motion and/or orientation output signals conveying motion and/or orientation information of the hilt. Motion and/or orientation information of the hilt may characterize one or more motion and/or orientation of the hilt. Motion of the hilt may include one or more of movement of the hilt, change in position of the hilt, and/or other motion of the hilt at a time or over a period of time. In some implementations, motion of the hilt may include distance between display12and the hilt at a time or over a period of time. Orientation of the hilt may include one or more of yaw, pitch, and/or roll of the hilt, change in yaw, pitch, and/or roll of the hilt, and/or other orientation of hilt at a time or over a period of time.

In some implementations, motion and orientation sensor14may include an inertial measurement unit and/or other motion and orientation sensors. An inertial measurement unit may include one or more of accelerometers, gyroscopes, magnetometers, and/or other motion and orientation sensors. An inertial measurement unit may include one or more of 3-DOF inertial measurement units, 6-DOF inertial measurement units, 9-DOF inertial measurement units, and/or other inertial measurement units. In some implementations, motion and orientation sensor14may include one or more distance sensors, such as infrared distance sensors, Lidar, ultrasonic distance sensors, and/or other distance sensors.

Gyroscope15may be carried by the hilt. In some implementations, gyroscope15may include a first gyroscope carried in a first portion of the hilt and a second gyroscope carried in a second portion of the hilt. The first portion and the second portion may be located in different ends of the hilt. For example,FIG. 3Aillustrates hilt30in a form of a cylinder. Hilt30includes first end31and second end32. Hilt30includes a chevron marking, which points toward the end of first end31. Hilt30includes first gyroscope33located in first end31, and second gyroscope34located in second end32. Other forms of hilt and other arrangements of gyroscopes carried by the hilt are contemplated.

The hilt may have different rotational axes. For example,FIG. 3Billustrates three rotational axes on hilt30. Hilt30includes a normal axis (yaw axis35a) that is perpendicular to the plane formed by other two axes of hilt30. Hilt30includes a lateral axis (pitch axis36a) that runs along the lateral midline of hilt30. Hilt30includes a longitudinal axis (roll axis37a) that runs along the vertical midline of hilt30.FIGS. 3C-3Eillustrate examples of rotations of hilt30around axes shown inFIG. 3B.FIG. 3Cillustrates an example of rotation around yaw axis35a(yaw rotation35b).FIG. 3Dillustrates an example of rotation around pitch axis36a(pitch rotation36b).FIG. 3Eillustrates an example of rotation around roll axis37a(roll rotation37b).

In some implementations, a landmark may be carried by the hilt. The landmark may indicate a reference point for the hilt that facilitates determination of a position and/or an orientation of the virtual blade. In some implementations, the landmark may include an augmented reality marker, a light emitting diode, the entire shape of the hilt, one or more parts of the hilt, and/or other landmarks. An augmented reality marker may be two-dimensional or three-dimensional. As a non-limiting example, an augmented reality marker may include one or more of a sticker, a label, a barcode, a quick response (QR) code, and/or other augmented reality markers. In some implementations, a hilt may include multiples types of landmarks. In some implementations, a landmark may be carried by the hilt as described in U.S. patent application Ser. No. 15/001,160, entitled “SYSTEMS AND METHODS FOR AUGMENTING AN APPEARANCE OF A HILT TO SIMULATE A BLADED WEAPON,” filed Jan. 19, 2016, the foregoing being incorporated herein by reference in its entirety. Other types of landmarks are contemplated.

In some implementations, a hilt may include multiples types of landmarks. For example, hilt40cmay include landmark41c(shown inFIG. 4C) on the front and landmark41d(shown inFIG. 4D) on the back. Other combinations of landmarks are contemplated.

Electronic storage16may include electronic storage media that electronically stores information. Electronic storage16may store software algorithms, information determined by processor11, information received remotely, and/or other information that enables system10to function properly. For example, electronic storage16may store visual information (as discussed elsewhere herein), information relating to virtual blades of virtual weapons, information relating to operation of gyroscope15, and/or other information.

Processor11may be configured to provide information processing capabilities in system10. As such, processor11may comprise one or more of a digital processor, an analog processor, a digital circuit designed to process information, a central processing unit, a graphics processing unit, a microcontroller, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information. Processor11may be configured to execute one or more computer program components. The computer program components may include one or more of position and orientation component20, blade effects component21, overlay component22, display component23, gyroscope control component24, and/or other components.

Position and orientation component20may be configured to determine a position and/or an orientation of a virtual blade of the virtual weapon based on the visual output signals and/or the motion and/or orientation output signals. When the hilt is within the field of view of image sensor13, position and orientation component20may be configured to determine a position and/or an orientation of the virtual blade based on the visual output signals conveying visual information within the field of view of image sensor13.

Position and orientation component20may detect a hilt based on the visual output signals. Position and orientation component20may determine a position and/or an orientation of the hilt. Position and orientation component20may include or retrieve information (for example, a database, etc.) that matches a detected hilt to a position and/or an orientation of a virtual blade. For example, position and orientation component20may determine a position of a virtual blade based on the position of the hilt, and/or an orientation of the virtual blade based on the orientation of the hilt.

For example, inFIG. 5A, position and orientation component20may detect hilt30a. Position and orientation component20may determine a position and/or an orientation of hilt30a. Based on the position and/or the orientation of hilt30a, position and orientation component20may determine a position and/or an orientation of virtual blade50afor hilt30a. As another example, inFIG. 5B, position and orientation component20may detect hilt30b. Position and orientation component20may determine a position and/or an orientation of hilt30b. Based on the position and/or the orientation of hilt30b, position and orientation component20may determine a position and/or an orientation of virtual blade50bfor hilt30b. The position of virtual blade50ainFIG. 5Ais closer to a user than the position of virtual blade50binFIG. 5B. The orientation of virtual blade50ainFIG. 5Ais vertical and the orientation of virtual blade50binFIG. 5Bis tilted to the right.

In some implementations, position and orientation component20may be configured to determine the position and/or the orientation of the virtual blade based on a position and/or an orientation of a landmark. The landmark may be carried by the hilt. The landmark may indicate a reference point for the hilt that facilitates determination of a position and/or an orientation (yaw, pitch, and/or roll) of the virtual blade. Position and orientation component20may include or retrieve information (for example, a database, etc.) that matches a detected landmark to a position and/or an orientation of a virtual blade relative to the position and/or the orientation of the landmark. For example, position and orientation component20may determine the position of a virtual blade based on the position of a landmark, and/or the orientation of the virtual blade based on the orientation of the landmark. In some implementations, the position and/or the orientation of the virtual blade may be determined using systems and methods described in U.S. patent application Ser. No. 15/001,160, entitled “SYSTEMS AND METHODS FOR AUGMENTING AN APPEARANCE OF A HILT TO SIMULATE A BLADED WEAPON,” filed Jan. 19, 2016, incorporated supra.

FIGS. 6A-6Eillustrate non-limiting examples of positions and orientations of virtual blades60determined based on positions and orientations of landmarks41. For example, position and orientation component20may detect landmark41a(shown inFIG. 6A) and determine a position and/or an orientation of landmark41a. Position and orientation component20may determine a position and/or an orientation of virtual blade60abased on the position and/or the orientation of landmark41a. Landmark41amay indicate the position and/or orientation of virtual blade60a. For example, the position of virtual blade60amay be indicated by certain distances from landmark41a(e.g., the right, left, top, and/or bottom positions of virtual blade60aare determined by certain distances to the right, left, above, and/or below landmark41a). The orientation of virtual blade60amay be indicated by the orientation of landmark41a(e.g., the pointed end of landmark41apoints toward virtual blade60a).

As another example, position and orientation component20may detect landmark41b(shown inFIG. 6B) and determine a position and/or an orientation of landmark41b. Position and orientation component20may determine a position and/or an orientation of virtual blade60bbased on the position and/or the orientation of landmark41b. Landmark41bmay indicate the position and/or orientation of virtual blade60b. For example, the position of virtual blade60bmay be indicated by certain distances from landmark41b(e.g., the right, left, top, and/or bottom positions of virtual blade60bare determined by certain distances to the right, left, above, and/or below landmark41b). The orientation of virtual blade60bmay be indicated by the orientation of landmark41b(e.g., the dot-side of landmark31bpoints toward one end of virtual blade60b).

As another example, position and orientation component20may detect both landmark41c(shown inFIG. 4C) and landmark41d(shown inFIG. 4D) on hilt40cand determine positions and/or orientations of landmark41cand landmark41d. Position and orientation component20may determine a position and/or an orientation of virtual blade60c(shown inFIG. 6C) based on the position(s) and/or the orientation(s) of landmark41cand/or landmark41d. Landmark41cand landmark41dmay indicate the position and/or orientation of virtual blade60c. For example, the position of virtual blade60cmay be indicated by certain distances from landmark41cand/or landmark41d(e.g., the right, left, top, and/or bottom positions of virtual blade60care determined by certain distances to the right, left, above, and/or below landmark41cand/or landmark41d). The orientation of virtual blade60cmay be indicated by the orientation of landmark41cand/or landmark41d(e.g., the end of hilt40cclose to landmark41cand/or landmark41dis close to the start of virtual blade60c, the front of virtual blade60cis indicated by landmark41c, and the back of virtual blade60cis indicated by landmark41d).

As another example, position and orientation component20may detect landmark41e(shown inFIG. 4E) and determine a position and/or an orientation of landmark41e. Position and orientation component20may determine a position and/or an orientation of virtual blade60e(shown inFIG. 6D) based on the position and/or the orientation of landmark41e. Landmark41emay indicate the position and/or orientation of virtual blade60e. For example, the position of virtual blade60emay be indicated by certain distances from landmark41e(e.g., the right, left, top, and/or bottom positions of virtual blade60bare determined by certain the right, left, above, and/or bottom positions of landmark41e). The orientation of virtual blade60emay be indicated by the orientation of landmark41e(e.g., the orientation of landmark41eis the orientation of virtual blade60e).

When the hilt is not within the field of view of image sensor13, position and orientation component20may be configured to determine the position and/or the orientation of the virtual blade based on the motion and/or orientation output signals. Position and orientation component20may determine changes in position and/or orientation of the hilt based on the motion and/or orientation output signals. Position and orientation component20may determine the position and/or the orientation of the hilt based on the changes in the position and/or the orientation of the hilt. In some implementations, position and orientation component20may determine the position and/or the orientation of the hilt with respect to display12based on systems and methods described in U.S. patent application Ser. No. 15/001,160, entitled “SYSTEMS AND METHODS FOR AUGMENTING AN APPEARANCE OF A HILT TO SIMULATE A BLADED WEAPON,” filed Jan. 19, 2016, incorporated supra.

In some implementations, position and orientation component20may be configured to determine a position and/or an orientation of a virtual blade based on the visual output signals and based on the motion and/or orientation output signals. For example, position and orientation component20may be configured to determine a position and/or an orientation of the virtual blade based on the visual output signals, and to adjust the position and/or the orientation of the virtual blade based on the motion and/or orientation output signals. Such a determination of the position and/or the orientation of a virtual blade may increase the accuracy of the position and/or the orientation of the virtual blade.

In some implementations, position and orientation component20may be configured to determine a position and/or an orientation of the virtual blade based on a previously determined position and/or a previously determined orientation of the virtual blade, and based on the motion and/or orientation output. For example, position and orientation component20may have determined a position and/or an orientation of the virtual blade, and may then determine a new position and/or a new orientation of the virtual blade based on the motion and/or orientation output signals. Such a determination of the new position and/or the new orientation of a virtual blade may allow position and orientation component20to avoid any latency arising from determining a position and/or an orientation of a virtual blade based on the visual output signals.

In some implementations, position and orientation component20may be configured to determine changes in a position and/or an orientation of a virtual blade. Changes in a position and/or an orientation of a virtual blade may include information regarding changes in a position and/or an orientation of a virtual blade at a time, over a period of time, at a location, or over a range of locations. For example, changes in a position and/or an orientation of a virtual blade may include one or more information regarding change in position, direction of position change, speed of position change, acceleration of position change, change in orientation, direction of orientation change, speed of orientation change, acceleration of orientation change, and/or other information regarding changes in a position and/or an orientation of the virtual blade.

Position and orientation component20may determine changes in a position and/or an orientation of a virtual blade based on the visual output signals and/or the motion and orientation signals. For example, when the hilt is within the field of view of image sensor13, position and orientation component20may determine changes in a position and/or an orientation of a virtual blade based on the visual output signals. As another example, when the hilt is not within the field of view of image sensor13, position and orientation component20may determine changes in a position and/or an orientation of the hilt based on the motion and/or orientation output signals.

In some implementations, position and orientation component20may determine changes in a position and/or an orientation of a virtual blade based on the visual output signals and based on the motion and/or orientation output signals. In some implementations, position and orientation component20may determine changes in a position and/or an orientation of the virtual blade based on a previously determined position and/or a previously determined orientation of the virtual blade, and based on the motion and/or orientation output.

Blade effects component21may be configured to select a virtual blade. A virtual blade may be a blade of a real weapon, such as a broadsword, an axe, or a katana, or a fictional weapon, such as a lightsaber. In some implementations, blade effects component21may be configured to select the virtual blade based on a user input received through an input device. An input device may refer to a device that allows a user to input information. For example, an input device may include a key entry device, a touch entry device, an imaging device, a sound device, and/or other input devices. A user input may refer to one or more information provided by a user through an input device.

A key entry device may include a device that allows a user to provide one or more user inputs by typing one or more of characters, numbers, and/or other symbols. A key entry device may include a separate device or a part of another device. For example, a key entry device may include a keyboard/button coupled to processor11. As another example, a key entry device may include a mobile device coupled to processor11. A user may provide one or more user inputs by typing one or more information. For example, a user may provide one or more user inputs by typing one or more of a type, shape, size, color, and/or other information about the virtual blade.

A touch entry device may include a device that allows a user to provide user inputs by touching a user interface of the touch entry device. A touch entry device may include a separate device or a part of another device. For example, a touch entry device may include a touch screen coupled to processor11. As another example, a touch entry device may include a mobile device coupled to processor11. A user may provide one or more user inputs by touching one or more portions of the touch entry device corresponding to one or more information. For example, a user may provide one or more user inputs by touching one or more portions of the touch entry device corresponding to one or more of a type, shape, size, color, and/or other information about the virtual blade.

An imaging device may include a device that allows a user to provide user inputs by using an image sensor of the imaging device. An imaging device may include a separate device or a part of another device. For example, an imaging device may include an image sensor coupled to processor11. As a non-limiting example, an imaging device may include image sensor13. As another example, an imaging device may include a mobile device coupled to processor11. A user may provide one or more user inputs by directing the field of view of the imaging device to objects that include information. For example, a user may provide one or more user inputs by directing the field of view of the imaging device to a landmark, an augmented reality marker, and/or other objects that include one or more of a type, shape, size, color, and/or other information about the virtual blade.

A sound device may include a device that allows a user to provide user inputs through voice and/or sounds. A sound device may include a separate device or part of another device. For example, a sound device may include a microphone coupled to processor11. As another example, a sound device may include a mobile device coupled to processor11. A user may provide one or more user input by speaking one or more information. For example, a user may provide one or more user inputs by speaking one or more of a type, shape, size, color, and/or other information about the virtual blade.

In some implementations, blade effects component21may be configured to select the virtual blade based on a landmark. Blade effects component21may include or retrieve information (for example, a database, etc.) that matches a landmark to a particular virtual blade. In some implementations, the virtual blade may be selected as described in U.S. patent application Ser. No. 15/001,160, entitled “SYSTEMS AND METHODS FOR AUGMENTING AN APPEARANCE OF A HILT TO SIMULATE A BLADED WEAPON,” filed Jan. 19, 2016, incorporated supra.

FIGS. 6A-6Eillustrate non-limiting examples of virtual blades selected by blade effects component21based on landmarks41. InFIG. 6A, blade effects component21may select virtual blade60abased on landmark41a. Virtual blade60ais cylindrical in shape and appears to extend outwardly from top of hilt40a. InFIG. 6B, blade effects component21may select virtual blade60bbased on landmark41b. Virtual blade60bis cylindrical in shape and appears to extend outwardly from top and bottom of hilt40b.

InFIG. 6C, blade effects component21may select virtual blade60cbased on landmark41cand landmark41d. Virtual blade60cis curved and appears to extend outwardly from top of hilt40c. Virtual blade60ccurves towards the back of hilt40c, with landmark41dindicating the back of hilt40cand landmark41cindicating the front of hilt40c.

InFIG. 6D, blade effects component21may select virtual blade60ebased on landmark41e(shown inFIG. 4E). Virtual blade60eis cylindrical in shape appears to extend outwardly from top of hilt40e. When virtual blade60eappears on top of hilt40e, landmark41emay not be visible. InFIG. 6E, blade effects component21may select virtual blade60fbased on landmark41f(shown inFIG. 4F). Virtual blade60fis cylindrical in shape appears to extend outwardly from top of hilt40f. When virtual blade60fappears on top of hilt40f, landmark41fmay not be visible.

In some implementations, blade effects component21may select virtual blade60e(shown inFIG. 6D) based on landmark41f(shown inFIG. 4F) or landmark41g(shown inFIG. 4G). When virtual blade60eappears on top of hilt40f, landmark41fmay not be visible. Virtual blade60emay be longer than landmark41f. When virtual blade60eappears on top of hilt40g, landmark41gmay not be visible. Virtual blade60emay be larger than landmark41g. Other selections of blades based on landmarks are contemplated.

Overlay component22may be configured to determine an overlay image. The overlay image may include a virtual blade of a virtual weapon determined by blade effects component21. The virtual blade may be placed within the overlay image according to the position and/or the orientation of the virtual blade. The position and/or the orientation of the virtual blade may change how the virtual blade appears within the overlay image. For example, the position and/or the orientation of the virtual blade may change one or more of the position, the size, the shape, the tilt, the rotation, and/or other appearances of the virtual blade.

Display component23may be configured to effectuate displaying of an overlay image on display12. The displaying may be effectuated so that the virtual blade appears to be attached to the hilt. In some implementations, display component23may be configured to effectuate displaying of an overlay image within one or more of an image, a video, and/or other visual information based on the visual output signals generated by image sensor13.

In some implementations, display component23may be configured to effectuate displaying of an overlay image on display12, which allows light to be passed through display12in portions in which the overlay image does not contain the virtual blade. For example, display12may include one or more of an optical head-mounted display and a user of display12may see light from the real world as well as the overlay image. In some implementations, display component23may be configured to change the transparency of one or more portions of display12. For example, display component23may change the transparency of one or more portions of display12corresponding to the virtual blade to block light from the real world passing through display12.

Gyroscope control component24may be configured to determine controls for gyroscope15based on the position and/or the orientation of the virtual blade. The controls for gyroscope15may determine one or more of speed and/or direction of rotation of gyroscope15around one or more axis at a time, over a period of time, at a location, or over a range of locations. For example, the controls for gyroscope15may determine one or more of direction and/or speed of yaw rotation, roll rotation, and/or pitch rotation of gyroscope15.

For example,FIGS. 5A and 5Billustrate examples of virtual blade50in different positions and orientations. ComparingFIGS. 5A and 5B, virtual blade50ais closer to the user than virtual blade50b. Virtual blade50ais oriented vertically while virtual blade50bis tilted to the right. Gyroscope control component24may determine different controls for virtual blade50aand virtual blade50b. As non-limiting examples, gyroscope control component24may determine low speed(s) of rotation around one or more axes for gyroscope15based on the position and/or orientation of virtual blade50a, and high speed(s) of rotation around one or more axes for gyroscope15based on the position and/or orientation of virtual blade50b. Low speed(s) of rotation by gyroscope15may provide low resistance to changing the orientation of hilt30a. High speed(s) of rotation by gyroscope15may provide high resistance to changing the orientation of hilt30b.

The resistance to changing the orientation of hilt30may simulate the feel of moving a virtual weapon. For example, for hilt30a, low resistance to changing the orientation of hilt30amay simulate the feel of moving a virtual lightsaber held close and upright (e.g., to simulate gravity on the virtual blade). As another example, for hilt30b, high resistance to changing the orientation of hilt30bmay simulate the feel of moving a virtual lightsaber held far and tilted (e.g., to simulate gravity on the virtual blade).

In some implementations, gyroscope control component24may be configured to determine the controls for gyroscope15based on changes in a position and/or an orientation of a virtual blade. Changes in a position and/or an orientation of a virtual blade may include information regarding changes in a position and/or an orientation of a virtual blade at a time, over a period of time, at a location, or over a range of locations. For example, changes in a position and/or an orientation of a virtual blade may include one or more information regarding change in position, direction of position change, speed of position change, acceleration of position change, change in orientation, direction of orientation change, speed of orientation change, acceleration of orientation change, and/or other information regarding changes in a position and/or an orientation of the virtual blade.

Gyroscope control component24may determine one or more of speed and/or direction of rotation of gyroscope15around one or more axis based on the changes in the position and/or the orientation of the virtual blade. For example, the position and orientation of virtual blade50may change from the position and orientation shown inFIG. 5Ato the position and orientation shown in andFIG. 5B. Based on the changes in the position and the orientation of virtual blade50, gyroscope control component24may increase the speed of rotation around one or more axes by gyroscope15.

As another example,FIGS. 7A and 7Billustrate examples of rotations of virtual blade71. InFIG. 7A, virtual blade71a, attached to hilt70a, may be rotated downwards. Based on virtual blade71adownward rotation, gyroscope control component24may determine low speed(s) of rotation around one or more axes for gyroscope15. Low speed(s) of rotation by gyroscope15may provide low resistance to changing the orientation of hilt70adownwards and may simulate the feel of moving the virtual weapon downwards (e.g., to simulate downward movement assisted by gravity).

InFIG. 7B, virtual blade71b, attached to hilt70b, may be rotated upwards and then downwards. When virtual blade71bis being rotated upwards, gyroscope control component24may determine high speed(s) of rotation around one or more axes for gyroscope15. High speed(s) of rotation by gyroscope15may provide high resistance to changing the orientation of hilt70bupwards and may simulate the feel of moving the virtual weapon upwards (e.g., to simulate upward movement opposed by gravity).

When virtual blade71breaches the top and begins to rotate downwards, gyroscope control component24may determine low speed(s) of rotation around one or more axes for gyroscope15. Low speed(s) of rotation by gyroscope15may provide low resistance to changing the orientation of hilt70bdownwards and may simulate the feel of moving the virtual weapon downwards (e.g., to simulate downward movement assisted by gravity).

In some implementations, system10may be configured to determine the controls for gyroscope15to simulate changes in a virtual blade. Blade effects component21may be configured to change the type, the shape and/or the size of the virtual blade. For example, as shown inFIGS. 8A and 8B, blade effects component21may change virtual blade81for hilt80. InFIG. 8A, virtual blade81aappears as a blade of an axe. InFIG. 8B, virtual blade81bappears as a short cylinder, simulating the top portion of the axe blade having broken off. Other types of changes in a virtual blade are contemplated.

Gyroscope control component24may be configured to determine the controls for gyroscope15based on the changed type, the changed shape and/or the changed size of the virtual blade. For example, inFIG. 8A, gyroscope control component24may determine high speed(s) of rotation around one or more axes for gyroscope15. High speed(s) of rotation by gyroscope15may provide high resistance to changing the orientation of hilt80and may simulate the feel of moving a heavy virtual weapon (e.g., to simulate heavy blade of an axe). InFIG. 8B, gyroscope control component24may determine low speed(s) of rotation around one or more axes for gyroscope15. Low speed(s) of rotation by gyroscope15may provide low resistance to changing the orientation of hilt80and may simulate the feel of moving a light virtual weapon (e.g., to simulate heavy blade of an axe being broken off).

In some implementations, system10may be configured to determine the controls for gyroscope15to simulate interactions between the virtual blade and an object. An object may be a physical object or a virtual object. A physical object may refer to an object in the real world. A physical object may include a static object (e.g., a still door, a wall, a rock, etc.) or a dynamic object (e.g., a moving door, a moving target, etc.). A virtual object may refer to an object generated by a computing device, such as processor11. A virtual object may include a static object (e.g., a virtual barrier, a still virtual blade of another hilt, etc.) or a dynamic object (e.g., a moving virtual blade of another hilt, a virtual blaster fire, a virtual moving robot, etc.).

Position and orientation component20may be configured to detect an object based on the visual output signals. Position and orientation component20may be configured to determine when the virtual blade touches the object. Gyroscope control component24may be configured to determine the controls for gyroscope15based on the virtual blade touching the object.

Position and orientation component20may determine a position and/or an orientation of the object based on the visual output signals conveying visual information within the field of view of image sensor13. Position and orientation component20may determine when the virtual blade touches the object based on the position and/or the orientation of the virtual blade, and the position and/or the orientation of the object. Other implementations to determine when the virtual blade touches the objects are contemplated.

Gyroscope control component24may determine controls for gyroscope15based on the virtual blade touching the object. Gyroscope control component24may determine one or more of speed and/or direction of rotation of gyroscope15around one or more axis based on the virtual blade touching the object. For example,FIG. 9shows virtual blade touching virtual barrier90. When virtual blade50is touching virtual barrier90, gyroscope control component24may determine high speed(s) of rotation around one or more axes for gyroscope15. High speed(s) of rotation by gyroscope15may provide high resistance to changing the orientation of hilt30and may simulate the feel of moving a virtual weapon into an object (e.g., to simulate virtual blade50slicing into virtual barrier90).

When virtual blade50is not touching virtual barrier90(e.g., before or after virtual blade50slices into virtual barrier90, or when virtual blade50is in gaps in virtual barrier90), gyroscope control component24may determine low speed(s) of rotation around one or more axes for gyroscope15. Low speed(s) of rotation by gyroscope15may provide low resistance to changing the orientation of hilt30and may simulate the feel of moving a virtual weapon in the air.

As another example,FIG. 10shows virtual blade50touching door100. InFIG. 10, virtual blade50is pushed into door100and moved in a circular shape. When virtual blade50touches door, gyroscope control component24may determine high speed(s) of rotation around one or more axes for gyroscope15. High speed(s) of rotation by gyroscope15may provide high resistance to changing the orientation of hilt30and may simulate the feel of moving a virtual weapon into an object (e.g., to simulate virtual blade50slicing into door100).

Gyroscope control component24may be configured to effectuate operation of gyroscope15in accordance with the controls for gyroscope15. Gyroscope control component24may effectuate speed and/or direction of rotation of gyroscope15around one or more axis. Gyroscope15may be operated to provide resistance to changing the orientation of the hilt. The resistance to changing the orientation of the hilt may simulate the feel of moving the virtual weapon, as described above.

In some implementations, system10may include a speaker. Blade effects component21may be configured to effectuate operation of the speaker. The operation of the speaker may be effectuated in response to changes in the position and/or the orientation of the virtual blade based one or more of the visual output signals and/or the motion and/or orientation output signals. The speaker may be operated to provide one or more audio effects. An audio effect may refer to one or more information that may be observed audibly. An audio effect may be static or dynamic, and may be audibly observable at a time, over a period of time, at a location, or over a range of locations. An audio effect may include one or more of a sound, a music, a word, a sentence, and/or other audio effect.

For example, blade effects component21may have selected a blade of a lightsaber as the virtual blade for a hilt. When the virtual blade is motionless, blade effects component21may effectuate operation of the speaker to produce a buzzing/humming sound of a motionless lightsaber. When the virtual blade is moving, blade effects component21may effectuate operation of the speaker to produce a “wah” sound of a moving lightsaber. Blade effects component21may be configured to change the intensity and/or length of the sound based on the movement of the virtual blade.

In some implementations, blade effects component21may be configured to effectuate operation of the speaker based on the virtual blade touching an object. For example, inFIG. 9, blade effects component21may effectuate the operation of the speaker to produce a cutting sound when virtual blade50is pushed into virtual barrier90. Other types of audio effects are contemplated.

In some implementations, system10may include a haptic generator. Blade effects component21may be configured to effectuate operation of the haptic generator in response to changes in the position and/or the orientation of the virtual blade based one or more of the visual output signals and/or the motion and/or orientation output signals. The haptic generator may be operated to provide one or more haptic effects. A haptic effect may refer to one or more information that may be observed haptically. A haptic effect may be static or dynamic, and may be haptically observable at a time, over a period of time, at a location, or over a range of locations. A haptic effect may include one or more of a vibration, a motion, a temperature, and/or other haptic effects.

For example, blade effects component21may have selected a blade of a lightsaber as the virtual blade for a hilt. Blade effects component21may effectuate operation of the haptic generator to produce a light vibration of a lightsaber. Blade effects component21may be configured to change the intensity and/or length of the vibration based on the movement of the virtual blade.

In some implementations, blade effects component21may be configured to effectuate operation of the haptic generator based on the virtual blade touching the object. For example, inFIG. 9, blade effects component21may effectuate the operation of the haptic generator to produce heat when virtual blade50is pushed into virtual barrier90. Other types of haptic effects are contemplated.

Although processor11, display12, image sensor13, motion and orientation sensor14, gyroscope15, and electronic storage16are shown to be connected to a bus17inFIG. 1, any communication medium may be used to facilitate interaction between any components of system10. One or more components of system10may communicate with each other through hard-wired communication, wireless communication, or both. For example, processor11may wirelessly communicate with motion and orientation sensor14. By way of non-limiting example, wireless communication may include one or more of radio communication, Bluetooth communication, Wi-Fi communication, cellular communication, infrared communication, or other wireless communication. Other types of communications are contemplated by the present disclosure.

Although processor11is shown inFIG. 1as a single entity, this is for illustrative purposes only. In some implementations, processor11may comprise a plurality of processing units. These processing units may be physically located within the same device, or processor11may represent processing functionality of a plurality of devices operating in coordination.

Processor11may be configured to execute one or more of position and orientation component20, blade effects component21, overlay component22, display component23, gyroscope control component24, and/or other components by software; hardware; firmware; some combination of software, hardware, and/or firmware; and/or other mechanisms for configuring processing capabilities on processor11.

It should be appreciated that although position and orientation component20, blade effects component21, overlay component22, display component23, and gyroscope control component24are illustrated inFIG. 1as being co-located within a single processing unit, in implementations in which processor11comprises multiple processing units, one or more of position and orientation component20, blade effects component21, overlay component22, display component23and/or gyroscope control component24may be located remotely from the other computer program components.

The description of the functionality provided by the different computer program components20,21,22,23, and/or24described herein is for illustrative purposes, and is not intended to be limiting, as any of computer program components20,21,22,23, and/or24may provide more or less functionality than is described. For example, one or more of computer program components20,21,22,23, and/or24may be eliminated, and some or all of its functionality may be provided by other computer program components20,21,22,23, and/or24. As another example, processor11may be configured to execute one or more additional computer program components that may perform some or all of the functionality attributed to one or more of computer program components20,21,22,23, and/or24.

Although display12is depicted inFIG. 1as a single element, this is not intended to be limiting. Display12may include one or more displays in one or more locations.

Although image sensor13is depicted inFIG. 1as a single element, this is not intended to be limiting. Image sensor13may include one or more image sensors in one or more locations.

Although motion and orientation sensor14is depicted inFIG. 1as single elements, this is not intended to be limiting. Motion and orientation sensor14may include one or more motion and orientation sensors in one or more locations.

Although gyroscope15is depicted inFIG. 1as single elements, this is not intended to be limiting. Gyroscope15may include one or more gyroscopes in one or more locations.

The electronic storage media of electronic storage16may be provided integrally (i.e., substantially non-removable) with one or more components of system10and/or removable storage that is connectable to one or more components of system10via, for example, a port (e.g., a USB port, a Firewire port, etc.) or a drive (e.g., a disk drive, etc.). Electronic storage16may include one or more of optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), electrical charge-based storage media (e.g., EPROM, EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.), and/or other electronically readable storage media. Electronic storage16may be a separate component within system10, or electronic storage16may be provided integrally with one or more other components of system10(e.g., processor11). Although electronic storage16is shown inFIG. 1as a single entity, this is for illustrative purposes only. In some implementations, electronic storage16may comprise a plurality of storage units. These storage units may be physically located within the same device, or electronic storage16may represent storage functionality of a plurality of devices operating in coordination.

FIG. 2illustrates method200for simulating a feel of moving a virtual weapon. The operations of method200presented below are intended to be illustrative. In some implementations, method200may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. In some implementations, two or more of the operations may occur substantially simultaneously.

Referring toFIG. 2and method200, at operation201, visual output signals conveying visual information within a field of view of an image sensor may be generated. Visual information may include one or more of an image, a video, and/or other visual information. In some implementations, operation201may be performed by one or more sensors the same as or similar to image sensor13(shown inFIG. 1and described herein).

At operation202, motion and/or orientation output signals conveying motion and/or orientation information of a hilt may be generated. In some implementations, operation202may be performed by one or more sensors the same as or similar to motion and orientation sensor14(shown inFIG. 1and described herein).

At operation203, a position and/or an orientation of a virtual blade of a virtual weapon may be determined based on the visual output signals and/or the motion and/or orientation output signals. In some implementations, operation203may be performed by a processor component the same as or similar to position and orientation component20(shown inFIG. 1and described herein).

At operation204, an overlay image comprising the virtual blade of the virtual weapon may be determined. The virtual blade may be placed within the overly image according to the position and/or the orientation of the virtual blade. In some implementations, operation204may be performed by a processor component the same as or similar to overlay component22(shown inFIG. 1and described herein).

At operation205, displaying of the overlay image on a display may be effectuated so that the virtual blade appear to be attached to the hilt. In some implementations, operation205may be performed by a processor component the same as or similar to display component23(shown inFIG. 1and described herein).

At operation206, controls for a gyroscope may be determined based on the position and/or the orientation of the virtual blade. In some implementations, operation206may be performed by a processor component the same as or similar to gyroscope control component24(shown inFIG. 1and described herein).

At operation207, operation of the gyroscope may be effectuated in accordance with the controls for the gyroscope to provide resistance to changing the orientation of the hilt. The resistance to changing the orientation of the hilt may simulate the feel of moving the virtual weapon. In some implementations, operation207may be performed by a processor component the same as or similar to gyroscope control component24(shown inFIG. 1and described herein).