INERTIAL AND OPTICAL MOTION SENSING DEVICE INTEGRATED INTO SPORTS EQUIPMENT

A motion sensing device with both inertial sensors and optical elements that integrates into a piece of sports equipment. Equipment with the device installed may look and feel like the original equipment. For example, for a baseball bat, a sensor package with a housing containing inertial sensors, cameras, and lights may be integrated into a knob that can be swapped with a standard bat knob; an additional sensor package may be integrated into a replacement bat cap. Sensor data and images from cameras in the device may be transmitted to a processor that calculates the equipment trajectory. External cameras may also capture images of lights in the device, which may flash or change in a pattern that is synchronized with the sensor data, and these images may also be used in trajectory calculations. Equipment with the motion sensing device may be used for virtual reality or for real sport activities.

BACKGROUND OF THE INVENTION

Field of the Invention

One or more embodiments of the invention are related to the fields of motion sensing devices and sensor data analysis. More particularly, but not by way of limitation, one or more embodiments of the invention enable an inertial and optical motion sensing device integrated into sports equipment.

Description of the Related Art

Virtual reality headsets and controllers enable users to have virtual experiences such as games or to practice activities in a virtual environment. Virtual reality experiences may include simulations of sporting events or games, or sport practice or training sessions. A limitation of existing virtual reality systems is that generic virtual reality controllers (typically held in the hands) are used to simulate a piece of sports equipment, such as a baseball bat or a golf club. Although a virtual reality headset can display an image of the sports equipment, a controller does not feel at all like the equipment, since its weight, size, and shape do not match the real equipment. Some virtual reality applications have attempted to address this limitation by attaching a virtual reality controller to a piece of sports equipment; these attachments are typically clumsy and bulky, they change the feel of the equipment significantly, and they are not sufficiently rigid to provide good tracking of rapid motions of the sports equipment.

For at least the limitations described above there is a need for an inertial and optical motion sensing device integrated into sports equipment.

BRIEF SUMMARY OF THE INVENTION

One or more embodiments described in the specification are related to an inertial and optical motion sensing device integrated into sports equipment. Embodiments of the invention may include sensor packages that may be integrated into or attached to sports equipment to enable tracking of the equipment's motion.

In one or more embodiments of the invention, an inertial and optical motion sensing device may have one or more housings, each of which may be coupled to or integrated into a piece of sports equipment. It may have control electronics contained within at least one of the housings. The control electronics may include a microprocessor, a wireless communications interface coupled to the microprocessor, and a memory coupled to the microprocessor. It may have motion sensing electronics contained with at least one of the housings. The motion sensing electronics may include a three-axis accelerometer and a three-axis gyroscope, each coupled to the microprocessor by a wired or wireless communication link. It may have multiple optical elements contained within at least one of the housings. Each optical element may be either a light or a camera, and it may be proximal to a region of its corresponding housing that is open or is transparent to a wavelength associated with the optical element. Each optical element may be coupled to the microprocessor by a wired or wireless communication link. The microprocessor may be configured to capture sensor data from the three-axis accelerometer and the three-axis gyroscope during a motion of the piece of sports equipment. When the optical elements include one or more cameras, the microprocessor may obtain images from the cameras during this motion and add these images to the sensor data. When the optical elements include one or more lights, the microprocessor may command the lights to emit light patterns during the motion and to synchronize the light patterns with the sensor data. The microprocessor may transmit the sensor data (which may include images) over the wireless communications interface to a processor that is configured to analyze the sensor data to calculate a trajectory of the piece of sports equipment.

In one or more embodiments, each housing may be located at or near a corresponding end of the longitudinal axis of the piece of sports equipment.

In one or more embodiments, coupling or integrating the one or more housings into the piece of sports equipment may not change the length of the longitudinal axis by more than 10%, and may not change the maximum diameter of the piece of sports equipment around this longitudinal axis by more than 10%.

In one or more embodiments of the invention, the piece of sports equipment may be a bat, and a first housing of the one or more housings may be coupled to or integrated into a knob of the bat. In one or more embodiments, a second housing of the one or more housing may be coupled to or integrated into a cap of the bat at an opposite end of the longitudinal axis from the knob.

In one or more embodiments, the piece of sports equipment may be a golf club, and a first housing of the one or more housings may be coupled to or integrated into the grip of the golf club.

In one or more embodiments of the invention, the optical elements may include one or more lights, and the processor may be configured to obtain one or more scene images from one or more external cameras that view the piece of equipment during at least a portion of the motion of the equipment. The processor may analyze the scene images to determine the location of the lights in the scene images, synchronize these light locations with the sensor data, and analyze the light locations and the sensor data to calculate the trajectory of the piece of sports equipment.

In one or more embodiments, the processor and the external cameras may be integrated into a virtual reality headset. The virtual reality headset may be configured to display a representation of the piece of equipment on a display of the virtual reality headset, and to update this representation over time according to the trajectory of the equipment.

In one or more embodiments, the optical elements may include three or more lights located with at least one of the housings. In one or more embodiments, the optical elements may include three or more cameras located with at least one of the housings.

DETAILED DESCRIPTION OF THE INVENTION

One or more embodiments of the invention may include motion sensing electronics that may be integrated into or attached to a piece of sporting equipment. The motion sensing electronics may measure or enable measurement of any attribute of motion, such as position, orientation, speed, angular velocity, acceleration, or angular acceleration, either directly or indirectly.FIG.1shows an embodiment of the invention integrated into a baseball bat. Embodiments of the invention may be integrated into any type of equipment, used for sports activities or any other type of activity. Baseball bat101is a normal baseball bat without any electronics. This bat has a longitudinal axis102that extends from the bottom of the bat knob104to the top of the bat end cap105. For many bats, the knob104and the end cap105are separate pieces that are attached to the main portion of the bat. These pieces may be removeable and replaceable. One or more embodiments of the invention may therefore use a bat knob and/or a bat cap that include motion sensing electronics; these components may be installed into a normal bat as replacements for the existing or standard bat knob and/or cap. In one or more embodiments the existing bat knob and/or cap may be retained, and electronic components may be integrated into or attached to the existing knob and/or cap.

In the example shown inFIG.1, bat101is equipped with motion sensing electronics by replacing original bat end cap105with cap115, and replacing original bat knob104with knob114. In one or more embodiments only a portion of the knob or end cap may be replaced or altered. In one or more embodiments, either or both of the knob and the end cap may be replaced or altered. Either or both of the replaced (or altered) items may have a housing that contains electronic components that enable tracking of the motion of the bat. Illustrative components in bat knob114may include for example, without limitation, motion sensing electronics such as inertial motion sensors accelerometer121(which may have three axes in one or more embodiments) and gyroscope122(which may have three axes in one or more embodiments), optical elements such as camera or cameras123and light or lights124. Some of the optical elements such as lights124may not track motion directly but may instead enable other elements such as external cameras to track the motion of the sports equipment. Knob114may also contain control electronics components such as a microprocessor125that is coupled to a memory126and to a wireless communications interface127. Microprocessor125may also be coupled to the sensors121and122, to cameras123, and to lights124, by wired or wireless communications links. The microprocess125may obtain data from the sensors121,122, and123during a motion of the bat, and it may transmit this sensor data over the wireless communications interface127to a processor that analyzes the data to determine the trajectory of the bat. Microprocessor125may also control the lights124, and it may for example control the lights to emit light patterns during motion (such as flashes or changes in intensities or colors), so that the light patterns are synchronized with the sensor data samples. External cameras observing the lights may therefore by synchronized with the sensor data stream using the known timing of light patterns relative to the sensor data samples.

In one or more embodiments, cap115may also contain electronic components that may provide data that can be used to track the motion of bat101. Illustrative components131through137may be identical to or similar to the components121through127contained within knob114. In one or more embodiments a cap115may contain only a subset of these components. In embodiments with both a knob and a cap, the electronics in the knob and cap may communicate with one another, or they may act independently and may both transmit their own data to another processor for analysis. In one or more embodiments, only one of the knob and the cap may have a microprocessor and a communications interface, and the sensors in the other may be connected to this single microprocessor. This connection between the knob components and the cap components (when both exist) may be wireless or wired. For wired connections, in one or more embodiments one or more connections140may be routed for example through a hollow core of the bat. Other components may be placed in the hollow core of the bat, such as a power source (battery)141, or vibration actuators that may be controlled to provide haptic feedback to a user.

The knob114and cap115may be configured to have similar size, shape, or appearance to the original knob104and cap105. A benefit of one or more embodiments of the invention is that the bat101(or other equipment) with the installed motion sensing components looks and feels like a real bat, rather than looking and feeling like a separate virtual reality controller or a hybrid with a controller attached to a bat. The dimensions of the bat with the replaced components114and115may be identical to or similar to those of the bat101with original components104and105. For example, the length112along the longitudinal axis of the bat with the additional motion sensing electronics may differ from the length of the original equipment along the longitudinal axis102by 10% or less. The maximum diameter113of the bat around (perpendicular to) the longitudinal axis with the additional motion sensing electronics may also differ from the maximum diameter103of the original equipment by 10% or less. Because the length112and maximum diameter113may be similar to (or identical to) the length and diameter of the original equipment, the user may largely not notice the added electronics, and may swing the bat with a very similar feel to that of the original bat. In one or more embodiments, added electronics and their housings may also be configured to have a minimal effect on other attributes of the bat, such as weight, center of mass, moment of inertia, or air resistance.

FIG.1illustrates an embodiment of the invention configured for a baseball bat. One or more embodiments of the invention may be configured to add motion sensing capabilities to any type of equipment, including for example, without limitation, any type of sports equipment such as a bat, club, racket, paddle, ball, or glove. One or more embodiments of the invention may add motion sensing electronics to equipment used for activities other than sports, such as medical or rehabilitation equipment. The motion sensing electronics may be integrated into or attached to the equipment in a manner that minimizes the effect on the appearance or feel of the equipment, so that a user of the equipment has an experience that does not change substantially when the motion sensing electronics is added. The electronics may be contained in one or more housings that are integrated into or attached to the equipment. Attachments may be relatively rigid, so that motion of the sensors in the housings is closely correlated with the motion of the equipment. Housings may also provide environment protection and shock isolation for the contained components.

FIG.2shows an illustrative embodiment of the invention integrated into a golf club201. Like a bat, a golf club has a longitudinal axis202, and it has a grip204at one end of this axis. Motion sensing electronics package214may be integrated into a replacement grip (or into a module that attaches to or integrates into an existing grip). This electronics package may have a housing that contains any or all of the components221through227, similar to the components121through127of the baseball bat knob package ofFIG.1. The length212of the golf club with the grip motion sensing package214may be identical to or similar to the length of the original golf club201along its longitudinal axis202. The maximum diameter203of the golf club around the longitudinal axis may not change, since the grip is small compared to the clubface. The user of the club therefore may notice very little or any difference when using the golf club with the motion sensing electronics package214. In this illustrative example, the golf club has a single motion sensing package214at only one end of the longitudinal axis; in other embodiments there may be additional housings containing other components, such as another package at or near the clubhead, at the opposite end of the longitudinal axis from the motion sensing package214.

FIGS.3through7show illustrative layouts for a baseball bat knob and end cap that contain motion sensing electronics.FIG.3shows bat300with the knob and cap electronics packages installed, and an exploded view with the bat body301separated from the knob package314and the cap package315. The illustrative bat body301is for example a metal bat with a hollow core. The illustrative knob314slips over or into the bottom end301bof the bat body; it may be attached using for example adhesive, a screw, or other fasteners. The illustrative cap315fits over or into the top end301aof the bat body; it may be attached using for example adhesive, a screw, or other fasteners. Knob and cap components may attach to a bat body in any desired manner and may be secured to the body with any type of attachment.

FIG.4shows the bat knob package314, with an exploded view that shows an outer ring401that covers electronics in the inner knob package410. The inner knob package may for example contain a circular arrangement of optical elements such as cameras and/or lights, surrounded by the outer ring401. The outer ring401may have holes or transparent regions near (in front of) the cameras or lights. For example, inner knob package410has three cameras412a,412b, and412c(camera412cis hidden from view inFIG.4). Outer ring401has three corresponding holes402a,402b, and402cthrough which the cameras can view the environment. Inner knob package410also has lights (such as LEDs) arrayed around the ring; embodiments may have any number of lights in any arrangement. For example, LEDs413a,413b, and413care visible inFIG.4(other LEDs are hidden from view). The lights may emit light of any wavelengths, including for example infrared. The portions of outer ring401in front of the lights may be open or may be transparent to the wavelengths emitted by the light. For example, region403of outer ring401may be made of a material through which light emitted by LEDs413aand413bcan shine. Similarly cameras412a,412b, and412cmay capture images in any wavelengths, and the outer ring in front of these cameras may be open or may be transparent to these wavelengths.

In one or more embodiments, knob314may also have one or more connectors such as a USB connector414. This connector may be used for battery charging, for transmission of data to or from the enclosed electronics, or both.

FIG.5shows an exploded view of inner knob package410. A sleeve501may have an inner hole that slips onto (or into) the bottom end of the bat, and it may have one or more features onto or into which electronic components511may be mounted.FIG.6shows an exploded view of the electronic components511of the bat knob package. The electronics include a ring602that mounts on the outside of the sleeve501, and inner components601that fit inside the sleeve501. The inner components may include for example a battery611and a circuit board612onto which inertial sensors, the microprocessor, the memory, and the communications interface are mounted. The ring602may have a supporting structure621onto which are mounted various cameras and/or lights, of any number and in any geometrical arrangement. In this example the cameras412a,412b, and412care mounted approximately 120 degrees apart. Lights413athrough413hare mounted at different locations around the ring621. By mounting three or more lights around the ring, one or more lights may be visible to external cameras regardless of the orientation of the bat. One or more connections (not shown) may be made between the inner electronics601and the outer components602, for example to supply power from the battery and to transmit data and commands to and from the microprocessor.

FIG.7shows an exploded view of an illustrative bat end cap package315. In this illustrative embodiment, the end cap package contains only cameras and lights; it does not include control electronics such as a microprocessor, or inertial sensors such as an accelerometer or a gyroscope. In other embodiments, the end cap package may contain any additional components such as a microprocessor or inertial sensors. The package315has a cap shell701, which fits around the end of the bat, and an enclosed inner ring702onto which cameras and/or lights are mounted. The configuration of the lights and cameras may be similar to that of the bat knob, although the number and arrangement of lights may be different so that views of the cap from external cameras can be distinguished from views of the knob.FIG.7shows illustrative cameras712a,712b, and712c, and illustrative LEDs713a,713b, and713c; other LEDs are hidden from view. The cap shell may have holes or transparent areas for the cameras and/or lights.

In one or more embodiments of the invention, data obtained from the motion sensors (such as accelerometer, gyroscope, and camera(s)) may be transmitted to an external processor for analysis. This processor may analyze the data to determine the trajectory of the piece of equipment in which the electronics package or packages are integrated or attached. This trajectory may for example include the position in three-dimensional space of any point of the equipment as a function of time. It may be of interest to calculate this trajectory for a movement of the equipment corresponding to a particular sports activity, such as the swing of a baseball bat or of a golf club.FIG.8illustrates this trajectory calculation for a swing of baseball bat300with attached bat knob314and bat end cap315that contain motion sensing electronics. Sensor data801is collected during the swing; this data may include acceleration values802and angular velocity values803at different points in time during the swing. Data may be sampled at any desired sampling rate, such as once per millisecond for example. This motion sensor data801may be transmitted to processor810for analysis, for example using the wireless communications interface in the knob and/or cap. The sensor data may also include images captured by one or more of the cameras in the knob and/or cap. These images may be sampled at any desired rates, and the sampling rate for images may not correspond to the sampling rate for inertial sensor data. Illustrative sensor data table801contains two images804aand804bcaptured at two different points in time from two different cameras. These images show views of the environment from the viewpoint of the bat knob or bat end cap. In one or more embodiments with lights in the knob and/or cap, additional tracking data may be available from external cameras oriented to view the scene of the equipment during its motion; these cameras may for example observe the lights of the knob and/or endcap during at least part of the equipment's motion. (For some motions, some of the lights may be occluded during some portions of the motion; however, the processor can use other sensor information during these periods of occlusion to calculate the trajectory throughout the motion.) Since patterns of lights such as flashing or intensity or color timing may be synchronized with the motion sensor samples, scene images of the lights may be correlated with the other sensor data to help determine the position and orientation of the bat at each point in time. Illustrative external camera811acaptures scene image812athat shows the bat end cap with two active lights, and illustrative external camera811bcaptures scene image812bthat shows the bat knob with one active light. The processor may analyze these images812aand812bto determine the locations of the lights in each image. These light locations may be synchronized with the sensor data. All of the data801(which includes inertial sensor data and camera images from cameras on the bat) and the synchronized light location data from external camera scene images812aand812b(possibly at different points in time) are transmitted to processor810, which performs analyses815to calculate the equipment trajectory816.

In one or more embodiments of the invention, equipment with one or more inertial and optical sensor packages may be used as virtual reality controllers, for example for games that simulate the sport that the equipment is used for. Processor810and external cameras811aand811bmay be integrated into a virtual reality headset, for example, and the headset may display a representation of the equipment on the headset display, using the calculated trajectory816to update this equipment representation over time. The trajectory816may also be used to determine virtual reality actions and events, such as determining whether a swung bat hits a virtual ball in a virtual pitching and hitting game.