Correcting distortions

A system comprising: a magnetic transmitter configured to generate magnetic fields; a magnetic sensor configured to generate signals based on characteristics of the magnetic fields; and one or more computer systems configured to: cause the magnetic transmitter to generate a first plurality of magnetic fields at a first frequency; receive a first plurality of signals from the magnetic sensor; determine data indicative of a position and orientation of the magnetic sensor at a first position of the magnetic sensor; determine a distortion term that corresponds to a first position of the magnetic sensor; cause the magnetic transmitter to generate a third plurality of magnetic fields at the first frequency; receive a third plurality of signals from the magnetic sensor; and determine a second position and orientation of the magnetic sensor relative to the magnetic transmitter, wherein the first frequency is greater than the second frequency.

TECHNICAL FIELD

This disclosure relates to correcting distortions, for example, correcting distortions in an Electromagnetic Tracking (EMT) system.

BACKGROUND

Augmented Reality (AR) and Virtual Reality (VR) systems can use Electromagnetic Tracking (EMT) systems to aid location of devices in various contexts (e.g., gaming, medical, etc.). Such systems utilize a magnetic transmitter in proximity to a magnetic sensor such that the sensor and the transmitter can be spatially located relative to each other. Improper calibration of the transmitter with respect to the sensor (or vice versa) can cause the EMT system to report incorrect positions for the sensor or transmitter.

SUMMARY

Electromagnetic Tracking (EMT) systems, including those that are employed as part of an Augmented Reality (AR) and/or a Virtual Reality (VR) system, can employ one or more techniques for improving the determination of the position and orientation of a magnetic sensor relative to a magnetic transmitter. For example, one or more techniques may be employed to reduce/eliminate positional errors caused by distortions in the tracking environment (e.g., due to the presence of metallic and/or magnetic object at or near the tracking environment).

To ensure that the transmitter and sensor can provide accurate position and orientation measurements to the user, such distortions can be compensated for in the system. For example, one or more terms indicative of distortion in the tracking environment (e.g., a distortion term) can be determined, and future measurements provided by the sensor can be corrected using the one or more distortion terms.

In general, in an aspect, a system includes a magnetic transmitter configured to generate magnetic fields; a magnetic sensor configured to generate signals based on characteristics of the magnetic fields received at the magnetic sensor; and one or more computer systems configured to: cause the magnetic transmitter to generate a first plurality of magnetic fields at a first frequency; receive a first plurality of signals from the magnetic sensor; determine data indicative of a position and orientation of the magnetic sensor at a first position of the magnetic sensor; determine, based on the first plurality of signals and the data indicative of the position and orientation of the magnetic sensor at the first position, a distortion term that corresponds to a first position of the magnetic sensor; cause the magnetic transmitter to generate a third plurality of magnetic fields at the first frequency; receive a third plurality of signals from the magnetic sensor; and determine, based on the third plurality of signals received from the magnetic sensor and the distortion term, a second position and orientation of the magnetic sensor relative to the magnetic transmitter, wherein the first frequency is greater than the second frequency.

Implementations can include one or more of the following features in any combination.

In some implementations, determining data indicative of a position and orientation of the magnetic sensor at a first position of the magnetic sensor comprises: causing the magnetic transmitter to generate a second plurality of magnetic fields at a second frequency; and receiving a second plurality of signals from the magnetic sensor.

In some implementations, determining data indicative of a position and orientation of the magnetic sensor at a first position of the magnetic sensor comprises: obtaining optical data related to the position and orientation of the magnetic sensor at a first position using an optical system; and determining the data indicative of the position and orientation of the magnetic sensor at the first position based on the optical data.

In some implementations, the first and second plurality of magnetic fields are generated when the magnetic transmitter remains at a first position and a first orientation, and the first and second plurality of signals are generated by the magnetic sensor while the magnetic sensor remains at the first position and a first orientation.

In some implementations, the first plurality of signals is represented as a first 3×3 matrix of data, the second plurality of signals is represented as a second 3×3 matrix of data, and the distortion term is represented as a 3×3 matrix of data.

In some implementations, the 3×3 matrix of data corresponding to the distortion term is calculated at least in part by subtracting the second 3×3 matrix of data from the first 3×3 matrix of data.

In some implementations, the magnetic transmitter and the magnetic sensor are each associated with an inertial measurement unit (IMU) configured to provide inertial data.

In some implementations, the 3×3 matrix of data corresponding to the distortion term is calculated at least in part by multiplying the difference between the first 3×3 matrix of data and the second 3×3 matrix of data by inertial data of the magnetic transmitter and inertial data of the magnetic sensor obtained while the magnetic transmitter remains at a first position and a first orientation and the magnetic sensor remains at the first position and a first orientation.

In some implementations, multiplying the difference between the first 3×3 matrix of data and the second 3×3 matrix of data by the inertial data while the magnetic transmitter and the magnetic sensor remain at their respective first positions and orientations results in the 3×3 matrix of data corresponding to the distortion term to be rotated into an initial reference frame that corresponds to the first orientation of the magnetic transmitter and the first orientation of the magnetic sensor.

In some implementations, the 3×3 matrix of data corresponding to the distortion term at the initial reference frame is multiplied by inertial data of the magnetic transmitter and inertial data of the magnetic sensor obtained when the magnetic transmitter is at a second position and a second orientation and the magnetic sensor is at the second position and the second orientation to obtain a distortion term at a second reference frame, wherein the distortion term at the second reference frame is represented as a 3×3 matrix of data.

In some implementations, multiplying the 3×3 matrix of data corresponding to the distortion term at the initial reference frame by the inertial data obtained when the magnetic transmitter and the magnetic sensor are at their respective second positions and orientations results in the 3×3 matrix of data corresponding to the distortion term at the initial reference frame to be rotated into the second reference frame, wherein the 3×3 matrix of data corresponding to the distortion term at the second reference frame corresponds to the second orientation of the magnetic transmitter and the second orientation of the magnetic sensor.

In some implementations, the third plurality of signals is represented as a third 3×3 matrix of data, and the third 3×3 matrix of data corresponds to the second reference frame.

In some implementations, the third plurality of signals include distortions due to presence of one or more conductive or magnetic objects at or near a tracking environment of the system, and a third position and orientation of the magnetic sensor relative to the magnetic transmitter that corresponds to the third plurality of signals includes inaccuracies in one or more dimensions.

In some implementations, the one or more computer systems are further configured to: determine, based on the third 3×3 matrix of data corresponding to the second reference frame and the 3×3 matrix of data corresponding to the distortion term at the second reference frame, an undistorted term that corresponds to the second position and orientation of the magnetic sensor; and determine, based on the undistorted term, the second position and orientation of the magnetic sensor relative to the magnetic transmitter.

In some implementations, the undistorted term is determined by subtracting the 3×3 matrix of data corresponding to the distortion term at the second reference frame from the third 3×3 matrix of data corresponding to the second reference frame.

In some implementations, the undistorted term corresponds to a correct position and orientation of the magnetic sensor, and the second position and orientation of the magnetic sensor represent the correct position and orientation of the magnetic sensor.

In some implementations, the second position and orientation of the magnetic sensor does not include inaccuracies that would otherwise be caused by distortions in the third plurality of signals due to presence of one or more conductive or magnetic objects at or near a tracking environment of the system if the undistorted term were not considered.

In some implementations, the first frequency is 30 KHz or greater.

In some implementations, the second frequency is 1.1 KHz or less.

In some implementations, the second frequency is 100 Hz.

In general, in another aspect, a method includes: causing a magnetic transmitter to generate a first plurality of magnetic fields at a first frequency; receiving, from a magnetic sensor, a first plurality of signals; determining data indicative of a position and orientation of the magnetic sensor at a first position of the magnetic sensor; determining, based on the first plurality of signals and the data indicative of the position and orientation of the magnetic sensor at the first position, a distortion term that corresponds to a first position of the magnetic sensor; causing the magnetic transmitter to generate a third plurality of magnetic fields at the first frequency; receiving, from the magnetic sensor, a third plurality of signals; and determining, based on the third plurality of signals received from the magnetic sensor and the distortion term, a second position and orientation of the magnetic sensor relative to the magnetic transmitter, wherein the first frequency is greater than the second frequency.

In general, in another aspect, one or more non-transitory computer-readable media store instructions operable to cause a computing device to perform operations comprising: causing a magnetic transmitter to generate a first plurality of magnetic fields at a first frequency; receiving, from a magnetic sensor, a first plurality of signals; determining data indicative of a position and orientation of the magnetic sensor at a first position of the magnetic sensor; determining, based on the first plurality of signals and the data indicative of the position and orientation of the magnetic sensor at the first position, a distortion term that corresponds to a first position of the magnetic sensor; causing the magnetic transmitter to generate a third plurality of magnetic fields at the first frequency; receiving, from the magnetic sensor, a third plurality of signals; and determining, based on the third plurality of signals received from the magnetic sensor and the distortion term, a second position and orientation of the magnetic sensor relative to the magnetic transmitter, wherein the first frequency is greater than the second frequency.

Advantages of the systems and techniques described herein include employing multiple modes of operation for the system. For example, in a first operating mode (e.g., a normal or typical mode of operation), the system may be configured to operate at a relatively high frequency (e.g., 30 KHz). Such a frequency may include one or more advantages, such as improved speed, better suitability for customer/applications, etc.). However, the first operating mode may be susceptible to errors. As such, the system may also be configured to operate in a second operating mode (e.g., a specialized operating mode). The second operating mode may be occasionally applied when circumstances warrant (e.g., when the magnetic sensor and magnetic transmitter temporarily cease movement). In the second operating mode, the system may be configured to operate at a comparatively lower frequency (e.g., 100 Hz), which may not be susceptible to the aforementioned errors, but may include one or more disadvantages that are not suited for a normal/typical operating mode (e.g., too slow). Information obtained during operation in the second mode at the second frequency, during which effects of potential distortions in the tracking environment are reduced or minimized, can be used to correct measurements obtained when the system is operating in the first mode at the first frequency. For example, one or more distortion terms can be determined and used to compensate for distortions in the tracking environment, thereby resulting in an accurate position and orientation of the magnetic sensor relative to the magnetic transmitter to be provided while the system is operating in the first operating mode.

In some implementations, rather than operating in a second operating mode, the system may be configured to employ one or more other techniques for obtaining measurements that are not impacted by environmental distortions, For example, an optical system (among others) can be used to determine a clean pose of the sensor and/or the transmitter based on optical data. In this way, the pose of the sensor as determined based on the optical data may be taken as truth data indicative of the actual pose of the sensor.

In some implementations, the tracking environment can be mapped prior to the pose of the sensor being determined. For example, distortion terms corresponding to various locations within the tracking environment can be determined ahead of time. When a true pose of the sensor is to be determined, a previously obtained distortion term that corresponds to the location of the sensor can be used to compute a clean pose of the sensor. In this way, distortion terms need not be determined in real time.

In some implementations, once a distortion term is identified, either the transmitter or the sensor (e.g., the receiver) can be “rotated” to a different pose. In other words, characteristics of the fields generated by the transmitter or data provided by the sensor can be modified such that it corresponds to a pose other than the current pose of the transmitter and/or the sensor. In this way, the transmitter and/or the sensor is rotated in a simulated manner.

DETAILED DESCRIPTION

An Electromagnetic Tracking (EMT) system can be used in gaming and/or surgical settings to track devices (e.g., gaming controllers, head-mounted displays, medical equipment, robotic arms, etc.), thereby allowing their respective three-dimensional positions and orientations to be known to a user of the system. AR and VR systems also use EMT systems to perform head, hand, and body tracking, for example, to synchronize the user's movement with the AR/VR content. Such systems use a magnetic transmitter in proximity to a magnetic sensor to determine the position and orientation (P&O) of the sensor relative to the transmitter.

Such systems can employ one or more techniques for improving the determination of the P&O of the sensor relative to the transmitter. For example, one or more techniques may be employed to reduce/eliminate positional errors caused by distortions in the tracking environment. For example, the EMT system may be sensitive to metallic objects, which can manifest as distortion in the tracking environment (e.g., distortions of the magnetic fields generated by the transmitter and/or sensed by the sensor). Distortion can include conductive distortion and ferromagnetic distortion. Conductive distortion is generally caused by eddy currents set up within conductive objects by alternating magnetic fields (e.g., such as those produced by the transmitter). The eddy currents generate additional magnetic fields, which can be indistinguishable from those produced by the transmitter. These additional fields can cause the EMT system100to report erroneous P&O results. Ferromagnetic distortion can be caused by magnetic reluctance of materials at or near the tracking environment106. Such magnetic reluctance “bends” the magnetic fields from their normal geometry. Such distortions cause the magnetic fields to depart from a magnetic field model on which a P&O algorithm is based, thereby causing erroneous P&O results to be reported.

To ensure that the transmitter and sensor can provide accurate P&O measurements to the user, such distortions can be compensated for in the system; for example, distortions can be compensated for by determining one or more terms indicative of distortion in the tracking environment (e.g., a distortion term) and using the one or more distortion terms to correct future measurements provided by the sensor.

In some implementations, a distortion term can be determined for an initial sampled location (e.g., an initial P&O of the sensor) and used to correct a P&O measurement at a subsequent sampled location (e.g., a subsequent P&O of the sensor) close to the initial sampled location. The distance between the initial location and the subsequent location may depend on the distortion gradients at the sampled locations. An undistorted (e.g., “clean”) pose output of the sensor at the initial sampled location may be acquired using one or more low distortion trackers (e.g., trackers that are minimally impacted by distortions or trackers that are not susceptible to distortions). In some implementations, the low distortion tracker may be configured to operate the system at a relatively low frequency, thereby minimizing or eliminating the effects of distortions in the tracking volume. In this way, a “trusted vector” (e.g., a true P&O) of the sensor is determined.

Alternatively or additionally, the undistorted pose output of the sensor at the initial sampled location may be acquired using one or more other techniques (e.g., one or more other “low distortion trackers”) that are minimally or not susceptible to distortion. For example, in some implementations, an optical system including one or more cameras mounted at the sensor, the transmitter, and/or one or more location(s) at or near the tracking volume can be used to determine the trusted vector indicative of the true P&O of the sensor based on visual data. In some implementations, other types of low distortion trackers can be used to compensate output of the sensor, for example, infrared tracking, acoustic tracking, or another P&O tracking method that is not susceptible or less susceptible to distortion effects of metal in the environment (e.g., as compared to the normal operating mode that employs high frequency EM tracking, as described herein), to name a few.

In some implementations, distortion terms can be determined for a plurality of locations within a tracking volume during a “mapping” or “initialization” routine. For example, distortion terms indicative of environmental distortion throughout the tracking volume can be obtained by moving the sensor throughout the tracking volume and recording the P&O output of the sensor (e.g., distorted P&O measurements). A corresponding clean P&O measurement indicative of the true P&O of the sensor can be determined for the various distorted P&O measurements, and a corresponding distortion term can be determined for a particular location within the tracking volume. The clean P&O measurements (e.g., corresponding to the trusted vectors) can be determined using a low distortion tracker (e.g., the optical system, the low frequency mode, etc.), as described briefly above and in more detail below. In this way, distortion terms can be correlated to various clean poses within the tracking volume, and in some implementations, additional distortion terms can be determined for other locations within the tracking volume (e.g., locations that were not specifically sampled) using one or more extrapolation techniques. The distortion terms mapped throughout the tracking volume can be used to determine an undistorted P&O of the sensor when the sensor is positioned, for example, at or near a location that corresponds to a distortion term, as described in more detail below.

FIG. 1shows an example of an EMT system100that can be used as part of an AR/VR system. The EMT system100includes at least a magnetic sensor112, an orientation measurement device (OMD)122, a magnetic transmitter114, and another OMD124. The OMDs122,124are relatively insensitive (or, e.g., not sensitive) to metal in the environment. Therefore, the OMDs122,124can be used to determine the clean orientation of the sensor112and/or the transmitter114. In some implementations, the OMDs122,124may include one or more inertial measurement units (IMUs) and/or an optical system that can measure an orientation of the sensor112relative to the transmitter114, or vice versa. In some implementations, the sensor112and OMD122are incorporated at a head-mounted display (HMD)102and the magnetic transmitter114and OMD124are incorporated at a controller104. The EMT system100also includes sensor processing142and transmitter processing144. The sensor processing142and transmitter processing144may be located at the HMD102and/or the controller104, either separately or together, or may alternatively be located at a separate electronic device (e.g., a computer system). The EMT system100may also include a low distortion tracker132. The low distortion tracker132is relatively insensitive (or, e.g., not sensitive) to metal in the environment. Therefore, like the OMDs122,124described above, the low distortion tracker132can be used to determine the clean P&O of the sensor112and/or the transmitter114. In some implementations, the low distortion tracker132may include an optical system with one or more cameras located at the HMD102and/or the controller104. Such an optical system can be used instead of or in addition to a low-frequency operating mode that is not susceptible to environmental distortions.

In general, the EMT system100is configured to compensate for distortions at or around a tracking environment. For example, a “distorted” P&O output of the sensor112at an initial pose may include errors due to environmental distortion. A “clean” P&O of the sensor112can be determined at the initial pose using a technique that is not susceptible to environmental distortions. For example, an optical system or a low-frequency operating mode can be used to determine the clean/true P&O of the sensor112(e.g., sometimes referred to as a trusted vector), an example of which is described in more detail below. Using both the distorted P&O and the clean P&O of the sensor112at the initial pose, a distortion term that corresponds to the initial pose can be determined. When the sensor112is subsequently positioned at a new pose, the distortion term that corresponds to the initial pose can then be used to correct a distorted P&O output of the sensor112at the new pose.

While two particular low distortion trackers are described herein, it should be understood that other low distortion trackers can be used. One of the functions of the low distortion tracker is to develop the trusted vector that corresponds to the clean/true P&O of the sensor112based on one or more measurements. The trusted vector represents a true position and orientation of the sensor112. Thus, any technique for determining the true position and orientation of the sensor112may be included in the EMT system100for the purposes described herein.

In some implementations, rather than or in addition to determining distortion terms substantially in real time as described above, the tracking environment may be mapped ahead of time. In this way, distortion terms indicative of environmental distortions that correspond to various poses within the tracking environment can be determined as part of an initialization routine, and such distortion terms can subsequently be applied to distorted P&O outputs of the sensor112to provide a clean P&O of the sensor112. Such an implementation is described in more detail below.

FIG. 2shows an example of the EMT system100ofFIG. 1. The EMT system100includes at least a head-mounted display (HMD)102that includes the magnetic sensor112and the OMD122, and a controller104that includes the magnetic transmitter114and the OMD124.

In some examples, a VR system uses computer technology to simulate the user's physical presence in a virtual or imaginary environment. VR systems may create three-dimensional images and/or sounds through the HMD102and tactile sensations through haptic devices in the controller104or wearable devices to provide an interactive and immersive computer-generated sensory experience. In contrast, AR systems may overlay computer-generated sensory input atop the user's live experience to enhance the user's perception of reality. For example, AR systems may provide sound, graphics, and/or relevant information (e.g., such as GPS data to the user during a navigation procedure). Mixed Reality (MR) systems—sometimes referred to as hybrid reality systems—may merge real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real-time.

The HMD102and the controller104are configured to track position (e.g., in x, y, and z) and orientation (e.g., in azimuth, altitude, and roll) in three-dimensional space relative to each other. For example, the transmitter114is configured to track the sensor112(e.g., relative to a reference frame defined by the position and orientation of the transmitter114), and/or the sensor112is configured to track the transmitter114(e.g., relative to a reference frame defined by the position and orientation of the sensor112). In some implementations, the system100is configured to track the position and orientation (e.g., the P&O) of the sensor112and/or the transmitter114in a tracking environment106of the EMT system. In this way, the P&O of the HMD102and/or the controller104can be tracked relative to each other and relative to a coordinate system defined by the EMT system100. For example, the HMD102and the controller104can be used to perform head, hand, and/or body tracking, for example, to synchronize the user's movement with the AR/VR content. While the tracking environment106is illustrated as being a defined space, it should be understood that the tracking environment106may be any three-dimensional space, including three-dimensional spaces without boundaries (e.g., large indoor and/or outdoor areas, etc.). The particular sensor112and transmitter114employed by the EMT system100may be determined by the procedure type, measurement performance requirements, etc.

In some implementations, the transmitter114includes three orthogonally wound magnetic coils, referred to herein as the X, Y, and Z coils. Electrical currents traveling through the three coils cause the coils to produce three orthogonal sinusoidal magnetic fields at a particular frequency (e.g., the same or different frequencies). In some implementations, time division multiplexing (TDM) may also be used. For example, in some implementations, the coils may produce magnetic fields at the same frequency (e.g., 30 KHz) but at non-overlapping times. The sensor112also includes three orthogonally wound magnetic coils, referred to herein as the x, y, and z coils. Voltages are induced in the coils of the sensor112in response to the sensed magnetic fields by means of magnetic induction. Each coil of the sensor112generates an electrical signal for each of the magnetic fields generated by the coils of the transmitter114; for example, the x coil of the sensor112generates a first electrical signal in response to the magnetic field received from the X coil of the transmitter114, a second electrical signal in response to the magnetic field received from the Y coil of the transmitter114, and a third electrical signal in response to the magnetic field received from the Z coil of the transmitter114. They and z coils of the sensor112similarly generate electrical signals for each of the magnetic fields generated by the X, Y, and Z coils of the transmitter114and received at/by they and z coils of the sensor112.

As described in more detail below, in some implementations, the transmitter114may be configured to use a particular frequency depending on a mode in which the transmitter114is currently operating. For example, in a first mode (e.g., during normal operation of the transmitter114as implemented in the EMT system100or in an AR or VR system), the transmitter114may be configured to generate magnetic fields at a frequency of 30 KHz or greater (e.g., 30 KHz, 34 KHz, etc.) In some implementations, in a second mode (e.g., for minimizing the effects of potential distorters in the environment), the transmitter114may be configured to generate magnetic fields at a frequency of 1.1 KHz or less (e.g., 1.1 KHz, 1 KHz, 100 Hz, etc.).

The data from the sensor112can be represented as a matrix of data (e.g., a 3×3 matrix), sometimes referred to as a measurement matrix, which can be resolved into the P&O (e.g., sometimes referred to as the pose) of the sensor112with respect to the transmitter114, or vice versa. In this way, the P&O of the sensor112and the transmitter114is measured. An example of a 3×3 signal measurement matrix (e.g., sometimes referred to as an S-matrix) is shown below, where each matrix element represents the sensor signal in the indicated coil of the sensor112(x, y, z) due to energizing a single coil of the transmitter114(X, Y, Z), and where the columns represent the signal produced by the coils of the transmitter114(X, Y, Z) and the rows represent signals measured by the coils of the sensor112(x, y, z):

It should be understood that the particular mathematic processes described herein are a result of merely one example technique for determining a pose of a sensor relative to a transmitter. The particular mathematical transforms performed may differ, as those skilled in the art would understand. The exemplary mathematics should not be interpreted as limiting the general inventive concept of using a low distortion tracker to correct a pose of the sensor and/or transmitter at a subsequent sampled location, as described herein.

The sensor processing (142ofFIG. 1) and the transmitter processing (144ofFIG. 1), which may be incorporated in the HMD102and/or the controller104or located separately from the HMD102and controller104, are configured to determine the P&O of the HMD102relative to the controller104and vice versa based on characteristics of the magnetic fields generated by the transmitter114and the various electrical signals generated by the sensor112.

In some implementations, the sensor processing142and the transmitter processing144may be implemented as one or more computer systems. For example, one or more computer systems may be configured to resolve the data from the sensor112into the P&O of the sensor112. In some implementations, the one or more computer system can include EM sensor processing functionality and/or EM transmitter processing functionality. In some implementations, one or more computer systems incorporated into the HMD102and/or the controller104(or, e.g., the sensor112and/or the transmitter114) may be configured to determine the P&O of the sensor112. In some implementations, EM sensor processing functionality and EM transmitter processing functionality may be incorporated into a single computer system (e.g., at the HMD102/sensor112, at the controller104/transmitter114, or at a separate computer system). The sensor112, the transmitter114, and/or the separate computer system may be configured to communicate information to each other (e.g., via a wireless connection, a wired connection, etc.). As described below, a separate computer system may also be configured to determine the P&O of the sensor112and transmitter114, and such information may be provided to the HMD102and/or the controller104.

The AR/VR system and/or the EMT system100can employ one or more techniques for improving the determination of the P&O of the sensor112relative to the transmitter114. For example, one or more techniques may be employed to reduce/eliminate positional errors caused by distortions in the tracking environment106. The EMT system100may be sensitive to metallic objects, which can manifest as distortion in the tracking environment106(e.g., distortions of the magnetic fields generated by the transmitter114and/or sensed by the sensor112). Distortion can include conductive distortion and ferromagnetic distortion. Conductive distortion is generally caused by eddy currents set up within conductive objects by alternating magnetic fields (e.g., such as those produced by the transmitter114). As mentioned above, the eddy currents can generate additional magnetic fields, which can be indistinguishable from those produced by the transmitter114. These additional fields can cause the EMT system100to report erroneous P&O results. For example, an algorithm for determining the P&O of the sensor112based on sensor signals may employ a field model of the magnetic fields generated by the transmitter114with no additional fields due to eddy current, and as such, the reported results do not provide an accurate representation of the P&O of the transmitter114and/or the sensor112when distortions are present.

Ferromagnetic distortion can be caused by magnetic reluctance of materials at or near the tracking environment106. Such magnetic reluctance “bends” the magnetic fields from their normal geometry, again causing the magnetic fields to depart from the magnetic field model on which the P&O algorithm is based, thereby causing erroneous results to be reported.

To ensure that the transmitter114and sensor112can provide accurate P&O measurements to the user, such distortions can be compensated for in the EMT system100, for example, by determining one or more terms indicative of distortion in the tracking environment106(e.g., a distortion term), and using the one or more distortion terms to correct future measurements provided by the sensor112. A particular distortion term may correspond to a particular position within the tracking environment106. In some implementations, a particular distortion term may correspond to a particular P&O of the sensor112relative to a particular P&O of the transmitter114. In some implementations, a particular distortion term may correspond to a particular position of the sensor112relative to a particular position of the transmitter114, and the particular distortion term can be mathematically adjusted to correspond to various orientations of the sensor112and/or the transmitter114at the particular position, as described in more detail below.

In some implementations, the EMT system100may determine an initial distortion term while the sensor112and the transmitter114are in an initial P&O (e.g., an initial reference frame). Thereafter, the sensor112and/or the transmitter114may move to a second P&O (e.g., a second reference frame). The distortion term obtained at the initial reference frame can be used to mathematically adjust the sensor measurements provided by the sensor112when the sensor112is at the second P&O to provide an accurate (e.g., correct, or “true”) position of the sensor112relative to the transmitter114. In other words, the sensor measurements provided by the sensor112when the sensor112is at the second P&O may otherwise include inaccuracies due to distortions in the tracking environment106. The distortion term can be representative of such distortions. Thus, the distortion term may be used to remove the effects of such distortions from the sensor signal when the sensor112is at the second P&O.

While the EMT system100is configured to determine the orientation of the sensor112and the transmitter114relative to each other by employing electromagnetic tracking techniques, the sensor112and the transmitter114are each associated with an orientation measurement device (OMD)122,124configured to provide information related to the orientation of the sensor112and the transmitter114. In some implementations, the OMD122,124are inertial measurement units (IMUs) that are configured to provide inertial data that corresponds to the sensor112and transmitter114. In some implementations in which IMUs are employed, each of the OMDs122,124may be configured to collect inertial data that corresponds to (e.g., is associated with) the sensor112and the transmitter114. In some implementations, the IMUs include one or more accelerometers and/or one or more gyroscopes configured to collect the inertial data. The inertial data can be used to determine, among other things, the orientation of the sensor112and the transmitter114. For example, the IMUs may be configured to measure specific force and/or angular rate, which can be used to determine an orientation, heading, velocity, and/or acceleration of the IMU (and, e.g., the HMD102and controller104). In some implementations, the determined velocity and/or acceleration can be used to assist in determining the position of the sensor112and the transmitter114. For example, the determined velocity and/or acceleration can be used to determine a change in position of the sensor112and/or the transmitter114over time. The inertial data can be communicated between the one or more computer systems described above. For example, in some implementations, the inertial data related to the sensor112may be wirelessly provided to the transmitter114and vice versa. In some implementations, a separate computer system may facilitate the exchange of inertial and other data between the sensor112and the transmitter114.

In some implementations, rather than IMUs being employed, the OMDs122,124may include an optical system that is used to determine the orientation of the sensor112relative to the transmitter114and vice versa. In this way, the true orientation of the sensor112and/or the transmitter114can be determined based on optical data rather than inertial data.

In some implementations, the sensor112may produce degraded (e.g., inaccurate) data due to operation in an EM distorted environment, thereby resulting in inaccurate pose output. Described herein are EM distortion compensation systems and techniques for determining a “clean” (e.g., undistorted) S-matrix representative of the pose of the sensor112. The clean/undistorted S-matrix representative of an accurate P&O of the sensor112at an arbitrary reference frame (i) (e.g., an arbitrary P&O) may be denoted as Scleani, which can be computed according to Equation (1):
Scleani=Sreci−Sdisti(1)
where Sreciis a degraded S-matrix (e.g., due to inaccuracies caused by distorters at or near the tracking environment106) when the sensor112is at the arbitrary reference frame i and Sdistiis a distortion term corresponding to the arbitrary reference frame i. As Sreciis measured, the systems and techniques described herein are directed to finding the distortion matrix Sdistiat the arbitrary reference frame.

The magnitude of a signal matrix from the sensor112remains constant over arbitrary rotations of rows and columns (e.g., via the Frobenius norm). In the systems and techniques described herein, the rows and columns of the S-matrices correspond to physical orientations of the sensor112and transmitter114. In some implementations, an S-matrix obtained when the sensor112and transmitter114are at an initial reference frame (e.g., initial reference frame0) may be used to compute a distortion term at the initial reference frame0, the distortion term at the initial reference frame0can be used to determine a distortion term at another reference frame (e.g., an arbitrary reference frame i), and the distortion term at the arbitrary reference frame i can be used to determine the P&O of the sensor112at the arbitrary reference frame i (e.g., at a later time relative to a time at which the distortion term at the initial reference frame0is obtained).

In the example illustrated inFIG. 2, the HMD102/sensor112and the controller104/transmitter114are at an initial reference frame0. In particular, the HMD102/sensor112are at an initial reference frame (S P&O0) and the controller104/transmitter114are at an initial reference frame (T P&O0). In the initial reference frames S P&O0and T P&O0, the sensor112and the transmitter114are each at a first (e.g., initial) position and orientation. In some implementations, the initial reference frame corresponds to a time at which the sensor112and the transmitter114have ceased or substantially ceased movement for a period of time (e.g., as determined by the OMDs122,124).

With the sensor112and the transmitter114at the initial reference frame, a first measurement is obtained by the sensor112. In particular, the coils of the transmitter114are configured to generate a first plurality of magnetic fields at a first frequency. The first frequency may be a frequency at which the EMT system100, AR, and/or VR systems are configured to operate under normal operating conditions (e.g., during typical use of the EMT system100). In some implementations, the first frequency may be used when the EMT system100is operating in a first/normal mode of operation. The first frequency may be a frequency of 30 KHz or greater (e.g., 30 KHz, 34 KHz, etc.). In some implementations, the first frequency is one that may be susceptible to inaccuracies due to potential distorters in the tracking environment106.

A first plurality of signals is received from the sensor112. For example, the sensor112is configured to generate signals based on characteristics of the magnetic fields received at the sensor112. The magnetic fields received at the sensor112may be largely based on the magnetic fields generated by the transmitter114at the first frequency. However, one or more potential distorters in the tracking environment106, among other things, may cause the generated magnetic fields to “bend” from their normal geometry. Such distortions may cause the first plurality of signals received from the sensor112to provide an incorrect P&O of the sensor112relative to the transmitter114. The first plurality of signals can be represented as a first 3×3 S-matrix of data, referred to herein as Srec0. For example, Srec0is received while the sensor112and transmitter114are at the initial reference frame S P&O0and T P&O0and the transmitter114generates magnetic fields at the first frequency.

With the sensor112and the transmitter114still at or close to the initial reference frame0, a true pose of the sensor112is determined (e.g., an actual pose of the sensor112without inaccuracies due to distortions). For example, in some implementations, a second measurement is obtained by the sensor112. In particular, the coils of the transmitter114are configured to generate a second plurality of magnetic fields at a second frequency. The second frequency may be a frequency at which the EMT system100, AR, and/or VR systems are configured to operate under a specialized operating condition (e.g., while the sensor112and the transmitter114are stationary or almost stationary, for example, while a user of the EMT system100temporarily stops moving). In some implementations, the second frequency may be used when the EMT system100is operating in a second/undistorted mode of operation. The second frequency may be a frequency of 1.1 KHz or less (e.g., 1.1 KHz, 1 KHz, 100 Hz, etc.). In some implementations, the second frequency is one that is unsusceptible (or, e.g., significantly less susceptible than the first frequency) to inaccuracies due to potential distorters in the tracking environment106.

A second plurality of signals is received from the sensor112. For example, the sensor112is configured to generate signals based on characteristics of the magnetic fields received at the sensor112. The magnetic fields received at the sensor112may be largely based on the magnetic fields generated by the transmitter114at the second frequency. Any potential distorters in the tracking environment106may have a limited impact on the magnetic fields generated using the second frequency. As such, potential distorters may not cause (or, e.g., may cause to a significantly lesser extent) the magnetic fields generated at the second frequency to “bend” from their normal geometry relative to the magnetic fields generated at the first frequency. Therefore, the second plurality of signals received from the sensor112provide an accurate P&O of the sensor112relative to the transmitter114. The second plurality of signals can be represented as a second 3×3 S-matrix of data, referred to herein as Sclean0. For example, Sclean0is received while the sensor112and transmitter114are at the initial reference frame S P&O0and T P&O0and the transmitter114generates magnetic fields at the second frequency. Sclean0is referred to as a “clean” S-matrix because it is assumed to accurately correspond to “clean” (e.g., undistorted) magnetic fields received at the sensor112. In other words, Sclean0theoretically represents the signals that would be provided by the sensor112in an environment that does not include any distortions. As such, it is expected that Sclean0can be resolved into an accurate (e.g., true, correct, actual, etc.) P&O of the sensor112when the sensor112is at the initial reference frame.

In some implementations, Sclean0can be determined using the low distortion tracker132. That is, rather than operating the sensor112and transmitter114in a low-frequency operating mode to determine the clean pose of the sensor112, an optical system including one or more cameras can be used to determine a clean, undistorted pose of the sensor112at the initial reference frame. The low distortion tracker132along with the sensor processing142and transmitter processing144can be used to represent the clean pose of the sensor112at the initial reference frame as a 3×3 matrix of data, as Sclean0.

In some implementations, the OMDs122,124corresponding to the sensor112and the transmitter114, when implemented as IMUs, may provide inertial data while the sensor112and the transmitter114are at the initial reference frame0. Such inertial data can be used to determine an orientation of the sensor112and the transmitter114. In some implementations, the orientations as determined based on the inertial data may be taken as accurate orientation data (e.g., the true orientation of the sensor112and the transmitter114). In some implementations, the IMUs may each be a 9-axis IMU, and the orientation data may be provided to the sensor processing142and/or the transmitter processing144.

As described above, in some implementations, the OMDs122,124may be implemented at least in part by an optical system that includes one or more cameras. The optical system can determine the true orientation of the sensor112and/or the transmitter114using optical data. In this way, using IMU(s) and/or an optical system, data indicative of the orientation of the sensor112and transmitter114can be determined.

A distortion term that corresponds to the initial reference frame0, Sdist0, may be calculated according to Equation (2):
Sdisto=Rso(Sreco−Scleano)Rto(2)
where Srec0is the S-matrix of the sensor112received while the sensor112and transmitter114are at the initial reference frame and while the transmitter114generates magnetic fields at the first frequency, Sclean0is the S-matrix of the sensor112received while the sensor112and transmitter114are at the initial reference frame and while the transmitter114generates magnetic fields at the second frequency, Rs0is data indicative of the orientation of the sensor112at the initial reference frame, and Rt0is data indicative of the orientation of the transmitter114at the initial reference frame. In particular, the difference between Srec0and Sclean0are rotated into the initial reference frame0by multiplying the difference by Rs0and Rt0.

The distortion term that corresponds to the initial reference frame0, Sdist0, may be stored (e.g., by the one or more computer systems) and used to calculate a distortion term at an arbitrary reference frame i, (e.g., Sdisti), for example, once the sensor112and/or the transmitter114resume movement. For example, after the sensor112and/or the transmitter114move to a second position and orientation that correspond to a second reference frame S P&Oiand T P&Oi, as illustrated inFIG. 3, the initial distortion term Sdist0can be rotated into the second reference frame i according to Equation (3):
Sdisti=RsiSdistoRsi(3)
where Rsiis data indicative of the orientation of the sensor112at the second reference frame i, and Rtiis data indicative of the orientation of the transmitter114at the second reference frame i. In other words, the distortion term that corresponds to the initial reference frame0, Sdist0, is multiplied by data indicative of the position and orientation of the sensor112and transmitter114at the second reference frame in order to rotate the initial distortion term, Sdist0, into the second reference frame i, the product of which is represented as Sdisit.

With the sensor112and the transmitter114at the second reference frame i, a third measurement is obtained by the sensor112. In particular, the coils of the transmitter114are configured to generate a third plurality of magnetic fields at the first frequency (e.g., in the first mode that uses the frequency at which the EMT system100, AR, and/or VR systems are configured to operate under normal operating conditions). As described above, in some implementations, the first frequency is one that may be susceptible to inaccuracies due to potential distorters in the tracking environment106.

A third plurality of signals is received from the sensor112. For example, the sensor112is configured to generate signals based on characteristics of the magnetic fields received at the sensor112. The magnetic fields received at the sensor112may be largely based on the magnetic fields generated by the transmitter114at the third frequency. However, one or more potential distorters in the tracking environment106, among other things, may cause the generated magnetic fields to “bend” from their normal geometry. Such distortions may cause the third plurality of signals received from the sensor112to provide an incorrect P&O of the sensor112relative to the transmitter114at the second reference frame i. The third plurality of signals can be represented as a third 3×3 S-matrix of data, referred to herein as Sreci. For example, Sreciis received while the sensor112and transmitter114are at the second reference frame S P&Oiand T P&Oiand the transmitter114generates magnetic fields at the third frequency.

Based on the third 3×3 S-matrix of data (e.g., Sreci), and based on the distortion term at the second reference frame i, (e.g., Sdisti), an undistorted term, Scleani, is determined. In particular, Scleaniis determined according to Equation (4):
Scleani=Sreci−RsiRzoT(Sreco−Scleano)RtoRtiT(4)
where the undistorted term, Scleani, is an S-matrix that is representative of an accurate (e.g., true, correct, actual, etc.) P&O of the sensor112when the sensor112is at the second reference frame i (e.g., at S P&Oi and T P&Oi, which correspond to the second position and orientation of the sensor112and the second position and orientation of the transmitter114). In other words, the third 3×3 S-matrix of data, Sreci, may include distortions due to presence of one or more conductive or magnetic objects at or near the tracking environment106of the EMT system100, and if a P&O of the sensor112were calculated based on Sreci, the P&O may include inaccuracies in one or more dimensions. As such, the distortion term at the second reference frame, Sdisti, is subtracted from Srecito produce a calculated S-matrix that can be resolved into an accurate P&O for the sensor112in the second reference frame (e.g., at the second P&O). In this way, the calculated second P&O of the sensor112does not include inaccuracies that would otherwise be caused by distortions in Srecidue to presence of one or more conductive or magnetic objects at or near the tracking environment106if the undistorted term, Scleani, were not considered.

In some implementations, as the sensor112and/or the transmitter114move within the tracking environment106(e.g., relative to the initial, 0, and second, i, reference frames), the initial distortion term, Sdist0, may be of minimal use at the subsequent position. For example, characteristics of the tracking environment106at subsequent positions may be significantly different than those at the initial reference frame (e.g., due to a relatively high distortion gradient), and as such, the initial distortion term, Sdist0, obtained at the initial reference frame may not be representative of distortion that are present at the subsequent positions. As such, additional distortion terms that correspond to various positions of the sensor112and/or transmitter114within the tracking environment106can be obtained. Such distortion terms can be obtained for various reference frames that correspond to a position and/or orientation of the sensor112and/or a position and/or orientation of the transmitter114. In some implementations, a distortion term can be obtained for a particular sensor112/transmitter114P&O when the sensor112and transmitter114temporarily cease movement during use.

The implementations described above are directed to techniques for determining a clean pose of the sensor112based on undistorted measurements obtained with the assistance of a low-frequency operating mode and/or an optical system that are not susceptible to environmental distortions. The distorted and undistorted measurements at an initial location are used to determine a distortion term at the initial location, and the distortion term is used to correct a distorted measurement when the sensor112is at a subsequent location. The distortion term is determined around the same time that the pose of the sensor112is being determined. However, in some implementations, distortion terms corresponding to various locations within the tracking environment106can be determined ahead of time. For example, a map of environmental distortions throughout the tracking environment106can be created. The map can be acquired as part of an initialization routine, or may be initiated by a user at a later time upon software command. When a true pose of the sensor112is to be determined, a previously obtained distortion term that corresponds to the location of the sensor112can be used to compute a clean pose of the sensor112. In this way, distortion terms need not be determined in real time.

To map the tracking environment106, the transmitter114may be situated at a fixed location at or near the tracking environment106. The sensor112is then moved (e.g., slowly) throughout the tracking environment106. The sensor112may be moved manually (e.g., by a user) or mechanically (e.g., according to a predetermined path within the tracking environment106, for example, by a robotic arm). By way of non-limiting example, the sensor112can move in a roughly circular path horizontally (e.g., co-planar) around the transmitter114. The sensor112can be moved around the transmitter114at different radii on different parallel planes in space. As the sensor112is moved throughout the tracking environment106, distorted pose sensor outputs (Srec0) are determined and saved for the various positions. At the same time, clean pose outputs (Sclean0) are determined (e.g., using an optical system and/or a low-frequency operating mode). In this way, the clean pose of the sensor112is determined for the various locations at which the distorted pose sensor outputs are obtained, and corresponding distortion terms (Sdist0) are correlated to the various clean poses. In this way, distortion data for the entire tracking volume may be taken, given a sufficient spacing of samples.

The system100can then produces one or more low bandwidth surface distortion maps having relatively low distortion gradients. For example, the system100produces a surface curve fit of the distortion terms Sdist0(e.g., the distortion matrices) using a low-order curve-fit function, for example a surface least-square fit (or other surface curve-fit). The process is repeated for various distortion matrix surfaces. In this way, the entire tracking volume (or, e.g., substantially the entire tracking volume) can be mapped to clean pose locations. In other words, the surface curve fit can be used to map substantially all of the tracking volume even though every location within the tracking volume may not necessarily have been sampled to determine a distortion term. The mapping data can be stored by the system100for later use, as described in more detail below.

Following the mapping, the surface maps can be used for environmental distortion correction. In particular, when the clean pose of the sensor112is to be determined, the system100can identify a distortion term that corresponds to a pose of the sensor112and use the distortion term to correct the distorted pose sensor outputs. For example, the system100first produces a sample of a clean pose output (Sclean0) at an initial sampled location in the tracking environment106. The clean pose output Sclean0can be determined as described above using an optical system and/or a low-frequency operating mode. The system100then correlates the initial clean pose output Sclean0to a particular location (e.g., a nearest location) in the environmental distortion map (e.g., within a tolerance) to select a matching initial distortion term Sdist0. When the sensor112is subsequently positioned at a new pose, a distorted pose sensor output Srecimeasurement is made. Using, e.g., Equation 3, a nominal distortion at the current pose is calculated. Then, using, e.g., Equation 4, the clean pose output of the sensor112at the current pose (Scleani) is calculated. The clean pose output (Scleani) can then be correlated to the particular location in the environmental distortion map (e.g., within a tolerance) to select the next matching distortion term in the global reference frame, and the process repeats to determine the pose of the sensor112at a subsequent position.

In some implementations (e.g., if the tracking volume is sufficiently mapped), the clean pose output of the sensor112at the current pose can be calculated using a distortion term that corresponds to the current pose. In other words, it may not be necessary to use the initial distortion term Sdist0to calculate the distortion term that corresponds to the current pose Sdistiusing Equation 3, but rather, the clean pose output (Scleani) can be calculated directly using the distortion term at the current pose Sdistias obtained from the mapping.

In some implementations, the various distortion terms (e.g., “initial” distortion terms obtained while the transmitter112and sensor114have stopped moving, as well as other distortion terms calculated based on “initial” distortion terms) and/or the distortion maps described above can be stored by the one or more computer system and/or by a database (e.g., a remote database) in communication with the one or more computer systems. In some implementations, the various distortion terms can together be used to build a distortion map of a particular tracking environment (e.g., the tracking environment106ofFIGS. 1-3). Distortion terms obtained for particular P&O of the sensor112and a particular P&O of the transmitter114can provide a data point for building the map.

Stored distortion terms that correspond to particular P&O of the sensor112and transmitter114can be useful if, at a later time, the sensor112and transmitter114subsequently return to the previously-determined P&O. For example, the stored distortion term for a particular P&O of the sensor112and transmitter114can again be used when the sensor112and transmitter114return to the same or similar P&O. Such distortion terms can be re-used when the characteristics of the tracking environment106have not changed over time. In some implementations (e.g., when the EMT sensor100is moved to a new location, and/or when conductive objects are added/removed from the tracking environment106or from areas proximate to the tracking environment106), the tracking environment106may be re-mapped to correspond to new distorters that may be present at or near the new tracking environment106.

In some implementation, the distortion terms may be used to create a numerical model of distortions in the tracking environment106in order to infer distortion terms that correspond to non-sampled P&O of the sensor112and transmitter114. For example, an initial distortion map may be created based on an initial set of distortion terms obtained for the tracking environment106. Based on the sampled distortion terms, other distortion terms may be inferred for positions and orientations within the tracking environment106that were not specifically sampled (e.g., positions and orientations near sampled positions and orientations). Additional distortion terms can be added to the distortion map to improve its reliability. As the additional distortion terms are added, the numerical model of distortions in the tracking environment106can be updated to reflect the additional data points that are available. Over time, the numerical model can be improved such that the EMT system100can provide P&O information for the sensor112having improved accuracy. In some implementations, an accurate P&O of the sensor112may be provided without necessarily obtaining a new initial distortion term (e.g., when the sensor112and transmitter114temporarily cease movement). For example, a stored distortion term may be used to correct the P&O of the sensor112on-the-fly based on the numerical model of the tracking environment106.

FIG. 4is a flowchart of an exemplary process400of determining a distortion term and determining a second (e.g., correct) position of a magnetic sensor relative to a magnetic transmitter (e.g., the magnetic sensor112and the magnetic transmitter114of the EMT system100ofFIGS. 1-3). One or more steps of the method may be performed by the one or more computer systems described herein.

At step402, a magnetic transmitter114generates a first plurality of magnetic fields at a first frequency. The first frequency may be a frequency at which the EMT system100is configured to operate under normal operating conditions (e.g., during typical use of the EMT system100and/or the AR and/or VR system). In some implementations, the first frequency may be used when the EMT system100is operating in a first/normal mode of operations. In some implementations, the first frequency is one that may be susceptible to inaccuracies due to potential distorters in the tracking environment106. For example, the first frequency is 30 KHz or more (e.g., 30 KHz).

At step404, a first plurality of signals are received from the magnetic sensor112. The signals are based on characteristics of the magnetic fields received at the magnetic sensor112. While the magnetic fields received at the magnetic sensor112may be largely based on the first plurality of magnetic fields generated by the magnetic transmitter114at the first frequency, potential distorters in the tracking environment106may cause the first plurality of signals to provide an incorrect P&O of the magnetic sensor112relative to the magnetic transmitter114. The first plurality of signals can be represented as a first 3×3 S-matrix of data, Srec0. For example, Srec0is received while the magnetic sensor112and the magnetic transmitter114are at an initial reference frame0(e.g., S P&O0and T P&O0) and while the magnetic transmitter114generates the first plurality of magnetic fields at the first frequency.

At step406, for example, with the magnetic sensor112and the magnetic transmitter114still at or close to the initial reference frame0, the magnetic transmitter114generates a second plurality of magnetic fields at a second frequency. The second frequency may be a frequency at which the EMT system100, AR, and/or VR systems are configured to operate under a specialized operating condition (e.g., while the magnetic sensor112and the magnetic transmitter114are stationary or almost stationary, for example, while a user of the EMT system100temporarily stops moving). In some implementations, the second frequency may be used when the EMT system100is operating in a second/undistorted mode of operation. In some implementations, the second frequency is one that is unsusceptible (or, e.g., significantly less susceptible than the first frequency) to inaccuracies due to potential distorters in the tracking environment106. For example, the second frequency may be a frequency of 1.1 KHz or less (e.g., 100 Hz). The second frequency is typically less than the first frequency (e.g., significantly less). For example, the first frequency may be two order of magnitude (or more) greater than the second frequency.

At step408, a second plurality of signals are received from the magnetic sensor112. The signals are based on characteristics of the magnetic fields received at the magnetic sensor112. The magnetic fields received at the magnetic sensor112may be largely based on the second plurality of magnetic fields generated by the magnetic transmitter114at the second frequency. Any potential distorters in the tracking environment106may have a limited impact on the second plurality of magnetic fields generated using the second frequency. As such, potential distorters may not cause (or, e.g., may cause to a significantly lesser extent) the second plurality of magnetic fields generated at the second frequency to “bend” from their normal geometry (e.g., as compared to the first plurality of magnetic fields generated at the first frequency). Therefore, the second plurality of signals received from the magnetic sensor112may provide an accurate P&O of the magnetic sensor112relative to the magnetic transmitter114. The second plurality of signals can be represented as a second 3×3 S-matrix of data, Sclean0. For example, Sclean0is received while the magnetic sensor112and the magnetic transmitter114are still at the initial reference frame0(e.g., S P&O0and T P&O0) and while the magnetic transmitter114generates the second plurality of magnetic fields at the second frequency. Sclean0is referred to as a “clean” S-matrix because it is assumed to accurately correspond to “clean” (e.g., undistorted) magnetic fields received at the magnetic sensor112. In other words, Sclean0theoretically represents the signals that would be provided by the magnetic sensor112in an environment that does not include any distortions. As such, it is expected that Sclean0can be resolved into an accurate (e.g., true, correct, actual, etc.) P&O of the magnetic sensor112when the magnetic sensor112is at the initial reference frame0.

In some implementations, Sclean0may be determined based on optical data provided by an optical system, as described above. For example, Sclean0can be determined using the low distortion tracker132, which may be implemented as an optical system with one or more cameras. Rather than operating the sensor112and transmitter114in a low-frequency operating mode to determine the clean pose of the sensor112, the optical system can be used to determine a clean, undistorted pose of the sensor112at the initial reference frame. The low distortion tracker132along with the sensor processing142and transmitter processing144can be used to represent the clean pose of the sensor112at the initial reference frame as a 3×3 matrix of data, as Sclean0.

At step410, a distortion term is determined based on the first plurality of signals and the second plurality of signals received from the magnetic sensor112. The distortion term corresponds to a first position of the magnetic sensor112. For example, the distortion term corresponds to the initial reference frame0(e.g., while the magnetic sensor112and the magnetic transmitter114are at the first position and orientation corresponding to the initial reference frame S P&O0, T P&O0). In some implementations, the distortion term that corresponds to the initial reference frame0is determined at least in part by subtracting Sclean0from Srec0.

In some implementations, when the OMDs122,124include IMUs, IMUs that correspond to the magnetic sensor112and the magnetic transmitter114may provide inertial data while the magnetic sensor112and the magnetic transmitter114are at the initial reference frame0. Such inertial data can be used to determine an orientation of the magnetic sensor112and the magnetic transmitter114at the initial reference frame0. In some implementations, the orientations as determined based on the inertial data may be taken as accurate orientation data (e.g., the true orientation of the magnetic sensor112and the magnetic transmitter114). The orientation data may be provided to the one or more computer systems (e.g., to the EM transmitter processing and/or the EM sensor processing and/or a separate computer system). In some implementations, the OMDs122,124may include an optical system that is used to determine the orientation of the sensor112and/or the transmitter114

In some implementation, data indicative of the orientation of the magnetic sensor112at the initial reference frame, Rs0, and data indicative of the orientation of the magnetic transmitter114at the initial reference frame, Rt0, is used to rotate the difference between Srec0and Sclean0into the initial reference frame0. In particular, the difference between Srec0and Sclean0are rotated into the initial reference frame0by multiplying the difference by Rs0and Rt0.

Once the difference is rotated into the initial reference frame0(or, e.g., one the difference is confirmed to be in the initial reference frame0), the distortion term that corresponds to the initial reference frame0is given as Sdist0.

In some implementation, the distortion term that corresponds to the initial reference frame, Sdist0, can be used to calculate a distortion term at a new reference frame (e.g., a second reference frame i). For example, the magnetic sensor112and/or the magnetic transmitter114may resume movement and move to a second position and second orientation that correspond to the second reference frame S P&Oi and T P&Oi, as illustrated inFIG. 3. The initial distortion term, Sdist0, can be rotated into the second reference frame i according to Equation (3) above. In particular, inertial data indicative of the orientation of the magnetic sensor112at the second reference frame, Rsi, and inertial data indicative of the orientation of the magnetic transmitter114at the second reference frame, Rti, can be multiplied by the initial distortion term, Sdist0, in order to rotate the initial distortion term, Sdist0, into the second reference frame i. The product of this multiplication is Sdisti, a distortion term that corresponds to the second frame i.

At step412, the magnetic transmitter114generates a third plurality of magnetic fields at the first frequency (e.g., in the first mode of operation). The third plurality of magnetic fields may be generated by the magnetic transmitter114after the magnetic sensor112and/or the magnetic transmitter114have moved to the second reference frame i. For example, after the magnetic sensor112and/or the magnetic transmitter114move to the second position and second orientation that correspond to the second reference frame S P&Oiand T P&Oi, the third plurality of magnetic fields are generated. As described above, in some implementations, the first frequency is one that may be susceptible to inaccuracies due to potential distorters in the tracking environment106.

At step414, a third plurality of signals are received from the magnetic sensor112. The signals are based on characteristics of the magnetic fields received at the magnetic sensor112. While the magnetic fields received at the magnetic sensor112may be largely based on the third plurality of magnetic fields generated by the magnetic transmitter114at the first frequency, potential distorters in the tracking environment106may cause the third plurality of signals to provide an incorrect P&O of the magnetic sensor112relative to the magnetic transmitter114. The third plurality of signals can be represented as a third 3×3 S-matrix of data, Sreci. For example, Sreciis received while the magnetic sensor112and the magnetic transmitter114are at the second reference frame i (e.g., S P&Oiand T P&Oi) and while the magnetic transmitter114generates the third plurality of magnetic fields at the first frequency.

At step416, the second position and orientation of the magnetic sensor112relative to the magnetic transmitter114(e.g., at the second reference frame i) are determined based on the third plurality of signals received from the magnetic sensor112and the distortion term. For example, the second position and orientation of the magnetic sensor112relative to the magnetic transmitter114are determined based on the third 3×3 S-matrix of data (e.g., Sreci), and based on the distortion term at the second reference frame i, (e.g., Sdisti).

In some implementations, determining the second position and orientation of the magnetic sensor112relative to the magnetic transmitter114at the second reference frame i includes determining an undistorted term, Scleani, which can be determined according to Equation (4) above. The undistorted term, Scleani, is an S-matrix that is representative of an accurate (e.g., true, correct, actual, etc.) P&O of the magnetic sensor112when the magnetic sensor112is at the second reference frame i (e.g., at S P&Oi and T P&Oi, which correspond to the second position and orientation of the magnetic sensor112and the second position and orientation of the magnetic transmitter114). In other words, the third 3×3 S-matrix of data, Sreci, may include distortions due to presence of one or more conductive or magnetic objects at or near the tracking environment106of the EMT system100, and if a P&O of the magnetic sensor112were calculated based on Sreci, the P&O may include inaccuracies in one or more dimensions. As such, the distortion term at the second reference frame, Sdisti, can be subtracted from Srecito produce a calculated S-matrix that can be resolved into an accurate P&O for the magnetic sensor112in the second reference frame (e.g., at the second P&O). In this way, the calculated second P&O of the magnetic sensor112does not include inaccuracies that would otherwise be caused by distortions in Srecidue to presence of one or more conductive or magnetic objects at or near the tracking environment106if the undistorted term, Scleani, were not considered.

In some implementations, distortion terms for various poses within the tracking environment106can be determined ahead of time as part of a mapping procedure, as described in more detail above. The distortion terms can be used to calculate undistorted terms for the sensor output when the sensor112is subsequently positioned.

As described above, the EMT system100can be operated using software executed by a computing device, such as one or more computer systems operating on the HMD102/sensor112and/or the controller104/transmitter114, and/or one or more separate computer system in communication with the sensor112and the transmitter114. In some implementations, the software is included on a computer-readable medium for execution on the one or more computer systems.FIG. 5shows an example computing device500and an example mobile computing device550, which can be used to implement the techniques described herein. For example, determining and/or adjusting distortion terms and determining the P&O of the sensor112may be executed and controlled by the computing device500and/or the mobile computing device550. Computing device500is intended to represent various forms of digital computers, including, e.g., laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device550is intended to represent various forms of mobile devices, including, e.g., personal digital assistants, cellular telephones, smartphones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the techniques described and/or claimed in this document.

Computing device500includes processor502, memory504, storage device506, high-speed interface508connecting to memory504and high-speed expansion ports510, and low speed interface512connecting to low speed bus514and storage device506. Each of components502,504,506,508,510, and512, are interconnected using various busses, and can be mounted on a common motherboard or in other manners as appropriate. Processor502can process instructions for execution within computing device500, including instructions stored in memory504or on storage device506, to display graphical data for a GUI on an external input/output device, including, e.g., display516coupled to high-speed interface508. In some implementations, multiple processors and/or multiple buses can be used, as appropriate, along with multiple memories and types of memory. In addition, multiple computing devices500can be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, a multi-processor system, etc.).

Memory504stores data within computing device500. In some implementations, memory504is a volatile memory unit or units. In some implementation, memory504is a non-volatile memory unit or units. Memory504also can be another form of computer-readable medium, including, e.g., a magnetic or optical disk.

Storage device506is capable of providing mass storage for computing device500. In some implementations, storage device506can be or contain a computer-readable medium, including, e.g., a floppy disk device, a hard disk device, an optical disk device, a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. A computer program product can be tangibly embodied in a data carrier. The computer program product also can contain instructions that, when executed, perform one or more methods, including, e.g., those described above with respect to determining and/or adjusting distortion terms and determining the P&O of the sensor112. The data carrier is a computer- or machine-readable medium, including, e.g., memory504, storage device506, memory on processor502, and the like.

High-speed controller508manages bandwidth-intensive operations for computing device500, while low speed controller512manages lower bandwidth-intensive operations. Such allocation of functions is an example only. In some implementations, high-speed controller508is coupled to memory504, display516(e.g., through a graphics processor or accelerator), and to high-speed expansion ports510, which can accept various expansion cards (not shown). In some implementations, the low-speed controller512is coupled to storage device506and low-speed expansion port514. The low-speed expansion port, which can include various communication ports (e.g., USB, Bluetooth®, Ethernet, wireless Ethernet), can be coupled to one or more input/output devices, including, e.g., a keyboard, a pointing device, a scanner, or a networking device including, e.g., a switch or router (e.g., through a network adapter).

Computing device500can be implemented in a number of different forms, as shown inFIG. 5. For example, the computing device500can be implemented as standard server520, or multiple times in a group of such servers. The computing device500can also can be implemented as part of rack server system524. In addition or as an alternative, the computing device500can be implemented in a personal computer (e.g., laptop computer522). In some examples, components from computing device500can be combined with other components in a mobile device (e.g., the mobile computing device550). Each of such devices can contain one or more of computing device500,550, and an entire system can be made up of multiple computing devices500,550communicating with each other.

Computing device550includes processor552, memory564, and an input/output device including, e.g., display554, communication interface566, and transceiver568, among other components. Device550also can be provided with a storage device, including, e.g., a microdrive or other device, to provide additional storage. Components550,552,564,554,566, and568, may each be interconnected using various buses, and several of the components can be mounted on a common motherboard or in other manners as appropriate.

Processor552can execute instructions within computing device550, including instructions stored in memory564. The processor552can be implemented as a chipset of chips that include separate and multiple analog and digital processors. The processor552can provide, for example, for the coordination of the other components of device550, including, e.g., control of user interfaces, applications run by device550, and wireless communication by device550.

Processor552can communicate with a user through control interface558and display interface556coupled to display554. Display554can be, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. Display interface556can comprise appropriate circuitry for driving display554to present graphical and other data to a user. Control interface558can receive commands from a user and convert them for submission to processor552. In addition, external interface562can communicate with processor542, so as to enable near area communication of device550with other devices. External interface562can provide, for example, for wired communication in some implementations, or for wireless communication in some implementations. Multiple interfaces also can be used.

Memory564stores data within computing device550. Memory564can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. Expansion memory574also can be provided and connected to device550through expansion interface572, which can include, for example, a SIMM (Single In Line Memory Module) card interface. Such expansion memory574can provide extra storage space for device550, and/or may store applications or other data for device550. Specifically, expansion memory574can also include instructions to carry out or supplement the processes described above and can include secure data. Thus, for example, expansion memory574can be provided as a security module for device550and can be programmed with instructions that permit secure use of device550. In addition, secure applications can be provided through the SIMM cards, along with additional data, including, e.g., placing identifying data on the SIMM card in a non-hackable manner.

The memory564can include, for example, flash memory and/or NVRAM memory, as discussed below. In some implementations, a computer program product is tangibly embodied in a data carrier. The computer program product contains instructions that, when executed, perform one or more methods, including, e.g., those described above with respect to determining and/or adjusting distortion terms and determining the P&O of the sensor112. The data carrier is a computer- or machine-readable medium, including, e.g., memory564, expansion memory574, and/or memory on processor552, which can be received, for example, over transceiver568or external interface562.

Device550can communicate wirelessly through communication interface566, which can include digital signal processing circuitry where necessary. Communication interface566can provide for communications under various modes or protocols, including, e.g., GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication can occur, for example, through radio-frequency transceiver568. In addition, short-range communication can occur, including, e.g., using a Bluetooth®, WiFi, or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module570can provide additional navigation- and location-related wireless data to device550, which can be used as appropriate by applications running on device550.

Device550also can communicate audibly using audio codec560, which can receive spoken data from a user and convert it to usable digital data. Audio codec560can likewise generate audible sound for a user, including, e.g., through a speaker, e.g., in a handset of device550. Such sound can include sound from voice telephone calls, recorded sound (e.g., voice messages, music files, and the like) and also sound generated by applications operating on device550.

Computing device550can be implemented in a number of different forms, as shown inFIG. 5. For example, the computing device550can be implemented as cellular telephone580. The computing device550also can be implemented as part of smartphone582, personal digital assistant, or other similar mobile device.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include one or more computer programs that are executable and/or interpretable on a programmable system. This includes at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

The systems and techniques described here can be implemented in a computing system that includes a backend component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a frontend component (e.g., a client computer having a user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or a combination of such backend, middleware, or frontend components. The components of the system can be interconnected by a form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

In some implementations, the components described herein can be separated, combined or incorporated into a single or combined component. The components depicted in the figures are not intended to limit the systems described herein to the software architectures shown in the figures.

A number of embodiments have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other embodiments are within the scope of the following claims.