Mobile device locationing

A mobile computing device includes: a tracking sensor; a proximity sensor; and a controller coupled to the tracking sensor and the proximity sensor, the controller configured to: obtain a sequence of sensor datasets, each sensor dataset including: (i) a location of the mobile computing device, in a local coordinate system, generated using the tracking sensor, (ii) a proximity indicator generated using the proximity sensor, defining a range to a fixed reference device, and (iii) a predefined location of the reference device in a facility coordinate system; determine, from the sequence, an adjusted pose of an origin of the local coordinate system in the facility coordinate system; and generate, using a current location of the mobile device in the local coordinate system and the adjusted pose, a corrected location of the mobile computing device in the facility coordinate system; and execute a control action based on the corrected location.

BACKGROUND

Mobile computing devices may be deployed in a wide variety of environments to perform tasks that rely at least in part on the location of the devices. For example, a mobile device may be carried by a warehouse worker and provide instructions to the worker to retrieve items in the warehouse, selected based on the current location of the device and the location of such items.

The above environments may impede the use of locationing technologies such as the global positioning system (GPS) by the devices, e.g. when the operating environment is indoors. Locationing techniques suitable for indoor use may also be subject to interference and/or other sources of error, which may reduce the accuracy thereof. Those techniques may therefore be insufficiently precise, and/or may lack the ability to detect a device orientation.

DETAILED DESCRIPTION

Examples disclosed herein are directed to mobile computing device including: a tracking sensor; a proximity sensor; and a controller coupled to the tracking sensor and the proximity sensor, the controller configured to: obtain a sequence of sensor datasets, each sensor dataset including: (i) a location of the mobile computing device, in a local coordinate system, generated using the tracking sensor, (ii) a proximity indicator generated using the proximity sensor, defining at least one of a range or a bearing angle to a fixed reference device, and (iii) an identifier indicative of a predefined location of the reference device in a facility coordinate system; determine, from the sequence, an adjusted pose of an origin of the local coordinate system in the facility coordinate system; and generate, using a current location of the mobile device in the local coordinate system and the adjusted pose, a corrected location of the mobile computing device in the facility coordinate system; and execute a control action based on the corrected location

Additional examples disclosed herein are directed to a method including: obtaining a sequence of sensor datasets at a mobile computing device having a tracking sensor and a proximity sensor, each sensor dataset including: (i) a location of the mobile computing device, in a local coordinate system, generated using the tracking sensor, (ii) a proximity indicator generated using the proximity sensor, defining at least one of a range or a bearing angle to a fixed reference device, and (iii) an identifier indicative of a predefined location of the reference device in a facility coordinate system; determining, from the sequence, an adjusted pose of an origin of the local coordinate system in the facility coordinate system; and generating, using a current location of the mobile device in the local coordinate system and the adjusted pose, a corrected location of the mobile computing device in the facility coordinate system; and executing a control action based on the corrected location.

Further examples disclosed herein are directed to a non-transitory computer-readable medium storing instructions executable by a controller of a mobile device having a tracking sensor and a proximity sensor, execution of the instructions causing the controller to: obtain a sequence of sensor datasets, each sensor dataset including: (i) a location of the mobile computing device, in a local coordinate system, generated using the tracking sensor, (ii) a proximity indicator generated using the proximity sensor, defining at least one of a range or a bearing angle to a fixed reference device, and (iii) an identifier indicative of a predefined location of the reference device in a facility coordinate system; determine, from the sequence, an adjusted pose of an origin of the local coordinate system in the facility coordinate system; and generate, using a current location of the mobile device in the local coordinate system and the adjusted pose, a corrected location of the mobile computing device in the facility coordinate system; and execute a control action based on the corrected location.

FIG.1illustrates an overhead view of a facility100in which a mobile computing device104(also referred to simply as the device104) is deployed. The facility100can include, for example, a retail facility, a warehouse, or the like, containing a plurality of support structures108such as shelves, tables or the like supporting items. Workers (e.g. human workers, autonomous or semi-autonomous devices, or the like) may travel the facility to place items on the support structures and or retrieve items from the support structures. Each worker may be equipped with a mobile computing device such as the device104. The device104can take a variety of forms, including a smart phone, a wearable computer, a tablet computer, or the like.

The device104can be configured to perform various actions related to the placement and retrieval of items on or from the support structures108, and/or related to other activities carried out by the worker operating the device104. For example, in the context of retrieving and placing items from or on the support structures108, the device104can present directional guidance to the worker, indicating a location of an item to be retrieved, a location for an item to be placed, or the like. To provide such directional guidance, the device104is configured to track its own location in the facility100, e.g. according to a facility coordinate system112previously established in the facility100. The device104may also periodically report its current location, e.g. to a server116via a network such as a wireless area network (WAN) or a wireless local area network (WLAN). The WLAN may be implemented by a set of wireless access points (APs)120-1,120-2,120-3deployed throughout the facility100and connected to the server116(or other network equipment, such as a router) via links such as the link124-3shown in connection with the access point120-3.

The server116and/or the device104can maintain a map of the facility100, including locations of the support structures108in the coordinate system112, enabling the presentation of device location, item locations and the like overlaid on the map.

Thus, various tasks performed by the device104may require knowledge of the current location of the device104in the coordinate system112. Certain locationing technologies, such as the global positioning system (GPS), may not be available to the device104, however. For example, the facility100may be an indoor facility, and/or may include metal structures (such as the support structures108) substantial enough to interfere with GPS signals. The device104is therefore configured to implement locationing techniques suitable for indoor use. As will be discussed below in greater detail, the device104is configured to combine distinct locationing techniques to increase one or both of the consistency and accuracy of the resulting location tracking, in comparison with the consistency and/or accuracy obtained by either technique alone.

Turning toFIG.2, certain internal components of the device104are shown. The device104includes a processor200(e.g. one or more central processing units), interconnected with a non-transitory computer readable storage medium, such as a memory204. The memory204includes a combination of volatile memory (e.g. Random Access Memory or RAM) and non-volatile memory (e.g. read only memory or ROM, Electrically Erasable Programmable Read Only Memory or EEPROM, flash memory). The processor200and the memory204each comprise one or more integrated circuits.

The device104also includes at least one input device208interconnected with the processor200. The input device208is configured to receive input (e.g. from an operator of the device104) and provide data representative of the received input to the processor200. The input device208includes any one of, or a suitable combination of, a touch screen integrated with the display124, a keypad, a microphone, or the like.

The device104also includes a camera212including a suitable image sensor or combination of image sensors including a Time of Flight sensor, stereoscopic cameras and the like. The camera212is configured to capture a sequence of images (e.g. a video stream) for provision to the processor200and subsequent processing. The subsequent processing can include the use of the images to track a pose of the device104, via one of the above-mentioned locationing techniques. The device104can also include an output assembly, such as a display214, which can include any suitable display panel or combination of display panels. In some examples the display214can be integrated with the input208, e.g. in the form of a touch screen. The device104can also include other output assemblies, such as a speaker, a notification LED, and the like (not shown). The device104also includes a communications interface216enabling the device104to communicate with other computing devices, such as the server116, via the APs120. The interface216therefore includes a suitable combination of hardware elements (e.g. transceivers, antenna elements and the like) and accompanying firmware to enable such communication.

The communications interface216may also be referred to as a proximity sensor herein, as the communications interface216can be configured to generate proximity indicators based on the signal strength and/or other properties of wireless signals detected by the communications interface216. For example, the communications interface216can detect beacons or other signals periodically emitted by the APs120, containing at least an identifier of the relevant AP120, and generate a received signal strength indicator (RSSI) or other proximity indicator that corresponds to a distance between the device104and the AP120. As will now be apparent, the APs120may be replaced by, or supplemented with, other wireless emitters, such as Bluetooth Low Energy (BLE) beacon devices, ultra-wideband (UWB) devices, or the like, and the communications interface216can be configured to detect signals emitted by such beacon devices, and generate the above-mentioned proximity indicators. In further examples, the reference devices can include magnetic beacons such as passive magnets or active, powered magnets, and the proximity sensor of the device104can be implemented as a magnetometer. Magnetic emitters can have distinct magnetic signatures from one another, enabling the device104to distinguish between reference devices in a manner similar to distinguishing between APs120via AP identifiers. More generally, the APs120and/or BLE beacons or other suitable emitters may be referred to as reference devices, and the device104includes a proximity sensor (e.g. the interface216) configured to detect signals from such reference devices and generate proximity indicators that correspond to a range between the device104and a reference device. The proximity indicators may be generated from signal strength measurements, time-of-flight measurements, or the like.

Further, the device104includes a motion sensor220for use in tracking the pose of the device104. The motion sensor220can include an inertial measurement unit (IMU) including one or more accelerometers, one or more gyroscopes, and/or one or more magnetometers. The motion sensor220can also include a depth sensor, such as a depth camera, a lidar sensor, or the like. Data collected by the motion sensor220is processed, in some examples along with images from the camera212, to determine a current pose of the device104. More generally, the device104is said to include a tracking sensor, which can be implemented by either of the motion sensor220and the camera212, or by a combination of the motion sensor220and the camera212.

The memory204stores computer readable instructions for execution by the processor200. In particular, the memory204stores a locationing application224(also referred to simply as the application224) which, when executed by the processor200, configures the processor200to perform various functions discussed below in greater detail and related to the tracking of the pose of the device104using a combination of locationing techniques, to generate poses (i.e. locations and orientations) of the device104in the coordinate system112. The application224may also be implemented as a suite of distinct applications in other examples. Those skilled in the art will appreciate that the functionality implemented by the processor200via the execution of the application224may also be implemented by one or more specially designed hardware and firmware components, such as FPGAs, ASICs and the like in other embodiments.

Turning toFIG.3, an example locationing technique employed by the device104is illustrated. In particular, the processor200can control the tracking sensor mentioned above (e.g. the camera212and the motion sensor220) to capture data representing the surroundings of the device104, as well as motion of the device104. For example, images captured by the camera212depict various features of the surroundings of the device104, such as a portion of a support structure108that falls within a field of view300of the camera212. The processor200detects one or more image features in the images from the camera212, such as gradient changes, edges, corners, and the like. Features can be detected via various suitable feature-detection algorithms. The processor200then tracks the changes in position of such features between successive images. The movement of the features between images, along with motion data such as acceleration and orientation change detected by the motion sensor220, is indicative of movement of the device104.

The positions of the above-mentioned features, as well as motion data from the motion sensor220, can be provided as inputs to a pose estimator implemented by the processor200, such as a Kalman filter. Various mechanisms will occur to those skilled in the art to combine image and/or motion sensor data to generate pose estimations. Examples of such mechanisms include those implemented by the ARCore software development kit provided by Google LLC, and the ARKit software development kit provided by Apple Inc.

The above pose tracking technique generates a sequence of poses of the device104relative to an arbitrary local coordinate system304, which is created by the device104when pose tracking begins. That is, the local coordinate system304may be generated at an initial pose of the device104, and the pose of the device in the local coordinate system304is tracked as the device travels, e.g. along a path308to a current position. The pose of the origin of the system304in the facility coordinate system112is not predetermined, and may therefore be unknown to the device104. In other words, although the device104may track its pose in the local coordinate system304, the device104may be dependent on further information to transform the tracked poses into the facility coordinate system.

In addition, the pose tracking mentioned above may be vulnerable to sensor drift. That is, while errors introduced between adjacent pose estimates may be small, over time such errors may accumulate to the point that the estimated pose of the device104in the local coordinate system no longer accurately represents the pose of the device104within the facility100. An exaggerated example of such drift is illustrated inFIG.3. In particular, the path308mentioned above is the path represented by successive tracked poses of the device104in the local coordinate system304. However, the true path travelled by the device104is shown as a path312, which deviates over time from the measured path308. As a result, the current pose of the device104is measured as shown in solid lines inFIG.3, while the true current pose of the device104is the pose316, which is sufficiently distant from the measured pose as to render the measured pose inadequate for at least some control actions that the device104performs based on location.

To discover the relation between the local coordinate system304and the facility coordinate system112, and to counteract the drift mentioned above, the device104is configured to implement a second locationing technique simultaneously with the technique shown inFIG.3. In particular, the device104is configured, using the communications interface216or other suitable proximity sensor, to detect a signals emitted by the APs120or other suitable reference devices as noted above. An example signal318emitted by the AP120-3is shown inFIG.3. As will be apparent, the signal318is detected by the device104at the true position316of the device104. Based on the detected signal, the device104can generate a proximity indicator corresponding to a range between the device104and the AP120-3. The fixed locations of the APs120in the facility coordinate system112are available to the device104, either in the memory204or via the server116. When multiple ranges are available, the device104may triangulate its current location according to the predefined locations of the relevant APs120. However, at any given time, the device104may detect too few signals to permit such triangulation. Further, the signals from the APs120or other reference devices may be affected by interference, multi-path artifacts, and the like, which can result in inaccurate proximity indicators (e.g. inaccurate estimates of the range between the device104and the relevant AP120). The proximity indicators are not, however, vulnerable to the sensor drift noted above in connection with the image and motion-based locationing technique. Stated another way, the feature-based locationing using the camera212and motion sensor220may be subject to low-frequency error in the form of drift, while the proximity-based locationing may be subject to high-frequency noise, but experiences little or no low-frequency drift.

The device104is therefore configured to combine aspects of feature-based locationing and proximity-based locationing to counteract the effect of sensor drift, while maintaining the ability of feature-based locationing to generate pose estimates with lower levels of noise than proximity-based locationing.

Referring toFIG.4, the device104is shown in isolation, to illustrate the representation of a pose of the device104according to the two distinct coordinate systems112and304. The pose of the device104is defined by a location400, and an orientation404, both of which are determined in the local coordinate system304, via the feature-based locationing technique using the above-mentioned tracking sensor.

The location400represents the location of a centroid of the device104in the local coordinate system304. In other embodiments, the location400can correspond to a different point of the device104. The location400is therefore defined by an X coordinate408in the local coordinate system304, a Y coordinate412in the local coordinate system304, and an angle in the XY plane, e.g. an angle416relative to the X axis of the local coordinate system304. As will be apparent, the local coordinate system304and the facility coordinate system112may be three-dimensional systems, and the pose of the device104may therefore be defined with an additional coordinate and two additional angles. However, in this example the vertical dimension is omitted for simplicity of illustration.

To make use of the pose of the device104for providing directional guidance or other functions within the facility100, the pose may be transformed to a pose in the facility coordinate system112. Such a transformation includes applying a translation and/or a rotation to the pose in the local coordinate system304. The transformation is defined by the pose of the origin of the local coordinate system304within the facility coordinate system112. The origin of the local coordinate system304has a pose defined by X and Y coordinates420and424, respectively, in the facility coordinate system112, as well as an angle428, e.g. relative to the X axis of the facility coordinate system112.

As will be apparent in the discussion below, the transformation between coordinate systems112and304may initially be unknown as the local coordinate system304is generated arbitrarily by the device104. Further, even when the above transformation is discovered (i.e. once the coordinates420and424, and the angle428, are available to the device104), sensor drift may result in the coordinates408and412and the angle416no longer accurately defining the true position of the device104once transformed via the coordinates420and424, and the angle428. The device104is therefore configured to periodically adjust or update the pose of the origin of the local coordinate system304within the facility coordinate system112, to counteract sensor drift.

Turning toFIG.5, a method500of sensor drift compensation is illustrated. The method500will be described in conjunction with its performance by the device104. However, in some examples, certain processing involved in the method500may be off-loaded to the server116, e.g. depending on the computational capabilities of the device104.

At block505, the device104is configured to initiate pose tracking. Specifically, at block505the device104creates the local coordinate system304, e.g. with an origin at the current location400of the device104and a Y axis aligned with the current orientation404of the device104. The pose of the local coordinate system304relative to the facility coordinate system112is, in this example, not yet known. Initiation of pose tracking at block505occurs in response to execution of the application224, e.g. in response to an input from the operator of the device104, a command from the server116, or the like.

To initiate pose tracking, the processor200controls the tracking sensor (e.g. the camera212and the motion sensor220, in this example) to begin capturing a stream of images and/or motion data. As discussed earlier, the captured data is employed to track a current pose of the device104in the local coordinate system304. Pose tracking, once initiated at block505, continues throughout the remainder of the method500. The frequency with which new pose estimates are generated by the device104varies, for example with the computational resources available to the device104, the frame rate of the camera212, and the like. For example, the device104may generate pose estimates at a frequency of about 30 Hz, although higher and lower frequencies are also contemplated.

At block510, the device104is configured to obtain a sensor dataset that will be employed in determining a pose of the device104in the facility coordinate system112. The sensor dataset includes at least the location400of the device104in the local coordinates system304, and may also include the orientation404of the device104. As will be apparent, the location400(and orientation404, when used) are obtained using the tracking sensor, e.g. the camera212and motion sensor220. The sensor dataset also includes a proximity indicator, generated using the proximity sensor of the device104(e.g. the communications interface216), and defining a range to a reference device, such as an AP120. Further, the sensor dataset includes a predefined location of the reference device in the facility coordinate system112. The predefined location of the reference device may be included in the signal from which the proximity indicator was derived. Alternatively, the signal from the reference device may contain a reference device identifier, and the predefined location may be retrieved from the memory204based on the reference device identifier.

FIG.6illustrates a first sensor dataset, obtained upon initiation of pose tracking. In particular, the first sensor dataset includes a location600of the device104, which in the illustrated example is at the origin of the local coordinate system304. Of particular note, the transform between the local and facility coordinate systems304and112may not yet have been discovered. Nevertheless, for illustrative purposes it is assumed that the device location shown inFIG.6is also the true position of the device104. The sensor dataset also includes a range602, determined from a proximity indicator such as an RSSI derived from a signal emitted by the AP120-1. The range602, as shown inFIG.6, is greater than the actual distance between the AP120-1and the device104, e.g. because of the various sources of error involved in such range measurements, as noted earlier. The sensor dataset also includes a position of the AP120-1in the facility coordinate system, defined by coordinates604and608.

Returning toFIG.5, at block515, the device104is configured to determine whether to adjust the origin of the local coordinate system304relative to the facility coordinate system112. Such adjustment is performed using multiple sensor datasets, and aims to counteract sensor drift that may occur in the pose tracking initiated at block505. The determination at block515can include, for example, whether a sequence of sensor datasets larger than a threshold number has been accumulated via successive performances of block510. For example, the threshold may be five datasets, in this example. A wide variety of other thresholds may be applied in other examples, however. For example, in some examples the threshold can be one hundred datasets, although smaller and larger thresholds are also contemplated. If the tracking sensor (e.g. the camera212and motion sensor220) loses track of the pose of the device104, e.g. due to low light levels, severe motion, camera obscuration, etc., the processor200can trigger an immediate update of the pose estimation, regardless of the threshold above which is designed to accommodate slow drifts of the local position estimation process.

In this example, the determination at block515is negative, because only one dataset has been captured. The device104therefore skips blocks520and525, and proceeds to block530. At block530, the device104uses the pose of the local origin in the facility coordinate system112to convert the location400(which is in the local coordinate system304) into a location in the facility coordinate system112. In this example, however, the pose of the local origin in the facility coordinate system112is not yet discovered, and block530may therefore be omitted, with the performance of the method500instead returning from block515to block510.

In other words, block510is repeated until a sufficient number of datasets is available to adjust the pose of the origin of the local coordinate system304in the facility coordinate system112. When the threshold number is reached, the determination at block515is affirmative, and the device104proceeds to block520.FIG.7illustrates four further performances of block510, for a total of five datasets. InFIG.7, the true location of the device104is shown in dashed lines, and the measured location of the device104in the local coordinate system304is shown in solid lines, illustrating an increasing amount of drift over time. Each of the additional datasets includes a location700(specifically, locations700-1,700-2,700-3, and700-4) in the local coordinate system304, as well as a range702(specifically, ranges702-1,702-2,702-3, and702-4) to an AP120. As seen inFIG.7, the ranges may be from different APs120as the device104moves through the facility100. In some examples, the device104can be configured to retain only one range (e.g. the smallest range, corresponding to the greatest signal strength, shortest time-of-flight or the like), when multiple APs120are close enough to be detected. In other examples, each dataset can include multiple ranges and corresponding AP120locations in the facility coordinate system112.

When the determination at block515is affirmative, at block520the device104selects a sequence of datasets, such as the preceding five datasets in this example (e.g. the current dataset, corresponding to the current location700-4and the five previous locations).

At block525, the device104is configured to generate an adjusted pose of the origin of the local coordinate system304, in the facility coordinate system112. The generation of the adjusted pose at block525includes selecting parameters defining the origin of the local coordinate system304so as to minimize an error metric, as will be defined in greater detail below. The parameters selected are the parameters discussed in connection withFIG.4as defining the pose of the local origin. Specifically, the parameters selected at block525are the coordinates420and424, as well as the angle428.

The error metric noted above is a difference (i.e. an error) between the true range from an AP120or other reference device to the device104, and a distance between the AP120and the device104calculated using the predefined location of the AP120and the location of the device104, as determined using the location400.FIG.8illustrates an example of the above-mentioned error.

FIG.8shows one dataset fromFIG.7in isolation. In particular, the dataset includes the location700-4, and a true position800of the device104. The location700-4is initially determined in the local coordinate system304, and its location in the facility coordinate system112may in fact not be known (e.g. if the transform between coordinate systems has not been discovered). However, it is assumed to be different from the true position of the device104, e.g. due to sensor drift in the feature-based locationing technique. The range702-4is also shown between the AP120-2and the true location800of the device104. Because the location700-4is inaccurate, a distance804between the location700-4and the AP120-2does not equal the range702-4. The difference between the range702-4and the distance804is the above-mentioned error. As will be apparent, selecting the parameters420,424, and428may shift the local coordinate system304such that the location700-4is shifted closer to the true location800, and the above error is reduced.

At block525, the device104is therefore configured to determine which values for the parameters420,424, and428result in a minimization of the above error. Further, because the device104is solving for multiple parameters, and because the ranges702include noise-related inaccuracies, the device104does not use a single dataset, but rather a plurality of datasets. The noise-related errors in the ranges702are assumed to be random in nature, and may therefore be counteracted by making use of a number of samples.

Thus, the device104is configured to select values for the parameters420,424, and428that minimize the above error across all of the selected datasets from block520. The error may be expressed as follows:

In the above, r is the range derived from a proximity indicator such as a signal strength (e.g. the range702-4shown inFIG.8) or round trip time delay (RTT). The values “x0” and “y0” are the parameters420and424, respectively. The angle theta is the parameter428mentioned above. The values “x1” and “y1” are the coordinates of the device104in the local coordinate system, and the values “xap” and “yap” are the coordinates of the relevant AP120in the facility coordinate system.

At block525, the device104can be configured to solve a set of equations as presented above, one for each dataset, to select the values x0, y0, and θ that minimize the total combined error (e.g. the sum of the squares of the errors for each dataset). The parameters above may be selected by executing a suitable non-linear mean square error (MSE) solver, as will be apparent to those skilled in the art. In other examples, a filter, such as a Kalman filter, may be employed to select the parameters420,424, and428at block525.

Turning toFIG.9, an example outcome of a performance of block525is shown. In particular, the local coordinate system304has been replaced with an adjusted local coordinate system304a, which has a different location and orientation in the facility coordinate system112(visible from the fact that the coordinate system304ais no longer centered on the initial location600). As a result of the adjustment of the local coordinate system304, the locations700-1,700-2,700-3, and700-4are closer to the true positions of the device104shown in dashed lines. Of particular note, the local coordinates of each dataset have not been altered. That is, the locations700are in the same positions as inFIG.7, relative to the local coordinate system304a. The adjustment of the origin of the local coordinate system304a, however, results in adjustment of the locations700relative to the facility coordinate system112to more closely align with the true positions of the device104, reducing the impact of sensor drift.

Returning toFIG.5, at block530the device104is configured to generate a corrected location, using a current location700in the local coordinate system and the parameters420,424and428selected at block525. In other words, the local coordinates for the location700-4, for example, are transformed to the facility coordinate system112using the parameters420,424, and428selected at block525. At block535, the device104can optionally select and execute a control action based on the corrected location. For example, the device104can present the corrected location on the display214, e.g. along with updated directional guidance to another location in the facility100, transmit the corrected location to the server116, or the like.

The device104can then return to block510. That is, the selection of new values for the parameters420,424, and428can be repeated periodically (including as frequently as with every new dataset once the threshold number of datasets mentioned above has been met). As a result, the origin of the local coordinate system112moves through the facility coordinate system112over time, to counteract the effect of sensor drift in the locations700obtained using the tracking sensor (e.g. the camera212and motion sensor220).

As will be apparent to those skilled in the art, performance of the method500by the device104enables the device104to track its location using a feature-based technique that may enable greater update rates and less high-frequency noise than the proximity-based technique, while employing the proximity-based technique to correct for low-frequency error (e.g. drift) that may otherwise accumulate when using the feature-based locationing technique. The above combination may enable the device104, in other words, to obtain greater locationing accuracy and consistency than via either technique in isolation.

In other implementations, the proximity indicators mentioned above can be supplemented with or replaced by other forms of proximity indicators, such as bearing angles from the device104to visual features in the facility100with predefined locations in the facility coordinate system112. Such an implementation is detailed below, with reference toFIG.10.
error=α−atan 2((yapl−yl),(xapl−xl))

The error above is therefore a difference between the bearing angle perceived by the device104in the local coordinate system, and the true bearing angle between the device104and the reference device. In the above, the angle alpha is a bearing angle1000(shown inFIG.10) between the device's heading (i.e. the orientation404) and the reference device120, in the local coordinate system. The arguments of the two-argument arctangent function include the values “xl” and “yl” which, as noted earlier, are the coordinates of the device104in the local coordinate system. The values “yapl” and “xapl” are the coordinates of the reference device in the local coordinate system. The device104may derive “yapl” and “xapl” as set out below.
xapl=(xap−x0)cos θ+(yap−y0)sin θ
yapl=(yap−y0)cos θ+(xap−x0)sin θ

The values “xap” and “yap” are the coordinates of the relevant AP120in the facility coordinate system, as noted earlier. The values “x0” and “y0” are the parameters420and424, respectively. The angle theta is the parameter428mentioned previously. The device104is configured to solve a set of equations as presented above, one for each dataset, to select the values x0, y0, and θ that minimize the total combined error (e.g. the sum of the squares of the errors for each dataset).

The reference devices, in the above example, can include lights configured to emit specific identifying patterns, fiducial markers, or the like. In some examples, a reference device can combine an AP120with the above visual features (e.g. an AP can include a housing bearing a fiducial marker). As will be apparent, a reference device with visual features as mentioned above may be detected with the camera212, rather than the communications interface216. The camera212, in other words, may therefore act as at least one of, and potentially both of, the tracking sensor and the proximity sensor.

In further implementations, at block520the device104can be configured to omit from the selected datasets and datasets with ranges702that deviate from preceding or following ranges702(e.g. based on signals from the same AP120) by more than a threshold. Such a deviation may indicate the presence of a multi-path artifact or other error that is sufficiently large to disqualify the dataset from use at block525.

In some deployments, reference devices such as the APs120may not have been installed with locationing as described above in mind. For example, the APs120may have been installed in the facility to provide network coverage, which may require less accurate knowledge of the facility coordinates of the APs120than is necessary for use in the method500. The device104can therefore also be configured to perform a surveying process, prior to performing the method500, to determine the locations of APs120with sufficient accuracy for use in the method500.

In particular, referring toFIG.11, a survey method1100is illustrated, which can be performed by the device104prior to performance of the method500. In particular, at block1105, the device104is configured to obtain coordinates, in the facility coordinate system112, of at least two anchor features in the facility. The anchor features are detectable by the tracking sensor, and can therefore include features detectable by the camera212, such as a corner of a support structure (e.g. a shelf), or the like. The coordinates obtained at block1105are measured, in the facility coordinate system112, prior to performance of the method1100, and can then be stored in the memory204or otherwise made available to the device104.

At block1110, the device104is configured to initiate pose tracking, as described above in connection with block505. At block1115, the device104is configured to detect the anchors mentioned above, and to determine the pose of the local coordinate system304, based on the local pose of the device104at the time(s) the anchors are detected, and the previously measured facility coordinates of the anchors. For example, referring toFIG.12, anchors1200-1and1200-2are shown at predefined locations in the facility, and whose coordinates in the facility coordinate system112are obtained by the device104at block1105. The device104is configured to use the facility coordinates of the anchors1200, as well as corresponding device positions in the local coordinate system304, to determine the pose of the local coordinate system304in the facility coordinate system112, for example as follows:
x0=(xa1−xa1L)
y0=(ya1−ya1L)
θ=arctan 2(ya2L−ya1L),(xa2L−xa1L))

In the above expressions, x0and y0are the parameters420and424. The “a1” values (x and y) represent the coordinates of the first anchor1200-1in the facility coordinate system112. The “a1L” values (x and y) represent the coordinates of the first anchor1200-1in the local coordinate system304. The “a2L” values represent the coordinates of the second anchor1200-2in the local coordinate system. The angle theta is the parameter428. From the above, the coordinates of the device104in the facility coordinate system112can be determined.

Returning toFIG.11, at block1120, the device104is configured to detect a plurality of proximity indicators for at least one of the APs120. As noted above, the proximity indicators can be RSSI values, e.g. an RSSI value determined from a signal1204(seeFIG.12) detected at the device104from the AP120-1. The proximity indicators can be collected as the device104travels through the facility. Having collected at least a predetermined number of proximity indicators for a given AP120, the device104is configured to determine the location of that AP120in the facility coordinate system112. For example, the device104can solve for values of the AP coordinates that minimize an error between the range from the device104to the AP120as determined from the proximity indicators, and the distance from the device104to the AP120as determined from the facility coordinates of the device and the coordinates of the AP120itself (which are not yet known). In other words, the device104can be configured to solve a plurality of expressions as set out below, one for each proximity indicator and corresponding device coordinates (in the facility coordinate system112)
error=r−√{square root over ((xd−xap)2×(yd−yap)2)}

In the above expression, r is the range derived from a proximity indicator such as a signal strength or round trip time delay (RTT). The values “xd” and “yd” are the coordinates of the device104in the facility frame of reference112, and the values “xap” and “yap” are the coordinates of the relevant AP120in the facility coordinate system112, which are to be solved for. Having solved for AP coordinates that minimize the error above, the device104can store the AP coordinates for later use in the method500. In some examples, the device104can determine whether the sum of the error from each proximity indicator used to derive the AP coordinates is below a threshold, before storing the AP coordinates for later use.

At block1125, the device104is configured to determine whether further APs120remain to be located. When the determination at block1125is affirmative, the device104returns to block1120, to continue traveling the facility and collecting proximity indicators (e.g. ignoring those of the AP120that has been successfully surveyed). In some examples, the device104can also periodically return to the anchors1200to update the pose of the local frame of reference304in order to avoid accumulating sensor drift. When the determination at block1125is negative, the method1100ends.