Patent Description:
Various use cases are known that rely on tracking a <NUM>-D position of a target. Example use cases include human-machine interfaces (HMIs): here, the <NUM>-D position of a user-controlled object, implementing the target, can be tracked. It can be determined whether the target performs a gesture. It could also be determined whether the target actuates an input element of a user interface (UI).

<CIT> discloses using an interacting multi-model for estimating the kinetic state of the user equipment. A <NUM>-mode drone movement model is created. The three models are 3D constant velocity movement Wiener process, 3D constant acceleration movement Wiener process, and a 3D constant position Wiener process.

<CIT> discloses a tracking unit configured to track captured positioning data. An interacting multi-model Kalman filter can be used. The interacting multi-model Kalman filter can consider different motion models, curve flying, flying straight on, rest.

<CIT> discloses autonomous target following methods and devices. measure the internal states of an autonomous following robotic device, e.g., a position, a velocity, and acceleration. GPS or inertial navigation systems can be used. The surrounding environment can be measured using laser scanners rate us, etc..

Accordingly, there may be a need for providing a robust and accurate estimate of the <NUM>-D position of a movable target such as hand, part of a hand, or a handheld pointing device.

This need is met by the features of the independent claims <NUM> and <NUM>.

It is to be understood that the features mentioned above and those yet to be explained below may be used not only in the respective combinations indicated, but also in other combinations or in isolation.

In the following, examples of the disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the following description of examples is not to be taken in a limiting sense. The scope of the disclosure is not intended to be limited by the examples described hereinafter or by the drawings, which are taken to be illustrative only.

Hereinafter, techniques will be described that facilitate estimating a <NUM>-D position of a movable target. According to the various examples described herein, it is possible to obtain an estimate of the <NUM>-D position of the movable target (hereinafter, simply target) at a given point in time or at multiple points in time, i.e., time resolved.

According to the various examples, the <NUM>-D position can be described by a state and state covariance - i.e., describing the estimation error of the state - of the target. The state can be expressed by a state vector that includes entries that describe the location in space, i.e., the <NUM>-D position. Optionally, the state vector could include further entries, e.g., velocity and/or acceleration.

In other words, according to the various examples disclosed herein, the <NUM>-D position of the moveable target can be tracked.

Tracking data that is determined by a respective tracking algorithm can include the at least parts of the state vector and optionally the state covariance. The tracking data can include the estimate of the <NUM>-D position of the moveable target.

As a general rule, the tracking data can include or, at least, indicate additional information. The additional information may be obtained from the tracking algorithm. For example, it would be possible that the tracking data is indicative of a likelihood that the target is in a given one of multiple predefined states of its motion (simply state hereinafter). This could be achieved by including a respective likelihood. It would also be possible that the information content or the structure of the information content is changed, depending on whether or not the target is in a given one of multiple predefined states. Such augmented information facilitates post-processing. For instance, it would be possible to select between multiple post-processing algorithms, depending on such additional information.

According to various examples described herein, the tracking data can be determined based on depth data that is indicative of a time-resolved measurement of a <NUM>-D position of the target in a field-of-view (FOV) of a depth sensor.

According to the various techniques described herein, various types of depth sensors can be used to obtain the depth data. Some options are summarized in TAB. <NUM> below.

It is possible to employ different depth sensors, cf. <NUM>, depending on the particular use case. In particular, the size of the FOV is correlating with the size of the target. For instance, LIDAR and radar sensors can be configured for short-range sensing or long-range sensing. Long-range sensing - e.g., having FOVs with dimensions in the range of meters or several tens or even hundreds of meters - may be preferable when tracking people in a traffic surrounding, while short-range sensing - e.g., having FOVs with dimensions in the range of centimeters or tens of centimeters - may be preferable when tracking a finger or a hand or a handheld device.

According to some examples, the tracking data is used to control an HMI. The HMI may detect gestures. A gesture can be defined by a certain movement (e.g., having a certain shape or form) and optionally velocities or accelerations performed by the target. The HMI may employ a UI. The UI includes one or more input elements that are defined with respect to the FOV of the depth sensor. For example, it is possible to determine, based on the tracking data, whether the target addresses a certain input element, e.g., by hovering without movement in an area associated with that input element. It could then be judged whether the certain input element is actuated, e.g., if the target addresses the certain input element for a sufficiently long time duration. A specific type of use case employing such an HMI would be the tracking of a palm or finger or a handheld pointing device (such as a stylus) on and above a touchscreen of an infotainment system or a screen for ticket machines for touchless sensing.

As will be appreciated from the above, along with the various possible use cases of the tracking data, different types of targets can be tracked. As a general rule, the target could be one of the following: a hand or a part of the hand of a person; a handheld pointing device; a person; a vehicle; etc..

Various techniques are based on the finding that it can be helpful to detect rest states of the target, e.g., in the various use cases described above. The rest state can be associated with the hand or finger not moving above an input element of a UE, or a person standing at the side of the street, or a user resting in a virtual-reality application. According to techniques described herein, it is possible to reliably detect the target being in a rest state. An accurate estimate of the <NUM>-D position can be determined, in particular when the target is in the rest state.

According to various examples, a tracking algorithm is employed. The tracking algorithm can iteratively process the depth data, for multiple time increments. , each iteration of multiple iterations can correspond to a respective time increment. The tracking algorithm provides the tracking data.

The tracking algorithm can predict, for each iteration of multiple iterations, the <NUM>-D position of the moveable target using one or more assumptions for the movement of the target and based on the depth data. This prediction can be based on the state and state covariance of the previous iteration, as well as up-to-data depth data of the respective iteration.

According to various examples described herein, a specific kind of tracking algorithm can be employed. Specifically, a multi-model (MM) tracking algorithm can be used to process the depth data obtained from the depth sensor. The MM tracking algorithm provides the tracking data.

As a general rule, the MM tracking algorithm employs multiple motion models that process the depth data simultaneously, but using different calculations. In particular, the multiple motion models can use different assumptions with respect to the observed motion state of the target. Different models can be parameterized differently. Different models can be used to track different types of motion that the target is expected to engage in. Each model predicts where the target will be after a time increment associated with each iteration of the MM tracking algorithm, if the target engages in the particular type of motion associated with the respective motion state associated with that model.

The individual outputs of the models - i.e., a respective predicted state and state covariance - are combined by a fusion module of the MM tracking algorithm - e.g., in a weighted manner based on the likelihood that the target performs the motion state modeled by the respective model -, to thereby provide an overall estimate of the <NUM>-D position of the moveable target.

In even further detail, the MM tracking algorithm may be implemented by an interacting MM (IMM) tracking algorithm. The IMM is a modification of the MM; here, the multiple models are interacting. This means that outputs of at least some of the multiple models influence the inputs of at least some of the multiple models in a subsequent iteration. A Markovian decision process may be used to consider the a-priori likelihood of the target being in the respective motion state of that model at each iteration. The transition probabilities of the Markovian decision process - describing the likelihood of a transition from a first motion state to a second motion state and describing the likelihood of the target remaining in a given motion state - can be predefined in the various examples described herein. This likelihood can then be used to determine an a-priori state estimate for the target, i.e., in particular an a-priori estimate of the <NUM>-D position of the target.

According to various examples described herein, the IMM tracking algorithm employs state models according to TAB.

In both models according to example I and II of TAB. <NUM>, it is possible to convert between the polar coordinates and Cartesian coordinates, where the depth data is available in polar coordinates due to the formatting of the raw measurement data and the sensor modality. Such conversion is described by Eqs. <NUM> and <NUM> below. <MAT> <MAT> r denotes the radial distance and θ describes the angular coordinate.

This defines the output transition matrix H - implementing a non-linear transformation - describing the measurement system providing the depth data, according to: <MAT> where Z is the projection of the predicted state X onto the measurement subspace. When obtaining depth data Y (r, θ, vr) at time t, then the innovation is given by Z-y, i.e., as the difference between actual measurement from the sensor and the projected predicted measurement from the IMM tracking algorithm. The innovation is also used for computing Mahalanobis distance - as will be explained below in connection with Eq. <NUM> - and thus model likelihoods.

For Eq. (<NUM>), unscented transformations can be used, e.g., as part of the unscented Kalman filter.

Using such IMM tracking algorithm can be helpful to facilitate robust and accurate tracking of the target. In particular, it is possible to adequately cover scenarios where the target does not move, but stands still. Various techniques are based on the finding that such a resting target can be relevant in many use cases. One such use case that can profit from these techniques will be described next in connection with <FIG>.

<FIG> schematically illustrates aspects with respect to a system including a radar depth sensor <NUM> (cf. <NUM>) and a UI <NUM> including multiple input elements <NUM>-<NUM>.

In the illustrated example, the radar sensor <NUM> includes two transmitters <NUM>, <NUM> that can transmit millimeter electromagnetic waves. A phased array antenna can be used. The radar sensor <NUM> also includes a receiver <NUM> that can detect backscattered electromagnetic waves. Beamforming can be used in order to detect the lateral position in the xy-plane of the object. The depth position - along the z-axis - can be judged from a phase shift of the backscattered electromagnetic waves with respect to the emitted electromagnetic waves. Thereby, it is possible to detect objects in a scene, in particular a target <NUM>. The FOV <NUM> of the radar depth sensor <NUM> is also illustrated.

Within the FOV <NUM>, the UI <NUM> is predefined. The UI <NUM> includes multiple input elements <NUM>-<NUM>. For instance, these input elements <NUM>-<NUM> could be associated with different buttons that are displayed on a screen <NUM>. For example, different tickets of a ticket vending machine may be associated with the different buttons, to give just one practical example.

There are various modes conceivable for the target <NUM> to interact with the UI <NUM>. Some modes are summarized in TAB.

<FIG> schematically illustrates aspects with respect to the rest state <NUM> of the target <NUM>, according to a TAB. <NUM>: example II. <FIG> is a schematic top view. <FIG> illustrates that the target <NUM> statically hovers in the region associated with the input element <NUM>. The respective <NUM>-D position <NUM> of the target <NUM> is illustrated.

Also illustrated is a measurement noise <NUM>. Typically, the radar sensor <NUM> exhibits measurement inaccuracies such as statistical fluctuations and, accordingly, the depth data obtained from the radar sensor <NUM> can experience a blur of the position <NUM> defined by the measurement noise <NUM>.

According to various examples, it is possible to provide an accurate estimate of the <NUM>-D position <NUM> and, more specifically, provide an accurate estimate of the target <NUM> addressing the input element <NUM>, even in view of the measurement noise <NUM>. Outliers - e.g., sudden jumps in the measured state due to noise - can be removed.

<FIG> schematically illustrates aspects with respect to a movement state <NUM> of the target <NUM>, according to TAB. <NUM>: example I. <FIG> is a schematic top view, corresponding to the schematic top view of <FIG> illustrates that the target <NUM> moves - at a certain velocity <NUM> - along a trajectory <NUM>, between a start position <NUM> and an end position <NUM>. The position <NUM> changes over the course of time. The target <NUM> may perform a swipe gesture such as a swipe-to-unlock or swipe-to-confirm gesture in the illustrated example.

<FIG> schematically illustrates the signal processing according to various examples. <FIG> illustrates a processing flow of multiple logical operations.

At box <NUM>, a depth sensor - e.g., the radar sensor <NUM> of <FIG>; also cf. <NUM> - is used to acquire raw measurement data <NUM>. The raw measurement data <NUM> is suitable for determining the <NUM>-D position of the target <NUM>. Thus, a Z-position of the target <NUM> can be measured.

The particular measurement modality used to obtain the raw measurement data <NUM> is not germane to the functioning of the techniques described herein.

The raw measurement data <NUM> is then pre-processed at box <NUM>, to obtain depth data <NUM>.

The depth data <NUM> comprises one or more observables indicative of the <NUM>-D position of the target <NUM>. For instance, the radial distance in a polar reference coordinate system could be indicated. It would also be possible that the xyz-position in a Cartesian coordinate system is indicated. A conversion according to Eqs. <NUM> and <NUM> can be made.

The depth data <NUM> can be subject to measurement noise stemming from imperfections of the depth sensor.

Next, tracking of the <NUM>-D position of the target is performed at box <NUM>. An IMM tracking algorithm can be employed. Thereby, tracking data <NUM> is obtained. The tracking data comprises an estimate of the <NUM>-D position of the target <NUM>. The measurement noise can be reduced.

The tracking data <NUM> can also be indicative of further information, e.g., a likelihood of the target <NUM> being in either the rest state <NUM> or the movement state <NUM>. Such additional information can be obtained from the IMM tracking algorithm of box <NUM>. Such additional information can be used to configure post-processing of the tracking data <NUM> at box <NUM>. For instance, a respective post-processing algorithm may be selected depending on the assistance information.

In detail, a use-case specific application is executed at box <NUM> based on the tracking data <NUM>. Various use cases have already been explained above, e.g., a UI including multiple input elements can be used to control an HMI (cf. It is possible that the application provides an output to the user such that a continuous user-machine interface - illustrated in <FIG> by the dashed arrows - is implemented.

<FIG> illustrates details with respect to an example implementation of box <NUM>. The raw measurement data <NUM> corresponds to multiple range Doppler maps obtained from multiple receivers of the radar sensor <NUM>. Based on Doppler information, a moving target indication (MTI) can be performed. Digital beamforming (DBF) can be used to separate waveform is associated with different directions in the scene. A Doppler analysis can be performed to obtain an indication of the velocity.

Next, object detection can be used using constant false alarm rate (CFAR) algorithms. In particular, in order-statistic (OS) CFAR algorithm can be used to suppress clutter. See, e.g.,<NPL>.

Clustering can be used to determine extensions of objects, including the target. One algorithm is DBSCAN, see <NPL>.

Then, parameters of the target can be estimated, e.g., the center (centroid estimation) and the angle of arrival.

Finally, the depth data <NUM> is obtained. In the illustrated example a vector is obtained that specifies distance/range, angle and speed of a center of the target; i.e., the <NUM>-D position is defined in a polar reference coordinate system.

<FIG> is only one example of pre-processing at box <NUM>. In particular, depending on the type of depth sensor used (cf. <NUM>), different pre-processing techniques may be applied at box <NUM>. Even when preprocessing radar raw measurement data <NUM>, different preprocessing techniques are available. The particular type of pre-processing is not germane for the various techniques described herein.

<FIG> schematically illustrates the tracking of the target based on the depth data <NUM> executed at box <NUM>. Specifically, <FIG> illustrates aspects with respect to a possible implementation of the IMM tracking algorithm <NUM>.

At the core of the IMM tracking algorithm <NUM> are the movement state model <NUM>, and the rest state model <NUM>, according to TAB. <NUM>: example I and example II, respectively.

Both models <NUM>, <NUM> process, in parallel, the depth data <NUM>, at multiple iterations <NUM> of the IMM tracking algorithm <NUM>. Per iteration <NUM>, a respective output of the process explained above in connection with box <NUM> may be obtained and processed as the depth data <NUM>; this depth data <NUM> then corresponds to the target being observed at the respective time instance associated with this iteration <NUM>. The depth data <NUM> thus provides the measured state of the target <NUM> (denoted Y above). The depth data <NUM> includes the measured state vector. The depth data <NUM> is used to make state predictions.

The movement state model <NUM> outputs a first state vector <NUM> and an associated second state covariance <NUM>, as explained in connection with Eq. <NUM> above. The first state vector <NUM> includes the <NUM>-D position <NUM> and one or more measures of the velocity <NUM>. For example, the polar velocity ω and the radial velocity vr can be used in the coordinated turn model explained in TAB. <NUM>: example I.

The rest state model <NUM> outputs a second state vector <NUM> and an associated second state covariance <NUM>, as explained above in connection with Eq. <NUM> above. The second state vector <NUM> output by the rest state model <NUM> includes the <NUM>-D position of the target, but does not include the velocity - because the velocity is, per definition, zero for the rest state model <NUM>.

The IMM tracking algorithm <NUM> generally combines state hypotheses from multiple filter models to get a better state estimate of targets with changing dynamics. A fusion module includes two sub-modules <NUM>, <NUM>. The fusion module allows to combine the state hypotheses of the two models <NUM>, <NUM>. Details with respect to the fusion module are explained below.

Firstly, a model likelihood module <NUM> of a fusion module determines a first likelihood <NUM> of the target <NUM> being in the movement state <NUM> associated with the movement state model <NUM> and further determines a second likelihood <NUM> of the target <NUM> being in the rest state <NUM> associated with the rest state model <NUM>.

This determination can be based on the respective state covariances <NUM>, <NUM>. For example, the larger the uncertainty, the less likely the respective state.

In particular, it would be possible to determine a first distance between the state estimate <NUM>, <NUM> of the movement state model <NUM> and the measured state of the depth data <NUM> - i.e., the innovation -, as well as to determine a second distance between the state estimate <NUM>, <NUM> of the rest state model <NUM> and the measured state of the depth data <NUM>. A Mahalanobis distance can be used, to be able to consider the uncertainty of the state estimates described by the state covariances <NUM>, <NUM>. These distances can serve as an estimate of the accuracy of each model <NUM>, <NUM>, i.e., quantify how much the estimated state differs from the true measured state. Based on the accuracies of the state estimates of each model <NUM>, <NUM>, it is then possible to determine the first likelihood <NUM> and the second likelihood <NUM>, respectively.

Optionally, it would be possible to consider an evolution of such accuracy over two or more previous iterations when determining these likelihoods <NUM>, <NUM>. For this, the model likelihood module <NUM> can employ a memory module <NUM>. The memory module <NUM> can store the respective values - e.g., state, state covariance, measured state - over multiple iterations. It would then be possible to track the evolution of the accuracy over the multiple iterations <NUM>.

In a specific implementation, the likelihood of the target being in the respective state may be given by: <MAT> where z is the measured state estimate from the depth data <NUM> and Z is defined in accordance with Eq. <NUM>. , Y-Z is the innovation. S is the covariance matrix of the state covariance <NUM> or <NUM>. The subscript "hist" describes the respective values of one or more previous iterations <NUM>.

Such tracking of the accuracies helps to increase the score of the model which matched well to the history and also prevents sudden jumps.

Second, it is then possible to determine the tracking data <NUM> based on a weighted combination using weighting factors that are set in accordance with the first likelihood <NUM> and the second likelihood <NUM>, in a combination module <NUM> of the fusion module of the IMM tracking algorithm <NUM>.

To be able to combine the first state vector <NUM> and the second state vector <NUM> with each other - e.g., in a weighted combination -, it would be possible that the second state vector <NUM> is padded using zero values for the respective entries associated with the at least one velocity measure. For example, vr = ω ≡ <NUM>, cf. <NUM> and <NUM>. Then, the first state vector <NUM> and the second state vector <NUM> have the same dimension and can be added.

The combination can be expressed as <MAT> where X<NUM> is the state estimate of the rest state model and X<NUM> is the state estimate of the movement model and µ<NUM> is the likelihood of the rest state model and µ<NUM> is the likelihood of the rest state model. Here, the likelihoods serve directly as weighting factors.

More generally, the weighting factors can be set so that for higher likelihoods <NUM>, <NUM>, the respective estimate of the <NUM>-D position included in the respective state vector <NUM>, <NUM> is considered more pronounced in the final estimate of the tracking data <NUM>. For instance, it would be possible that the tracking data includes the estimation of the <NUM>-D position of obtained from the particular model <NUM>, <NUM> that has a higher likelihood <NUM>, <NUM> for the respective iteration <NUM>; in other words, one of the two state vectors <NUM>, <NUM> could be discarded.

A weighted combination typically provides for increased accuracy when transitioning between the rest state and the movement state; while a selection between the two state estimates can provide an increased accuracy when the target is statically in the rest state or in the movement state.

As illustrated in <FIG>, a feedback is provided so that the IMM tracking algorithm <NUM> updates, for each iteration <NUM>, the a-priori estimate of the <NUM>-D position of the movable target based on the first output <NUM> of the state vector estimated by the movement state model <NUM> at that iteration <NUM>, the second output <NUM> of the state vector estimated by the rest model <NUM> at that iteration <NUM>, the first likelihood <NUM>, the second likelihood <NUM>, as well as the state covariances <NUM>, <NUM> estimated by the movement state model <NUM> and the rest model <NUM>, respectively.

Based on these values, a Markovian decision process <NUM> can be used that determines an a-priori estimate of the <NUM>-D position of the movable target, by considering the likelihood that the target is either in the dynamic state <NUM> or the rest state <NUM>, respectively and combining the state estimates of the state vector <NUM> and the state vector <NUM>, respectively. Thereby, transitions between the states <NUM>, <NUM> can be modeled. The models <NUM>, <NUM> are interacting.

Above, scenarios have been described in which the state estimates provided by the movement state model <NUM> and the rest state model <NUM> define the <NUM>-D position of the target <NUM> in the Cartesian coordinates. According to various examples, it is possible that the rest state model <NUM> determines the <NUM>-D position of the target <NUM> with respect to one or more input elements of a user interface, e.g., with respect to the input elements <NUM>-<NUM> of the user interface <NUM>, as discussed above. In particular, the state estimate <NUM> can include an indication of a respective input element being addressed by the target <NUM>. Then, the tracking data <NUM> can - e.g., in addition to the <NUM>-D position in Cartesian coordinates - also provide an indication of the respective input element. For example, the equation <NUM> could be modified so that the state vector indicates the respective input element: <MAT>.

Here, lX, ly are indices that label a <NUM>-D matrix of input elements. For instance lX = <NUM>, ly = <NUM> would identify the input element in the first column, second row. For a <NUM>-D vector of input elements (cf. <FIG>), a single index would suffice.

Such an approach facilitates providing a stable indication of a respective input element being addressed by the target <NUM> in the rest state. In particular, fast changes between different input elements - e.g., in view of measurement noise <NUM> (cf. <FIG>) can be avoided.

<FIG> is a flowchart of a method according to various examples. The method of <FIG> can be executed by a device comprising a processor. For instance, the processor may load program code from a memory and execute the program code to then execute the method of <FIG>. The device could be a computing unit of an HMI, coupled to one or more depth sensors. The method of <FIG> facilitates tracking of an object.

At box <NUM>, depth data is obtained. For example, the depth data <NUM> as discussed in connection with <FIG> in <FIG> could be obtained. The depth data could be obtained from logic as described in connection with <FIG>, box <NUM>.

At box <NUM>, the depth data is processed using an IMM tracking algorithm. Thereby, tracking data is obtained, e.g., the tracking data <NUM> as discussed in connection with <FIG>. The tracking data includes an estimate of the <NUM>-D position of the target.

At box <NUM>, postprocessing can be applied. This can facilitate one or more use-case specific applications, as previously explained in connection with <FIG>: box <NUM>.

To facilitate the postprocessing at box <NUM>, the tracking data may include additional information. For instance, the tracking data may be indicative of a likelihood of the target being either in a rest state or a movement state. Such an example is explained in connection with <FIG>.

<FIG> is a flowchart of a method according to various examples. <FIG> illustrates an example implementation of box <NUM>.

At box <NUM> a determination is made whether the target is more likely in a movement state (cf. <FIG>: movement state <NUM>) than in a rest state (cf. <FIG>: rest state <NUM>). If yes, the method commences at box <NUM>; if not, the method commences at box <NUM>. Box <NUM> and box <NUM> are associated with different post-processing algorithms for post-processing the tracking data.

The selection at box <NUM> is made based on the tracking data being indicative of the likelihood of the target being in the movement state or the rest state.

The likelihood that the target is in the respective state can be derived from the model likelihoods, as explained in connection with <FIG>: model likelihood module <NUM>.

There are various options to implement the tracking data to be indicative of this likelihood.

In one example, the tracking data could also include an explicit indication of a respective likelihood.

In a further example, an implicit indication could be provided. For instance, the information content of the tracking data may vary depending on the likelihood. To give an example, if the target is likely in the movement state <NUM>, it would be possible to output a time trajectory of the <NUM>-D position as the tracking data, by concatenating the estimated <NUM>-D positions of multiple iterations of the IMM tracking algorithm. Conversely, if the target is likely to be in the rest state <NUM>, it would be possible to include the indication of the respective input element of the UI, e.g., without any time resolution (cf. Eq. <NUM>). , the indication of a selected input element may be selectively included in the tracking data depending on the likelihood of the moveable target being in the rest state.

In the illustrated example of <FIG>, depending on the likelihood that the target is in the movement state (and, thus, by definition, not in the rest state), it would be possible to select between either a classification of a gesture recognition - box <NUM> -, or a determination of an actuation of an input element - box <NUM>.

At box <NUM>, it would be possible that a gesture classification is selectively applied depending on a likelihood that the target is in the movement state. For instance, the gesture classification could be based on a dynamic trajectory that is defined by the estimates of the <NUM>-D position of the target provided in multiple iterations of the IMM tracking algorithm. An example gesture classification could use a neural network to classify such dynamic trajectories. Another example may employ a rule-based matching algorithm.

Likewise, it would be possible, at box <NUM>, to determine whether one or more input elements are actuated. This can be based on an indication of the respective input element in the tracking data, cf. For example, in this connection, it would be possible to apply a time-domain low-pass filter to the indication. Thereby, it can be checked whether the target <NUM> remains relatively stable, i.e., stable within one of the regions associated with the input elements - for a respective time duration. For example, it could be required that the target remains position in one of these input elements for a time duration that is longer than a predefined threshold time duration, to trigger a respective action of the HMI.

Also, at box <NUM>, the <NUM>-D position in a Cartesian coordinate system could be mapped to respective positions of the input elements, e.g., in case an explicit indication of an input element is not already included in the tracking data.

<FIG> schematically illustrates a device <NUM> configured to perform techniques described herein. The device <NUM> includes a processor <NUM> and a memory <NUM>. The device <NUM> also includes an interface. For example, it would be possible that the device <NUM> receives raw measurement data <NUM> or depth data <NUM> via the interface <NUM>. The processor <NUM> can load program code from the memory <NUM> and execute the program code. Upon executing the program code, the processor performs techniques as described herein, e.g.: processing depth data using an IMM tracking algorithm, preprocessing raw measurement data to obtain the depth data; postprocessing tracking data obtained from the IMM tracking algorithm, e.g., to control an HMI, e.g., by providing respective control instructions via the interface <NUM>. The processor <NUM> could perform the methods of <FIG>.

<FIG> illustrates a device <NUM> that is configured to perform techniques described herein. The device <NUM> includes multiple modules <NUM>-<NUM> that could be implemented in software and/or in hardware. For instance, the modules <NUM>-<NUM> could be implemented by software, e.g., by respective portions of program code that can be loaded and executed by a processor.

The module <NUM> is for obtaining depth data. Accordingly, the module <NUM> may implement box <NUM> of the method of <FIG>.

The module <NUM> is for processing the depth data. Accordingly, the module <NUM> may implement box <NUM> of the method of <FIG>.

The module <NUM> is for postprocessing tracking data, e.g., as obtained from module <NUM>. Accordingly, the module <NUM> may implement box <NUM> of the method of <FIG>.

Summarizing, it has been disclosed to determine an estimate of a <NUM>-D position of a moveable target using an IMM tracking algorithm. This is based on measurements of depth data. At least one model of the IMM tracking algorithm models a rest state of the moveable target.

While conventionally IMM tracking algorithms are used for mode matching for different types of motion of a moving target, the disclosure enables to model two orthogonal motion state. One or more first motion states recognize the movement of the target and a second motion state is used is to accurately localize the target - e.g., a finger - when it is not moving.

The metric to match the observed state to a respective model has been enhanced. The likelihood of the target being observed in a respective motion state can be accurately determined, by considering an evolution of accuracies across multiple iterations.

Also, the post-processing can be based modal probabilities of the target being in respective motion state. If the target is moving (pr(movement state) > pr(rest state)) the trajectory of the target can be provided as tracking data. If the target is static, an indication of an input element addressed by the target can be provided as output in the tracking data.

Claim 1:
A method, comprising:
- obtaining depth data indicative of a time-resolved measurement of a <NUM>-D position of a moveable target (<NUM>) in a field-of-view (<NUM>) of a depth sensor (<NUM>), and
- processing the depth data (<NUM>) using an interacting multi-model, IMM, tracking algorithm (<NUM>), the IMM tracking algorithm (<NUM>) providing, for each one of multiple iterations (<NUM>), tracking data (<NUM>) comprising a respective estimate of the <NUM>-D position of the moveable target (<NUM>),
wherein the IMM tracking algorithm (<NUM>) comprises a first model (<NUM>) providing a first output (<NUM>, <NUM>), a second model (<NUM>) providing a second output (<NUM>, <NUM>), and a fusion module (<NUM>, <NUM>) fusing the first output (<NUM>, <NUM>) and the second output (<NUM>, <NUM>) to thereby provide the tracking data (<NUM>),
wherein the first model (<NUM>) models a movement state (<NUM>) of the moveable target (<NUM>),
wherein the second model (<NUM>) models a rest state (<NUM>) of the moveable target (<NUM>),
wherein the second model (<NUM>) determines the second output (<NUM>, <NUM>) based on the estimate of the <NUM>-D position (<NUM>) of the moveable target (<NUM>) with respect to one or more input elements (<NUM>-<NUM>) of a user interface (<NUM>) predefined within the field-of-view (<NUM>),
wherein the moveable target (<NUM>) is selected from the group consisting of: hand; part of a hand; and handheld pointing device.