Patent Description:
During rehabilitation, one of the challenges is to control rehabilitation exercises.

From <CIT> an exercise information display system and exercise information display method is already known. The system includes a sensor device, which obtains data associated with a motion status of a human body during an exercise, a data processing device which generates plural types of exercise information based on the data obtained by the sensor device and a viewing device which displays from among the plural types of exercise information at least first information indicating a posture of the human body during the exercise and second information associated with the first information in the display format, where the first information and a second information are displayed in connection with each other.

A system capable of tracking human motion requires the user to attach a set of sensors to a set of straps he is wearing. Each of those sensors must be placed on anatomical landmarks depending upon the limb whose motion is to be tracked. The same system also requires the user to attach the sensors to a sensor station for the purpose of charging them as well as performing an inspection thereof. As such, the following is required for using this system:.

Due to these requirements and the fact that it is not clear for the user what he must do at a given point while using the system, the following problems arise:.

These events result in a corruption of the motion tracking procedure, an invalid sensor inspection and a lack of battery of the sensors. In sum, using the system correctly requires great understanding of its requirements (which impairs the user experience) and failing to do so has severe consequences. The solution provided by <CIT> has these drawbacks too. In that document, sensors are attached to parts of the body of the person, and then a mobile device is arranged adjacent to each sensor for registering the sensor and configuring its coordinate system by deriving a matrix transformation system between measurements of the sensors and measurements of the mobile device. If the user attaches the sensors somewhere else, does not attach the sensors at all, does not place the mobile device properly, etc. the sensors will not be adequately attached to the person and the motion tracking procedure will be riddled with errors that are not adverted by the user or the system.

It is therefore an option of the present invention to provide a system for tracking human motion which provides an enhanced solution for correct sensor placement and avoiding misplacement of the sensors.

This object is solved according to present invention by a method of performing sensor placement error detection and correction.

Accordingly, the method of performing sensor placement error detection and correction for a system for tracking human motion is performed such that the method comprises at least the following steps forming a method routine:.

The system comprises at least a sensor unit and at least a sensor holder.

The invention is based on the basic idea that incorrect sensor placement can be checked in a very easy and at the same time effective way. In particular, the inventors have found that incorrect sensor placement in most cases has only a limited amount of reasons and within this list of reasons, there are reasons with higher numbers than others. It is therefore possible to prepare a list of potential reasons for the incorrect sensor placement and to present these reasons the user, then check whether the sensor is now correctly placed and confirm with user feedback, whether the system is now in a condition, where one or all sensors or at least one sensor is now put into correct condition.

Moreover, the reasons are hierarchically ranked. In particular, it would be possible to use the information, which reasons have a higher likelihood and to put them in the ranking on top whereas other reasons, with less likelihood are put in lower ranking positions. By this, the advantages achieved that with a higher likelihood only a few check routine steps must be done.

Furthermore, it is possible that it is checked, which reasons have already been presented. Thus, iterations must not be done and can be avoided.

Additionally, it can be checked, whether still reasons are left. By this step, it can be avoided that after complete check-up the system starts again. Also, if it is found that no reasons are left, then a general remark regarding the manual can be provided in this plate.

In case that no reasons are left, the method routine can be ended. Then, the method routine can be stopped.

If, however, further reasons are left, then the hierarchically top ranked reason may be selected and presented to the user. With the hierarchically top ranked reason preferably the reason is meant, which is within the ranking the next top ranked reason in the ranking, considering the already shown ranked reasons, which are not under consideration, but only those reasons, which are still left and not yet been shown.

A further possible step of the routine may be receiving a feedback from the user regarding fulfillment of the present unmet condition. Moreover, the invention relates to a system for tracking human motion with the features of claim <NUM>. Accordingly, the system comprises:.

Additionally, the storage element may be configured such that the reasons are stored within the storage element in a hierarchical ranking. Such a hierarchical ranking may be provided based on experience or test results and pre-stored in the storage element.

Furthermore, the hierarchical ranking may be also connected with a learning system and updated from time to time. Also such updates may be done automatically.

Furthermore, it is possible that the selection module may be configured such that it can be checked by means of the selection module, which reasons have already been presented to the user.

Additionally, it is possible that the selection module is configured such that it can be checked by means of the selection module, whether still reasons are left.

The presenting module can be further capable to wait for the user to acknowledge feedback.

In case that no reasons are left, the method routine can be ended by the system.

Additionally, it is possible that the selection module is configured such that if reasons are left, the hierarchically top ranked reason is selected and presented to the user by means of the presenting module.

Further details and advantages will be shown in the figures, which should not be interpreted as restricting the scope of the invention, but just as examples of how the invention can be carried out.

<FIG> shows in a perspective view an embodiment of a system <NUM> for tracking human motion according to the present invention.

The system <NUM> comprises in the shown embodiment two sensor units <NUM> and <NUM>.

Furthermore, there are sensor holders <NUM>, <NUM>, which are configured to receive and be attached with the sensor unit <NUM> or <NUM>.

In the shown embodiment, sensor holders <NUM>, <NUM> are configured and arranged to correspond and be attached during a work cycle/tracking cycle with sensor units <NUM>, <NUM>.

As long as sensor units <NUM>, <NUM> are used, they can be received and placed in the sensor holders <NUM>, <NUM>, which form the respective counterelement.

Moreover, the system <NUM> comprises a main unit <NUM>, cf. also <FIG>.

The main unit <NUM> is in the shown embodiment realized by a tablet PC.

Additionally, the system <NUM> comprises a sensor station <NUM>.

The sensor station <NUM> comprises a plurality of receiving slots 22e, which are configured to receive the sensor units <NUM>, <NUM>.

The sensor station <NUM> is configured and arranged for re-charging of the sensor units <NUM>, <NUM>.

The sensor units <NUM>, <NUM> comprise each a sensor communication interface <NUM>, <NUM> and the sensor holders <NUM>, <NUM> also comprise each a counter communication interface <NUM>, <NUM>.

The sensor units <NUM>, <NUM> are identical. This applies in particular to the technical features and also to the design of the sensor units <NUM>, <NUM> (and other sensor units).

Each sensor unit <NUM> or <NUM> has an L-form.

However, each sensor unit <NUM> or <NUM> is not personalized/individualized.

It is, however, possible that sensors can differ from each other and are personalized and/or individualized. Also, they can be named and/or provided with specific attachment information, where the sensor unit shall be placed.

The sensor unit <NUM>, <NUM> has a color coding field 12a, 14a which has a color. This field can also be used for naming or branding.

Moreover, each sensor unit <NUM> or <NUM> has its own electronic identification circuit 12b or 14b.

With the electronic identification circuit 12b or 14b the sensor unit <NUM>, <NUM> can identify a counterelement.

As counterelement for each electronic identification circuit 12b or 14b, there is an individualized resistor R1, R2 in each sensor holder <NUM>, <NUM> as passive element.

Due to the fact that the sensor holders <NUM>, <NUM> have individualized counterelements, it is not necessary that the sensor units are individualized.

To the contrary, in the shown embodiment the sensor units are completely identical as no strict correspondence between sensor units <NUM>, <NUM> and sensor holders <NUM>, <NUM> is needed.

Moreover, only the sensor holders <NUM>, <NUM> comprise an identification of the limb, where they should be placed on and also an identification of the side of the body, to which they belong to and on which they should be placed.

The sensor holders <NUM>, <NUM> are embodied such that they comprise wearable elastic straps 16a, 18a.

The wearable straps 16a, 18a comprise each at least one length adjustment element 16b, 18b, which is here a clamp 16b, 18b for length adjustment of the straps 16a, 18a.

With wearable straps 16a, 18a the sensor holders <NUM>, <NUM> may be placed on the body or a limb of the patient.

On each elastic strap 16a, 18a a sensor unit attachment element 16c, 18c is provided, which serves as a docking element or docking station for one of sensor units <NUM>, <NUM>. Each sensor unit attachment element 16c, 18c has a form-fit or snap-fit for the sensor units <NUM>, <NUM>.

Here the form-fit and snap-fit is realized by means of elastic holding walls 16d, 18d and a recess 16e, 18e for the L-form protrusion 12c, 14c extending out of the main body 12d, 14d of the sensor unit <NUM> or <NUM>.

Furthermore, there are snap-fit fingers 16f, 18f in the walls 16d, 18d, which are protruding out of the walls 16d, 18d and are arranged such that they realize a snap-fit contact with a sensor unit <NUM>,<NUM>, once a sensor unit <NUM>, <NUM> is inserted into the sensor unit attachment element 16c, 18c.

The sensor unit attachment elements 16c, 18c are attached to wearable straps 16a, 18a.

The sensor station <NUM> comprises a ON-OFF-Button 22a. This is, however, not mandatory.

Furthermore, the sensor station <NUM> comprises in each of its slots 22e an attachment communication interface 22b.

Also, there is a communication line 22c for connection with the main unit <NUM>, which is here represented by a cable 22c.

Additionally, there is a Bluetooth link 22d as shown in <FIG>.

Further (wireless) Bluetooth links B1, B2, B3, B4, B5 are established between each sensor unit <NUM>, <NUM> and further sensor units <NUM>, <NUM>, <NUM> on the one side and the main unit <NUM> on the other side/in the center of the system <NUM>.

All sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are completely identical.

It would also be possible, to have one Bluetooth connection between one sensor master unit and the main unit and connect the other sensor units with the sensor master unit.

All sensors units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are each equipped with at least one inertial sensor.

Alternatively or additionally, as sensor may be used at least one of the sensor types gyroscope, acceleration sensor, piezo sensor, magnetic sensor, optical sensor or the like. In particular, the sensors/sensor units may comprise accelerometers and/or gyroscopes and/or magnetometers. Additionally, the system <NUM> comprises at least one analyzing module <NUM> and at least one presenting module <NUM> with at least one storage element <NUM>.

The analyzing module <NUM> is configured and arranged such that it can be analyzed, whether there is a condition for correct sensor unit placement met or not met.

Furthermore, the storage element <NUM> of the presenting module <NUM> (e.g. the or a display of the main unit <NUM>) is equipped with potential reasons for the incorrect sensor unit placement, wherein the presenting module <NUM> is further configured and arranged such that in case of detecting an unmet condition out of the storage element <NUM> a least of potential reasons for the incorrect sensor placement may be provided.

Furthermore, the system <NUM> comprises a selection module <NUM>.

Furthermore, the presenting module <NUM> is capable to.

The functionality can be described as follows:.

As described below in connection with <FIG>, it is not necessary to have the sensor units and the sensor holders assigned to each other before the procedure.

The inspection procedure is a procedure where the sensor units are kind of "quality controlled" from the point of view that the measured values/signals provided by the sensor units are correct.

The main unit <NUM> is configured such that motion data as well as the information regarding the attachment of the sensor unit <NUM>, <NUM> to the sensor holder <NUM>, <NUM> can be tracked and/or processed.

The main unit <NUM> processes the motion data as well as the information regarding the attachment of each sensor unit <NUM>, <NUM>, <NUM>, <NUM>, <NUM> and provides feedback to the user via the display of a tablet device, here forming the main unit <NUM>.

Sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> (also called motion sensors) are placed in anatomical landmarks, they collect the user's motion data as well as information regarding which component they are attached to and send it to the main unit <NUM> through a wireless communication protocol (here the sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are embodied as devices comprising a microcontroller, Bluetooth connectivity, inertial sensors and the attachment communication interface).

The sensor holders <NUM>, <NUM> with the wearable elastic straps 16a, 18a securely attach each sensor unit <NUM>, <NUM>, <NUM>, <NUM>, <NUM> to the corresponding anatomical landmark (of a patient).

Each sensor holder <NUM>, <NUM> is equipped with a counter communication interface <NUM>, <NUM> (also called attachment communication interface <NUM>, <NUM>).

The sensor station <NUM> holds the sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> for charging and inspection purposes. Equipped with the attachment communication interface 22b, this component communicates with the main unit <NUM> through the cable/ communication line 22c and/or the wireless communication protocol, here the additional Bluetooth link 22d.

This interface, i.e. the attachment communication interface 22b of the sensor station <NUM> or the counter communication interface <NUM>, <NUM> of the sensor holder <NUM>, <NUM> provides the advantage that the sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> have the capability of inferring to which piece of hardware they are connect (if any).

In its simplest form, it can be embodied as a couple of terminals connected by a resistor on the sensor holder/sensor station side and an active circuit on the sensor unit side. With such configuration and by having a different resistor value for each of the straps as well as the sensor station, the microcontroller in the sensor is able to measure resistance between its terminals continuously and thus infer whether it is disconnected, connected to one of the straps or to the sensor station. By having a passive circuit on the sensor holder side, one can avoid the need of charging the straps which is obviously of interest for the sake of the user experience.

Each of the sensor holders <NUM>, <NUM>, as well as the sensor station <NUM>, is equipped with a hardware interface that connects to a homologous interface in the sensors upon their attachment. This interface and the connection provide each individual sensor unit with the capability of inferring whether it is attached to a specific strap, attached to the base station or detached. As a result the system is able to:.

These features have an advantageous and beneficial impact on the user experience as the system is now able to guide the user through each step by providing relevant and adaptive feedback whilst ensuring proper functioning.

Generally speaking the method of using the system <NUM> has at least the following steps:.

With the established communication link L1, L2 an individualized element of the sensor holder <NUM>, <NUM>, here the resistors R1, R2, is/are checked and identified by means of the sensor unit <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

Further details are explained in connection with <FIG>.

<FIG> shows a flow chart of the system setup procedure (also including a sensor unit inspection routine).

This routine comprises the necessary steps a user must perform in the shown embodiment of the system to start tracking its motion with the system <NUM>. Similar setups and routines are generally possible.

Here, at least one or more or all of the sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> is/are attached to the sensor station <NUM>, i.e. the at least one sensor unit <NUM>, <NUM>, <NUM>, <NUM>, <NUM> is put in one of the slots 22e.

In step S100 it is checked, whether all sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> (here all, it might in an alternative embodiment be also possible that for the check only one or several sensor units or all sensor units must be placed into the sensor station <NUM>) are attached to the sensor station <NUM>.

If this is not the case, the procedure continues in step S101, i.e. the user is invited to place each of the detached sensors into the slots 22e of the sensor station <NUM>. Then, it is continued with step S100 again.

If all sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are then attached to the sensor station <NUM>, then in step S102 the sensor unit inspection is performed.

After the sensor inspection of step S102 a decision is taken in step S103, whether the sensor inspection is passed or not.

If the sensor inspection is not passed, then in step S104 the user is invited to calibrate the sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

If the sensor inspection is passed, then in step S105 it is checked, whether each sensor unit <NUM>, <NUM>, <NUM>, <NUM>, <NUM> is attached to one sensor holder <NUM>, <NUM>.

This is checked as described above by means of the sensor communication interface <NUM>, <NUM> and the passive counter communication interface <NUM>, <NUM> of the sensor holder <NUM>, <NUM>.

If the check of step S105 reveals that not all sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are correctly placed, the user is invited in step S106 to place each of the detached sensors units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> to one sensor holder, <NUM>, <NUM> (one sensor unit to one sensor holder).

Steps S101 ("Advert the user to place each of the detached sensors on the station"), S104 ("Advert the user that the sensors must be calibrated") and S106 ("Advert the user to place each of the detached sensors on one of the straps") can involve the step of displaying such a message by means of the display on the main unit <NUM>.

The detailed sensor placement error detection and correction procedure of the system <NUM> is described below in connection with <FIG>.

If the system <NUM> comes in check S105 to the result that each sensor is attached to a sensor holder <NUM>, <NUM>, then in step S107 each sensor unit <NUM>, <NUM>, <NUM>, <NUM>, <NUM> is assigned to the anatomical landmark of the sensor holder <NUM>, <NUM> it is attached to.

After that, in step S108 the motion tracking procedure is started.

During this procedure, there is a check routine in step S109, which is following step S108. Here, it is continuously (or alternatively intermittently) checked, whether each sensor unit <NUM>, <NUM>, <NUM>, <NUM>, <NUM> is still attached to a sensor holder <NUM>, <NUM>.

In step S110, the motion tracking procedure is stopped, when in step S109 it is found that one or more of the sensor units <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are no longer attached to the respective sensor holder <NUM>, <NUM>.

If so, it is switched back to step S106.

Then the procedure may continue from there on.

The procedure for sensor placement error detection and correction works as follows:
The problem to be solved has to do with the fact that the system <NUM>, which is capable of tracking human motion, requires the user to place a set of sensor units on anatomical landmarks as described above. depending upon the limb whose motion is to be tracked.

Due to these requirements and the fact that it is not clear for the user what he must do at a given point while using the system <NUM>, the following problems arise:.

These events result in a corruption of the motion tracking procedure.

In order to overcome this issue, the method and also the system rely on an algorithm, which assesses the orientations reported by the sensor units at a step before the actual motion tracking procedure where the patient is asked to stand still in a specified position according to the exercises he will perform (e.g. standing upright, seated upright, lying on his back, etc). At that point these orientations are analysed and a set of metrics is computed. Each of these metrics is then compared against an allowable interval (which might be open on one of the limits) and the placement is considered valid if the value each metric falls within the respective interval. These intervals/metrics are essentially a set of conditions that the reported orientations meet, in either an absolute or a relative manner, when the sensors are correctly placed.

This is e.g. based on predefined conditions regarding angles of the sensor units (e.g. to each other or within the threedimensional space). There can be also predefined conditions based on angles and predefined intervals, in which specific conditions must be met.

Each condition can be e.g. defined as a logical AND or a logical OR of one or more intervals of angles and allowable intervals.

Each of these conditions is associated with one or more possible causes of its violation. All causes of violation (even those regarding different conditions) have a hierarchical relation between them. This hierarchy expresses the order through which the causes of violation will presented to the patient and is a result of the likelihood of the cause as well as the fact that some errors must be corrected before other errors may be detected (e.g. a patient must first fix an error related to switching a couple of sensor units before an error related to a misplacement of a single sensor unit may be detected).

When one or more causes of violation are eligible - because the violated condition has more than one possible cause and/or because more than one condition was violated - the system selects the hierarchically superior cause and presents it to the patient. Then, the system expects the patient to check whether he may have misplaced the sensor units as indicated by the presented cause and, if so, act accordingly.

When he is done he interacts with the system <NUM> to signal another analysis may be triggered. At this point the system gathers all the possible causes of violation once more but this time removes the cause that has already been presented to the patient (if it is still violated) before selecting the next cause of violation to be presented. This process may continue recursively until all the conditions are satisfied or until a specified number of attempts is performed.

For the purpose of illustrating the disclosure an example will be provided for the case of a sensor setup comprising on sensor unit <NUM> on the left lower arm, one sensor unit <NUM> on the lateral side of the left upper arm and one sensor unit <NUM> on the chest, as shown in <FIG>.

To that end, it will be assumed that the axes of reference of each a sensor unit are those illustrated in <FIG> and that all the conditions will be expressed as angles between two axes of reference of different sensor units or between an axis of reference and a global vector. In regards to the global reference frame, hereinafter will be assumed that x points towards the north, z points up and y is given by their cross product.

The list of conditions that must be satisfied in order for the placement to be considered valid are presented in Table <NUM>. As an example, the first condition is that the angle between the (chest) red sensor's x vector and the global z vector (the 'up' vector) must be lower than 90o. At the step in which the patient is asked to stand still in an upright position the system asserts the validity of each of these conditions and if any of them is invalid the sensor placement is deemed incorrect.

The provision of feedback is accomplished by the algorithm described in <FIG>. As one can see, the algorithm has two possible ending scenarios E1 and E2, the first is when at some point all the conditions are satisfied and the second is when even after being presented with all the possible error causes the user still does not manage to place the sensors correctly. There could also be another scenario where the number of possible causes presented to the user is limited and the algorithm ends when that number is reached.

In detail, the following steps are performed according to one embodiment of the method of the present invention, performed on a system <NUM> according to the present invention: In Step S200, the algorithm is started.

In Step S201, it is checked by means of the analyzing module <NUM>, whether there is any unsatisfied condition.

If this is not the case, the endpoint E1 is reached (i.e. ending scenario E1). Then, there is no problem at all regarding unsatisfied conditions in connection with sensor placement.

If there is an unsatisfied condition detected by means of the analyzing module, then the list of possible causes is provided for all the unsatisfied conditions out of the storage element <NUM> of the presenting module <NUM> in step S202.

The list of conditions along with the respective possible causes is exemplarily shown in Table <NUM> below:.

In step S203 the possible causes are removed that have already been presented to the user by the selection module <NUM>.

After that, in step S204 it is checked, whether there is any possible cause left.

If this is not the case, then the endpoint E2 (i.e. ending scenario E2) is reached. Then, there is no longer a problem at all regarding unsatisfied conditions in connection with sensor placement or the reason cannot be detected this way.

If there is still a possible cause left, then it is continued in step S204.

In step S205, the hierarchically superior cause (i.e. the cause, which is among the left over ones and which is most prominently ranked) is chosen by the selection module <NUM> and presented to the user by means of the presenting module <NUM>.

After that in step S206 it is waited for the user input to acknowledge the feedback and it is then jumped back to step S201.

Claim 1:
A method of performing sensor placement error detection and correction for a system (<NUM>) for tracking human motion comprising
- at least one sensor unit (<NUM>,<NUM>),
- at least one sensor holder (<NUM>,<NUM>),
characterized in that the method comprises at least the following steps forming a method routine:
- Analyzing, whether there is at least one unmet condition, indicating an incorrect sensor unit placement;
- In case of detecting such at least one unmet condition preparing a list of potential reasons for the incorrect sensor unit placement;
- Remove the possible reasons for incorrect sensor unit placement already presented to the user;
- Select the one of the reasons and present it to a user; and
- After one of the reasons has been selected and presented, repeating the analyzing step to analyze, by assessing at least one orientation reported by the at least one sensor unit at a step before a motion tracking procedure where the user is asked to stand still in a specified position, if there is at least one unmet condition, and if there is, repeating the detecting, removing and selecting steps.