Recurrent Inertial Graph-Based Estimator (RING): A Single Pluripotent Inertial Motion Tracking Solution
Authors: Simon Bachhuber
Ive Weygers
Dustin Lehmann
Mischa Dombrowski
Thomas Seel
Venue: Accepted by TMLR ()

Abstract: This paper introduces a novel ML-based method for Inertial Motion Tracking (IMT) that fundamentally changes the way this technology is used. The proposed method, named RING (Recurrent Inertial Graph-Based Estimator), provides a pluripotent, problem-unspecific plug-and-play IMT solution that, in contrast to conventional IMT solutions, eliminates the need for expert knowledge to identify, select, and parameterize the appropriate method. RING's pluripotency is enabled by a novel online-capable neural network architecture that uses a decentralized network of message-passing, parameter-sharing recurrent neural networks, which map local IMU measurements and nearest-neighbour messages to local orientations. This architecture enables RING to address a broad range of IMT problems that vary greatly in aspects such as the number of attached sensors, or the number of segments in the kinematic chain, and even generalize to previously unsolved IMT problems, including the challenging combination of magnetometer-free and sparse sensing with unknown sensor-to-segment parameters. Remarkably, RING is trained solely on simulated data, yet evaluated on experimental data, which indicates its exceptional ability to zero-shot generalize from simulation to experiment, while outperforming several state-of-the-art problem-specific solutions. For example, RING can, for the first time, accurately track a four-segment kinematic chain (which requires estimating four orientations) using only two magnetometer-free inertial measurement units. This research not only makes IMT more powerful and less restrictive in established domains ranging from biomechanics to autonomous systems, but also opens its application to new users and fields previously untapped by motion tracking technology. Code and data is available at https://github.com/simon-bachhuber/ring_supplementary_material.

Published in Transactions on Machine Learning Research (10/2024)
Recurrent Inertial Graph-Based Estimator (RING): A Single Pluripotent Inertial Motion Tracking Solution
Simon Bachhuber
smmonbacchkuer@@ffo..d
Deportment of Artificial Intelligence in Biomedical Engineering
Friedrich- Areearich-leeammder Alezonder-Univvesstti Universität Erlangen-Noruen
Ive Weygers
ive. www.weygerssffu.ccom de
Department of Artificial Intelligence Biomedical Engineering Friedrich- Alezander- Creerichh-lezanderr
Universität Selangen-Nürnbeee
Dustin Lehmann
dustin. deretinilelmman@@tuuuoo
Control Systems Group
Technical Universiby Berlin
Mischa Dombrowski
mischha.dombbouusi@ffo...ddd
Department of Artificial Intelligence in Biomedical Engineering
Friedrich- Friedrich-Alccander- Universität Brlangen-Nürnbery
Thomas Scel
Institute of Mechatronics Systems Leibniz Universitää Hannover
Reviewed on OpenReview: /ttps://openreeee/ net/ forum? id= h2C33tmmnn
Abstract
This paper introduces novel ML-based mothod for Inortial Motion Tracking (IMT) that fundamentally changes the way this technology is used. The proposed method, named RING (Recurrent Inertial Graph-Based Estimator), provides a pluripotent, problem-unspett plug-and-play IMT solution that, in contrast to conventional IMT solutions, eliminates the need for expert knowledge to identify, select, and parameterize the appropriate method. RING's pluripotency is enabled by novel online-capable neural network architecture that uses a decentralized network of message-passing, parameter-sharing recurrent neural networks, which map local IMU measurements and ncarest-ncighbou messages to local orientations. This architecture enables RING address a broad range of IMT problems that vary greatly in aspects such as the number of attached sensors, or the number of segments in the kinematic chain, and even generalize to previously unsolved IMT problems, including the challenging combination of magnetometer-ffee free and sparse sensing with unknown sensor-to-segment parameters. Remarkably, RING is trained solely on simulated data, yet evaluated on experimental data, which indicates its exceptional ability to zero-shot generalize from simulation to experiment, while outperforming several state-of-the-art problem-specific solutions. For example, RING can, for the first time, accurately track a four-segment kinematic chain (which requires estimating four orientations) using only two magnctometer- free inertial measurement units. This research not only makes IMT more powerful and less restrictive in established domains ranging from biomechanics to autonomous systems, but also opens its application to new users and fields previously untapped by motion tracking technology. Code and data is available here
I lnne track them

---

Published in Transactions on Machine Learning Research (10/2024)
Real- W World Inertial Motion Tracking Problems
Applications
X IMUs Axes
Graph
A
After Training in Simulation
RING
RING
A Single Solution
Pose of Chain
Y+
9
Appl 9
Figure 1: RING MI.-based method that provides versatile, pluripotent IMT solution applicable across broad range of challenging LMT problems, designed for use without the need for expert knowledge. Remarkably, RING is trained solcly on simulated data, yet zero-shot generalizes to real-world experiments and outperforms several problem-specific oate-oo-thhearr solutions.
1 Introduction
In the domain of multi-agent systems, structural policies have shown great potential for the control of complex agents; meanwhile, Recurrent Neural Networks (RNNs) are an establishod choice for sequential data. The potential of their combination for the analysis of structural sequential data has rarely been investigated and exploited. This combination seems particularly promising for state estimation in graph-structured systems, such as, for example, for IMT of Kinematic Chains (KCs).
The need for reliable and accurate estimation of the orientation, attitude, or pose of articulated objects in three-dimensional (3D) space spans across various application domains ranging from aerospace engineering
(Euston et al. 2008; Givens & Coopmans 2019 to health applications (Buke et al., 2015 López-Nas Nava & Muñoz-Meléndes 2016 Seel et al.] (2020). Incrtial Measurement Units (IMUs), which typically comprise a 3D accelerometer, 3D gyroscope, and 3D magnetometer, have become smaller and less expensive within the last two decades and have therefore rapidly become the most promising tochnology for accurate, reliable, and inexpensive motion tracking in rigid bodies and KCs, especially since camera-basel systems are typically more expensive, more restrictive, and suffer from occlusion (von Marcard et al. 2017 Huang et al. 2018
However, fusing the available measurement signals to estimate the dcsired orientations requires advanced IMT algorithms that typically need to overcome a combination or all of the following three main IMT challenges (Seel et al. 2020)): (1) inhomogeneous magnetic fields in indoor environments and proximity of ferromagnetic material or electric devices; (2) sensor-to-sen to-og-tt-segment calibration, i.e. identifying the joint position and axis orientations in local sensor coordinates; (3) solving sparse problems in which some segments of the KC are not equipped with sensor, to improve the usability and reduce costs.
In recent years, numerous highly specialized methods have been proposed to address these challenges. Magnetometer-fieee methods have been developed to estimate the relative orientation between two adjacent segments by exploiting different kinematic raints (Kok et al. 2014 Laidig et al. 2017 Lehmann et al.
2020 2024). Morcover, numcrous gencral-purpose magnctometer-fiee attitude estimators have been proposod (Mahony et al., 2008 Madgwick 2010 Seel & Ruppin 2017 Weber et al.] 2021 Laidig & Seel 2023 Several algorithms were developed to achieve censor-to-seguneg
calibration for specific kinematics with full sensor
2020). Finally, variety of sparse IMT methods have been developed that either use a limited number of sensors while still depending on magnetometers (yon)
setups (Tetzz et al., 2016 McGrath et al. 2018 Olsson et al.
Marcard et al. 2017 Huang et. al.) 2018: Sy et. al.) 2020 2021 Zheng et al. 2021) or are magnetometer-free
Grapentin et al. 2020 Yi et al. 2021 2022 Bachhuber et al. 2023 Van Wouwe et al. 2023
In summary, there exists a plethora of methods, each tailored to very specific application, such as magnetometer-free tracking of a single-degree-offfreeeem (1-DoP) joint (Lehmann al. 2020), human
2

---

Published in Transactions on Machine Learning Research (10/2024)
pose estimation from six IMUs, detailed in Yi et al. (2021) To apply IMT, the user must successfully identify the method that is suitable for the given problem and typically specify various parameters such as joint axes directions and sensor placement. Therefore, the user must be an expert in the field of IMT, which strongly limits the use of IMUs in many application domains. To make matters even worse, a given problem might require a nontrivial combination of methods which may exclude each other, e.g., the tracking of a sparse three-segment KC currently requires known joint axes directions (Bachhuber et al.) 2023). but the method that estimates the joint axes directions does not allow for a sparse sensor setup (Olsson al. 2020
What if, instead of a plethora of methods, we had single pluripotent method that can be used for, e.g., magnetometer-free tracking of 1-DoF joints, and the tracking of sparse sensor sctups with known or even unknown joint axes directions? What if we had one to track them all?
In this work, we demonstrate that ML methods can be used to, for the first time, achieve this goal. We propose a mothod, named RING, that combines novel neural network architecture, named RINGCell, with an elaborate training data simulation, the Random Chain Motion Generator (RCMG). The key insight that enables the pluripotency of RING is that given system in Inertial Motion Tracking Problem (IMTP) can be viewed as a graph where nodes represent segments and edges represent single DoF. Then, a shared set of paramcters can be applied on decentralized, per-node level, where only local IMU measurements are observed and only the local estimation problem is solved, i.e., the orientation relative to the parent is estimated. Information exchange between nodes is enabled by passing messages along the edges of the graph.
We show that RING can be used to plug-and-play solve a range of challenging magnetometer-Fi free sparse or non-sparse motion tracking problems with single trained neural network and that it even solves previously unsolved challenging IMTPs, such as, c.g., the tracking of triple-hinge-jonnt system with only two IMUs. We remonstrate that although RING is trained solely on simulated data, it zero-shot generalizes to experimental data and aligns with various state-of-thee th- (SOTA) results.
2 Related Works
As mentioned above, there exists a plethora of highly specialized methods in the field of IMT, but there is no single pluripotent solution that solves a variety of IMTPs. Even the use of ML methods for 1MT has so far only led to specific solutions for single IMTPs. RNNs have been used in Weber et al. (2021) to achieve SOTA attitude estimation, and in Bachhuber et al. (2023) to successfully track specific sparse KC. Deep learning has also been used for human motion capture, where the full-body pose is ostimated from typically six
more IMUs, and previous work has shown promising results (von Marcard et al. 2017 Huang et al.) 2018 Zheng et al., 2021 Yi et al. 2021 2022 Van Wouwe et al. 2023 Puchert & Ropinski 2023). However, while addressing a challenging problem, these methods are limited to human motion capture with one specific sensor sctup and assume statistical patterns of human motion (yon Marcard et al. 2017), or full-body biomechanical models (Yi el al. 2022) to constrain the estimated pose.
From a methodological viewpoint, RING uses a decentralized network of message-passing RNNNs with shared parameters that are trained via supervised lcarning. The concept of decentralized networks, communication, and collaboration is at the heart of multi-agent systems, and the means of communication can either be
Wang et al. 2019) or, more recently, learned (Sukhbaatar et al. 2016 Foerster et al. 2016 Wang et al. 2018 Pathak et al. 2019 Huang et al. 2020 In deep Reinforcement
prescribed (Panait & Luke 2005
Learning (RL), feedforward networks have been used to parameterize structured policies that pass messages
along the edges of graph in (Sukhbaatar et al.] (2016): Foerster et al. (2016); Wang et al. (2018); Huang
ee al. 2020), and distinct advantages of mressage-psssing have been shown. In particular, in Huang al. (2020) it was investigated whether centralised control can emerge from decentralized policies, and they show that it is possible to learn global policy that achieves locomotion across various agent morphologies. It is interesting to note that policies must collaborate in order to achieve global task, e.g., locomotion, and are motivated by a global reward. In the present work, the decentralizes RNNs must collaborate by exchanging information in order to solve the task of motion tracking, and are motivated by a decentralized loss function. The advantages of communication and global coordination emerging in a decentralized structure were also investigated in [Sukhbaatar et al. (2016): Foerster et al. (2016). In particular, the work of Sukhbaatar et al. (2016) uses supervised learning instead of RL for learning communication protocols.
3

---

Published in Transactions on Machine Learning Research (10/2024)
3 Preliminaries
3.1 Notation
We use a typical notation with scalars denoted by vectors by æ, matrices (or higher dimensional tensors) by X, and quaternions (or higher dimensional arrays quaternions) by q. Additionally, note that the symbol 1 defines either the unity element of given space, or the indicator function, such that, e.g., I)(()) is one for 0 and zero else. The symbol @ is used to denote the direct product of vector spaces, and additionally denotes quaternion multiplication, see Definition B.2 Further details can be found in Appendix A.1
3.2 An Inertial Motion Tracking Problem
We define an IMTP as the task of estimating the trajectory of the complete rotational state of an Articulated Rigid-Body System (ARBS) from inertial data. Conceptually, an ARBS is collection of multiple rigid (or assumed to be rigid) bodies that are interconnected via joints that allow for relative motion between these bodies. Let there be singleton, inertial reference coordinate system named base (sometimes world, or Earth), then an orientation of a body relative to the base is referred to as an absolute orientation. An orientation of a body relative to the coordinate system of another body is referred to as a relative orientation. Then, assuming that no additional information about the types of joints is provided, to fully deseribe the rotational state of an ARBS that consists of N bodies al a single moment in time, N orientations must be specified. In this work, we will utilize N --1 relative orientations and one absolute orientation.
An IMTP is solved by estimating the rotational state from at most N IMUs that are connected to the bodies of the ARBS (at most one IMU per body), and that provide 3D measurements of angular rates, specific forces, and the magnetic field density in their local sensor coordinates. A magnetometer-fiee method solves an IMTP without the use of the magnetometer, and such IMUs are referred to as 6D IMUs. In this case, the rotational state of the ARBS can only be estimated up an absolute heading error, that is, up to an arbitrary rotation around the gravity (or vertical) direction, and the one absolute orientation is referred to the absolute attitude. This is because both accelerometer and gyroscope measurements are invariant under rotation around the vertical direction. Finally, note that if at least one body of the ARBS does not have an IMU attached, then the IMTP is said to be sparse.
3.3 Graph Connectivity
The topology of an ARBS is represented by Connectivity Graph (CG) (Featherstone) 2008 2010), which is an undirected graph where the nodes represent the bodies that constitute the ARBS and the edges represent its joints.
Before the CG can be encoded, the bodies must be numbered. We adopt the broadly adopted standard numbering scheme and notation from Featherstone (2008 2010 which for an ARBS with N bodies proceeds follows:
1. The base is assigned the number 0 and it serves as the root node.
2. The remaining bodies consecutively numbered from 1 to N, so that each body has a higher number than its parent.
The CG may then be encoded via a parent array À E NN where A[i] is the body number of the parent of body i. Additionally, we define the function yu() to return the set of the body numbers of the children of body i, that is
(x ti ADJ = Vj 1}.
(1)
Definition 3.1. The body i of an ARBS with parent array À is said to be an outer body if it has no children bodies, i.e., (1) = U, or its parent is the base, i.e., Ali] = O; otherwise it said to be an inner body.
As an example, consider an ARBS that is a three-segment KC (see Figure 2 There, the three bodies are numbered increasingly from top-tobbottom, and then, for this numbering, the parent is given by X = (0, 1, T)
4

---

Published in Transactions on Machine Learning Research (10/2024)
Note that if the inner body (middle segment) is assumed to connect to the base, then the parent array is different and given by X = (0, T,T.
3.4 Minimal and Maximal Coordinates
We refer to the generalized coordinates position vector as the minimal coordinates, denoted by q- It fully captures the kinematic state of the ARBS using minimal amount of coordinates. The size of depends on the Dol's of the joints in the ARBS. In this work, we will consider ARBSs that can move freely in space (without any constraints). Therefore, the corresponding CG, the edges that connect to the base are 6-DoF free joints and all the remaining edges represent single DoF.
Definition 3.2. For an ARBS with N bodies with a CG given by À where the joints that connect to the base are 6-DoFF and 1-DoF otherwise, the size of q is given by N₂ = -;; 71o(A(i)) -1- Lo(X())). Similarly, the size of the velocity of minimal coordinates q is given by Ng =4=E; E; ( + 1o((())
For an ARBS with N bodies, we refer to the of all euclidean positions and rotational states from the base to all bodies in the system as the maximal coordinates, denoted by t (H (*) The size of t depends only on the number of bodies in the system.
3.5 Representing Orientations
We represent 3D orientations with quaternions q due to various advantages over equivalent representations using orthogonal matrices or Euler angles (Kuipers) 2002).. In particular, we use lq E H to denote the absolute orientation from the base to the body one's coordinate system. Similarly, we use l/ to denote the relative orientation from body one body two. In Appendix B we define various utilized quaternion-velated operations.
3.6
Loss Function For Orientations
In order train and evaluate ML methods that prodict orientations (represented by quaternions), suitable metric function must. be identified. We can compare the difference between a ground truth orientation q and the corresponding predicted orientation & by computing the angle of the smallest rotation that makes ground truth and prediction identical. It is given by angle(q @ q²) where @ denotes quaternion multiplication, denotes the complex conjugate, and angle extracts the rotation angle of a quaternion (see Appendix B.5) Then, we use the following mean-squared-errrr function loss q E H. 9 E H - R2 to calculate a scalar error between two quaternions, given by
loss q, 9) = angle(q & 9¹)2
(2)
The loss function for a single orientation entire rotational state of the KC in eq.
eq. (2 then used to compute the mean-squaard-eerror for the
4 Method
In this section, define the problem under consideration and the proposed method which consists of three components:
A virtually infinite, simulated training data set (see Section
4.2
A novel, online-capable neural network architecture, named RINGCelI, purpose-built for state estimation in ARBS (scc Section 4.3 In contrast to typical RNNs that map fixed number of input features to a fixed number of output features using a centralized logic, RINGCell leverages the graph-structure of ARBS and employs a decentralized, message message-passing logic with a shared set of parameters. This design maps a fixed number of input features per body to a fixed number of output features per body, and it allows RINGCell adapt the size and topology of the ARBS without. the need for retraining.
5

---

Published in Transactions on Machine Learning Research (10/2024)
0
sys =
n, e N
J RN3
Figure 2: System object example (sec Defini-
R R2Nx3
tion 4.2) for a three-segment KC with N = 3 bodies and parent array X3 (0, T.
2
K, r E BRN
Tg E R
3
A powerful IMT solution, named RING, that can solve a broad range of IMTPs in the real world (scc Section 4.4 RING is obtained by training RINGCell on the simulated training data.
Finally, Section 1.5 provides software implementation of the entire method and, more specifically, RING. Most notably, provide single file that trains the here benchmarked version of RING from scratch without requiring any files that contain real-world training data.
4.1 Problem Formulation: A Class of Inertial Motion Tracking Problems
In this work, we consider the following class of IMTPs. We consider an N-segment KC where N E (1,2,3.4) with unknown physical geometry and with segments that are interconnected via hinge joints. The axes directions of these hinge joints may be known or unknown. For each inner body al most one and for each outer body exactly one 6D IMU, with unknown sensor-to-body position, is rigidly or nonrigidly attached to each body. For cach body and with the KC at rest, the constant sensor-to-body orientation is assumed be known. The initial pose of the ARBS is assumed to be unknown. Then, the task is to estimate, for every timestep, the rotational state of the KC (consisting of N orientations) from the available IMU measurements and the available joint axes directions. Note that the task requires providing accurate estimates at each timestep, thus necessitating an online-capable solution focused on online, real-time processing (filtering) as opposed to retrospective data processing (smoothing). To summarize, two TMTPs of this class of IMTPs can differ in: 1) the number of bodies N in the KC, 2) the number of attached LMUs, 3) the number known joint axes directions, and 4) whether IMUs are attached rigidly or nonrigidly. A broad range IMTPs from this class of IMTPs are illustrated in Figure [ within the grey ellipsoid.
Definition 4.1. The parent arrays AN of the N-segment KC where N E (1,2,3,4) are given by, without loss of generality, A = (0) A2 = (0, 1)T, As = (0, 1,22) A = 20,1,2,3),, respectively.
4.2 Training Data: The RCMG Algorithm
The RING network is trained on data obtained from simulated random motion of one- two-, three-, and four-segment KCs. The procedure that generates this training data (from only PseudoRNG) is called the Random Chain Motion Generator (RCMG) [Bachhuber al. 2022). The RCMG procedure (see Algorithm has three main steps that execute consccutively:
Randomized KC (see Section 4.2.1 p. A randomized KC is drawn to manipulate the downstream simulation and achieve various forms of domain randomization that enable bridge the sim-to-real gap from simulation (training) to real-world (testing).
The KC is simulated to randomly move in space and the maximal coordinates of all bodics relative to the base are computed for every timestop.
Random Motion Generation (see Section 4.2.2
3. Get X, Y Data (see Section 4.2.3 From the maximal coordinates of all bodies, the IMU, joint axes, and pose data is computed, and returned as training data.
Internally, the RCMG procedure uses system object sys (see Definition 4.2 which the collection of various attributes such as, e.g., joint axes array J E Nx3

---

Published in Transactions on Machine Learning Research (10/2024)
Randomly Attached Bases O0
A
randBaae
X2
3 00
Oa
O
Co O0
Figure Each sequence of training data is simulated using the RCMG. In the first step of the RCMG randomized KC is created which involves randomizing the node in the KC to which the base attaches. Afterwards, the nodes in the graph are numbered according to Section 3.3 For example here, for \1, the new numbering array is = (2, 3,),,) and the new parent array X = 1,33,
Definition 4.2. We define a system object sys with N bodies as the collection of the following attributes:
numbering array of integers n E N which stores a permutation of the numbers going from 1... N,
& parent array of integers À E NN,
joint axes array J E Nx3 that contains the hinge joint axis direction,
pelative-topparenl position array R E R2Nx3 that contains the position vector of the body's 's coordinate system relative to its parent (expressed in the parent's coordinate system). In Section 4.2.1 it is outlined that for each of the N bodies there is a second IMU body. In the R array, the first N values are used to specify the position vector between two non-IMU bodics (segment-to-segmann positions, physical geometry of the KC), while the last N values are used to specify the position vector from non-IMU body to the correspouding IMU body (segment-to-be body positions, IMU attachment).. This is done to independently randomize both the physical geometry and the IMU attachment, forcing the network to learn to generalize to scenarios such arm robot with unknown segment lengths, as well as to calibrate for an unknown IMU attachment,
an array of stiffness parameters for N free joints K Nx6
an array of damping parameters for N free joints r E Nx6
float representing the sampling time T, E IR, here always 0.01 s. An exemplary system object is shown in Figure
4.2.1 Step 1) Randomized Kinematic Chain
In order to enrich the simulated training data, several forms of domain randomization are achieved by drawing a KC with randomized system attributes.
The first domain randomization is the randomization of all downstream forward kinematics applications, and additionally, randomization of the absolute random translation and orientation in the generation of random motion. This has been shown improve the training data such that the trained network more effectively closes the sim-to-real gap (Bachhuber et al. 2023) This is achieved by re-altaching the bases randomly and afterwards, the nodes in the graph are re-numbered according to Section 3.3 An example of this is shown in Figure 3 Secondly, the N randomized hinge joint axes J are drawn. Thirdly, the position array R. is randomized by drawing values from uniform ranges.
Finally, the stiffness K and damping arrays are randomized. These values are used to model nonrigidly attached IMUs by connecting additional nodes that are connected via passive spring-damper free joints. For each node in the CG, we add an additional IMU node with body number i N that child of node and that is connected to node via passive saring-damper free joint. The stiffness K[i] and damping [[
7

---

Published in Transactions on Machine Learning Research (10/2024)
Kinematic chains with added IMU nodes (green)
Frozen Spring. Damper
Passi ve Free Joint
25%
chance
Spring-Damin
Du
IMU node 2
Figure 4: For each node i in the KOs, there exists a second IMU node (that is not counted in the body count N of the system; here in green) with body number i + N. The IMU node is connected via a passive spring-damper free joint to the original node order to simulate nonrigidly-atlaches IMUs. There 25% chance that the damping and stiffness parameters of the passive free joint are chosen such that the IMU is effectively rigidly attached and the passive free joint is frozen.
parameters are used during the forward simulation of the KC as the parameters of the 3D spring-damper dynamics between node i and (IMU) node + N. For each node i, the stiffness K[ and damping F[i] parameters are randomized a way such that, either the IMU effectively rigidly attached, or such that the IMU moves relative to the segment node (which then models nonrigid attachment).
4.2.2 Step 2) Random Motion Generation
As a second step, the RCMG procedure generates random motion of the previously obtained randomized KC. Random motion is obtained by drawing a random reference trajectory for all joints in the KC, The generation of such randomized reference trajectories influenced and constrained by various parameters, e.g., upper limits angular velocities lower limits on the amount of motion (to avoid jittering). Additional details the reference trajectory generation be found in Appendix C11 Afterwards, a dynamical forward simulation is performed where joint forces are computed using PD control such that the random reference is tracked. Note that the additionally added nodes to model nonrigid IMU attachment are passive free joints and as such they are not actuated. Finally, the trajectories of maximal coordinates of all N bodies and IMU bodies, given by T (H @ R3) R 2NxT (from base to body), are computed.
4.2.3 Step 3) Get X, Y Data
In the last step, the training tuples of X, Y are computed from the trajectories maximal coordinates and afterwards returned. They are:
X X where X(;, i, 666 is the simulated 6D IMU data for body as measured in its IMU body i+N (but for each inner body, there is chance that the IMU data gets dropped out and replaced by zeros), and where X/: i, 6:] is the joint axis direction J[i] of the hinge joint between body and its parent A[i] and zeros if the parent is the base (but for each body, there chance that the joint axis direction data gets dropped out and replaced by zeros), and
Y e HTXN where Y|: is the timeserics of: 1) absolute attitudes (q(t) if the parent A[i is the base; 2) relative orientations (q(1) from body i to its parent p A[i1, and
where T is the number of timesteps (here, 60s at 100 Hz, thus T = 6000), and N is the number of bodies in the KC. IMU and joint axes data is dropped out in order to force RING to learn to solve IMTPs with sparse IMU placement (an inner body may not have an IMU attached to it), and learn to solve IMTPs with require sensor-to-sogment calibration. Finally, multiple sequences are stacked build training batch. These input and output arrays (X and Y) directly align with the provided software implementation of RING, see A Appendix 4.5
8

---

Published in Transactions on Machine Learning Research (10/2024)
Algorithm RCMG (Generate One Training Data Pair)
Input: AN
Output: X EERTA RT Y TH'N
sys +- randSys (Av) (see Algorithm
T + randMotion (sys) {see Algorithm
5: X, Y + gsy, AN. T) {see Algorithm
RINGParams
RINGCellParams
MessageMLP
StackedGRUCol
QuaternionML
RING
E11
RINGCell
&t
E
RINGCelSState
E
M.
9o
Et
hg
E1-1
pa G.uld) t..lo)
MessageMLP
Concatenate
StackedGRUCell
QuaternioMM
E
Y:
X
M.
M.
À co
X
X.le X.
Y
i
X.Ip|
po
Alp
Figure 5: The architecture of the plug-and-play IMT solution RING. It consists of the RNN cell RINGCell and a final MLP head that returns one quaternion per node in the graph. RING is trained to estimate child-to-parent orientations from the available local IMU and joint axis data and nearest noighbour messages. this end, RINGCell applies a shared set of parameters on a decentralized, per-node level and passes messages along the edges of the graph to allow for information exchange. Note that while the parameters are shared, the hidden states not shared across nodes. This architecture allows RING to apply single set of parameters across broad range of IMTPs, which may vary in aspects such as the number of segments, and it ultimately enables RING's remarkable pluripotency.

---

Published in Transactions on Machine Learning Research (10/2024)
4.3 The Architecture of RING
RING is based on recentralized network of crassage-asssigg RNNs which allows for information exchange along the edges of a graph (as given by the CG). RING can be applied to an arbitrary CG (see Section 3.3 and it maps per-node-input to per-nodo-output. Most importantly, the parameters of RING are shored across the nodes of the graph such that the number of parameters of RING does not depend on the given CG. The hidden state, however, is not shared across the nodes. This allows for the training of a single RING network which then solves variety of IMTPs (see Section 4.1
The introduction of RING stems from the requirement of acquiring single pretrained network that solve different IMTPs with varying number of inputs and outputs, e.g., estimate two orientations for a two-segment KC, and three orientations for a three-segment KC. This is possible because on a node (or segment) level, we can guarantee static input (at most one IMU, at most a known joint axis direction) and output shapes (one orientation relative to parent). But the per-node-outpit cannot be estimated from only the per-node-input as the orientation relative to the parent depends on the orientation state of the parent, and orientation w.r.b. the base camnot be estimated from 6D IMU data, additionally, the segment may not have IMU attached. Thus, we have to allow information exchange between nodes and propose scheme which again gives rise to local static input and output shapes of messages. Finally, we claim that the estimation of the entire pose in hierarchical approach is inherently natural and provides an advantageous structural prior to subsequent parameter learning which we explore in Section 5.4
RING consists novel RNNN cell, named RINGCell, and a final Multi-Layer Perceptron (MLP) head such that the network's per-node-output is a single unit quaternion (per timestep). Note that the network head is independent of the RINGCell and may easily be replaced to suit different needs.
Let À be CG with N E N nodes, let F N be the number of input features per node, let H N be the half-hidden state dimensionality, and let M E N be the dimensionality of the latent messages passed inside the cell based on the edges in the CG. Then, let 6-11 E RNx2H be the hidden state of the RINGCell from the previous timestep t- 1, and let X, E NN be the F inputs for all N nodes at time t. Thon, the next hidden state &, is obtained by
& = ringCelll ( Vt
(3)
with So 0.
Internally, a RINGCell has the parameters of
a Messogg-M.P-netwoot fo TR + M (three layers, single hidden layer, hidden layer size H, ReLU activations, no final activation), and
a Stack go R22H R2 + R2H which consists of the sequence of Gated- Recurrent- Unit( GRU)Cell, LayerNorm, GRUCell (Cho al., 2014]. Note that is symbolic for the whole set of parameters of the Caacked-GRUUCell--eeewowo and that it different to the parameters of fe.
Note that the two GRUCells each have a hidden state dimensionality of H, thus the hidden state of the Stackod- Stackod-GRUCal GRUColl-network is of dimensionality of 2H.
Then, RINGCell has three consecutive steps, Vi N.
Messages M, E N are computed.
M 1 = fo (i,
(4)
Messages are passed and latent input X E Nx(2M+F) computed.
X
concat
M, [ 0+
Σ
Mr X.lil
(5)
cE
where M, [[i]] is 0 if A 0.
10

---

Published in Transactions on Machine Learning Research (10/2024)
3. Hidden state is updated.
f go (6)- x(i))
(6)
The architecture of RING is finished by piping the hidden state &, through final network head, the Quaternion-MI.P that combines
a Layernorm, and a MLP-network hg R + R (three layers, single hidden layer, hidden layer size H, ReLU activations, no final activation).
The Quaternion- has two consecutive steps, Vi = =1..
Unnormalized output Y E RRxx computed.
Y = he [layernorm (E, H:])
(7)
2. Normalize to allow interpretation as unit quaternions. One unit quaternion per node. Final RING output Y, E HHN
Y
(8)
Note that the normalizing operation, due to its square-root operation, requires special care to allow for successful backpropagation. Also note that the employed loss function, see Section 3.6 is based on a arctan expression, which does not require any special care, in contrast to seemingly equivalent expressions to extract the angle from a quaternion that are based on arccos.
Finally, by combining the equations B3, RING network a function that maps the timeseries of available input data X RTxNxF and CG X NN to the timeseries of predicted output data Ý HTXN ie.
and unrolling the RNN in time, we can view the entire
Ý = ringo (X,A)
(9)
where the parameters of RING are given by the set (fe. 9e, ha). The hyperparameters of RING are H N and M E N. Note that the set of parameters of RING is influenced by the hyperparameters of RING and the number of input features per node F, but not by the GC X or the number of bodies. For example, a single RING network can be used for predicting the orientations of both two-segment or threo-segment KCs even though the number bodies and, consequently, the dimensionality of input and output arrays different, o.g., X E Fx2xx compared to X RT33FF
4.4 RING: Single IMT Solution to a variety of IMTPs
In this section, we use the simulatedd training data from Section 4.2 to train a network based on the RINGCell architecture (see Section 4.3). The trained network is then called RING and, with a single of parameters, RING shows its pluripotency in range of experimental scenarios, from simple single-joint tracking complex four-segment KCs, without reliance magnetometers as will be shown in Section
For each of the four KCs A (see Definition 4.1). we use the RCMG (see Algorithm to simulate 512 input-output pairs and stack them to create single batch of training data. This batch of training data is used to update the parameters of the RING network (see Section 4.3) with // = 400 and M = 200 (total parameter count: 3337 404) by minimizing the meaa--quuarrellrrrenaiiinntnttte
i.e.,
min 10T
Σ
E X.Y X.YGACH((A)
loss (Y, ring(X, Ax)) [t
(10)
NE(122,34)
where RCMG is given by Algorithm [ loss is given by eq.
2 ring is given by eq. 9 and where the
expectation E is estimated using 512 draws.
11

---

Published in Transactions on Machine Learning Research (10/2024)
1-DoF
rigidly/ nonriiidly attached IMUs
Segment
Figure 6: Experimental five-segment KC with len IMUs (orange boxes) and 20 OMC markers (grey spheres). It uses a single spherical joint followed by three hinge joints, each oriented along the x, y, and z axes, respectively. Each segment of the KC was equipped with two IMUs: one firmly attached to the segment and another affixed nonrigidly using foam padding. The experimental KC is moved in space, and the inertial and optical data is recorded to validate RING and compare it to SOTA methods across broad range of IMTPs.
We use the LAMB optimizer (You al. 2020 combined with global (threshold 0.1) and adaptive (threshold 0.2) gradient clipping, a cosine decaying learning rate (initial learning rate 1 x 10-3 and Truncated Backpropagation Through Time (TBTT). Borrowing the notation from Williams & Peng 1990 utilize TBTT(10s, 10s), i... the gradients are stopped and applied after 10s instead of the total length of s. This results in every batch generation (or episode) corresponding to six parameter updates. We train for 5000 episodes using a single node with eight A40 GPUs (each 48 gigabytes of VRAM). Due to the large amounts of training data, overfitting is seemingly impossible and any form of early stopping did not prove to be required. This has been also confirmed with a separate validation dataset. Training has been stopped after 5000 episodes as the loss has longer improved.
For practical applications, the inference time and computational requirements of RING are more important than its training requirements. To this end, a theoretical and empirical analysis of the time complexity of RING is conducted in Appendix D and the findings are summarized here. The theoretical time complexity to advance the prediction of RING by one timestep is O x H x (H + M + This translates in practice an efficient NN that enables real-world online application even on low-end hardware. For example, on single-core Intel Xeon at 2 GHz, RING can comfortably enable motion tracking of a four-segment KC more than 500 Hz, which is well above typical IMU sampling rates that range from 90 to 286 Hz (Laidig et al. 2021
4.5
Openly-available Code and Data
The code and experimental validation data is hosted at hampleeentary_mateerr and the repository contains implementations of the RCMG, RINGCell, and RING as decoupled components. Additionally, it includes the code of SOTA methods and validation data to create the experimental validation results (AMAEs and RMAEs) of RING and the SOTA methods that are discussed in Section 5 Finally, we also provide files to retrain RING from scratch, without requiring any real-world training data files. The software, most notably, uses the JAX and Haiku frameworks (Bradbury et al. 2018 Hennigan et al. 2020 The caso-of-use of RING is demonstrated through a code example in the Appendix
5 Experimental Validation of RING
In this section, we evaluate the accuracy of RING with one common set of pretrained parameters (see Section [4.4p across broad range of experimental IMTPs (see Section 4.1]. In general, RING shows remarkable pluripotency in zero-shot generalization to real-world experiments across diverse IMTPs. This underscores its broud applicability.
In Section 5.1 we describe the experimental setup used to evaluate RING on real-world data. We show that RING successfully solves multiple previously solved challenging IMTPs and competes with the current SOTA methods (Section 5.2). Impressively, RING further achieves accurate tracking in even more challenging IMTPs, including two IMTPs that have not been solved before (Section 5.3
12

---

Published in Transactions on Machine Learning Research (10/2024)
5.1 Experimental Setup and Evaluation Metric
Experimental evaluation is conducted on a five-segment KC which is shown in Figure G The KC is constructed by a concatenation of single spherical joint followed by three hinge joints, each oriented along the x, and 2 axcs, respectively. Each segment of the KC was equipped with two IMUs (MTw Awinda, Xsens, Enschede, the Netherlands): one firmly attached to the segment and another affixed nonrigidly using foam padding, This foam-attachment of TMU was conducted to investigate the reduction of motion artifacts. The left panel in Figure G shows a close-up of one segment of the KC with one exemplary nonrigidly attached IMU. Each segment is equipped with least three noncollinearly placed reflective markers that are used to obtain ground truth orientations from twelve camera Optical Motion Capture (OMC) system (OptiTrack Prime x22, NaturalPoint Inc., Corvalis, USA).
Two distinct trials were conducted, involving random movements of the five-segment KC, by introducing manual alterations of the KC pose by two experienced scientists. The first trial, spanning a duration of 66s, fcatured diverse range of motions, extending from very slow to notably rapid movements. Morcover, the second trial, with a length of 68s, not only encompassed a variety of motions but also incorporated random intervals of complete stillness, wherein the KC remained motionless.
All trials were preprocessed in the following way: NaN values were removed, an offline time-synchronization was employed via cross-correlation between measured (IMUs) and approxinated angular velocities (OMC), and all collected data was resampled to a uniform sampling rate of 100 Hz. Additionally, we correct for any small misaligument between the rigidly attached LMUs local coordinate systems and the corresponding segments' body coordinate systems as spanned by the OMC markers. Simultancously, we align the OMC's reference coordinate system with the earth reference coordinate system (the base), as observed by the IMUs. For more comprehensive details regarding these preprocessing steps and the software implementations utilized, readers are referred to the study Laidig et al. (202))
The rich experimental data obtained from the five-segment KC enables us to perform evaluations on subsets of data and the KC, which represent a varioty of different IMTPs. For example, for validation of a one-segment KC, we use the recorded data of all five segments of the five-segment KC independently for evaluation, which effectively increases the amount of validation data by a factor of five. Similarly, for validation of a two-segment KC with hinge joint, we use three different sub-KCs with a joint axis direction along the x, y, and axes (the two-segment sub-KC of segment one and segment two is excluded due to its spherical joint). Similarly, for a three-segment KC with double hinge joints, we use two different sub-KOs, and for a four-segment KC with triple hinge joint we use the sub-KC of segment two to five.
Assessing KC pose estimation performance of RING in comparison with existing SOTA and thus solving the TMTP under consideration, requires the usage of a suitable evaluation metric. As already discussed in Section 3.6 the expression angle(q @ Q") can be used to compare the difference between a ground truth orientation q and the corresponding predicted orientation q. Thus, in order to compare the timescries of ground truth Y E HTTN and predicted pose Y E TTTN of an N-segment KC with GC AN, we compute (AMAE) and
the Maeea-an-bbooottorrrrtttttttttte (RMAE) which are given by
T
AMAE Y. 8
Σ
angle ZeeroHead (Y[t, @ zeroHead (911.1)))
(11)
=
t= 500
T
N
RMAE Y.
Σ
Σ
angle (Y(1, @ [(.1)=)
(12)
TN
I=500 2=2
where I.I denotes the absolute value and zeroHead removes the heading component (see Definition B.7 Note
that initial 5s (equaling to an index of 500 at 100 Hertz) of each timeseries were deliberately excluded from the AMAE and RMAE calculations. This decision was made to ensure that the recorded errors accurately reflected the method's performance after convergence.
From Section 3.2 recall that magnetometer-frre IMT estimates one absolute attitude, and N relative orientations. This difference is captured by the two metrices AMAE and RMAE. Mathematically, AMAE
13

---

Published in Transactions on Machine Learning Research (10/2024)
reports zero angle error if the ground truth orientation and estimated orientation equal up to an arbitrary heading difference. Whereas, RMAE reports zero angle error if and only if both orientations are exactly identical. The implications are that a low AMAE indicates that the inclination of the system is correctly estimated but the entire system might still be rotated with an arbitrary yaw angle. A low RMAE means that the entire internal pose of the system accurately estimated.
5.2
RING Unifies Prior Work
5.2.1
Attitude Estimation
Attitude estimation in real-world enviromments using inertial sensors vital prerequisite for a wide range of applications, including tracking human movement and enabling autonomy in air and land vehicles. The attitude estimation problem is defined estimating the orientation of an object with respect to the horizontal plane with single sensor (Weber et al. 2021
The widely employed and long-standing methods from Madgwick (0010); Mahony et al. (2008); Seel &
Ruppin (2077) have recently been outperformed with SOTA approaches by Weber et al. (2021) ;; Laidig Seel (2023). Table shows that. RING aligns with SOTA performance in attitude estimmation, as indicated by the experimental ASA.s. More specifically, the AMAE of the attitude for Madgwick] (2010) (2.25 + 0.81) for
Laidig & Scel 2023) is (1.61 + 1.04),, for Weber et al. (2021) is (2.06 + 1.03),, ", for MMhony et al. (2008) is
(2.09 + 0.87)",
for Seel & Ruppin 2017 is (2.56 + 0.93 and for RING is (2.13 + 0.91).
5.2.2
Magnetometer-free Tracking of 1-DoF Joint
IMT of connected segments without relying on magnetometers is desirable for numerous applications where the magnetic field is typically disturbed, including control of industrial robotic manipulators, interconnected drones in automated warehouses, and human motion analysis in hospital environments.
Here, we consider the IMTP of tracking the relative motion between two segments from two 6D IMUs, assumiin single hinge joint with a known axis direction connects both segments.
Overcoming the use of magnctometers is typically achieved by combining SOTA approaches for attitude estimation and utilizing joint-specific constraints that exploit knowledge of the hinge joint axis direction Laidig al. 2017 Lehmann et al. 2020). Additionally, magnetometer--eliane method uses 9D VQF (Laidig & Scel 2023 for both IMUs independently and does not exploit any kinematic constraint between the two body parts. The RMAEs are reported in Table I and they are: for the VQF-basck baseline (19.36 + 8.02) for Lehmann et al. (2020) (4.15 + 2.05). for Laidig et al. (2017) (3.32 + 1.),,, and for RING (3.52 + 1.00) This shows that RING aligns with the SOTA methods, which are already a non-trivial combination of two separate methods which, in contrast to applying RING, requires expert knowledge.
5.2.3 Magnetometer-free Tracking of 1-DoF Joint with Unknown Joint Axis Direction
The IMTP of Section 5.2.2 can be made more challenging by assuming that the hinge joint axis direction is not known. Then, it can first be estimated from the LMU data, and then subsequently a method that assumes a known joint axis direction can be applied.
A SOTA method for hinge joint axis direction estimation is given by Olsson et al. 2020) RING out-of-tho-box supports an unknown hinge joint axis direction by its versatility to drop out the respective node input and replacing with zeros.
The experimental RMAEs are given in Table 1 combining Olsson al. 2020 with Lchmann et al. 2020 results in 24.36+).223 and with Laidig el al. 2017 results in (3.1842.05) o and RING achieves (3.92 1.40) This shows that RING aligns with SOTA performance, which comprises three distinct methods.
5.2.4
Three-Segment Sparse IMT
Only few recent works have achieved the combination of sparse and magnetometer-fee free IMT. One such challenging IMTP is, the tracking of all relative segment orientations of three-segment KC, with double
14

---

Published in Transactions on Machine Learning Research (10/2024)
hinge joints and known hinge joint axes directions, by using only two IMUs placed on the outer segments.
The SOTA method for this IMTP is given by Bachhuber et al. (2023). The absolute attitude can be easily tracked using any of the methods from Section 5.2.1
As shown in Table ! the experimental RMAE using Bachhuber et al. 2023 is (5.60 + 2.35),, and using RING the RMAE reduced to (4.14 + 0.53) and, consequently, RING outperforms the SOTA in this IMTP.
5.3
RING Goes Beyond the SOTA
5.3.1
Motion Artifact Reduction
The efficacy of all SOTA IMT methods heavily rely on rigid sensor-to-body attachment. In many practical scenarios, such human motion analysis, this assumption is rapidly violated, resulting in strongly degraded estimation accuracies, attributed to a model-reality mismatch.
Here, we consider the IMTP of Section 5.2.2 however here, the IMUs are not rigidly attached to the two segments, while the estimation target remains the relative orientation between the two segments (and not between the coordinate systems of the two IMUs), and the absolute attitude of one of the segments.
Currently, there exist method that does not make the assumption of rigid IMU attachment. Still, the methods from Section 5.2.1 can be applied to estimate the absolute attitude, and we use the most accurate estimator VQF (Laidig & Seel 2023) for comparison purposes. The methods from Section 5.2.2 are applied to estimate the relative orientation. To compensate for the violation of the rigid-IMU-atttchmeen assumption, we additionally apply an intuitive low-pass filter (LPF) step to the estimated absolute attitude and relative orientation for each baseline method to suppress unwanted high frequency artifacts. The cutoff frequency was grid searched and we report only the best result for each baseline. Alternatively, the use of the LPF on the estimated orientations prior to computing the relative orientation did not yield better performance.
In Table 1 column 5. reports the experimental AMAE in the attitude estimate, and demonstrates RING's superior performance of (7.59 + 2.85) over the combination of VQF+LPF with an AMAE of (9.19 + 2.31).. Similarly, column 5.3.1 reports experimental RMAEs, they are: Lehmann et al. (2020) achieves (8.00 = 2.78),, Laidig et al. (2017) achieves (7.00 1.57)", and RING achieves (5.56 = 2.33) S This shows that RING outperforms the SOTA methods. Note that the reduced AMAE shows that RING has
learned to fuse the information of the second segment's IMU into the attitude estimation of the first segment.
5.3.2 Three- Segment Sparse IMT with Unknown Joint Axes Directions
can be made even more challenging by not assuming known joint axes The IMTP considered in Section 5.2.4 4 directions.
To the best of the authors' knowledge, there currently exists no method that is applicable in such a challenging IMTP. RING can solve this IMTP with only a modest increase in error (Table D given the increased complexity of the task. Note that the direction of the joint axis cannot be estimated using Olsson et al. (2020) such that the method from Section 5.2.4 may be applied, as Olsson et al. (2020) does not allow for a sparse sensor setup which inherently requires a pluripotent approach such as RING. We report a RMAE value of (5.37 + 0.71) for RING in this challenging IMTP.
5.3.3 Four- Sparse IMT: 3-DoFs between IMUs
Increasing sensor sparsity will naturally make an IMTP problem more complex. A KC configuration with four segments and only two 6D IMUs results in three DoFs (three consecutive hinge joints) between the two outer-segment 6D IMUs and it represents the limit of accurate sparse IMT. Despite the complexity, the estimation target remains to capture all three relative orientations.
To the best of the authors' knowledge, there currently is no method that applicable in such challenging IMTP. RING can solve this problem formulation sufficiently well. When assuming known joint axes directions for the three hinge joints, RING achieves RMAE of (6.78 + 1.41) An exemplary trial is shown in Figure 7
15

---

Published in Transactions on Machine Learning Research (10/2024)
Table Motion tracking accuracy (in degrees) of RING compared various SOTA methods across a variety of IMTPs. While previous methods are problem-spocific and Not Applicable (NA) many IMTPs, RING is the only method that accurately solves all problems. All columns report the RMAE, see eq. 12 as metric, except for the columns 5.2.1 and a.d5.3311 5.3.IA which report AMAE metric, see eq.
11
IMTPs
5.2.1
5.2.2
5.2.3
5.2.4
5.3.1
5.3.1
5.3.2
5.3.3
19 61
Method
1q
iq
19
19
(1)
2.06 1.03
NA
NA
NA
2(5)
NA
NA
NA
(2)
2.25 0.81
2(5)
25))
NA
2(5)
NA
NA
(3)
2.09 0.87
2(5)
(5)
NA
2(5)
NA
NA
NN
(4)
2.56 = 0.93
(5)
(5)
NA
2(5)
NA
NA
NN
(5)
1.61 1 1.04
19.3 1 8.02
NA
9.20 L 2.31
24.9 1 17.6
NA
NN
(5)+(6)
3.32 2.12
NA
NA
7.00 1.57
NA
NA
(5)+(7)
4.15 2.05
NA
NA
8.00 2.78
NA
NA
(G)
2.05
NA
2.60
NA
(7)+
4.06 + 2.23
NA
7.90
2.48 NA
NA
NA
(9)
NA
NA
NA
3.60 + 2.35
NA
NA
NA
RING
2.13 = 0.91 3.52 + 1.00 3.92 # 1.40
4.14 0.33
7.59 + 2.85
5.56 + 2.33
5.37 + 0.71
6.78 1.41
Methods: Weber et al (2023 (8) [Mdd 22110 (2), Mahoaa at (20ns (3)) [Seel & lupppin 2017 (4), Laidig & Seal
20023 (3),
Laidig et al
2017 (6).
Lehmann the AMAE RMAE ot
2020 diachibbe
to
RMAR the mnetladd
indicdde that AMAR RMAF. equall
right
t=0s
40s
50s
Real-world Experiment.
Render Optical Motion
Capture
Render RING's
prediction
Exemplary frames that showcase RING's performance for the IMTP that involves a four-segment KC with sparse IMU attachments and known joint axes directions (see Section 5.3.3) It is remarkable first that RING can accurately estimate the four orientations (one absolute and three relative orientations) from only two magnetometer-fiee IMUs. video of the trial available here
Figure 7:
When assuming unknown joint axes directions, RING achieves (13.66 This shows that with unknown joint axes directions, this IMTP pushes the limits of observability (Bachhuber et al. 2022
5.4 The Decentralized Approach of RING Provides an Advantageous Structural Prior
In this sention, we showcase second advantage of the decentralized approach of RING over centralized approach that is typically employed, e.g., by stacking multiple LSTM- or GRU-Cells. First, recall that RING is based on a decentralized network of message-passing RNNs which allows for the training of a single set of parameters despite a varying number of bodies the KC and, while the latter aspect is the core motivation behind this architecture is, interestingly, not the sole motivation behind the decentralized approach. The second motivation is that the estimation of the entire pose in RING's decentralized approach provides an advantageous structural prior compared to RNN that utilizes centralized approach. For this purpose, we compare RING to the RNN-based Observer (RNNO), deep GRU network with intermediate Layernorm layers, as it was proposed in BBahhhber et al. (2023). RNNO maps all available input data to the entire targeted pose data and does not utilize the specific graph structure of the IMTP. RNNO has been proposed as the solution of a specific IMTP (see Section 5.2.4) and, for this IMTP, both RING and RNNO achieve similar performance. However, this not the case the IMTP becomes vastly more complex. Consider, e.g., the IMTP of Section 5.3.3, which is arguably the most challenging IMTP under consideration in this work. We have trained RNNO the subset of training data of RING that corresponds to this IMTP. Despite
16

---

Published in Transactions on Machine Learning Research (10/2024)
Table 2: Motion tracking accuracy of RING solving the IMTP defined in Section 5.2.3 on external datasets.
RING provides consistently low errors demonstrating robustness across different IMU hardwarc.
Dataset
IMU Hardware
T(s]
Native Rate [Hz]
RMAE [
Ours
Xsens MTw Awinda
402
40
3.92 + 1.40
RepolMU (1)
microlMU (2)
390
90
3.44 + 0.06
OpenAXES (3)
Bosch BMI160 + Analog Devices ADXL355
227
~ 125
2.52 + 0.29
Szczesna et al.
(2016) (1), Jedrasiak et al. (2013) (2), Webering et al. (2023
(3)
varying several hyperparameters, the best reported RMAE of RNNO is (18.72 5.1)). This is significantly higher compared to RING's RMAE of (6.78 + 1.41)"; even though RING is maintaining its applicability to a broad range of IMTPs, and is not purpose-trained for a single IMTP. This showcases that the decentralized approach of RING provides an advantage even if only the solution of a single, specific IMTP required.
5.5 RING's Robustness to Different IMUs
IMUs in general, and especially across vendors, differ properties like noise density and bias offset.. RNN- based inertial sensor fusion has been demonstratod to generalize across different sensor hardware (Weber et al.) 2021p. Nelethe to ensure broad real-world applicability, we investigate RING's robustness to different IMU hardware. First, we analyze how RING's performance scales as noise and bias properties are incrementally increased. Then, we evaluate RING on openly-available, real-world datasets that use IMUs from different vendors.
To simulate reasonable noise density and bias offsct ranges, we constructed a worst-case IMU by combining the worst properties of various IMU manufacturers, as summarized in Table 3 Then, those worst-case noise and bias values are incrementally (in seven equidistant steps) increased from 0 to 120%, and a corresponding amount of simulated noise and bias is added to real-world TMU data (see Section 5.1), which yields seven modified datasets. RING and all SOTA methods are validated on the modified datasct, and this procedure is repeated using (en different seeds. The AMAEs and RMAEs of all met are plotted for all IMTPs as a function of the seven steps in Figure 9 The figure shows that, whilst the performance of all methods (as expocted) slightly worsens as noise and bias are increased, RING maintains accuracy comparable to SOTA methods in all IMTPs and, especially in two-segieent KC tracking, substantially outperforms them.
The RepoLMU dataset (Szczesna et al. 2016) contains real-world IMU and ground truth data from passive motions of a swinging pendulum. It uses non-commercial micro IMUs (Jedrasiak et al. 2013) with focus low cost and small size over accuracy. The OpenAXES Robot Dataset contains data from motions of robot arm drawing different shapes at different speeds. IMUs are attached to each segment, and the ground truth is known from the robot's 's encoders. Table 2 reports RING's performance for both external datasets.
6 Discussion
We have shown that single RNN, named RING, can accurately solve a broad range of IMTPss even if two IMTPs do not have same onput-output dimensionality. This pluripotent behavior originates from a decentralized architecture that provides an advantagoous structural prior compared to the more typical centralized approach which, in contrast to RING, has the additional limitation that it can only be utilized if the solution of single IMTP is sufficient. In summary, for the set of IMTPs under consideration (see Section 4.1). we have shown that RING can enable accurate IMT for one- two-, and three-segment KCs long as the IMUs are rigidly attached. Most notably, this includes an IMTP that is sparse, magnetometer-fice and requires sensor-to-segmeut calibration. If IMUs are nonrigidly attached, then RING is shown to provide accurate orientation estimates for a two-segment KC. RING can achieve accurate IMT for a four-segment KC, provided that the IMUs are rigidly attached and sensor-to-segment calibration not required. It is a remarkable first, that RING can track the pose of the four-segment KC (which requires in total four distinct orientations) using only two IMUs. Note that RING does not require problem specific priors, although their
17

---

Published in Transactions on Machine Learning Research (10/2024)
introduction, i.c., known joint axes direction, additionally attached IMU is straightforwardly possible as additional input data to improve the motion tracking accuracy even further. This can be seen in Table where, e.g., for a three-segment KC, providing joint axes directions decreases the mean RMAE from 5.37" (Section 5.3.2 to 4.14 (Section 5.2.4
7
Conclusion
In the presented work, we combined ideas from the domain of multi-agent systems with RNNs to propose an architecture based on decentralized network of message-passingg parameter-sharing RNNs. We have successfully exploited this combination for the analysis of structural sequential data and solved challenging state estimation problem, which is IMT of KCs, by letting the decentralized network map local IMU measurements and nearest-ncighbour messages to local orientations. In particular, we introduced RING, a pluripotend IMT solution that, unlike all previous, problemn-ppecffic approaches, enables plug-and-play non-expert use. RING outperforms a range of problem-speccfic SOTA solutions and even generalizes to previously unsolved scenarios, including the challenging combination of magnctometer-free and sparse sensing with unknown sensor-lo-segment parameters. Remarkably, RING demonstrates the ability to zero-shot generalize to experimental scenarios, despite being trained solely on simulated data. For example, RING can, for the first time, accurately track real-world four-segment kinematic chain (which requires estimating four orientations) using only two magnetometer-fiee free IMUs.
RING's pluripotency greatly simplifies the application of IMT by eliminating the need for expert knowledge to identify, select, and fine-tune problem-specific methods. This is expected to not only make IMT more powerful and less restrictive in established domains but also to facilitate the accessibility of IMT technology by non-expert users and broadens its applicability to previously untapped domains.
References
Simon Bachhuuer, Daniel Weber, Ive Weygers, and Thomas Seel. RNN-based Observability Analysis for Magnelomeeter-Free Sparse Inertial Motion Tracking. In 2022 25th International Conference Information Fusion (FUSION), pp. 1-8, Linköping, Sweden, July 2022. IEEE. ISBN 978-1-73774-972-1. leee. org//occmentt/3841555
doi: D0.23119/FUSION49953 2022 URL htttps://eeexpp::
Simon Bachhuber, Dustin Lehmann, Eva Dorschky, Anne D. Koelewijn, Thomas Seel, and Ive Weygers. Plug-and-Play Sparse Inertial Motion Tracking With Sim-to-Real Transfer. IEFE Sensors Letters, 7(10): ieee
1 4, October 2023. ISSN 2475-1472. doi: 10.1109/LSENS. 1..1009/LSENSS222233307777 2023.3307122. URL
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlass, Skye Wanderman-Minn and Qiao Zhang. JAX: Composable transformations of Python+NumPy programs, 2018. URL http:://gtthbb.
com wtw google/jgaaa http://tttub.cooogggg//////////e
Ao Buke, Fang Gaoli, Wang Yongcai, Song Lei, and Yang Zhiqi. Healthcare algorithms by wearable inertial sensors: survey. China Communications, 12, April 2015. ISSN 1673-5447. doi:
11.1199/CC.2015.7114440 URL https::/ieeeepploree ieee. org/ absstract//ocuuueet/14444
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Damitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning Phrase Representations using RNN Racoder-Decoder for Statistical Machine Translation, September 2014. URL http:://arxx...rgg///// / /406..140...11 1078
Mark Euston, Paul Coote, Robert Mahony, Jonghyuk Kim, and T. Hamel. Complementary Filter for Attitude Estimation of a Fixed-Wiing Wing UAV. In IEEE/RS. International Conference on Intelligent Robots and Systems, October 2008. doi: 10.1109 N..1109/IROS.. TROS 2608.4650766.
Roy Featherstone. Rigid Body Dynamics Algorithms. Springer, New York, 2008. ISBN 978-0-337-74314- 978-0-387-74315--.
18

---

Published in Transactions on Machine Learning Research (10/2024)
Roy Featherstone. A Beginner's Guide to 6-D Vectors (Part 2) [Tutorial]. IEEE Robotics & Automation Magazine, 1788) 99, December 2010. ISSN 1070-9932, 1558-223X. doi: 10.
URL
https://iseexploore leee.
Jakob Foerster, loannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson. Learning to Communicate with Deep Agett-Agent Reinforcement Learning. In Advances in Newral Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL
htwprccedddnnns.uuurr....... cc/ppappr/201//aaaee hash/ ((550tt9922a2cccff24444//111aaa2 Abbsseeerttbbbbaaaaaaattttt Abstract html
Matthew W. Givens and Calvin Coopmans. A Survey of Inertial Sensor Fusion: Applications in sUAS Navigation and Data Collection. In 2019 International Conference on Unmanned Aircraft Systems (ICUAS). pp. 1054 1060, June 2019. doi: RTA.1///ICUS..1....TTT. @ocument/8798225)
URL hattp:://ieeepp:: ieee. org/ orge
Aaron Grapentin, Dustin Lehmann, Ardjola Zhupa, and Thomas Seel. Sparse Magnetometer-FFee Real-Time Incrtial Hand Motion Tracking. In 2020 IEEE International Conference on Multisensor Fusion and
Inlegration for Intelligent Systems (MFI), pp. 94 100, September 2020. doi: 10.1109/MP149255.0000 9235262. URL https:///eeeexploree ieee org.eegeeeeeeeeeeeeeeeeeeeeee ortp/bccmmmet//92222222222222222
Tom Hennigan, Trevor Cai, Tamara Norman, Lena Martens, and Igor Babuschkin. Haiku: Sonnet for JAX, 2020. URL http://github com/deepmind//tmmbbeeee
Wenlong Huang, Tgor Mordatch, and Deepak Pathak. One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control, July 2020. URL http:://axiiv wrg/abs////...o0/////// httt:://arxvv.oog//bb/////..... 04976
Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael Black, Otmar Hilliges, and Gerard Pons- Moll. Deep inertial poser: Learning reconstruct human pose from sparse inertial measurements in real time. ACM Transactions on Graphics, 3766):115:11185:115 December 2018. ISSN 07330-0301. doi: 10.1145/3222127.32777
URL
hctp//dllacmm..ooooomom http:///.......rr////////0./.1.. 21111111111111722221222777111111 3275108
Karol Jędrasiak, Krzysztof Daniec, and Aleksander Nawrat. The low cost. miero inertial measurement unit. In 2013 IEEE 8th Conference Industrial Electronics and Applications (ICIEA), pp. 403 408, June 2013. doi: 10.1109/ICIEA 2013.65664033 URL http::////eeeeppore..eeeeeer/e/p
Manon Kok, Jeroen D. Hol, and Thomas B. Schön. An oplimizaation-baased approach to human body motion capture using inertial sensors. IFAC' Proceedings Volumes, 47(3):79 85, January 2014. ISSN 1474-6670. doi:
pi1/$14744677164415588 7A-10003.22022222 URL htttps:/wwwt wwt.://eeieeetttcc/////eeeeeee/e
J.B. Kuipers. Quaternions and Rotation Sequences: A Primer with Applications to Orbils, Aerospace and Virtual Reality. Princeton University Press, 2002. ISBN 978-0-691-10298-6, URL https://press princeton.edu/booneeoo....o httpa//ww.....aceeee//o/oooooo/n
Daniel Laidig and Thomas Seel. VQF: Highly accurate IMU orientation estimation with bias estima- tion and magnetic disturbance rejection. Information Fusion, 91:187 March 2023. ISSN 1566- 2535. doi: 10.1106/j.inffus.. wwt.:if...22..1....ooo 10.2.1
$156625352200188
Daniel Laidig, Thomas Schauer, and Thomas Seel. Exploiting kinematic constraints to compensate magnetic disturbances when calculating joint angles of approximate hinge joints from orientation estimates of inertial sensors. In 2017 International Conference on Rehabilitation Robotics (ICORR), pp. 971-976, July 2017. doi: 10.1109/1CORRR.0117 8009375. URL https://ieeexplooo wew.e wtt..iie..r.ocoemee/////////
Daniel Laidig, Marco Caruso, Andrea Cercatti, and Thomas Scel. BROAD-A Benchmark for Robust Inertial Orientation Estimation. Data, 6(7):72, July 2021. ISSN 2306-5729. doi: (..3300/dtta677002 URL
https:// :/www http://wwwwwpp..... wwtp.//30--557///77777
19

---

Published in Transactions on Machine Learning Research (10/2024)
Dustin Lehmann, Daniel Laidig, and Thomas Seel. Magnetometer-frre motion tracking of one-dimensional joints by exploiting kinematic constraints. Proceedings on Aulomation in Medical Engincering. 1(1):027- 027, February 2020. view/335
Dustin Lehmann, Daniel Laidig, Simon Bachhuber, Thomas Seel, and Ive Weygers. Magnetomnttrr-Prre Inertial
Motion Tracking of Kinematic Chains, April 2024. URL https://papers: ssmpperrssssss..o.mm sarn. com/abstraatt=47557777
Irvin Hussein López-Nava and Angélica MelozzMeléndez, Wearable Inertial Sensors for Human Motion Analysis: A Review. IEEE Sensors Journal, 16(22) November 2016. ISSN 1558-1748. doi:
N0.1109/JSEN.2016.222 26099999.... URL httpps///eeeeexeporrr leee /tt..orr//occcmee///666661611
Sebastian O H Madgwick. An efficient orientation filter for inertial and inertial/magnetic sensor arrays. 2010.
Robert Mahony, Tarek Hamel, and Yean-Michel Pftimlin. Nonlinear Complementary Filters on the Special Orthogonal Group. TEEE Transactions on Automatic Control, 53(5):1203 1218 Jume 2008. ISSN 1558-2523. doi: 10.1109/TAC. TTC.//A.......2333333 URL
Timothy McGrath, Richard Fineman, and Teia Stirling. An Auto-Calibrating Knee Flexion-Fxxtessonn Axis Estimator Using Principal Component Analysis with Inertial Sensors. Sensors, 18(6):1882, June 2018. ISSN 1424-8220. doi: 10.3390/s18061888 URL https: :
Fredrik Olsson, Manon Kok, Thomas Seel, and Kjartan Halvorsen. Robust Plug-and-Play Joint Axis Estimation Using Inertial Sensors. Sensors, 20(1))33534, January 2020. ISSN 1424-8220. doi: 10.3390/ s20123534. URL
http://wwwmmiiic.co/ com 1awpoo//1444--2200000 http://wwaii..o///////00///////0
Liviu Panait and Sean Luke. Cooperative Multi-Agent Learning: The State of the Art. Autonomous Agents and Multi-Agent Systems, 11(3):387-4344 November 2005. ISSN 1573-7454. doi: 10. 10.1007/110458--055-2833
URL /ttt::://bii.org/00.10000 110.0..000000111100 00000/0000000000000-1------222
Decpak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexci A Efros. Learning to Control Self- Assembling Morphologies: A Study of Generalization via Modularity. In Adoances in Newral Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL httttp://pprceeedngg. hwwpiss...r.pp... cc/ html
Patrik Puchert and Timo Ropinski. A3GC-IP: Attention-oriented adjacency adaptive recurrent graph convolutions for human pose estimation from sparse inertial measurements. Computers & Graphics, 117:96 104, December 2023. ISSN 0097-8493. doi: 10016/j.cag.2023 09.009.. URL
Thomas Seel and Stefan Ruppin. Eliminating the Effect of Magnetic Disturbances on the Inclination Estimates of Inertial Sensors**This work was conducted within the research project BeMobil, which is supported by the German Federal Ministry of Research and Education (FKZ 16SV7069K). IFAC- PapersOnLine, 50(1):8798 8803, July 2017. ISSN 2405-8963. doi: 10.10166...facol..o.o www..oo...........11 URL attps://ww..cieeceedie
com http:///eeeeeeccc//////////e//ee http://wcw..accoooocccc.c.ctaaaa
Thomas Seel, Manon Kok, and Ryan S. McGinnis. Inertial Sensors Applications and Challenges in a Nutsholl. Sensors, 20221):6211, January 2020. ISSN 1424-8220. doi: 10.3300/s20216221. URL
https://www. www.. wtt. mttppi..oo//1222-2222/////11//1
Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. Learning multiagent communication with back- propagation. In Proceedings of the 30th International Conference on Newral Information Processing Systems, NIPS'16, pp. 2252-2260, Red Hook, NY, USA, December 2016. Curran Associates Inc. ISBN 978-1-5108-3881-9.
Luke Sy, Michael Raitor, Michael Del Rosario, Heba Khamis, Lauren Kark, Nigel H. Lovell, and Stephen J. Rodmond. Estimating Lower Limb Kinematics Using Reduced Wearable Sensor Count. IEEE transactions on bio-medical engineering, 68(4):1293-133 1304, April 2021. ISSN 2531. doi: 10.1109/TBME.2000 3026464.
20

---

Published in Transactions on Machine Learning Research (10/2024)
Luke Wicent F. Sy, Nigel H. Lovell, and Stephen J. Redmond. Estimating Lower Limb Kinematics Using a Lie Group Constrained Extended Kalman Filter with Reduced Wearable IMU Count and Distance Measurements. Sensors (Basel, Switzerland), 20(23):6829, November 2020. ISSN 1424-8220. doi: 10.3390/ $20236829.
Agniesıka Saczesna, Przemyslaw Skurowski, Przemyslaw Pruszowski, Damian Peszor, Marcin Paszkuta, and Konrad Wojciechowski. Reference Data Set for Accuracy Evaluation of Orientation Estimation Algorithms for Inertial Motion Capture Systems. volume 9972, pp. 509-520, September 2016. ISBN 978-3-319-46417-6. 45.
doi:
Bertram Taetz, Gabriele Bleser, and Markus Miezal. Towards Self- Self-Calibrating Inertial Body Motion Capture, June 2016. URL http::/arxii..o/// org/ awspaab.....0000 03754
Tom Van Wouwe, Seunghwan Lee, Antoine Falisse, Scott Delp, and C. Karen Liu. Diffusion Inertial Poser: Human Motion Reconstruction from Arbitrary Sparse IMU Configurations, August 2023. URL
http://arxiv. http:/aaxxvvvooogaaaa/ httg/abb/2300...00 16682
Timo von Marcard, Bodo Rosenhahn, Michael J. Black, and Gerard Pons-Moll. Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs, March 2017. URL http://arxiv.ont org/abs/
1703.08014
He Wang, Wenwu Yu, Guanghui Wen, and Guanrong Chen. Fixed-Time Consensus of Nonlinear Multi-Agent Systems With General Directed Topologies. IEEE Transactions on Circuits and Systems II: Express Briefs, 0689)11587-1591, September 2019. ISSN 1558-3791. doi:
URL
https://ieeexplore.eee ieee. wtgg/documen//8574550
Tingwu Wang, Renjie Liao, and Sanja Fidler. NerveNet: Learning Structured Policy with Graph Neural Networks. In ICLR, 2018.
Daniel Weber, Clemens Gühmann, and Thomas Seel. RIANN A Robust Neural Network Outperforms Attitude Estimation Filters. A1, 2(3): 463, September 2021. ISSN 2673-2688. doi: 00.3390/ai2030022 URL httpt:::/s.tt.dottt wwwpmdwwwwppaaeeaamommmmm...t.tp 200/2257722288/2///2222
Fritz Webering, Christian Fahnemann, and Nils Stanislawski. OpenAXES example robot dataset, 2023. URL
https://date: //attaa///aa.../ htta://aatt.uiimmmmoooeeeeeae maii-hannver..deee http://awaaaaaaaaacooo//////////
Ronald J. Williams and Jing Peng. An Efficient Cradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories. Neural Computation, 2(4):490-501, December 1990. ISSN 0899-7667. doi: 10.1162/
199..1990.2.4.499. URL https::/ieeexpploree wewpeexwwwpoeeeeeeeeeeeee.eeeeee org. wwt..orr/roomeeee///75/5155
Xinyu Yi, Yuxiao Zhou, and Feng Xu. TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors, May 2021. URL httt:://arxi..org/aaaa wtgp/bs////....ooo///// 04655
Xinyu Yi, Yuxiao Zhou, Mare Habermann, Soshi Shimada, Vladislav Golyanik, Christian Theobalt, and Feng Xu. Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors, March 2022. URL httt:://arxxii org/ 2203. 08528
Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. Large Batch Optimization for Deep Learning: Training BERT in 76 minutes, Jamuary 2020. URL http://arxiv org/ org/abs/1904.0095
Zhaolong Zheng, Hao Ma, Weichao Yan, Haoyang Liu, and Zaiyue Yang. Training Data Selection and Optimal Sensor Placement for Deep-Learning- Based Sparse Inertial Sensor Human Posture Reconstruction. Entropy, 55888 May 2021. ISSN 1099-4300. doi: 0009944400/235////
URL https::///// http::://www.mm.....o com/
21

---

Published in Transactions on Machine Learning Research (10/2024)
A Preliminaries
A.1 Notation
Scalars lowercase uppercase, italic, non-bold, e.g,, R, or, e.g., N
(Column) vectors are lowercase, italic, bold, e.g,, æ E RR, and, e.g. 21,
Matrices (or higher) are uppercase, upright, bold, e.g., X and X E p3x4,
Individual quaternions are denoted with q E HI. One-dimensional arrays of quaternions with q, e.g,, E HS Two dimensional arrays of quaternions with q, e.g., q E H5x5
(Programming) functions and structures are written in typewriter typestyle, c.g., sys.
The equal symbol overloaded and used for definitions, assignments, and comparison, and the context defines the current meaning.
The symbol 0 defines an arbitrarily large array of zeros that is automatically broadcasted to the required dimensionality.
The symbol 1 defines either the unity element of given space, or the indicator function, such that, e.g., (i) is one for i and zero else.
The symbol & is used to denole the direct product of vector spaces, and additionally denotes quaternion multiplication, see Definition B.2
A.2 Array Indexing and Slicing
Vectors and matrices are array-like objects that can be indexed and sliced dynamically. Indexing starts with and slicing inclusive both sides. For example, let X R3x4, then X[1]E TRA,, or X(1:2) R2x4
Additionally, define the following auto-completion rules by example, such that X|:2| is equivalent to X[1:2], and X[2:] is equivalent to X(2:3), and X|:] is equivalent to X(1:3). Finally, multiple dimensions can be indexed or sliced simultaneously and are separated by a comma, e.g., 3(:33:J R3x2
B Quaternion Algebra
Definition B.1. We use HI to denote the space of all unit quaternions, and we denote unit quaternion with q Yu + i + 4y + yzk. Definition B.2. Let q1, 92 H be two unit quaternions, then we denote quaternion multiplication of the two unit quaternions, that is
q1 q2 (fwwlww Nzllex Plyl2 (1=(2=) + (2iw42s + Nxl22w + Ny922 q1=92y)i
(2n Q1a422 Ny q1=42x0 (qww)2 + Q1a 11422 q1=(2w)) )
Note that the space of unit quaternions in combination with quaternion multiplication @ forms a closed group, i.e. (q) q2) Aqi. q2-
Definition B.3. The inverse of a quaternion q is given by the complex conjugate denoted by q*
Definition B.4. The quaternion that corresponds to a certain rotation around an axis j = (ju. Ju. J e R by an angle a E R is given by = quat(jjaaap a) = cos( +) + (iz sin())) ( + (j,, sin( + (jz sin ())
Definition B.5. Extracting the angle from a given quaternion q (the inverse operation of B.4 can be / +G2
done using an)le(q) = 2 aretan
gm
22

---

Published in Transactions on Machine Learning Research (10/2024)
Definition B.6. We define the projection project of quaternion q onto (primary) axis the decomposition into two quaternions, the primary rotation 4p and the residual rotation Qr, such that = q, & qp while the angle of the residual rotation is minimized. This can be done with
1. ap arctan
jogs tJu +J
Fw
2. 9p quat(jiap)
3. 9r q 9
Definition B.7. We define the function zeroHead as the function that maps a quaternion q the quaternion qr with zero heading component and returns qr. It can be obtained via
qp: qr + project (q, (0,
where project is given in Definition B.6
Definition B.8. The function randQuat that returns a random quaternion, uniform the sphere, be obtained by drawing i.i.d. four numbers from normal distribution, interpreting them as the components Qwi lx ty 9= of a quaternion, and then normalizing the quaternion to obtain unit quaternion.
Definition B.9. The function rotate(q,") r) applics quaternion q vector r E R3 If the quaternion is interpreted as Iq (from to 1) and the vector expressed using the unit-vectors of coordinate system 0, then the function rotate returns the same vector but using the unit-vectors of coordinate system 1. Let (Ta, Ty: T,, then the rotate(q,r) function is given by (q & (0, T2) @ qq") [1:].
C Training Data: The RCMG Algorithm
Algorithm randSys (RCMG First Step)
1: Input: Av
2: Output: sys
3: sys + initSys(AN) fallocate empty structure)
sys f- randBase(sys, Ax) (see Definition
for 1 N do
sys.J [i] rotate(randQuat(). εx) {random hinge joint axis direction; unused if sys.A[i] =0} for d = 1 to 3 do sys. R.s. d] randSSegmentTooegeennt()() (see Definition C.2
8:
9: Ryi N, d] randSensooTToSeement((( {see Definition C.2
10: end for
11: (IMU of node the standard system is always rigidly attached, as there no second IMU whose
measurements may be fused effectively climinate motion due the nonrigid attachment.) if sys.n|1] = or randBermoulli then
12:
13:
k: + getRigidStif() (see Definition C.4
14:
7 getRigidDamp( (see Definition C.4
15:
else
16:
k: < randloonRig:dSttif(( {see Definition
C.5
17:
7 + randNonRigidDamp(( {see Definition
C.5
18:
end
19: K|i
20: Sys.T k (element-wise multiplication)
21: end for
23

---

Published in Transactions on Machine Learning Research (10/2024)
Algorithm randMotion (RCMG Second Step)
1: Input: sys, (motionConfig) {motionConfig object is used only by randFreeTraj and randHingTraj and it influences and constraints the random motion (see Appendix C.1
123)
Output: T
+- int sys.T,
(# timesteps; duration of training sequences seconds)
4: Q = 0 (timeseries of minimal coordinates Q E N of system without IMU bodies; see Definition 3.2)
a
for 1 to N do
if sys.Ali 0 then
b+ 7 Q/a:b] randFreeTraj (motionConfig. T) {see Definition C.6
10: else
11:
a b b
12:
[[a:b] = randHingTraj (motionConfig, 7) {see Definition C.6
13: end if
14: m--ll b
15: end for
16: Q 0 (Limeseries of minimal coordinates Q E (NN,(N,+7N) of system with IMU bodies; see Definition 3.2
17: q +- 0
18: q + 0 ( E NN+6N, see Definition 3.2
19: for
1 do
20:
+ PDControl (sys, Q f ([:N,]) {see Definition C.7 + concat (7, E 16N T (N IMUs' passive free joints)
21:
22: q. q t forDyn (sys, q. q. +) (see Definition C.8
23: Q
24: end for
25: + forkin (sys, Q) {see Definition C.8
24

---

Published in Transactions on Machine Learning Research (10/2024)
Algorithm getXY (RCMG Third Step)
1: Input: sys, Av. E H R 2NxT
Output: X Nx9x7 Ye NXT
X 4
Y+0
5: for i = 1 to N do
6: + Avli]
7: i + sys.n|i]
8: p p+0
if p then
10:
- D+8sysmmj sys.n/pl
11:
end if
12:
O isOuter(i,sys..) (true if body i is outer body; see Definition 3.1
13:
if O; or randBernoulli {inner IMU data might not be made available) then
14:
j- i N {body number IMU node]
15:
X[i,:6] simIMU (Tb)) sys.T.) {see Definition C.9
16:
end if
17:
if p > and randBernoulli(0.5) ffor hinge joints, joint axis might not be made available) then
18:
X 7:] sys.J
19:
end if
20:
0q
+
21:
if # then
22:
9q <- [-T
23:
end
qq Qq-Tia Tū,
24:
.9 --"009 99" (note that the expression 29" @ 9q can not be used instead of 99 @ 999", it
25:
dramatically reduce the network's 's ability to learn. The reason is that in the expression b 99" the joint axis direction is expressed in the (more meaningful) local coordinate system and not in the base's coordinate system.}
26:
if p p= 0 then
27:
1 < zeroHead (jq) (see Definition B.7
28: end if
29: Y|i ;
30: end for
25

---

Published in Transactions on Machine Learning Research (10/2024)
MoticaConig #1
ModionConlig #2
RCMG
Simulates Random Motion
Hinge Joint Angle a(() [sad]
Randon 6DuF
Cotion #3
MoticnConãn 84
Random Hinge
Random Hinge 2
Time [s] 10 15
Time [s]
10
15
Figure 8: RCMG generates random motion by drawing random trajectories of the minimal coordinates of the system. Exemplary random trajectories of the hinge joint's minimal coordinates are shown for the four different motionConfigs (see Appendix C.Ip. They are drawn using the function randHingeTraj, see Definition C.6 RCMG also draws the random trajectory of the minimal coordinates of the free joint, but they are not. shown here for simplicity.
C.1 The motionConfig Object
In the second step of the RCMG (see Section 4.2.2 random motion of the KC is simulated and, subsequently, training data is generated. Random motion of the KC is obtained by using PD control during dynamical forward simulation such that the minimal coordinates of the KC track a randomly drawn set of reference trajectorics for the minimal coordinates. The functions randFreeTraj and randHingTraj (sce Definition C.6 are used to draw the minimal coordinates reference trajectories in Algorithm 3 Furthermore, the lype of motion generated by these functions can be manipulated with a motionConfig object which defines various parameters, o.g., upper limits on angular velocitios or lower limits on the amount of motion. For an exhaustive list of parameters, the reader is referred the software implementation (see Appendix 4.5)
In this work, use in total four different motionConfigs to generate training data. For each generated sequence, randomly and uniformly draw from these four. The four motionConf are used to ensure that wide range of different motion patterns are covered in the training data. Exemplary trajectories of the hinge joints' minimal coordinates are shown in Figure
C.2 Support Functions used Algorithms 2 3/4
In the following, the functions used in Algorithms 2 23 will be discussed. And while no pscudo-code is provided for these support functions, it should be noted that the majority of these functions should be understandable despite textual description only. Additionally, the reader may always refer to the software implementation for additional details (sce Appendix 4.5
The function randBase(sys, AN) randomly re-attaches the base of the system sys and afterwards the nodes in the graph are re-numbered according to Section 3.3 This process yields a new parent array A. Additionally, the permutation the new numbers of the nodes expressed in the numbering scheme that was used to obtain Av is captured in the numbering array n.
Definition C.I.
Consider the three-segment KC given in Figure The parent array is given by X (0, 1) and we assume, without loss of generality, the numbering scheme that is shown in the figure. Here, the function randBase has three choices for attaching the base. The first is trivial and given by node 1. The second is given by node 2. In this case the parent array is always given by X (0, 1)T, and two scenarios for the numbering array are possible. They are n 1,3) and n = (3, 1,,)2 2)T. The third and last option is given by node 3. In this case the parent array is unchanged but n = (3, 2, 1)T.
A sccond example is given in Figure
26

---

Published in Transactions on Machine Learning Research (10/2024)
Definition C.2. The functions randSoggentToSSggnent and randSensorToSegment randomize the body-to- body- and sensor-to-body positions expressed in the local coordinate, respectively. Internally, they both draw these vectors randomly from uniform values ranges. These value ranges are chosen such that there exists a dominant longitudinal direction (x-component), and two equal transversal directions.
Definition C.3. The function randBernoulli(y) returns 1 (or true) with probability p and zero (or false) else.
Definition C.4. The functions getRigidStif and getRigidDamp return the stiffness and damping parameters (both are six-dimensional) of the free joint that connects body i to IMU body i+ N. This mechanism is used to simulate nonrigid IMU attachment. However, these functions provide a fixed set of values that leads to highly stiff and critically damped spring-damper system such that there is effectively no motion between body and IMU body i N.
Definition C.5. The functions randRigidStif and randRigidDamp return the stiffness and damping parameters of the free joint that connects body i to LMU body i N. Internally, both functions draw these six-dimensional voctor randomly from log-uniform values ranges.
Definition C.6. The function (anarreeeTrajoommioooooooon (for free joint minimal=maximal) coordinates E (H @ IR3)" with T timesteps for a free 6-DoF joint. The function fandHHingerraaffotiooooooo
7') returns a random trajectory of minimal
T) returns a random trajectory of minimal coordinates E RT" with T timesteps for a hinge joint. Internally, they both use the motionConfig (see Appendix C.1 constraint the random motion to physically relevant motion, i.c., the, e-g., angular velocity is bounded from above. Exemplary trajectories are shown in Figure
g For additional details, the reader is referred to the software
implementation, see Appendix 4.5
Definition C.7. The function PDControl (sys, q. 9) computes the generalized force vector R.N using decentralized scheme using N independent PD controllers, and where qr E IR, denotes the reference minimal coordinates and E R. denotes the observed minimal coordinates. Note that the provided system object sys has 2N bodies but only the first N bodies are actuated. The remaining bodies are passive free joints.
Definition C.8. The function forDyn (sys, q. q. T) applies forward dynamics in the system sys and integrates the minimal coordinates position and velocity vector I by sys.Tg. The function forKin (sys, Q) applics forward kinematics in the system sys, i.c., it provides a map from minimal Q to maximal coordinates T. Here, it is additionally vectorized over the time dimension. text-book reference for both well-known algorithms can be found in Featherstone (2008).
Definition C.9. The function SimIMU T.) simulates a 6D IMU from a trajectory of maximal coordinates. First, the maximal coordinates are butter-worth low-pass-flleredd (both the quaternion and position trajectory). Then, a second-order numerical differentiation for both gyroscope and accelerometer is used. The accelerometer is low-pass-filteet filterdd Gravity and simulated noise and bias terms are addod. The cutoff frequencies have been optimized such that experimental IMU data is recovered with the highest fidelity from the maximal coordinate trajectories from OMC. Additional details can be found in Bachhuber et al. (2022).
D RING's Time Complexity and Computational Requirements at Inference
For practical applications, the inference time and computational requirements of RING are critical to enable woal-world online applications. Therefore, we conduct theoretical and empirical time complexity analysis of RING at inference.
The theoretical time complexity of RING depends on the operations involved when advancing the prediction by one timestep. First, we assume a naive matrix multiplication complexity, i.e, let A E CxD and B E DxE then AB OCC x D x E). Now, recall that N the number of bodies, F the number of features per body (here F 9), M is the message dimension, and H the hidden state dimension, then
eq. is X
(fo).
eq- 5 is O(N M) (summation operation, tree with N nodes has at most N edges),
27

---

Published in Transactions on Machine Learning Research (10/2024)
Table 3: Non-exhaustive list of IMU hardware and their typical (typ.) noise and bias properties provided in the manufacturers' technical specifications. Unfortunately, not all manufacturers provide this information.
Hardware
Gyr. N.D. /////v //z/Hzz
Gyr. Of P
Acc. N.D. lug/ hog/vHz)
Acc. Of.
[mg]
Bosch BMI160 (1)
0.007
#3
180
+00
A.D. ADXL355 (2)
Acc. only
Acc. only
22.5
22
Xsens MTi 10 (3)
0.03
+0.2
60
+5
Xsens MTi 100 (3)
0.01
2022
15
Xsens MTw (ours) (4)
0.01
N.P.
200
N.P.
Movella Dot (5)
0.007
N.P.
120
N.P.
K. KXTC9-2050 (6)
Acc. only
Acc. only
125
N.P.
max (worst-case)
0.03
#3
200
+40
Noise Density (N.D.), Offset (Of.), Not Provided (N.P.)
Accelerometer units are micro-gravity per square-root Hertz and milli-gravity
Sources in supplementary materials: bosch bmi160.pdf. (1), analog devices adx1355.pd (2),
xsens xsens_mtinn xsens_mti.pdf (3), xsens_mmtx (4), movella_dott (5), kionix kxtc9 2050.pdf (6)
IMTP defined in Section 5.2.1
IMTP defined in Section 3.2.2
IMTP defined in Section 5.2
IMTP defined in Soction 5.2.4
30
30
(
(
(9-(7) RING
[5
AMAB (dog.
10-
(20 (8)
RMAB [dog]
RMAB ideg
(0
(8)
INNG
HUNC
20
20
RMAE
3
8
21
(19)
1LTTOCK
INPTP defined In Scction 5.
IMTP defined Section 5.3.1B
IMTP definod In Soction 5.3.2
IVTP defined In Soction 5.3.3
30
12.5
[3)
(2)(00)-(0)
HUNG
AMAB [dog]
10.0
RMAB [dlgg]
20
+53+
(3)())())
RING
RMAE (deg)
RMAE [ddg]]
7.5
10
5.0
2.5
15)
RLNG
20
40
60
80
100
120
20
10
00
80
100
1700
70
40
60
80
100
120
200
40
60
80
100
120
Noise and Bias [X6]
Noise and Bins [%6
Noise and Bins [%6
Noise and Bias [%6]
Figure 9: Comparison of the robustness of RING and SOTA methods as the level of noise and bias is increased.
To simulate reasonable noise density and bias offsct ranges, we constructed a worst-case IMU by combining the worst properties of various IMU manufacturers, and bias values are incrementally (in seven equidistant steps) increased from 0 to 120%, and a corresponding amount of simulated noise and bias is added to our real-world IMU data (sce Section 5.1]. which yiclds seven modified datasets. RING and all SOTA methods are validated on the modified dataset, and this procedure is repeated using ten different seeds. The AMAFs and RMAFs of all methods are plotted for all IMTPs as a function of the seven steps. The 55% 55%% /75%- across all trials and seeds are shown, and they show that RING maintains accuracy comparable SOTA methods all IMTPs and, espocially in two-segment KC tracking, substantially outperforms them. Methods are Weber el. al. (2021 (1), Madgwick
summarized in Table 3 Then, those worst-case noise
(2010) Mahony et al. 20088 (3), SSel & Ruppin (2017 (4), Laidig & Seel 2002 Laidig et al. (207))())
Lehmnan et al. 20200
Olsson et al. (020(())
Bachhuber et al. (023 (9).
28

---

Published in Transactions on Machine Learning Research (10/2024)
Table 4: Performance comparison across different hardware configurations.
Computation Time [us]
Latency [1s]
Hardware
A1
A2
A
Aa
A
A2
A
A
(1)
131 8
737 + 143
881 128
916 + 228
158 =
789 + 137
912 84
975 + 172
(2)
+ 20
192
206 =
=
267 -
(3)
78 5
127 8
129 +
127
152 = 14
199 + 20
197 = 11
201 =
(4)
376 4 90
681 # 148
647 + 96
879 + 361
603 = 167
834 = 174
947 211
1106 = 377
(5)
214 55
327 + 69
325 + 51
338 60
494 = 53
692 = 133
684 426
704 153
Apple M2 Pro (1), W-1390F (2), W-1390P RTX A5000 (3), Intel Xcon @ 2.0 Ghz, 1 Core, Threads (Google Colab) (4),
Intel Xeon 2.0 Gliz, Core, Threads
Tesla T4 GPU (Google Colab) (5)
eq. 6 is O(N H (2M +)+ N H²) (first GRU coll of go), O(N H) (Layernorm), O(N
+ N H²) (second GRU cell of go),
eq. 7 is O(N x H) (Layernom)), O(N H + H Nx4x (ha),
eq. is O(N) (normalization).
This leads to overall complexity of O x H + F)). Note that the leading N term is imple- mented in way such that it corresponds to an efficient batch operation and not for loop. This is crucial for performance (especially GPUs).
We conduct the empirical analysis for various types hardware and report computation time and latency required for advancing the prediction by one timestep. Latency includes overheads such as conversion of the NumPy array the www.baarinng-faawwwr--ss transfer overheads. Effectively, latency measures the time required from NumPy array input to NumPy array prediction, i.e.,
array type and potential to-and-from-device
X, np .ndaray device jar.Array
X.
jar.Array to hest Y,
ring
np.ndarray.
Computation Time
Latency
Consequently, latency needs to be lower than the time delta due to the IMU sampling rate to enable lag-free frrefree real-time application (excluding delay of one frame). Table 4 reports the timings for various hardware and the different IMTPs that include cither one- two-, three-, or four-segment KCs.
E
Software Example
This example code uses the published software (sce Section 4.5 and showcases how RING is applied in Section 5.3.2. i.e., it solves an IMTP that consists of a three-segment KC with sparse 6D IMU atlachment. and with unknown joint axes directions.
import ring
import numpy as mp
T
int
30
# sequence Length
[s]
Ts
float
0.01
# sampling interval [s]
B
int =
# batch size
lam:
1ist [int] [-1, o, 1] parent array: because Python's conventsons body counting starts at 0,
conseguence the base body is sndicated by -1 and not 0
N int
=1amm # number of bodses int(T/Ts) # number of timesteps
T_i: int
np..zeros(BB, T_i, N, 9))
12
structured as follows:
29

---

Published in Transactions on Machine Learning Research (10/2024)
Xf.... :3]
xf..... 3:6] 99r
15 .f. 6:9] josntazis
17 # let's aasume we have an on each outer segment of the
18 # three-segment ksnematic chass
19 x[..., 0, :3] acc_segmenti
20 x[..., 2, :3] acc_segments
21 X[..., 0, 3:6] gyr_segmenti
22 X[..., 2, 3:6] gyr_segment3
23
24 ringnet = ring. RIN, Ts)
25 yhat, = ringnet (
# yhat: uni't guaternions, shape (B, 7,3, H1, 4)
30


Reviews:
[{"review_id": "en3cSGVXkU", "reviewer": "Reviewer_TEny", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1: The decentralized architecture of RING is a significant advancement in the field of IMT. By allowing for parameter sharing and message passing among networks, RING can adapt to various configurations and problems, showcasing its pluripotency.\n\n2: One of the most impressive aspects of RING is its ability to generalize from simulated training data to real-world experimental data without additional training. This capability is crucial for practical applications, as it reduces the need for extensive data collection and model retraining.\n\n3: The versatility of RING opens up new possibilities for IMT applications across various fields, including biomechanics and autonomous systems. This could lead to wider adoption of motion tracking technologies in previously untapped areas.\n\nWeaknesses:\n\n1: While the paper emphasizes the zero-shot generalization capability, the experimental validation appears to be limited to specific scenarios. More extensive testing across diverse real-world conditions would strengthen the claims made regarding RING's robustness and adaptability.\n\n2: Although RING is designed to be a plug-and-play solution, the underlying architecture may still pose challenges for users without a strong background in machine learning or neural networks. Providing more detailed guidelines or user-friendly tools for implementation could enhance accessibility. It is thus suggested to provide time complexity analysis, both theoretically and empirically.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a novel method for Inertial Motion Tracking (IMT) called the Recurrent Inertial Graph-Based Estimator (RING). This method aims to revolutionize the field of IMT by providing a versatile, problem-unspecific solution that does not require expert knowledge for implementation. RING utilizes a decentralized network of message-passing, parameter-sharing recurrent neural networks to process local inertial measurement unit (IMU) data and generate local orientations. The authors claim that RING can effectively tackle a wide range of IMT problems, including those that are magnetometer-free and involve sparse sensing configurations.\n\nBasically, the RING method represents a solid advancement in Inertial Motion Tracking technology, offering a versatile and user-friendly solution that can adapt to a wide range of problems. Its ability to generalize from simulated to experimental data is particularly noteworthy and positions RING as a promising tool for future research and applications. However, further experimental validation would enhance the paper's contributions and facilitate broader adoption. Overall, this work is a solid work.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It is thus suggested to provide time complexity analysis, both theoretically and empirically.", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ly2HfKQZAU", "reviewer": "Reviewer_zZGD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ The use of a decentralized network of message-passing RNNs is novel and effectively handles the complexities of various IMT problems. The architecture's ability to generalize across different kinematic chains and sensor setups without retraining is impressive.\n\n+ RING's ability to zero-shot generalize from simulation to experimental data and outperform specialized solutions in various scenarios is a significant contribution.\n\n- Although RING shows remarkable generalization from simulated to real-world data, the reliance on simulated training data may limit its performance in highly specialized or unforeseen real-world scenarios. \n\n- The paper focuses on accuracy but does not provide detailed insights into the real-time performance and computational efficiency of RING, which are critical for practical applications.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces the Recurrent Inertial Graph-Based Estimator (RING), a ML-based method for inertial motion tracking (IMT). RING offers a pluripotent, problem-unspecific solution that simplifies the use of IMT technology by eliminating the need for expert knowledge in selecting and parameterizing appropriate methods. The architecture utilizes a decentralized network of message-passing, parameter-sharing RNN that map local IMU data and nearest-neighbor messages to local orientations. This design enables RING to generalize across a broad range of IMT problems and outperforms several state-of-the-art solutions, even under challenging conditions such as magnetometer-free and sparse sensing with unknown sensor-to-segment parameters.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- My main concern is the robustness of RING in zero-shot learning when dealing with raw IMU data from different sensor configurations. The performance of a model that uses raw IMU data as input can indeed be sensitive to variations in sensor configurations, including differences in calibration, placement, and orientation between devices from different manufacturers. From my experience, the IMU data captured from apple and meta's platforms have large difference. These differences can impact the raw sensor readings, potentially affecting the model's ability to generalize effectively in zero-shot learning scenarios.  The authors either needs to prove their model is robust to sensor configurations and why. Or verify that strategies like domain randomization, data transformation, and fine-tuning may be necessary. [critical to secure my recommendation for acceptance]\n\n- The paper focuses heavily on the accuracy and generalization capabilities of RING but provides limited information on its real-time performance, such as latency, computational efficiency, and resource utilization. These factors are crucial in many practical applications, particularly those requiring real-time feedback or low-latency responses. The authors should evaluate the system for real-time performance.  [critical to secure my recommendation for acceptance]", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7RNrVard6H", "reviewer": "Reviewer_n9Fd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- The introduction of a \"pluripotent\" model for IMT is an appealing idea and definitely of interest to some of the TMLR audience.\n- Experimental results are quite interesting, showing zero-shot generalisation of the general model to concrete real life examples.\n- The figures are mostly clear and help to understand the model, which has several moving components.\n\n**Weaknesses:**\n- The writing can be hard to understand at certain points. In particular, it requires quite a bit of familiarity within the field of motion tracking, which the general TMLR audience will not have. The paper is most certainly geared towards researchers in robotics than a general ML audience. See also __Requested changes__ below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a problem-agnostic ML model for inertial motion tracking (IMT). This is based on representing systems in IMT by a graph and using message passing within an RNN cell to extract hidden representations and making predictions. The parameter sharing within the message passing makes the model independent of the system that it can predict. The network is trained on synthetic data  generated randomly by the Random Chain Motion Generator (RCMG) procedure, yet can achieve zero-shot generalisation to several real-world settings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- The abbreviation \"IMU\" is not introduced\n- \"see 2\" (page 4) => \"see Figure 2\"\n- Throughout the paper I find the use of the **Definition** heading a little bit strange. I don't understand how the statements in Definitions 3.2 and 4.1 are definitions. They seem more like remarks.\n- Compared to the other figures, I find Figure 2 to be less informative. In particular, the `sys` representation next to the figure doesn't really tell us much. Additionally, the matrix size of __K__ and __$\\Gamma$__ is cut off.\n- I don't understand the relative-to-parent position array in __Definition 4.2__. In particular, the last sentence. What are the first $N$ values due to the geometry of the KC and what are \"sensor-to-body\" positions and why do we need it? This can be a little confusing is one is not already familiar with the field.\n- Could you please give some more explanation of what the AMAE and RMAE (equations (8) and (9)) tells us? For example, why do we treat  the first component $i=1$ separately from the others and build separate metrics for $i=1$ and $i=2, ..., N$? What is the implication of having a low or high AMAE/RMAE score?\n- \"This pluripotent behavior origins from a\" => \"This pluripotent behavior *originates* from a\" (page 16).", "broader_impact_concerns": "As far as I can see, there are not ethical concerns of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The AE and reviewers support acceptance of this manuscript. The proposed methodology is well-suited for the task and has demonstrated useful generalization properties. Including the additional experiments provided during the author response in a minor revision will further improve the manuscript.
Decision: Accept (poster)