# Multi-step Forecasting Contact Wrenches and Motion Synergies for Proactive Control in Human Robot Collaboration

Dimitris Papanagiotou, Sotiris Manitsaris, Alina Glushkova

► To cite this version:

Dimitris Papanagiotou, Sotiris Manitsaris, Alina Glushkova. Multi-step Forecasting Contact Wrenches and Motion Synergies for Proactive Control in Human Robot Collaboration. Workshop: Human-in-the-Loop Robot Learning in the Era of Foundation Models: Challenges and Opportunities, held at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025), Oct 2025, Hangzhou, China. hal-05507702

HAL Id: hal-05507702

https://hal.science/hal-05507702v1

Submitted on 12 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Multi-step Forecasting Contact Wrenches and Motion Synergies for Proactive Control in Human Robot Collaboration

Dimitris Papanagiotou<fnref n="1" />, Sotiris Manitsaris<fnref n="1" />, Alina Glushkova<fnref n="1" />

## I. INTRODUCTION

Human-Robot Collaboration (HRC) has gained a significant growth in industrial applications where collaborative robots are expected to work safely and efficiently in shared workspaces. Currently, most control strategies rely primarily on feedback from force/torque sensors or vision systems. This dependence on instant signals makes the robot reactions sensitive to sensor noise and processing delays, thereby reducing adaptability, robustness and the ability of these methods to generalize for similar tasks.

Recent systematic reviews [1,2,3,4,5] have examined the evolution of control strategies for HRC systems, covering low-level impedance-based control strategies to high-level learning-based methods. Although these approaches improve compliance and safety, their response is inherently reactive, as interaction is detected only after it occurs. To date, only a few studies have investigated predictive models capable of forecasting human behavior [6,7,8,9,10], which would enable the transition toward proactive control.

To address these limitations, we investigate the use of human behavior forecasting in HRC frameworks. Anticipating contact forces and torques from human motion can enable proactive control and smoother interaction. Our Gesture Operational Model (GOM) [11] combines biomechanical and stochastic principles within a state-space formulation that considers the human body as a dynamic system whose parameters evolve across time and space. Building on this idea, we propose a two-phase framework: in the pre-contact phase, human motion synergies are exploited to forecast the operator's complementary arm motion and retarget it onto the robot's configuration space, while in the contact phase, forecast wrench signals are integrated into a force control scheme. In this work, we extended our previous GOM by considering kinematic and kinetic descriptors of both human and robot as variables of the same dynamic system. Therefore, contact forces and torques are now integrated into it and can be forecast.

To achieve this, we employ a SARIMAX model (Seasonal Auto-Regressive Integrated Moving Average with exogenous variables), a stochastic time-series forecasting method that captures temporal dependencies while incorporating external signals. Each wrench component is modeled as a univariate process, with its past values as endogenous inputs and human joint angles plus other wrench components as exogenous variables. This lightweight approach is trained

once on synchronized motion-force data and deployed for rolling multi-step forecasting. Preliminary results on a polishing task show accurate short-horizon forecasts (up to 20 s) with negligible latency, making the method suitable for real-time integration and future adaptive control strategies.

The remainder of this paper presents the proposed methodology, experimental validation of the force forecasting component, and a discussion of future work integrating motion forecasting and control. Although our method does not directly employ foundation models, it can serve as a complementary component to them. In particular, Section IV introduces a two-phase proactive HRC framework that combines motion synergy modeling and proactive force control, outlining the next step beyond the forecasting results presented here.

## II. METHODOLOGY

### A. Stochastic modeling of human motion

The present approach treats physical HRC as a dynamic system with continuously evolving parameters across time and space. This extension of GOM focuses on the evolution of contact wrench signals depending on their own past values and selected motion variables. Unlike the original GOM, we exclude inter-joint and inter-limb couplings to keep the model lightweight. In this context, joint-angle measurements are used to forecast the evolution of physical interaction. This simplification retains the temporal reasoning of GOM while adapting it to contact wrench forecasting, enabling integration with stochastic time-series forecasting techniques.

### B. Dynamic forecasting of contact wrenches

To forecast contact wrenches during physical HRC, we employ a SARIMAX model (1). The forecasting of each wrench component is based on: (i) its own past values, (ii) human motion variables, such as joint angles ($P_i(t-1) = [P_{x,i}, P_{y,i}, P_{z,i}]$) from $N$ selected limbs, and (iii) previous values of contact forces ($F_j$) and torques ($T_j$), treated as additional exogenous inputs.

$$ F_x(t) = \underbrace{a_1 F_x(t-1) - a_2 F_x(t-2)}_{\text{Endogenous}} + \underbrace{\sum_{i=1}^{N} \beta_i^\top P_i(t-1)}_{\text{Exogenous 1}} + \underbrace{\sum_{j \in \{y,z\}} \gamma_j F_j(t-1) + \sum_{j \in \{x,y,z\}} \delta_j T_j(t-1)}_{\text{Exogenous 2}} \quad (1) $$

<footnote n="1">¹Centre for Robotics, MINES Paris - PSL Université, 60 Bd Saint-Michel, Paris, France</footnote>
This structure enables the model to account for both human-driven motion intent and ongoing physical interaction dynamics. The model is trained once on a fixed-length dataset of synchronized motion and force signals. Through this training force/torque representation is parametrized by coefficients $a_1, a_2, \beta_i, \gamma_j, \delta_j$. During inference, forecasts are generated recursively using only past forecasts and exogenous inputs, mimicking real-time deployment where ground-truth is not accessible.

### III. EXPERIMENTAL VALIDATION

We validate our approach on a polishing task involving small objects, where precise joint-angle estimation and accurate contact force control are critical. An expert performed 5 hours of polishing operations, creating a dataset for one-shot training. Joint angles were extracted from inertial sensors (IMUs), while wrenches were measured with an F/T sensor. Both modalities were sampled at 30 Hz, resulting in synchronized motion-force time series. Fig. 1 illustrates indicative rolling multi-step forecasts for force $F_x$ and torque $T_z$ over a horizon of 600 frames (≈ 20s at 30 Hz). The model is initialized using only the first two ground-truth values of each time series (both the endogenous force/torque values and the exogenous inputs, including human joint angles), after which it generates predictions autonomously. Despite the limited

<figure><img src="image_2.png" /><figcaption>Fig. 1: Forecasting of $F_x$ and $T_z$ over a horizon of 600 frames (≈ 20 s).</figcaption></figure>

input data, the model manages to generate satisfying forecast curves for both $F_x$ and $T_z$ with Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) less than 0.05. The state-space human modeling approach has proven effective for short-horizon forecasting of contact wrenches.

### IV. PROPOSED FRAMEWORK FOR PROACTIVE CONTROL

While the results so far focus on force forecasting, our future goal is to create a two-phase control framework that is illustrated in Fig. 2 and can be generalized to various collaborative tasks. This framework is based on the extension of GOM to integrate both kinematic (through inter-limb synergies) and kinetic (through force forecast) parameters. It is divided into two phases: (i) pre-contact and (ii) direct or indirect physical contact. This approach is designed to integrate humans into the control loop of the collaborative arm enabling it to anticipate human actions.

<figure><img src="image_3.png" /><figcaption>Fig. 2: Proposed proactive HRC framework: Phase I (motion synergy and retargeting) and Phase II (contact wrench forecasting) are integrated into a unified human-robot control method.</figcaption></figure>

#### A. Phase I: Motion synergy and retargeting

In the pre-contact phase, the robot motion is guided based on real-time human motion data from 5 IMU-tracked joints (providing 15 Euler angles per arm). GOM is trained offline to estimate the coefficients of the autoregressive model that encodes the inter-limb synergy. This procedure enables the creation of a compact model that maps the joint angle evolution of one arm to the complementary motion of the other, with the objective to make human and robotic hands interchangeable. During online execution, only the motion of one arm (e.g. the right arm) is captured via IMUs, and the corresponding complementary motion (e.g. the left arm) is predicted through the synergy coefficients of the GOM. The predicted 15 joint angles are then retargeted to the configuration space of a 6-axis collaborative robot. Two alternative methods are envisioned:

* The most common strategy is to derive the robot configuration from the estimated wrist pose using inverse kinematics. Specifically, through the human forward kinematics function (2) we obtain the end-effector (wrist) pose.

$$x_{\text{wrist}}(t) = [p(t), R(t)] = FK_{\text{wrist}}(P_{\text{wrist}}(t)) \quad (2)$$

where $p(t) \in \mathbb{R}^3$ is the Cartesian position of the hand and $R(t) \in SO(3)$ its orientation matrix. The estimated hand pose is then expressed in the robot task frame by applying a calibrated rigid transformation $T_{h \to r}$.

$$x_{\text{robot}}(t) = T_{h \to r} \cdot x_{\text{wrist}}(t) \quad (3)$$

The desired robot configuration is obtained by solving the following inverse kinematics (IK) problem.

$$\hat{q}(t) = \arg \min_{q \in \mathbb{R}^6} \| FK_{\text{robot}}(q) - x_{\text{robot}}(t) \|_W \quad (4)$$

where $FK_{\text{robot}}(q)$ denotes the forward kinematics of the 6-DoF robot arm given joint angles $x_{\text{robot}}(t)$ is the desired end-effector pose in the robot frame, and $W$ is a positive-definite weighting matrix that balances position and orientation errors. This method ensures that the collaborative robot explicitly reproduces the position and orientation of the human wrist, providing consistent
guidance during the pre-contact phase which requires solving the IK problem continuously during execution.

* An alternative method is to directly map the predicted human joint angles to those of a 6-DoF collaborative robotic arm through a linear transformation. $P_{human}(t) \in \mathbb{R}^{15}$ denotes the vector of human joint angles from the five joints of the tracked arm. The corresponding robot joint configuration $q(t) \in \mathbb{R}^6$ is calculated based on:

$$q(t) = A \cdot P_{human}(t) + b \quad (5)$$

where $A \in \mathbb{R}^{6 \times 15}$ is an offline learned coefficient matrix and $b \in \mathbb{R}^6$ is a calibration term to align reference postures between the human and the robot. This mapping can be estimated offline by collecting paired datasets of human joint angles and robot joint configurations while executing representative tasks. During online execution, the transformation provides real-time estimates of robot joint angles directly from human motion, with optional post-processing such as joint-limit saturation or smoothing to ensure feasibility. Although this approach does not preserve exact end-effector geometry, it is computationally efficient and well suited for fast retargeting in the pre-contact stage.

## B. Phase II: Proactive force control

Once contact with the environment is established, the second phase is activated. The extension of GOM, as it was discussed in Section II, is used to forecast the contact wrenches. These wrenches can be used as feedforward signals for a force controller. Forecast contact wrenches allow the controller to anticipate human behavior, improving settling time and increasing safety during physical HRC. This proactive component is expected to enhance both stability and transparency of the interaction.

## C. Integration and outlook

The proposed framework integrates proactive force control with adaptive continuous regulation, resulting in a unified control strategy. At a higher level, the system combines kinetic and kinematic contributions to generate smooth, anticipatory robot responses that balance safety, stability, and transparency. This integration can be abstractly represented by:

$$\dot{x} = \alpha_{kinematics} \cdot P + \alpha_{kinetics} \cdot W r \quad (6)$$

where $P$ denotes the predicted motion intention derived from synergies, $Wr$ the forecast wrench response, and $\alpha_{kinematics}, \alpha_{kinetics}$ weighting factors balancing their contributions.

## V. CONCLUSION

In this work, we proposed a lightweight method to forecast short-horizon contact wrenches from human motion, enabling proactive behaviors in physical HRC. Building on the state-space rationale of the GOM, we formulated each wrench component as a SARIMAX process driven by exogenous variables comprising selected joint angles (from wearable IMUs) and the remaining wrench components at the previous step. We trained the model once on synchronized motion-force data collected during expert polishing and deployed a rolling multi-step forecast using only past forecasts and exogenous inputs. On approximately 5 hours of recordings, our approach anticipated up to 20 seconds ahead over 600 frames with low error (e.g., RMSE, MAE) and negligible runtime, making it suitable for real-time integration.

Beyond force prediction, this research also introduces a proactive HRC framework that combines motion synergy and force prediction into a unified two-phase control strategy (Section IV). The proposed method integrates pre-contact motion prediction, based on inter-limb synergies assumption of GOM, with contact-phase adaptation using forecast wrenches. Future work will focus on implementing and experimentally validating this framework on a 6-DoF collaborative robot in industrial scenarios such as polishing. Our approach can function as a safety and efficiency module that complements large-scale models with real-time physical interaction forecasting, aiming towards adaptive and proactive human-robot collaboration.

## ACKNOWLEDGMENT

This work was supported by the French Government as part of the ‘Skills and Jobs of the Future’ call for projects (AMI), within the France 2030 programme, operated by the Caisse des Dépôts.

## REFERENCES

[1] Przemysław A. Lasota, Terrence Song, Julie A. Shah, 2017. 'A Survey of Methods for Safe Human-Robot Interaction', now, doi: 10.1561/23000000052.

[2] V. Villani, F. Pini, F. Leali and C. Secchi, 2018. 'Survey on human-robot collaboration in industrial settings: Safety, intuitive interfaces and applications', Mechatronics, vol. 55, pp. 248-266.

[3] D. Rodríguez-Guerra, G. Sorrosal, I. Cabanes and C. Calleja, 2021. 'Human-Robot Interaction Review: Challenges and Solutions for Modern Industrial Environments', in IEEE Access, vol. 9, pp. 108557-108578. doi: 10.1109/ACCESS.2021.3099287.

[4] S. Proia, R. Carli, G. Cavone and M. Dotoli, 2022. 'Control techniques for safe, ergonomic, and efficient human-robot collaboration in the digital industry: A survey', IEEE Transactions on Automation Science and Engineering, vol. 19, no. 3, pp. 1798-1819.

[5] A. Hameed, A. Ordys, J. Možaryn and A. Sibilska-Mroziewicz, 2023. 'Control System Design and Methods for Collaborative Robots: Review', Applied Sciences, 13(1):675. doi: 10.3390/app13010675.

[6] R. Luo, R. Hayne and D. Berenson, 2018. 'Unsupervised early prediction of human reaching for human-robot collaboration in shared workspaces', 2018. Autonomous Robots 42, 631-648. doi: 10.1007/s10514-017-9655-8

[7] C. Escobedo, M. Strong, M. West, A. Aramburu and A. Ron-cone, 'Contact Anticipation for Physical Human-Robot Interaction with Robotic Manipulators using Onboard Proximity Sensors', 2021. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, pp. 7255-7262, doi: 10.1109/IROS51168.2021.9636130.

[8] R. Pandya, Z. Wang, Y. Nakahira, and C. Liu, 2024, May. 'Towards proactive safe human-robot collaborations via data-efficient conditional behavior prediction', In 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 12956-12963). IEEE.

[9] M.K. van Den Broek, and T.B. Moeslund, 2024. 'What is proactive human-robot interaction?-a review of a progressive field and its definitions'. ACM Transactions on Human-Robot Interaction, 13(4), pp.1-30.
[10] G. Xia, Z. Ghrairi, A. Heuermann, and K.D. Thoben, 2025. 'Enhancing sustainability of human-robot collaboration in industry 5.0: Context-and interaction-aware human motion prediction for proactive robot control', Journal of Manufacturing Systems, 82, pp.376-388.
[11] B. E. Olivas-Padilla, S. Manitsaris, and A. Glushkova, "Explainable ai in human motion: A comprehensive approach to analysis, modeling, and generation," Pattern Recognition, vol. 151, p. 110418, 2024.