Dynamics-regulated kinematic policy for egocentric pose estimation

1 Introduction . From a video captured by a single head-mounted wearable camera ( e.g. , smartglasses , action camera , body camera ) , we aim to infer the wearer ’ s global 3D full-body pose and interaction with objects in the scene , as illustrated in Fig . 1 . This is important for applications like virtual and augmented reality , sports analysis , and wearable medical monitoring , where third-person views are often unavailable and proprioception algorithms are needed for understanding the actions of the camera wearer . However , this task is challenging since the wearer ’ s body is often unseen from a first-person view and the ⇤Work done at Carnegie Mellon University . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , virtual . body motion needs to be inferred solely based on the videos captured by the front-facing camera . Furthermore , egocentric videos usually capture the camera wearer interacting with objects in the scene , which adds additional complexity in recovering a pose sequence that agrees with the scene context . Despite these challenges , we show that it is possible to infer accurate human motion and human-object interaction from a single head-worn front-facing camera . Egocentric pose estimation can be solved using two different paradigms : ( 1 ) a kinematics perspective and ( 2 ) a dynamics perspective . Kinematics-based approaches study motion without regard to the underlying forces ( e.g. , gravity , joint torque ) and can not faithfully emulate human-object interaction without modeling proper contact and forces . They can achieve accurate pose estimates by directly outputting joint angles but can also produce results that violate physical constraints ( e.g . foot skating and ground penetration ) . Dynamics-based approaches , or physics-based approaches , study motions that result from forces . They map directly from visual input to control signals of a human proxy ( humanoid ) inside a physics simulator and recover 3D poses through simulation . These approaches have the crucial advantage that they output physically-plausible human motion and human-object interaction ( i.e. , pushing an object will move it according to the rules of physics ) . However , since no joint torque is captured in human motion datasets , physics-based humanoid controllers are hard to learn , generalize poorly , and are actively being researched [ 36 , 57 , 51 , 52 ] . In this work , we argue that a hybrid approach merging the kinematics and dynamics perspectives is needed . Leveraging a large human motion database [ 29 ] , we learn a task-agnostic dynamics-based humanoid controller to mimic broad human behaviors , ranging from every day motion to dancing and kickboxing . The controller is general-purpose and can be viewed as providing low-level motor skills of a human . After the controller is learned , we train an object-aware kinematic policy to specify the target poses for the controller to mimic . One approach is to let the kinematic model produce target motion only based on the visual input [ 58 , 51 , 56 ] . This approach uses the physics simulation as a post-processing step : the kinematic model computes the target motion separately from the simulation and may output unreasonable target poses . We propose to synchronize the two aspects by designing a kinematic policy that guides the controller and receives timely feedback through comparing its target pose and the resulting simulation state . Our model thus serves as a high-level motion planning module that adapts intelligently based on the current simulation state . In addition , since our kinematic policy only outputs poses and does not model joint torque , it can receive direct supervision from motion capture ( MoCap ) data . While poses from MoCap can provide an initial-guess of target motion , our model can search for better solutions through trial and error . This learning process , dubbed dynamics-regulated training , jointly optimizes our model via supervised learning and reinforcement learning , and significantly improves its robustness to real-world use cases . In summary , our contributions are as follows : ( 1 ) we are the first to tackle the challenging task of estimating physically-plausible 3D poses and human-object interactions from a single front-facing camera ; ( 2 ) we learn a general-purpose humanoid controller from a large MoCap dataset and can perform a broad range of motions inside a physics simulation ; ( 3 ) we propose a dynamics-regulated training procedure that synergizes kinematics , dynamics , and scene context for egocentric vision ; ( 4 ) experiments on a controlled motion capture laboratory dataset and a real-world dataset demonstrate that our model outperforms other state-of-the-art methods on pose-based and physics-based metrics , while generalizing to videos taken in real-world scenarios . 2 Related Work . Third-person human pose estimation . The task of estimating the 3D human pose ( and sometimes shape ) from third-person video is a popular research area in the vision community [ 2 , 21 , 24 , 14 , 40 , 59 , 10 , 34 , 12 , 31 , 23 , 28 , 11 , 54 , 58 ] , with methods aiming to recover 3D joint positions [ 25 , 46 , 14 , 34 ] , 3D joint angles with respect to a parametric human model [ 2 , 21 , 23 , 28 ] , and dense body parts [ 11 ] . Notice that all these methods are purely kinematic and disregard physical reasoning . They also do not recover the global 3D root position and are evaluated by zeroing out the body center ( root-relative ) . A smaller number of works factor in human dynamics [ 42 , 43 , 38 , 49 , 4 ] through postprocessing , physics-based trajectory optimization , or using a differentiable physics model . These approaches can produce physically-plausible human motion , but since they do not utilize a physics simulator and does not model contact , they can not faithfully model human-object interaction . SimPoE [ 58 ] , a recent work on third-person pose estimation using simulated character control , is most related to ours , but 1 ) trains a single and dataset-specific humanoid controller per dataset ; 2 ) designs the kinematic model to be independent from simulation states . Input : Video sequence : Intialization Kinematic Policy Camera Poses Image Features Initialization Pose Object State Initialization Per-step input : Egocentric human pose estimation . Compared to third-person human pose estimation , there are only a handful of attempts at estimating 3D full body poses from egocentric videos due to the ill-posed nature of this task . Most existing methods still assume partial visibility of body parts in the image [ 48 , 39 , 53 ] , often through a downward-facing camera . Among works where the human body is mostly not observable [ 20 , 55 , 56 , 33 ] , Jiang et al . [ 20 ] use a kinematics-based approach where they construct a motion graph from the training data and recover the pose sequence by solving the optimal pose path . Ng et al . [ 33 ] focus on modeling person-to-person interactions from egocentric videos and inferring the wearer ’ s pose conditioning on the other person ’ s pose . The works most related to ours are [ 55 , 56 , 19 ] which use dynamics-based approaches and map visual inputs to control signals to perform physically-plausible human motion inside a physics simulation . They show impressive results on a set of noninteractive locomotion tasks , but also observe large errors in absolute 3D position tracking–mapping directly from the visual inputs to control signals is a noisy process and prone to error accumulation . In comparison , our work jointly models kinematics and dynamics , and estimates a wider range of human motion and human-object interactions while improving absolute 3D position tracking . To the best of our knowledge , we are the first approach to estimate the 3D human poses from egocentric video while factoring in human-object interactions . Humanoid control inside physics simulation . Our work is also connected to controlling humanoids to mimic reference motion [ 36 , 37 , 5 , 57 , 52 , 15 , 6 ] and interact with objects [ 5 , 30 ] inside a physics simulator . The core motivation of these works is to learn the necessary dynamics to imitate or generate human motion in a physics simulation . Deep RL has been the predominant approach in this line of work since physics simulators are typically not end-to-end differetiable . Goal-oriented methods [ 5 , 30 , 1 ] does not involve motion imitation and are evaluated on task completion ( moving an object , sitting on a chair , moving based on user-input etc. ) . Consequently , these frameworks only need to master a subset of possible motions for task completion . People , on the other hand , have a variety of ways to perform actions , and our agent has to follow the trajectory predefined by egocentric videos . Motion imitation methods [ 36 , 37 , 52 , 57 , 52 ] aim to control characters to mimic a sequence of reference motion , but have been limited to performing a single clip [ 36 , 37 , 52 , 57 ] or high-quality MoCap [ 52 ] motion ( and requires fine-tuning to generalize to other motion generators ) . In contrast , our dynamics controller is general and can be used to perform everyday motion and human-object interactions estimated by a kinematic motion estimator without task-specific fine-tuning . 3 Method . The problem of egocentric pose estimation can be formulated as follows : from a wearable camera footage I1 : T , we want to recover the wearer ’ s ground truth global 3D poses bq1 : T . Each pose bq t , ( brpos t , brrot t , bj rot t ) consists of the root position brpos t , root orientation brrot t , and body joint angles bj rot t of the human model . Here we adopt the popular SMPL [ 27 ] human model and the humanoid we use in physics simulation is created from the kinematic structure and mean body shape defined by SMPL . Our framework first learns a Universal Humanoid Controller ( UHC ) from a large MoCap dataset ( Sec . 3.1 ) . The learned UHC can be viewed as providing the lower level muscle skills of a real human , trained by mimicking thousands of human motion sequences . Using the trained UHC , we learn our kinematic policy ( Sec . 3.2 ) through dynamics-regulated training ( Sec . 3.3 ) . At the test time , the kinematic policy provides per-step target motion to the UHC , forming a closed-loop system that operates inside the physics simulation to control a humanoid . The result of the UHC and physics simulation is then used as input to the kinematic policy to produce the next-frame target motion , as depicted in Fig . 2 . As a notation convention , we usee· to denote kinematic quantities ( obtained without using physics simulation ) , b· to denote ground truth quantities , and normal symbols without accents to denote quantities from the physics simulation . 3.1 Dynamics Model - Universal Humanoid Controller ( UHC ) . To learn a task-agnostic dynamics model that can be tightly integrated with a kinematic model , we design our controller ’ s state space to only rely on the current simulated pose q t and target posed bq t+1 and remove all phase or sequence-level information found in prior arts [ 57 , 36 , 37 ] . This design allows us to train on an order of magnitude larger dataset of human motion [ 29 ] with only pose information and significantly improve our models ’ ability to mimic diverse and unseen motions . Formally , we model controlling a humanoid to follow a reference motion bq1 : T as a Markov Decision Process ( MDP ) defined as a tuple M = hS , A , Pphysics , R , i of states , actions , transition dynamics , reward function , and a discount factor . The state S , reward R , and transition dynamics Pphysics are provided by the physics simulator , while action A is computed by the policy ⇡UHC . At each timestep t , the agent in state st takes an action sampled from the policy ⇡UHC ( at|st , bqt+1 ) while the environment generates the next state st+1 and reward rt . We employ Proximal Policy Optimization ( PPO ) [ 41 ] to find the optimal policy ⇡⇤UHC that maximizes the expected discounted return E [ P T t=1 t 1 rt ] . State . The state st , ( qt , q̇t ) of the humanoid contains the character ’ s current pose qt and joint velocity q̇ t . Here , the state st encapsulates the humanoid ’ s full physical state at time step t. It only includes information about the current frame ( q t , q̇ t ) and does not include any extra information , enabling our learned controller to be guided by a target pose only . Action . The action at specifies the target joint angles for the proportional derivative ( PD ) controller [ 45 ] at each degree of freedom ( DoF ) of the humanoid joints except for the root ( pelvis ) . We use the residual action representation [ 15 ] : qd t = bq t+1 + at , where qdt is the final PD target , at is the output of the control policy ⇡UHC , and bqt+1 is the target pose . The torque to be applied at joint i is : ⌧ i = kp ( qd t q t ) kd q̇ t where kp and kd are manually specified gains and is the element-wise multiplication . As observed in prior work [ 57 , 58 ] , allowing the policy to apply external residual forces ⌘ t to the root helps stabilizing the humanoid , so our final action is at , ( eqdt , ⌘t ) . Policy . The policy ⇡UHC ( at|st , bqt+1 ) is represented by a Gaussian distribution with a fixed diagonal covariance matrix ⌃ . We first use a feature extraction layer Ddiff ( bqt+1 , qt ) to compute the root and joint offset between the simulated pose and target pose . All features are then translated to a root-relative coordinate system using an agent-centic transform T AC to make our policy orientationinvariant . We use a Multi-layer Perceptron ( MLP ) as our policy network to map the augmented state T AC q t , q̇ t , bq t+1 , Ddiff ( bqt+1 , qt ) to the predicted action at . Reward function . For UHC , the reward function is designed to encourage the simulated pose q t to better match the target pose bq t+1 . Since we share a similar objective ( mimic target motion ) , our reward is similar to Residual Force Control [ 57 ] . Training procedure . We train our controller on the AMASS [ 29 ] dataset , which contains 11505 high-quality MoCap sequences with 4000k frame of poses ( after removing sequences involving human-object interaction like running on a treadmill ) . At the beginning of each episode , a random fixed length sequence ( 300 frames ) is sampled from the dataset for training . While prior works [ 52 , 51 ] uses more complex motion clustering techniques to sample motions , we devise a simple yet empirically effective sampling technique by inducing a probability distribution based on the value function . For each pose frame bq j in the dataset , we first compute an initialization state sj1 : sj1 , bq j ,0 , and then score it using the value function to access how well the policy can mimic the sequence bq j : T starting from this pose : V ( s j 1 , bqj+1 ) = vj . Intuitively , the higher vj is , the more confident our policy is in mimicing this sequence , and the less often we should pick this frame . The probability of choosing frame j , comparing against all frames J in the AMASS dataset , is then P ( bq j ) = exp ( vj/⌧ ) PJ i exp ( vi/⌧ ) where ⌧ is the sampling temperature . More implementation details about the reward , training and evaluation of UHC can be found in Appendix C .

Summary: This paper proposes a method for pose estimation from first-person camera view. The work builds on a universal controller that learns to imitate a wide variety of behaviors. A kinematic policy is then either trained fully via supervised learning or in a hybrid manner, using a combination of supervised learning and reinforcement learning. The reinforcement learning optimization consists of a motion imitation and a physics-based objective. Specifically, the universal controller’s output is utilized to regularize the kinematic policy to become more physically plausible. The main contributions are a universal controller and the dynamics-regulated training procedure for egocentric pose estimation in human-object interaction scenarios.