Dynamics-regulated kinematic policy for egocentric pose estimation

1 Introduction . From a video captured by a single head-mounted wearable camera ( e.g. , smartglasses , action camera , body camera ) , we aim to infer the wearer ’ s global 3D full-body pose and interaction with objects in the scene , as illustrated in Fig . 1 . This is important for applications like virtual and augmented reality , sports analysis , and wearable medical monitoring , where third-person views are often unavailable and proprioception algorithms are needed for understanding the actions of the camera wearer . However , this task is challenging since the wearer ’ s body is often unseen from a first-person view and the ⇤Work done at Carnegie Mellon University . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , virtual . body motion needs to be inferred solely based on the videos captured by the front-facing camera . Furthermore , egocentric videos usually capture the camera wearer interacting with objects in the scene , which adds additional complexity in recovering a pose sequence that agrees with the scene context . Despite these challenges , we show that it is possible to infer accurate human motion and human-object interaction from a single head-worn front-facing camera . Egocentric pose estimation can be solved using two different paradigms : ( 1 ) a kinematics perspective and ( 2 ) a dynamics perspective . Kinematics-based approaches study motion without regard to the underlying forces ( e.g. , gravity , joint torque ) and can not faithfully emulate human-object interaction without modeling proper contact and forces . They can achieve accurate pose estimates by directly outputting joint angles but can also produce results that violate physical constraints ( e.g . foot skating and ground penetration ) . Dynamics-based approaches , or physics-based approaches , study motions that result from forces . They map directly from visual input to control signals of a human proxy ( humanoid ) inside a physics simulator and recover 3D poses through simulation . These approaches have the crucial advantage that they output physically-plausible human motion and human-object interaction ( i.e. , pushing an object will move it according to the rules of physics ) . However , since no joint torque is captured in human motion datasets , physics-based humanoid controllers are hard to learn , generalize poorly , and are actively being researched [ 36 , 57 , 51 , 52 ] . In this work , we argue that a hybrid approach merging the kinematics and dynamics perspectives is needed . Leveraging a large human motion database [ 29 ] , we learn a task-agnostic dynamics-based humanoid controller to mimic broad human behaviors , ranging from every day motion to dancing and kickboxing . The controller is general-purpose and can be viewed as providing low-level motor skills of a human . After the controller is learned , we train an object-aware kinematic policy to specify the target poses for the controller to mimic . One approach is to let the kinematic model produce target motion only based on the visual input [ 58 , 51 , 56 ] . This approach uses the physics simulation as a post-processing step : the kinematic model computes the target motion separately from the simulation and may output unreasonable target poses . We propose to synchronize the two aspects by designing a kinematic policy that guides the controller and receives timely feedback through comparing its target pose and the resulting simulation state . Our model thus serves as a high-level motion planning module that adapts intelligently based on the current simulation state . In addition , since our kinematic policy only outputs poses and does not model joint torque , it can receive direct supervision from motion capture ( MoCap ) data . While poses from MoCap can provide an initial-guess of target motion , our model can search for better solutions through trial and error . This learning process , dubbed dynamics-regulated training , jointly optimizes our model via supervised learning and reinforcement learning , and significantly improves its robustness to real-world use cases . In summary , our contributions are as follows : ( 1 ) we are the first to tackle the challenging task of estimating physically-plausible 3D poses and human-object interactions from a single front-facing camera ; ( 2 ) we learn a general-purpose humanoid controller from a large MoCap dataset and can perform a broad range of motions inside a physics simulation ; ( 3 ) we propose a dynamics-regulated training procedure that synergizes kinematics , dynamics , and scene context for egocentric vision ; ( 4 ) experiments on a controlled motion capture laboratory dataset and a real-world dataset demonstrate that our model outperforms other state-of-the-art methods on pose-based and physics-based metrics , while generalizing to videos taken in real-world scenarios . 2 Related Work . Third-person human pose estimation . The task of estimating the 3D human pose ( and sometimes shape ) from third-person video is a popular research area in the vision community [ 2 , 21 , 24 , 14 , 40 , 59 , 10 , 34 , 12 , 31 , 23 , 28 , 11 , 54 , 58 ] , with methods aiming to recover 3D joint positions [ 25 , 46 , 14 , 34 ] , 3D joint angles with respect to a parametric human model [ 2 , 21 , 23 , 28 ] , and dense body parts [ 11 ] . Notice that all these methods are purely kinematic and disregard physical reasoning . They also do not recover the global 3D root position and are evaluated by zeroing out the body center ( root-relative ) . A smaller number of works factor in human dynamics [ 42 , 43 , 38 , 49 , 4 ] through postprocessing , physics-based trajectory optimization , or using a differentiable physics model . These approaches can produce physically-plausible human motion , but since they do not utilize a physics simulator and does not model contact , they can not faithfully model human-object interaction . SimPoE [ 58 ] , a recent work on third-person pose estimation using simulated character control , is most related to ours , but 1 ) trains a single and dataset-specific humanoid controller per dataset ; 2 ) designs the kinematic model to be independent from simulation states . Input : Video sequence : Intialization Kinematic Policy Camera Poses Image Features Initialization Pose Object State Initialization Per-step input : Egocentric human pose estimation . Compared to third-person human pose estimation , there are only a handful of attempts at estimating 3D full body poses from egocentric videos due to the ill-posed nature of this task . Most existing methods still assume partial visibility of body parts in the image [ 48 , 39 , 53 ] , often through a downward-facing camera . Among works where the human body is mostly not observable [ 20 , 55 , 56 , 33 ] , Jiang et al . [ 20 ] use a kinematics-based approach where they construct a motion graph from the training data and recover the pose sequence by solving the optimal pose path . Ng et al . [ 33 ] focus on modeling person-to-person interactions from egocentric videos and inferring the wearer ’ s pose conditioning on the other person ’ s pose . The works most related to ours are [ 55 , 56 , 19 ] which use dynamics-based approaches and map visual inputs to control signals to perform physically-plausible human motion inside a physics simulation . They show impressive results on a set of noninteractive locomotion tasks , but also observe large errors in absolute 3D position tracking–mapping directly from the visual inputs to control signals is a noisy process and prone to error accumulation . In comparison , our work jointly models kinematics and dynamics , and estimates a wider range of human motion and human-object interactions while improving absolute 3D position tracking . To the best of our knowledge , we are the first approach to estimate the 3D human poses from egocentric video while factoring in human-object interactions . Humanoid control inside physics simulation . Our work is also connected to controlling humanoids to mimic reference motion [ 36 , 37 , 5 , 57 , 52 , 15 , 6 ] and interact with objects [ 5 , 30 ] inside a physics simulator . The core motivation of these works is to learn the necessary dynamics to imitate or generate human motion in a physics simulation . Deep RL has been the predominant approach in this line of work since physics simulators are typically not end-to-end differetiable . Goal-oriented methods [ 5 , 30 , 1 ] does not involve motion imitation and are evaluated on task completion ( moving an object , sitting on a chair , moving based on user-input etc. ) . Consequently , these frameworks only need to master a subset of possible motions for task completion . People , on the other hand , have a variety of ways to perform actions , and our agent has to follow the trajectory predefined by egocentric videos . Motion imitation methods [ 36 , 37 , 52 , 57 , 52 ] aim to control characters to mimic a sequence of reference motion , but have been limited to performing a single clip [ 36 , 37 , 52 , 57 ] or high-quality MoCap [ 52 ] motion ( and requires fine-tuning to generalize to other motion generators ) . In contrast , our dynamics controller is general and can be used to perform everyday motion and human-object interactions estimated by a kinematic motion estimator without task-specific fine-tuning . 3 Method . The problem of egocentric pose estimation can be formulated as follows : from a wearable camera footage I1 : T , we want to recover the wearer ’ s ground truth global 3D poses bq1 : T . Each pose bq t , ( brpos t , brrot t , bj rot t ) consists of the root position brpos t , root orientation brrot t , and body joint angles bj rot t of the human model . Here we adopt the popular SMPL [ 27 ] human model and the humanoid we use in physics simulation is created from the kinematic structure and mean body shape defined by SMPL . Our framework first learns a Universal Humanoid Controller ( UHC ) from a large MoCap dataset ( Sec . 3.1 ) . The learned UHC can be viewed as providing the lower level muscle skills of a real human , trained by mimicking thousands of human motion sequences . Using the trained UHC , we learn our kinematic policy ( Sec . 3.2 ) through dynamics-regulated training ( Sec . 3.3 ) . At the test time , the kinematic policy provides per-step target motion to the UHC , forming a closed-loop system that operates inside the physics simulation to control a humanoid . The result of the UHC and physics simulation is then used as input to the kinematic policy to produce the next-frame target motion , as depicted in Fig . 2 . As a notation convention , we usee· to denote kinematic quantities ( obtained without using physics simulation ) , b· to denote ground truth quantities , and normal symbols without accents to denote quantities from the physics simulation . 3.1 Dynamics Model - Universal Humanoid Controller ( UHC ) . To learn a task-agnostic dynamics model that can be tightly integrated with a kinematic model , we design our controller ’ s state space to only rely on the current simulated pose q t and target posed bq t+1 and remove all phase or sequence-level information found in prior arts [ 57 , 36 , 37 ] . This design allows us to train on an order of magnitude larger dataset of human motion [ 29 ] with only pose information and significantly improve our models ’ ability to mimic diverse and unseen motions . Formally , we model controlling a humanoid to follow a reference motion bq1 : T as a Markov Decision Process ( MDP ) defined as a tuple M = hS , A , Pphysics , R , i of states , actions , transition dynamics , reward function , and a discount factor . The state S , reward R , and transition dynamics Pphysics are provided by the physics simulator , while action A is computed by the policy ⇡UHC . At each timestep t , the agent in state st takes an action sampled from the policy ⇡UHC ( at|st , bqt+1 ) while the environment generates the next state st+1 and reward rt . We employ Proximal Policy Optimization ( PPO ) [ 41 ] to find the optimal policy ⇡⇤UHC that maximizes the expected discounted return E [ P T t=1 t 1 rt ] . State . The state st , ( qt , q̇t ) of the humanoid contains the character ’ s current pose qt and joint velocity q̇ t . Here , the state st encapsulates the humanoid ’ s full physical state at time step t. It only includes information about the current frame ( q t , q̇ t ) and does not include any extra information , enabling our learned controller to be guided by a target pose only . Action . The action at specifies the target joint angles for the proportional derivative ( PD ) controller [ 45 ] at each degree of freedom ( DoF ) of the humanoid joints except for the root ( pelvis ) . We use the residual action representation [ 15 ] : qd t = bq t+1 + at , where qdt is the final PD target , at is the output of the control policy ⇡UHC , and bqt+1 is the target pose . The torque to be applied at joint i is : ⌧ i = kp ( qd t q t ) kd q̇ t where kp and kd are manually specified gains and is the element-wise multiplication . As observed in prior work [ 57 , 58 ] , allowing the policy to apply external residual forces ⌘ t to the root helps stabilizing the humanoid , so our final action is at , ( eqdt , ⌘t ) . Policy . The policy ⇡UHC ( at|st , bqt+1 ) is represented by a Gaussian distribution with a fixed diagonal covariance matrix ⌃ . We first use a feature extraction layer Ddiff ( bqt+1 , qt ) to compute the root and joint offset between the simulated pose and target pose . All features are then translated to a root-relative coordinate system using an agent-centic transform T AC to make our policy orientationinvariant . We use a Multi-layer Perceptron ( MLP ) as our policy network to map the augmented state T AC q t , q̇ t , bq t+1 , Ddiff ( bqt+1 , qt ) to the predicted action at . Reward function . For UHC , the reward function is designed to encourage the simulated pose q t to better match the target pose bq t+1 . Since we share a similar objective ( mimic target motion ) , our reward is similar to Residual Force Control [ 57 ] . Training procedure . We train our controller on the AMASS [ 29 ] dataset , which contains 11505 high-quality MoCap sequences with 4000k frame of poses ( after removing sequences involving human-object interaction like running on a treadmill ) . At the beginning of each episode , a random fixed length sequence ( 300 frames ) is sampled from the dataset for training . While prior works [ 52 , 51 ] uses more complex motion clustering techniques to sample motions , we devise a simple yet empirically effective sampling technique by inducing a probability distribution based on the value function . For each pose frame bq j in the dataset , we first compute an initialization state sj1 : sj1 , bq j ,0 , and then score it using the value function to access how well the policy can mimic the sequence bq j : T starting from this pose : V ( s j 1 , bqj+1 ) = vj . Intuitively , the higher vj is , the more confident our policy is in mimicing this sequence , and the less often we should pick this frame . The probability of choosing frame j , comparing against all frames J in the AMASS dataset , is then P ( bq j ) = exp ( vj/⌧ ) PJ i exp ( vi/⌧ ) where ⌧ is the sampling temperature . More implementation details about the reward , training and evaluation of UHC can be found in Appendix C .

Summary: In this paper, the authors presents a new method for egocentric physically-plausible 3d human motion estimation. Specially, they designed a general-purpose humanoid controller (UHC) and a dynamics-regulated kinematic policy that can be directly trained and deployed inside a physics simulation. The proposed universal humanoid controller is trained on large-scale Mocap datasets and is for general-purpose. In the experiments, the authors evaluate their proposed method on both Mocap dataset and their collected real dataset. The authors show good performance improvements compared with PoseReg and EgoPose, as well as good imitation success rate compared with DeepMimic on Human3.6M.