Estimating Dynamics On-the-Fly Using Monocular Video

We present an optimization framework to help estimate on-the-fly both the motion and physical parameters of an articulated multibody system using uncalibrated monocular image sequences. The algorithm takes video images of a physical system as input and estimates the motion together with the physical system parameters, given only the underlying articulated model topology. A valid initial pose of the system is found using a sequential optimization framework and used to bootstrap the successive pose estimation as well as estimation of physical system parameters (kinematic/geometric lengths as well as mass, inertia, damping coefficients). We also address the issue of robustly estimating a dynamically-equivalent system using partial state information (solely from noisy visual observations) and without explicit inertial parameter information. This framework results in a robust dynamically-equivalent system with good predictive capabilities when tested on a double pendulum system. INTRODUCTION System identification of articulated multibody systems has largely relied on existence of well-defined and persistent external excitation of the system [2]. Under the persistent excitation criterion, techniques for kinematic and inertial parameter estimation using joint sens∗Address all correspondence to this author. ing are well established [1]. However, estimation of inertial parameters of articulated multi-body biological systems, such as humans, animals or insects, has been possible only when augmented with specialized sensor systems. While kinematic identification methods for robotic systems using visual observation alone are well developed [8], they rely on calibrated camera setups. Inertial parameter estimation using visual observation alone with uncalibrated cameras still remains a challenge [9]. Nevertheless, accurate and efficient kinematic and inertial parameter estimation has many applications in telepresence [3], robot navigation [4, 5], vision-based humanoid control, multi-robot cooperation [6] and imitation based robot control [7]. Prior work has addressed the problem of articulated multi-body pose estimation using filtering techniques based on fiducials, interest-points, edges, and silhouettes. Broida et al. [10] used tracked feature point trajectories over multiple images and constructed a state space model incorporating both kinematic and structural states to estimate state vector using an iterated extended kalman filter (IKEF). Pathirana et al. [11] used robust extended Kalman filter (RKEF) and fused information from two cameras and ultrasonic sensor to estimate the kinematic states of mobile object from a moving camera. Extended Kalman Filter (EKF) has been also used by Azarbayejani et al. [12] to estimate motion and focal length of the camera. In this paper, we address the issue of estimating the kinematic and inertial parameters of a double-link pendulum using visual information from uncalibrated monocular video. We then seek to use these estimates to predict future poses of the double-link pendulum with no prior information about the physical system parameters or input. Well established system identification techniques fail to solve this problem due to: (i) partial state knowledge (only position information); (ii) noise due to imaging inaccuracies; (iii) unknown and insufficient input excitation (both structure and frequency); (iv) low sampling frequency; and (v) inherent dynamics nonlinearity. Further, most of the EKF/IKEF/RKEF based methods applied to motion estimation, rely on availability of point feature correspondences established over a long sequence of images, which is non-trivial for self-occluding articulated objects. Hence, we leverage an optimization-based framework to solve the problem of parameter and input identification which resulted in a robust dynamically equivalent system. The presented methodology can use the learnt equivalent model to: (i) efficiently predict future pose; (ii) help with gap-filling when occulusions are present; and (iii) develop model-based control strategies. PROBLEM STATEMENT Given the information about the topology and type of body geometries in the system, estimate the camera location; orientation, size, mass, inertia of each body; and torque input/damping coefficient at each joint in the system using monocular images. We use the optimization based framework for first estimating the camera parameters and then the initial pose of the multibody articulated system. The initial pose estimation technique is then extended to estimate the pose of the system in a video. Finally, using the estimated pose, the physical parameters of the system are estimated using dynamic-simulation-based optimization. Fig. 1 shows an outline of the overall approach. CAMERA PARAMETER ESTIMATION Camera Depth Estimation: We align the principle axes of the camera with the world coordinate system of the model. The camera depth (z coordinate) estimation is based on the fact that an actual body with proportional dimensions and similar orientation in space will roughly occupy a similar area in an actual image as that of the model in the synthetic image. We set up an optimization problem based on the difference in the silhouette area in R M l Vid aw onocu ar eo