We present a novel, non-intrusive approach for estimat ing contact forces during hand-object interactions relying solely on visual input provided by a single RGB-D camera. We consider a manipulated object with known geometrical and physical properties. First, we rely on model-based vi sual tracking to estimate the object's pose together with that of the hand manipulating it throughout the motion. Fol lowing this, we compute the object's first and second order kinematics using a new class of numerical dif f erentiation operators. The estimated kinematics is then instantly fed into a second-order cone program that returns a minimal force distribution explaining the observed motion. However, humans typically apply more forces than mechanically re quired when manipulating objects. Thus, we complete our estimation method by learning these excessive forces and their distribution among the fingers in contact. We provide afull validity analysis of the proposed method by evaluating it based on ground truth data from additional sensors such as accelerometers, gyroscopes and pressure sensors. Exper imental results show that force sensing from vision (FSV) is indeed feasible.
With known geometrical and physical properties, Pham et al. REF demonstrated the estimation of the contact force in a hand-object interaction; the estimate was solely based on the visual input provided by a single RGB-D camera.
Towards force sensing from vision: Observing hand-object interactions to infer manipulation forces
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Computer Science