Robust Learning of Tactile Force Estimation through Robot Interaction

Current methods for estimating force from tactile sensor signals are either inaccurate analytic models or taskspecific learned models. In this paper, we explore learning a robust model that maps tactile sensor signals to force. We specifically explore learning a mapping for the SynTouch BioTac sensor via neural networks. We propose a voxelized input feature layer for spatial signals and leverage information about the sensor surface to regularize the loss function. To learn a robust tactile force model that transfers across tasks, we generate ground truth data from three different sources: (1) the BioTac rigidly mounted to a force torque (FT) sensor, (2) a robot interacting with a ball rigidly attached to the same FT sensor, and (3) through force inference on a planar pushing task by formalizing the mechanics as a system of particles and optimizing over the object motion. A total of 140k samples were collected from the three sources. We achieve a median angular accuracy of 3.5 degrees in predicting force direction (66% improvement over the current state of the art) and a median magnitude accuracy of 0.06 N (93% improvement) on a test dataset. Additionally, we evaluate the learned force model in a force feedback grasp controller performing object lifting and gentle placement. Our results can be found on  I. MOTIVATION & RELATED WORK Tactile perception is an important modality, enabling robots to gain critical information for safe interaction in the physical world [1–3]. The advent of sophisticated tactile sensors  with high fidelity signals allows for inferring varied information such as object identity and pose, surface texture, and slip between the object and robot [5–13]. However, using these sensors for force feedback control has been limited 1 NVIDIA, USA. 2 University of Utah Robotics Center and the School of Computing, University of Utah, Salt Lake City, UT, USA. bala@cs.utah.edu 3 Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, GA, USA. 4 University of Washington,  or learning taskspecific feedback policies on the tactile signals [15–17]. One limiting factor has been the inaccuracy of functions to map the tactile signals to force robustly across different tasks. Current methods for force estimation on the SynTouch BioTac [18] fail to cover the entire range of forces applied during typical manipulation tasks. Analytic methods [19, 20] tend to produce very noisy estimates at small force values and their accuracy decreases as the imparted force angle relative to the sensor surface normal becomes large (i.e., a large shear component relative to the compression force). On the other hand, learned force models [21, 22] tend to overfit to the dataset used in training and have not been sufficiently validated in predicting force across varied tasks. More specifically, Wettel and Loeb [21] use machine learning techniques to estimate the force, contact location, and object curvature when a tactile sensor interacts with an object. Lin et al.  improve upon [21], formulating analytic functions for estimation of the contact point, force, and torque from the BioTac sensor readings. Navarro et al.  explore calibration of the force magnitude estimates by recording the DC pressure signal when the sensor is in contact with a force plate. They use these values in a linear least squares formulation to estimate the gain. While they can estimate the magnitude of force, they cannot estimate force direction. Su et al.  explore using feed-forward neural networks to learn a model that maps BioTac signals to force estimates. The neural network more accurately estimates forces than the linear model from [19] and is used to perform grasp stabilization. Importantly, none of these methods validate their force estimates using a data source different from the method used to generate the training data. They also lack experimental comparison between different approaches in the context of robotic manipulation tasks. In this paper, we attempt to address these shortcomings, by collecting a large scale ground truth dataset from different methods and by leveraging the sensor surface and spatial information in our proposed neural network architecture. For one of our collection methods, we infer force from the motion of an object on a planar surface, by formalizing the interaction as a system of particles, a deviation from the well-established velocity model for planar pushing [23] which does not reason about force magnitude. This scheme of force estimation allows us to obtain accurate small-scale forces (0.1-2N), enabling us to learn a precise force prediction model. Motivated by [24], we compare our proposed method with the current state-of-the-art methods for force estimation for the BioTac sensor. We specifically compare the analytic model from [19] and the best performing feed-forward neural network model from [22]. We compare both in terms of force estimation accuracy on our dataset and also empirical experiments on a robot manipulation task. To summarize, this paper makes the following contributions: 1) We provide a novel method to infer force from object motion on a planar surface by formalizing the mechanics as a system of particles and solving for the force in a least squares minimization problem, given the object motion and the point on the object where the force is imparted. 2) We introduce a novel 3D voxel grid, neural network encoding of tactile signals enabling the network to better leverage spatial relations in the signal. We further tailor our learning to the tactile sensor through the introduction of a novel loss function used in training that scales the loss as a function of the angular distance between the imparted force and the surface normal. 3) We collected a large-scale dataset for the BioTac sensor, consisting of over 600 pushing episodes and 200 interactions between an arm-hand system equipped with the BioTac sensors and a force torque sensor. We validate these contributions on our dataset and in an autonomous pick and place task. We show that our proposed method robustly learns a model to estimate forces from the BioTac tactile signals that generalize across multiple robot tasks. Our method improves upon the state of the art [19, 22] in tactile force estimation for the BioTac sensor achieving a median angular accuracy of 3.5 degrees in predicting force direction (66% improvement over the current state of the art) and a median magnitude accuracy of 0.06 N (93% improvement) on a test dataset. II.  We describe the sensor’s states in the following section, followed by a formal definition of the problem. We then describe the computation of ground truth force from planar pushing in Sec. II-C and our network architecture in Sec. II-D.