A Visuo-haptic Sensor for the Exploration of Deformable Objects

We present a linear tactile sensor for grasping applications, which is based on a passive rubber foam material mounted on the fingers of a gripper and a low-cost camera pointed towards the actuator. This concept allows to visually measure local deformation, geometry and forces, without the need for costly dedicated sensors in the actuator. In an experiment, the geometry and deformation behavior of a deformable object (plastic bottle) is explored with the proposed sensor. 1) Introduction: Autonomous grasping of partially unknown, deformable objects in human environments is a complex problem. In general it requires an object model, a grasp planner as well as visual and haptic feedback. The latter is needed during the grasping process to obtain the current state and provided by a dedicated laminar tactile sensor such as [1], which is mounted onto the fingers and measures local pressure/forces applied to the object. A haptic object model of geometry and deformation behavior (local stiffness) can be obtained by exploration, i.e. by acquiring grasp samples on the object surface. A two-finger gripper is sufficient for many objects and allows for relatively simple grasp configurations. Such models are essential for objects like bottles or cups which require substantially different grasping parameters, depending on their material (glass, plastic or paper) and the weight of their content. Geometric models can also be obtained visually, if the object is not transparent and the surface exhibits diffuse reflection. During a grasp, the desired grasping force and pattern are derived from the object model. The sensor feedback is used to adjust the forces and to verify the grasping pattern. In this work, we present a visuo-haptic sensor, which relies on a passive and inexpensive deformable rubber foam material mounted onto the fingers of a gripper, as well as a low-cost camera which observes the fingers. A similar approach has been presented for a mobile platform in [2], [3]. Contact forces, local pressure and object deformation along the fingers are determined from visual measurements as well as from the known deformation characteristics of the foam. The position of the fingers is also determined visually. The sensor can be used for grasping with haptic (force) feedback, as well as for haptic exploration tasks. The proposed setup allows for very low-cost grippers, since dedicated sensors for position and force can be saved. Especially laminar force sensors are quite costly and require cabling through the robot arm. On humanoid robots, the system even takes advantage of existing cameras, since the head cameras can be pointed to the hand. Furthermore, the system offers compliance, due to the rubber foam, and provides coherent measurements between the haptic and visual modalities. 2) Visuo-haptic sensor: The setup of the proposed grasping system is depicted in Fig. 1: A two-finger gripper with a linear driver is equipped with strips of rubber foam on the inside of the fingers. The foam is the passive part of the sensor; when the gripper closes, it comes in contact with the object and deforms depending on the local pressure. Cameras (one for each finger) are mounted above the gripper, such that they observe the top side of the fingers and the top surface of the foam material. To prevent occlusion of the foam by the object, they are located slightly outside of the grasping range. Visual This work has been supported, in part, by the European Research Council under the European Union’s Seventh Framework Programme (FP7/20072013) / ERC Grant agreement no. 258941. Fig. 1. Left: The proposed sensor consists of one or two low-cost webcams (top-left) and passive rubber foam strips on the fingers of a gripper (bottom). Right: Detailed view of the foam strip with its front painted in green. The deformation in the contact region at the center is measured visually using the camera. In the lower part, the red lines indicate the locations of the two contours tracked by snakes (reference and front). snakes are used to track the two long contours on the top side of the foam – i.e. the reference contour ri between the metallic finger and the foam, as well as the front contour si between the foam and object (or an artificial internal edge, see below). Distances are converted from the 2D image to world coordinates [in meters] by the intrinsic camera parameters and by a projection onto the plane spanned by the finger’s major axis and its motion axis. The absolute position of the finger is known from ri, and the current deformation δ along the front contour s is calculated by δi = (si − ri)− (s i − r ref i ). The reference configuration ref is obtained during initialization, see below. Since the stiffness of the rubber foam is also known, object deformation and the applied pressure can be obtained simultaneously. The deformation characteristics of foam materials are wellinvestigated [4] and generally expressed by a non-linear relation between normalized strain (compression) and stress (pressure [Pa]). We measure the strain-stress curve of the used rubber foam as outlined in [3] and approximate it by a third-order polynomial f , which yields the local pressure applied to the object. The curve obtained for rubber foam is more linear compared to plastic foams and has a slope of 0.08 1 in its central region. The total force applied to the object is obtained by integration over the pressure using f , the normalized deformation δ w along the front counter s and the material width/height w, h: F = h ∫