Generating an image of an objectxe2x80x99s appearance from somatosensory information during haptic exploration Visual occlusions caused by the environment or by the robot itself can be a problem for object recognition during manipulation by a robot hand. Under such conditions, tactile and somatosensory information are useful for object recognition during manipulation. Humans can visualize the appearance of invisible objects from only the somatosensory information provided by their hands. In this paper, we propose a method to generate an image of an invisible objectxe2x80x99s posture from the joint angles and touch information provided by robot fingers while touching the object. We show that the objectxe2x80x99s posture can be estimated from the timeseries of the joint angles of the robot hand via regression analysis. In addition, conditional generative adversarial networks can generate an image to show the appearance of the invisible objects from their estimated postures. Our approach enables userfriendly visualization of somatosensory information in remote control applications.

GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments Recent advancements have led to a proliferation of machine learning systems used to assist humans in a wide range of tasks. However, we are still far from accurate, reliable, and resourceefficient operations of these systems. For robot perception, convolutional neural networks (CNNs) for object detection and pose estimation are recently coming into widespread use. However, neural networks are known to suffer from overfitting during the training process and are less robust under unforeseen conditions (which makes them especially vulnerable to adversarial scenarios). In this work, we propose Generative Robust Inference and Perception (GRIP) as a twostage object detection and pose estimation system that aims to combine the relative strengths of discriminative CNNs and generative inference methods to achieve robust estimation. Our results show that a second stage of samplebased generative inference is able to recover from false object detections by CNNs, and produce robust estimations in adversarial conditions. We demonstrate the efficacy of GRIP robustness through comparison with stateoftheart learningbased pose estimators and pickandplace manipulation in dark and cluttered environments.