Abstract-We propose position-velocity encoders (PVEs) which learn-without supervision-to encode images to positions and velocities of task-relevant objects. PVEs encode a single image into a low-dimensional position state and compute the velocity state from finite differences in position. In contrast to autoencoders, position-velocity encoders are not trained by image reconstruction, but by making the position-velocity representation consistent with priors about interacting with the physical world. We applied PVEs to several simulated control tasks from pixels and achieved promising preliminary results. While position and velocity are fundamental components of state representations in robotics, robots cannot directly sense these properties. Instead, they need to extract taskrelevant position and velocity information from sensory input. For robots to be versatile, they must be able to learn such representations from experience. Deep learning allows to learn position-velocity representations in principle-but most existing approaches depend crucially on labelled data. In this paper, we investigate how robots could learn positionvelocity representations without supervision. We approach this problem by using robotics-specific prior knowledge about interacting with the physical world, also known as robotic priors [7] , in order to learn an encoding from high-dimensional sensory observations to a low dimensional state representation. Our contribution is to split the state representation into a velocity state and a position state and to incorporate robotic priors about position and velocity in the form of model constraints and learning objectives. Our method, the position-velocity encoder (PVE), implements a hard model constraint by estimating velocity states from finite differences in position states. This constraint fixes the relation between these two parts of the state representation. Additionally, PVEs include soft objectives that measure consistency with robotic priors. These objectives are optimized during learning and shape which information is encoded and how multiple state samples relate to each other. Both ingredients work together to learn an encoding into a structured state representation that includes position states, which describe information from a single observation, and velocity states, which describe how this information changes over time. Figure 1 shows the position encoder that maps observations (blue rectangles) to position states (blue dots). The velocity state-the time derivative of the position state-is approximated from finite differences in position. This structured state space allows us to formulate new robotic priors, specifically for positions and velocities, in the form of learning objectives.
REF developed an encoder that computes object positions and velocities from images of objects, by enforcing these properties to be compliant with common sense.
PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations
ArXiv
ArXiv
Engineering Computer Science