  Learning physically structured representations of dynamical systems that
include contact between different objects is an important problem for
learning-based approaches in robotics. Black-box neural networks can learn to
approximately represent discontinuous dynamics, but they typically require
large quantities of data and often suffer from pathological behaviour when
forecasting for longer time horizons. In this work, we use connections between
deep neural networks and differential equations to design a family of deep
network architectures for representing contact dynamics between objects. We
show that these networks can learn discontinuous contact events in a
data-efficient manner from noisy observations in settings that are
traditionally difficult for black-box approaches and recent physics inspired
neural networks. Our results indicate that an idealised form of touch feedback
-- which is heavily relied upon by biological systems -- is a key component of
making this learning problem tractable. Together with the inductive biases
introduced through the network architectures, our techniques enable accurate
learning of contact dynamics from observations.
