An interaction neural network system implemented by one or more computers for analyzing current states and predicting future states of interacting real-world objects, wherein the interaction neural network system includes multiple neural networks that are trained to receive data representing current states of the objects at time t, together with data representing their interactions and to process the received data to generate as output data representing predicted subsequent states of the objects at time t+1 that are predicted consequences of the interactions between the objects; wherein, the system comprising: an interaction component comprising an interaction component neural network configured to: receive as input (i) states of one or more receiver entities and one or more sender entities, and (ii) attributes of one or more relationships between the one or more receiver entities and one or more sender entities, wherein the one or more receiver entities and the one or more sender entities are moving or stationary objects; and process the received input to produce as output multiple effects of the relationships between the one or more receiver entities and one or more sender entities; a dynamical component comprising a dynamical component neural network configured to: receive as input (i) the states of the one or more receiver entities and one or more sender entities, and (ii) the multiple effects of the relationships between the one or more receiver entities and one or more sender entities; and process the received input to produce as output a respective prediction of a subsequent state of each of the one or more receiver entities and one or more sender entities; wherein outputting the respective prediction of the subsequent state of each of the one or more receiver entities and one or more sender entities comprises outputting a prediction of the dynamics of the one or more receiver entities and one or more sender entities, so as to detect a collision between the objects based on rigid body dynamics. an interaction component comprising an interaction component neural network configured to: receive as input (i) states of one or more receiver entities and one or more sender entities, and (ii) attributes of one or more relationships between the one or more receiver entities and one or more sender entities, wherein the one or more receiver entities and the one or more sender entities are moving or stationary objects; and process the received input to produce as output multiple effects of the relationships between the one or more receiver entities and one or more sender entities; receive as input (i) states of one or more receiver entities and one or more sender entities, and (ii) attributes of one or more relationships between the one or more receiver entities and one or more sender entities, wherein the one or more receiver entities and the one or more sender entities are moving or stationary objects; and process the received input to produce as output multiple effects of the relationships between the one or more receiver entities and one or more sender entities; a dynamical component comprising a dynamical component neural network configured to: receive as input (i) the states of the one or more receiver entities and one or more sender entities, and (ii) the multiple effects of the relationships between the one or more receiver entities and one or more sender entities; and process the received input to produce as output a respective prediction of a subsequent state of each of the one or more receiver entities and one or more sender entities; wherein outputting the respective prediction of the subsequent state of each of the one or more receiver entities and one or more sender entities comprises outputting a prediction of the dynamics of the one or more receiver entities and one or more sender entities, so as to detect a collision between the objects based on rigid body dynamics. receive as input (i) the states of the one or more receiver entities and one or more sender entities, and (ii) the multiple effects of the relationships between the one or more receiver entities and one or more sender entities; and process the received input to produce as output a respective prediction of a subsequent state of each of the one or more receiver entities and one or more sender entities; wherein outputting the respective prediction of the subsequent state of each of the one or more receiver entities and one or more sender entities comprises outputting a prediction of the dynamics of the one or more receiver entities and one or more sender entities, so as to detect a collision between the objects based on rigid body dynamics.