A computer-implemented method for training a policy generator neural network to select actions to be performed by a mechanical agent interacting with a real-world environment, the method comprising: obtaining data identifying a set of one or more trajectories, each trajectory comprising a set of observations characterizing a set of states of the environment and corresponding actions performed by another agent in response to the states, as state action pairs; obtaining data identifying an encoder of a trained variational autoencoder neural network, wherein the encoder is configured to process trajectories to determine probability distributions over embeddings of the trajectories; for each trajectory: passing the trajectory through the encoder to determine a probability distribution over embeddings; determining an embedding for the trajectory by sampling from the probability distribution encoded for the trajectory by the encoder; determining an imitation trajectory by applying a policy defined by the policy generator neural network to the embedding for the trajectory; and determining a return from the imitation trajectory using a reward function that is conditioned on the embedding, wherein the reward function makes use of a discriminator configured to discriminate between the imitation trajectory and the trajectory, wherein the discriminator is conditioned using the embedding; and train the policy generator neural network using the reward function that is conditioned on the embedding, by adjusting parameters of the policy generator neural network based on the reward function that is conditioned on the embedding, to improve the return from the reward function. obtaining data identifying a set of one or more trajectories, each trajectory comprising a set of observations characterizing a set of states of the environment and corresponding actions performed by another agent in response to the states, as state action pairs; obtaining data identifying an encoder of a trained variational autoencoder neural network, wherein the encoder is configured to process trajectories to determine probability distributions over embeddings of the trajectories; for each trajectory: passing the trajectory through the encoder to determine a probability distribution over embeddings; determining an embedding for the trajectory by sampling from the probability distribution encoded for the trajectory by the encoder; determining an imitation trajectory by applying a policy defined by the policy generator neural network to the embedding for the trajectory; and determining a return from the imitation trajectory using a reward function that is conditioned on the embedding, wherein the reward function makes use of a discriminator configured to discriminate between the imitation trajectory and the trajectory, wherein the discriminator is conditioned using the embedding; and passing the trajectory through the encoder to determine a probability distribution over embeddings; determining an embedding for the trajectory by sampling from the probability distribution encoded for the trajectory by the encoder; determining an imitation trajectory by applying a policy defined by the policy generator neural network to the embedding for the trajectory; and determining a return from the imitation trajectory using a reward function that is conditioned on the embedding, wherein the reward function makes use of a discriminator configured to discriminate between the imitation trajectory and the trajectory, wherein the discriminator is conditioned using the embedding; and train the policy generator neural network using the reward function that is conditioned on the embedding, by adjusting parameters of the policy generator neural network based on the reward function that is conditioned on the embedding, to improve the return from the reward function.