A computer implemented method for controlling a robot by selecting an action to be performed by a reinforcement learning agent that interacts with an environment by receiving observations characterizing a current state of the environment and, in response, performing actions from a predetermined set of actions, wherein the reinforcement learning agent is the robot and the environment is a real-world environment, and wherein the method comprises: receiving an observation characterizing a current state of the environment; receiving a goal characterizing a target state from a set of target states of the environment; processing the observation using an observation neural network to generate a numeric representation of the observation, wherein the observation neural network is configured to receive the observation as input and to process the observation to generate the numeric representation of the observation; processing the goal using a goal neural network to generate a numeric representation of the goal, wherein the goal neural network is configured to receive the goal as input and to process the goal to generate the numeric representation of the goal; combining the numeric representation of the observation and the numeric representation of the goal to generate a combined representation; processing the combined representation using an action score neural network to generate a respective score for each action in the predetermined set of actions; wherein the action score neural network is configured to receive as input the combined representation and an action and to process the combined representation to generate the respective score for the action, or wherein the action score neural network is configured to receive as input the combined representation and to process the combined representation to generate the respective score for each action in the predetermined set of actions; selecting the action to be performed using the respective scores for the actions in the predetermined set of actions; and having the robot perform the selected action. receiving an observation characterizing a current state of the environment; receiving a goal characterizing a target state from a set of target states of the environment; processing the observation using an observation neural network to generate a numeric representation of the observation, wherein the observation neural network is configured to receive the observation as input and to process the observation to generate the numeric representation of the observation; processing the goal using a goal neural network to generate a numeric representation of the goal, wherein the goal neural network is configured to receive the goal as input and to process the goal to generate the numeric representation of the goal; combining the numeric representation of the observation and the numeric representation of the goal to generate a combined representation; processing the combined representation using an action score neural network to generate a respective score for each action in the predetermined set of actions; wherein the action score neural network is configured to receive as input the combined representation and an action and to process the combined representation to generate the respective score for the action, or wherein the action score neural network is configured to receive as input the combined representation and to process the combined representation to generate the respective score for each action in the predetermined set of actions; selecting the action to be performed using the respective scores for the actions in the predetermined set of actions; and having the robot perform the selected action.