A computer-implemented method of selecting an action to be performed by a mechanical agent interacting with a real-world environment, the method comprising: receiving a current observation characterizing a current state of the real-world environment, the observation being derived from one or more sensors of the mechanical agent; for each action of a plurality of actions that can be performed by the mechanical agent to interact with the real-world environment: processing the action and the current observation using a distributional Q network having a plurality of network parameters, wherein the distributional Q network is a deep neural network that is configured to process the action and the current observation in accordance with current values of the network parameters to generate a network output that defines a probability distribution (114, 116, 118. 120) over possible Q returns for the action - current observation pair, and wherein each possible Q return is an estimate of a return that would result from the mechanical agent performing the action in response to the current observation, and determining a measure of central tendency of the possible Q returns with respect to the probability distribution for the action - current observation pair; and selecting an action from the plurality of possible actions to control the mechanical agent in response to the current observation using the measures of central tendency for the actions. receiving a current observation characterizing a current state of the real-world environment, the observation being derived from one or more sensors of the mechanical agent; for each action of a plurality of actions that can be performed by the mechanical agent to interact with the real-world environment: processing the action and the current observation using a distributional Q network having a plurality of network parameters, wherein the distributional Q network is a deep neural network that is configured to process the action and the current observation in accordance with current values of the network parameters to generate a network output that defines a probability distribution (114, 116, 118. 120) over possible Q returns for the action - current observation pair, and wherein each possible Q return is an estimate of a return that would result from the mechanical agent performing the action in response to the current observation, and determining a measure of central tendency of the possible Q returns with respect to the probability distribution for the action - current observation pair; and processing the action and the current observation using a distributional Q network having a plurality of network parameters, wherein the distributional Q network is a deep neural network that is configured to process the action and the current observation in accordance with current values of the network parameters to generate a network output that defines a probability distribution (114, 116, 118. 120) over possible Q returns for the action - current observation pair, and wherein each possible Q return is an estimate of a return that would result from the mechanical agent performing the action in response to the current observation, and wherein the distributional Q network is a deep neural network that is configured to process the action and the current observation in accordance with current values of the network parameters to generate a network output that defines a probability distribution (114, 116, 118. 120) over possible Q returns for the action - current observation pair, and wherein each possible Q return is an estimate of a return that would result from the mechanical agent performing the action in response to the current observation, and determining a measure of central tendency of the possible Q returns with respect to the probability distribution for the action - current observation pair; and selecting an action from the plurality of possible actions to control the mechanical agent in response to the current observation using the measures of central tendency for the actions.