A computer-implemented neural network system for reinforcement learning, wherein the neural network system is used to select actions to be performed by an electromechanical agent interacting with a real-world environment to perform a task in an attempt to achieve a specified result, the system comprising: an input for receiving state data characterizing the environment and comprising image data obtained from an image sensor as the agent interacts with the real-world environment, the image data being defined by arrays of pixels and including one or more values for each pixel; an input neural network for extracting from the image data, for each of multiple entities potentially present in the environment, respective entity data (e 1 -e N ) indicative of the presence of the entity in the environment, the input network comprising at least one convolutional layer arranged to generate from the state data, for each cell of an array of cells at least one feature value, the cells corresponding to entities that are different respective subsets of the environment, the input network being arranged to generate, for each entity, the respective entity data as a data set comprising at least one feature value of the cell that corresponds to the entity and data indicative of a position of the cell in the array of cells; a relational neural network comprising at least one attention block , each attention block comprising a respective transform neural network (411-41N) for each of the entities, each transform neural network being arranged to generate respective modified entity data for the corresponding entity, the modified entity data being based on the entity data for a corresponding plurality of the entities; and an output neural network arranged to receive data output by the relational neural network, and use the received data to select a respective action defining control inputs to control the electromechanical agent based on the modified entity data. an input for receiving state data characterizing the environment and comprising image data obtained from an image sensor as the agent interacts with the real-world environment, the image data being defined by arrays of pixels and including one or more values for each pixel; an input neural network for extracting from the image data, for each of multiple entities potentially present in the environment, respective entity data (e 1 -e N ) indicative of the presence of the entity in the environment, the input network comprising at least one convolutional layer arranged to generate from the state data, for each cell of an array of cells at least one feature value, the cells corresponding to entities that are different respective subsets of the environment, the input network being arranged to generate, for each entity, the respective entity data as a data set comprising at least one feature value of the cell that corresponds to the entity and data indicative of a position of the cell in the array of cells; a relational neural network comprising at least one attention block , each attention block comprising a respective transform neural network (411-41N) for each of the entities, each transform neural network being arranged to generate respective modified entity data for the corresponding entity, the modified entity data being based on the entity data for a corresponding plurality of the entities; and an output neural network arranged to receive data output by the relational neural network, and use the received data to select a respective action defining control inputs to control the electromechanical agent based on the modified entity data. <EOS>