A method of training a convolutional neural network (125, 600), comprising: identifying, by one or more processors, a plurality of training examples generated based on sensor output from one or more robots (180A, 180B) during a plurality of grasp attempts by the robots, each of the grasp attempts being performed over a plurality of instances of time and being performed in respect of a respective one of one or more environmental objects (191A, 191B), each of the training examples including training example input comprising: an image (I t, 661A) for a corresponding instance of time of a corresponding grasp attempt of the grasp attempts, the image capturing a robotic end effector (182A, 182B) and the one or more environmental objects at the corresponding instance of time, and an end effector motion vector (v t, 662) defining motion of the end effector to move from an instance of time pose of the end effector at the corresponding instance of time to a final pose of the end effector for the corresponding grasp attempt, each of the training examples including training example output comprising: a grasp success label (l i) indicative of success of the corresponding grasp attempt; training, by one or more of the processors, the convolutional neural network based on applying the training example input of the training examples to the convolutional neural network and performing backpropagation on the convolutional neural network based on the training example output of the training examples. identifying, by one or more processors, a plurality of training examples generated based on sensor output from one or more robots (180A, 180B) during a plurality of grasp attempts by the robots, each of the grasp attempts being performed over a plurality of instances of time and being performed in respect of a respective one of one or more environmental objects (191A, 191B), each of the training examples including training example input comprising: an image (I t, 661A) for a corresponding instance of time of a corresponding grasp attempt of the grasp attempts, the image capturing a robotic end effector (182A, 182B) and the one or more environmental objects at the corresponding instance of time, and an end effector motion vector (v t, 662) defining motion of the end effector to move from an instance of time pose of the end effector at the corresponding instance of time to a final pose of the end effector for the corresponding grasp attempt, each of the training examples including training example output comprising: a grasp success label (l i) indicative of success of the corresponding grasp attempt; each of the training examples including training example input comprising: an image (I t, 661A) for a corresponding instance of time of a corresponding grasp attempt of the grasp attempts, the image capturing a robotic end effector (182A, 182B) and the one or more environmental objects at the corresponding instance of time, and an end effector motion vector (v t, 662) defining motion of the end effector to move from an instance of time pose of the end effector at the corresponding instance of time to a final pose of the end effector for the corresponding grasp attempt, an image (I t, 661A) for a corresponding instance of time of a corresponding grasp attempt of the grasp attempts, the image capturing a robotic end effector (182A, 182B) and the one or more environmental objects at the corresponding instance of time, and an end effector motion vector (v t, 662) defining motion of the end effector to move from an instance of time pose of the end effector at the corresponding instance of time to a final pose of the end effector for the corresponding grasp attempt, each of the training examples including training example output comprising: a grasp success label (l i) indicative of success of the corresponding grasp attempt; training, by one or more of the processors, the convolutional neural network based on applying the training example input of the training examples to the convolutional neural network and performing backpropagation on the convolutional neural network based on the training example output of the training examples.