Robots have transformed many industries, most notably manufacturing 1 , and have the power to deliver tremendous benefits to society, such as in search and rescue 2 , disaster response 3 , health care 4 and transportation 5 . They are also invaluable tools for scientific exploration in environments inaccessible to humans, from distant planets 6 to deep oceans 7 . A major obstacle to their widespread adoption in more complex environments outside factories is their fragility 6, 8 . Whereas animals can quickly adapt to injuries, current robots cannot 'think outside the box' to find a compensatory behaviour when they are damaged: they are limited to their prespecified self-sensing abilities, can diagnose only anticipated failure modes 9 , and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots 6, 8 . A promising approach to reducing robot fragility involves having robots learn appropriate behaviours in response to damage 10,11 , but current techniques are slow even with small, constrained search spaces 12 . Here we introduce an intelligent trialand-error algorithm that allows robots to adapt to damage in less than two minutes in large search spaces without requiring selfdiagnosis or pre-specified contingency plans. Before the robot is deployed, it uses a novel technique to create a detailed map of the space of high-performing behaviours. This map represents the robot's prior knowledge about what behaviours it can perform and their value. When the robot is damaged, it uses this prior knowledge to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a behaviour that compensates for the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new algorithm will enable more robust, effective, autonomous robots, and may shed light on the principles that animals use to adapt to injury. Current damage recovery in deployed robots typically involves two phases: self-diagnosis, followed by selection of the best pre-designed contingency plan 9,13-15 . Such self-diagnosing robots are expensive, because self-monitoring sensors are expensive, and are difficult to design, because robot engineers cannot foresee every possible situation: this approach often fails either because the diagnosis is incorrect 13, 14 or because an appropriate contingency plan is not provided 15 . Injured animals respond differently (see Fig. 1a ): they learn by trial and error how to compensate for damage (for example, learning which limp minimizes pain) 16, 17 . Similarly, trial-and-error learning algorithms could allow robots to creatively discover compensatory behaviours without being limited to their designers' assumptions about how damage may occur and how to compensate for each type of damage. However, state-of-the-art learning algorithms are impractical because of the ''curse of dimensionality'' 12 : the fastest algorithms, from fields like reinforcement learning 12 and modular robotics 10,11 , depend on human demonstrations 12 or take 15 min or more even for relatively small search spaces (for example, 6-12 parameters, requiring 15-30 min 10,11 ). Algorithms without these limitations take several hours 12 . Damage recovery would be much more practical and effective if robots adapted as creatively and quickly as animals do (for example, in less than 2 min) in larger search spaces and without expensive, self-diagnosing sensors. Here we show that rapid adaptation can be achieved by guiding an intelligent trial-and-error learning algorithm with an automatically generated, pre-computed behaviour-performance map that predicts the performance of thousands of different behaviours (Supplementary Video 1). Current learning algorithms either start with no knowledge of the search space 12 or with minimal knowledge from a few human demonstrations 12, 18 . Our hypothesis is that animals understand the space of possible behaviours and their value from previous experience 19 , and that animals adapt by intelligently selecting tests that validate or invalidate whole families of promising compensatory behaviours. The key insight here is that robots could do the same. Our robots store knowledge from previous experience in the form of a map of the behaviour-performance space. Guided by this map, a damaged robot tries different types of behaviours that are predicted to perform well and, as tests are conducted, updates its estimates of the performance of those types of behaviours. The process ends when the robot predicts that the most effective behaviour has already been discovered. A key assumption is that information about many different behaviours of the undamaged robot will still be useful after damage, because some of these behaviours will still be functional despite the damage. The results of our experiments support this assumption for all the types of damage we tested, revealing that a robot can quickly discover a way to compensate for damage (for example, see Fig. 1c ) without a detailed mechanistic understanding of its cause, as occurs with animals. We call this approach 'Intelligent Trial and Error' (IT&E) (see Fig. 1d ). The behaviour-performance map is created using a novel algorithm and a simulation of the robot, which can be either a standard physics simulator or automatically discovered 14 . The robot's designers need only describe the dimensions of the space of possible behaviours and a performance measure. For instance, walking gaits could be described by how much each leg touches the ground (a behavioural measure) and speed (a performance measure). An alternative gait behavioural measure could be the percentage of time a robot's torso has positive pitch, roll, and yaw angles. For grasping, performance could be measured by the amount of surface contact, and it has been demonstrated that 90% of effective poses for the 21-degree-of-freedom human hand can be captured by a three-dimensional behavioural space describing the principal components of ways in which hand-poses commonly vary 20 . To fill in the behaviour-performance map, an optimization algorithm simultaneously searches for a high-performing solution at each point in the behavioural space ( Fig. 2a, b and Extended Data Fig. 1 ). This step requires simulating millions of behaviours, but needs to be performed only once per robot design before deployment (Supplementary Methods). A low confidence is assigned to the predicted performance of behaviours stored in this behaviour-performance map because they have not been tried in reality ( Fig. 2b and Extended Data Fig. 1 ). During the robot's mission, if performance drops below a user-defined threshold (owing either to damage or to a different environment), the robot
Among others, Cully et al. developed a walking robot that can quickly adapt its gait after been damaged using an intelligent trial and error algorithm REF .
Robots that can adapt like animals
Nature
Nature
Computer Science Medicine Biology