We compare several adaptive design strategies using a data set of 223 M 2 AX family of compounds for which the elastic properties [bulk (B), shear (G), and Young's (E) modulus] have been computed using density functional theory. The design strategies are decomposed into an iterative loop with two main steps: machine learning is used to train a regressor that predicts elastic properties in terms of elementary orbital radii of the individual components of the materials; and a selector uses these predictions and their uncertainties to choose the next material to investigate. The ultimate goal is to obtain a material with desired elastic properties in as few iterations as possible. We examine how the choice of data set size, regressor and selector impact the design. We find that selectors that use information about the prediction uncertainty outperform those that don't. Our work is a step in illustrating how adaptive design tools can guide the search for new materials with desired properties. Regression methods have provided a foundation for modeling structure-property relationships in materials science 1-6 . These methods take as input a data set of known material compositions along with some property (e.g. strength, band gap, melting temperature, etc). Each material, in turn, is described in terms of one or more features that represent aspects of structure, chemistry, bonding and/or microstructure in an abstract, high-dimensional space. The success of regression is based on its ability to capture the relative variation in a property as a function of the features, eventually culminating in the prediction of new materials with desired properties. Materials design is an optimization problem with the goal of maximizing (or minimizing) some desired property of a material, denoted by y, by varying certain features, denoted by x. Optimizing a material for targeted applications generally proceeds by iterating three steps: 1) develop a model (or a set of models) that enables prediction of y from x of known materials, 2) based on these models, optimally select a value of x, which corresponds to a new and yet-to-be synthesized material, and 3) measure y at this value of x, and add (x, y) to the database of known properties. The primary bottleneck in material design is step 3, measuring y, because it requires synthesis and characterization of new materials, which can be expensive and time consuming. Thus, we want to synthesize as few compounds as possible. Alternatively, in the case of a complex code (e.g. high-throughput first principles calculations 7-10 ), we would want to make as few function calls or tractable number of calculations as possible. Thus, using the experiment or complex code as a means to exhaustively search (in a brute force manner) the vast space to optimize a given property is to be avoided. Instead, it is necessary to use a relatively cheap surrogate or inference model to learn from the available data, and in conjunction with some form of design, guide experiments to minimize the number of new materials that need to be tested. Currently, the work in materials informatics is largely based on performing step 1, which requires finding a regressor that fits existing data to predict the best y. Advances in operations research and machine learning suggest that merely choosing the predictions out of the regressor is not the best way to select x when the predicted values have uncertainties 11 . This is relevant for materials design because relatively small data sets are often used to extrapolate to a vast unexplored search space 6 , which means that model uncertainties will be important. When we can estimate the errors associated with the y-values, it is typically better to choose the x that gives the best "expected improvement"-a value that depends on both the predicted y and the predicted error in our estimate of y. The advantages of maximizing the expected improvement are amplified when the function predicting y from x has local optima-local optima are difficult to escape by many optimization methods. Both Efficient Global Optimization (EGO) [11] [12] [13] and the closely related Knowledge Gradient (KG) algorithms 14 use information about prediction error to select the next x to test. Both
REF use a regressor-selector pair to locate the desired material in the least number of iterations by alternating between exploration and exploitation strategies.
Adaptive Strategies for Materials Design using Uncertainties