Systems and methods for active learning from sparse training data

A method for active learning using sparse training data can include training a machine learning model using less than ten first training data points to generate a candidate machine learning model. The method can include performing a Monte Carlo process to sample one or more first outputs of the candidate machine learning model. The method can include testing the one or more first outputs to determine if each of the one or more first outputs satisfy a respective convergence condition. The method can include, responsive to at least one first output not satisfying the respective convergence condition, training the candidate machine learning model using at least one second training data point corresponding to the at least one first output. The method can include, responsive to the one or more first outputs each satisfying the respective convergence condition, outputting the candidate machine learning model.

TECHNICAL FIELD

The present invention relates generally to systems and methods of training models, such as machine learning models. Certain embodiments relate to active learning from sparse training data.

BACKGROUND

Machine learning approaches, such as to train a neural network, may use large training datasets. For example, a dataset may involve hundreds of thousands to millions of data points. This can limit the ability to effectively generate trained models for situations where such datasets are not available or difficult to generate (e.g., if the dataset results from performing a real-world experiment).

SUMMARY

At least one aspect relates to a system. The system can include one or more processors configured to train a machine learning model using a first training data point to generate a candidate machine learning model, sample a first output of the candidate machine learning model, determine whether the first output satisfies a convergence condition, modify, responsive to the first output not satisfying the convergence condition, the candidate machine learning model using a second training data point corresponding to the first output, and output, responsive to the first output satisfying the convergence condition, the candidate machine learning model.

At least one aspect relates to a method for machine learning using sparse training data. The method can include training, by one or more processors, a machine learning model using a first training data point to generate a candidate machine learning model, sampling, by the one or more processors, a first output of the candidate machine learning model, determining, by the one or more processors, whether the first output satisfies a convergence condition, responsive to the first output not satisfying the convergence condition, modifying, by the one or more processors, the candidate machine learning model using a second training data point corresponding to the first output, and responsive to the first output satisfying the convergence condition, outputting, by the one or more processors, the candidate machine learning model.

At least one aspect relates to a method. The method can include training a machine learning model using less than ten first training data points to generate a candidate machine learning model, performing a Monte Carlo process to sample one or more first outputs of the candidate machine learning model, testing the one or more first outputs to determine if each of the one or more first outputs satisfy a respective convergence condition, responsive to at least one first output not satisfying the respective convergence condition, training the candidate machine learning model using at least one second training data point corresponding to the at least one first output, and responsive to the one or more first outputs each satisfying the respective convergence condition, outputting the candidate machine learning model.

DETAILED DESCRIPTION

Embodiments described herein relate generally to methods of active learning using sparse training data. Effectively training machine learning models, such as neural networks, can require very large datasets (e.g., on the order of hundreds of thousands of data points). Some approaches, such as query by committee, can reduce the number of data points required for training, but do not address the underlying problem of generating the large dataset itself. This may particularly limit the ability to train models where the dataset is based on performing real-world experiments or performing computationally intensive simulations of a system, where it may be impractical to prepare a sufficient number training data points before training a machine learning model.

Systems and methods performed in accordance with the present solution can enable active learning from sparse training data, such as to perform machine learning even with a relatively sparse dataset generated from experimental data. For example, the active learning can train a neural network with a small dataset (e.g., dataset of one point), query a sample of validation data points, and evaluate the performance of the neural network using the sampled data points. Responsive to the evaluation, poorly performing data points can be iteratively added to the training dataset until the evaluation of the neural network satisfies a convergence condition. Systems and methods in accordance with the present solution can reduce the number of data points needed to perform the machine learning to the order of a few hundred, such as by focusing training of the model on portions of the sample space for which the model initially performs poorly. This can enable models to be trained more quickly or with lesser computational resources, and to achieve target performance without requiring large amounts of experimental data to be generated.

FIG.1depicts a system100for active learning from sparse training data. The system100includes one or more processors104and memory108, which can be implemented as one or more processing circuits. The processor104may be a general purpose or specific purpose processor, an application specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable processing components. The processor104may be configured to execute computer code or instructions stored in memory108(e.g., fuzzy logic, etc.) or received from other computer readable media (e.g., CDROM, network storage, a remote server, etc.) to perform one or more of the processes described herein. The memory108may include one or more data storage devices (e.g., memory units, memory devices, computer-readable storage media, etc.) configured to store data, computer code, executable instructions, or other forms of computer-readable information. The memory108may include random access memory (RAM), read-only memory (ROM), hard drive storage, temporary storage, non-volatile memory, flash memory, optical memory, or any other suitable memory for storing software objects and/or computer instructions. The memory108may include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present disclosure. The memory108may be communicably connected to the processor104and may include computer code for executing (e.g., by processor104) one or more of the processes described herein. The memory108can include various modules (e.g., circuits, engines) for completing processes described herein. The one or more processors104and memory108may include various distributed components that may be communicatively coupled by wired or wireless connections; for example, various portions of system100may be implemented using one or more client devices remote from one or more server devices. The memory108can include various software or firmware engines, modules, scripts, databases, or other computer code to implement the various components described herein, including the training database112, machine learning model116, model sampler120, outputter124, convergence condition128, or features or functions thereof. Various aspects of the system100may be distributed across one or more devices, which may be connected using wired or wireless networks.

The system100can include a training database112. The training database112can include training data points that can be used to train a model (e.g., model116). For example, the training data points can be represented using a data structure that includes one or more data point values for each of one or more parameters that are to be provided as inputs to the model, and an output value (e.g., free energy value) associated with the data point values. The training data points can represent material properties, environmental properties, or various other properties of the models or systems to be optimized. For a given model to be trained, the training database112can include training data points up to the order of thousands (e.g., rather than several tens or hundreds of thousands or more).

The training database112can correspond to experimental values, values from simulations, or combinations thereof. For example, the training database112can include experimental data points that include input values for one or more input parameters to a system to be represented by the model and output values for one or more outputs corresponding to the input values. For example, a training data point can include inputs such as an identifier of a material, a measured temperature of the material, a measured pressure of the material, and outputs such as a free energy of the material measured at the measured temperature and the measured pressure. The output may include a force associated with the component or system to be modeled (e.g., a force in eV/Angstrom).

The system100can include a model116. The model116can be a machine learning model for which the system100is to perform supervised learning. The model116can be any of a variety of machine learning models that can be trained using supervised learning, including but not limited to neural networks, regression models, support vector machines, decision trees, random forest models, and naïve Bayes models. The system100can train the model116by providing training data points from the training database112as input to the model, causing the model116to generate a candidate output using the input, and modifying various components of the model116to satisfy a convergence condition (e.g., until the candidate output changes less than a threshold amount between subsequent iterations—this may be different than convergence condition128described below).

The model116can include a neural network, which may include an input layer, one or more hidden layers, and an output layer. Each layer can include nodes that receive respective inputs (which may be from a previous layer) and generate outputs (which may be provided to a subsequent layer). The nodes of the hidden layers can have weights and biases that the nodes apply to the received inputs to generate output. The system100can train the neural network by applying an optimization function to the neural network to cause the weights and biases to be adjusted until an optimization condition is satisfied.

The model116can be trained using a relatively small number of training data points. For example, the system100can retrieve one or more first training data points from the training database112and provide the one or more first training data points as input to the model116(e.g., in a first iteration). A number of the one or more first training data points can be less than one hundred, less than fifty, less than twenty, less than ten, less than five, or one.

The system100may group at least some of the training data points of the training database112into training points (which may be used to train the model116) and validation data points (which may be used by model sampler120as described below to select inputs to use to sample the trained model116, based on which sampled outputs of the trained model116can be compared to outputs of the validation data points to evaluate convergence condition128).

The model sampler120can sample the trained model116to retrieve one or more outputs of the trained model116. The model sampler120can perform a random or pseudo-random process to retrieve the one or more outputs. The model sampler120can sample the trained model116by identifying one or more inputs to provide to the trained model116so that the trained model116provides the one or more outputs responsive to the one or more inputs. For example, the model sampler120can identify the one or more inputs using input values of training data points from the training database112(which may not necessarily include the training data points used to train the model116) and query the trained model116to receive outputs corresponding to the input values of training data points from the training database112.

The model sampler120can perform a Monte Carlo process to retrieve the one or more outputs. For example, the model sampler120can identify a domain of inputs that includes inputs of the training data points of the training database112. The model sampler120can randomly select one or more inputs from the domain of inputs, such as by selecting the one or more inputs using a probability distribution of the domain of inputs or based on a distance from inputs used to sample the model116in previous iterations.

The outputter124can determine whether to output the trained model116or continue training of the trained model116by evaluating convergence condition128using the sampled outputs of the trained model116. The convergence condition128may include a condition based on a difference between the sampled output(s) of the trained model116and the outputs of the training data points having the inputs used by the model sampler120to sample the trained model116.

For example, the outputter124can compare the one or more outputs sampled from the trained model116to corresponding one or more outputs of the training data points, determine a difference based on the comparison, compare the difference to a threshold represented by the convergence condition128, and determine to output the trained model116responsive to the difference being less than the threshold. The outputter124can compare the one or more outputs to a ground truth value, such as a predetermined output value (e.g., predetermined based on user input, experimental data, or output of a different model used as a baseline for comparison).

The outputter124can compare the one or more outputs with a reference output or reference data point (e.g., synthetic data point; synthetic output) generated by a reference high-fidelity model. The reference high-fidelity model can be a model that is more computationally expensive for generating output than the model116(and, in turn, may have greater accuracy). The reference high-fidelity model can include at least one of a density functional theory model, a quantum Monte Carlo model, or a coupled clusters model.

The outputter124can determine the difference in various manners, such as by determining a Euclidean norm between the one or more outputs outputted by the trained model116and the corresponding one or more outputs of the training data points. Responsive to the trained model116satisfying the convergence condition128, the outputter124can output the trained model116. The outputter124can evaluate the convergence condition128by determining a mean absolute error between the sampled output from the trained model116and the actual values of the training data points corresponding to the sampled output.

For example, the training data points may include a data structure that includes temperature and pressure as input values and free energy as an output value. The output of the model116(e.g., as sampled by the model sampler120) can include free energy values that correspond to input values of temperature and pressure. The system100can evaluate the convergence condition by comparing, for each sampled output, the free energy value outputted by the model116for a particular temperature and particular pressure to the corresponding free energy value of the training data point having the same particular temperature and particular pressure.

Responsive to the trained model116not satisfying the convergence condition128, the system100can continue to train the model116. For example, the system100can identify the training data points of the training database112that correspond to the sampled outputs of the trained model116that the model sampler120sampled in order for the outputter124to evaluate the convergence condition128. The system100can provide the identified training data points as input to the model116to train the model116. The system100can also include the one or more first training data points as input to the model116(in addition to the identified training data points) to train the model116. For example, the system100can provide training data points having the input values of the points sampled by model sampler120and corresponding output values as training data points to the model116.

By providing the identified training data points as input to the model116, the system100can efficiently cause the model116to more effectively be trained in portions of the space over which the model116is to be trained for which it had performed poorly, while maintaining a relatively small number of data points used to train the model116. The system100can iteratively proceed through training the model116, sampling the model116(e.g., using model sampler120), determining whether the model116has converged (e.g., based on evaluating sampled outputs of the model116using the convergence condition128), and outputting the model116if the model116has converged (e.g., if the sampled data points of the model116perform well against the corresponding outputs of the training data of the training database112) or using the sampled data points to further train the model116if the model116has not converged (e.g., if the sampled data points did not perform well against corresponding outputs of the training data of the training database112).

FIGS.2A-2Bdepict an example of performance of a model trained in accordance with present disclosure. As depicted in chart200, a mean absolute error between output of the model and the actual values of the training data points can decrease below 0.01 with less than 25 training data points, towards 0.003 with 300 training data points. Chart250depicts performance of a model trained using 300 data points. The model performed with close correspondence between free energy values as determined by the model compared to free energy values of the training data points used to train the model.

FIG.3depicts a method300for active learning. The method300can be performed using various systems described herein, including the system100. Various steps in the method300may be repeated, omitted, performed in various orders, or otherwise modified.

At305, a machine learning model can be trained to generate a candidate machine learning model. The machine learning model can be trained using a relatively small number of training data points (e.g., sparse training data). For example, the machine learning model can be trained (e.g., in a first iteration) using at least one and less than or equal to ten training data points. Each training data point can be a data structure that includes one or more input values (e.g., material identifiers, temperature, pressure, or other characteristics of the material or component to be modeled) associated with one or more output values (e.g., free energy). The training data points (e.g., one or more first training data points for the first iteration) can be provided as input to the machine learning model to train the machine learning model to generate the candidate machine learning model.

The machine learning model may include a neural network that is trained by providing the training data point(s) as input to an input layer of the neural network, and using an optimization function (e.g., cost function) to adjust weights and biases of one or more hidden layers of the neural network to generate the candidate machine learning model as a candidate neural network (e.g., a candidate neural network in which a difference between outputs of the candidate neural network and outputs of the training data are minimized).

At310, an output of the candidate machine learning model can be sampled. For example, in the first iteration, a first one or more candidate outputs of the candidate machine learning model can be sampled. The output can be sampled by providing input values corresponding to training data points (which are associated with predetermined output values, such as output values from experimentation or simulation) to the candidate machine learning model to receive candidate outputs responsive to the input values.

The output of the candidate machine learning model can be sampled by a random or pseudo-random. For example, a Monte Carlo process can be applied to sample the candidate machine learning model. The Monte Carlo process can be applied to randomly select training data points (e.g., input values of training data points) and to sample the candidate machine learning model using the randomly selected training data points. For example, the input value(s) of the randomly selected training data point(s) can be applied as input to the candidate machine learning model to generate sampled output from the candidate machine learning model.

At315, it can be determined whether the sampled output (e.g., first output from a first iteration) satisfies a convergence condition. The convergence condition can represent the performance of the candidate machine learning model relative to the training data that the candidate machine learning model is to model. The sampled output can be evaluated by comparing the sampled output to reference values, such as reference values determined by a reference high-fidelity model.

For example, the convergence condition can be based on a threshold difference between the sampled output and the training data output corresponding to the sampled output. The convergence condition can be evaluated by determining a difference between the sampled output and the training data output corresponding to the sampled output, comparing the difference to the threshold difference, and determining that the sampled output satisfies the convergence condition responsive to the difference being less than or equal to the threshold difference. The difference can be determined as a norm between the sampled output and the training data output (e.g., determining a difference between each sampled output and the respective training data output corresponding to the same input value(s) as the sampled output, and aggregating the differences across the comparisons, such as to perform a Euclidean norm). The difference can be determined as a mean absolute error. The difference being greater than the threshold difference, such that the convergence condition is not satisfied, can indicate that the candidate machine learning model does not perform sufficiently well in representing the behavior of the system to be modeled, such that further training of the machine learning model should be performed.

The convergence condition may be evaluated for each sample data point—training data point pair, such as to identify particular data points for which the model did not perform effectively. For example, five sampled outputs may be sampled from the candidate machine learning model, and three of the five sampled outputs can be determined to not satisfy a respective convergence condition responsive to differences between those three sampled outputs and the corresponding training data outputs being less than respective threshold differences.

At320, the candidate machine learning model can be modified responsive to the sampled output not satisfying the convergence condition (e.g., the candidate machine learning model not performing to target performance levels relative to the training data representing the system to be modeled). The candidate machine learning model can be modified by performing further training of the candidate machine learning model (e.g., in a second or additional iteration). Modifying the candidate machine learning model can include reinitializing the machine learning model to an untrained state and training the untrained machine learning model using training data points corresponding to the sampled output(s).

The candidate machine learning model can be modified by training the candidate machine learning model using at least one training data point corresponding to the sampled output used to evaluate the convergence condition (e.g., in a previous iteration). For example, the training data points having input values that were used to sample the candidate machine learning model can be provided as input to the candidate machine learning model to train the candidate machine learning model. These training data points may represent data points that the model failed to model effectively, such that retraining using these data points (which may not have previously been used to train the model) can allow the model to be retrained in a targeted manner. The training data points having input values that were used to sample the candidate machine learning model can be provided as input in addition to or alternatively to the one or more first training data points used to train the candidate machine learning model in the previous iteration. The training data points corresponding to one or more particular sampled outputs that did not satisfy respective convergence conditions may be used to retrain the candidate machine learning model (e.g., while other training data points corresponding to one or more particular sampled outputs that did satisfy respective convergence conditions are not used for retraining).

At325, responsive to the sampled output satisfying the convergence condition, the candidate machine learning model can be outputted. The sampled output satisfying the convergence condition can indicate that the candidate machine learning model has reached appropriate performance, such as appropriate performance relative to the system to be modeled (which may be represented by the training data points).

Definitions

The term “coupled,” as used herein, means the joining of two members directly or indirectly to one another. Such joining may be stationary (e.g., permanent or fixed) or moveable (e.g., removable or releasable). Such joining may be achieved with the two members coupled directly to each other, with the two members coupled to each other using a separate intervening member and any additional intermediate members coupled with one another, or with the two members coupled to each other using an intervening member that is integrally formed as a single unitary body with one of the two members. Such members may be coupled mechanically, electrically, and/or fluidly.