A method for valuating training samples, the method comprising: obtaining, at data processing hardware, a set of training samples; and during each of a plurality of training iterations: sampling, by the data processing hardware, a batch of training samples from the set of training samples; for each training sample in the batch of training samples, determining, by the data processing hardware, using a data value estimator, a selection probability for the training sample based on estimator parameter values of the data value estimator, wherein the selection probability for each training sample is determined based on a priority value assigned to each training sample based on sample quality of each training sample; selecting, by the data processing hardware, based on the selection probabilities of each training sample, a subset of training samples from the batch of training samples; determining, by the data processing hardware, using a predictor model with the subset of training samples, performance measurements, wherein determining the performance measurements using the predictor model comprises determining loss data by a loss function; adjusting, by the data processing hardware, model parameter values of the predictor model based on the performance measurements; and updating, by the data processing hardware, the estimator parameter values of the data value estimator based on the performance measurements, wherein updating the estimator parameter values of the data value estimator based on the performance measurements comprises: determining, from the loss data, a reinforcement signal; and updating estimator parameter values of the data value estimator based on the reinforcement signal. obtaining, at data processing hardware, a set of training samples; and during each of a plurality of training iterations: sampling, by the data processing hardware, a batch of training samples from the set of training samples; for each training sample in the batch of training samples, determining, by the data processing hardware, using a data value estimator, a selection probability for the training sample based on estimator parameter values of the data value estimator, wherein the selection probability for each training sample is determined based on a priority value assigned to each training sample based on sample quality of each training sample; selecting, by the data processing hardware, based on the selection probabilities of each training sample, a subset of training samples from the batch of training samples; determining, by the data processing hardware, using a predictor model with the subset of training samples, performance measurements, wherein determining the performance measurements using the predictor model comprises determining loss data by a loss function; adjusting, by the data processing hardware, model parameter values of the predictor model based on the performance measurements; and updating, by the data processing hardware, the estimator parameter values of the data value estimator based on the performance measurements, wherein updating the estimator parameter values of the data value estimator based on the performance measurements comprises: determining, from the loss data, a reinforcement signal; and updating estimator parameter values of the data value estimator based on the reinforcement signal. sampling, by the data processing hardware, a batch of training samples from the set of training samples; for each training sample in the batch of training samples, determining, by the data processing hardware, using a data value estimator, a selection probability for the training sample based on estimator parameter values of the data value estimator, wherein the selection probability for each training sample is determined based on a priority value assigned to each training sample based on sample quality of each training sample; selecting, by the data processing hardware, based on the selection probabilities of each training sample, a subset of training samples from the batch of training samples; determining, by the data processing hardware, using a predictor model with the subset of training samples, performance measurements, wherein determining the performance measurements using the predictor model comprises determining loss data by a loss function; adjusting, by the data processing hardware, model parameter values of the predictor model based on the performance measurements; and updating, by the data processing hardware, the estimator parameter values of the data value estimator based on the performance measurements, wherein updating the estimator parameter values of the data value estimator based on the performance measurements comprises: determining, from the loss data, a reinforcement signal; and updating estimator parameter values of the data value estimator based on the reinforcement signal. determining, from the loss data, a reinforcement signal; and updating estimator parameter values of the data value estimator based on the reinforcement signal.