A computing device, comprising: one or more processors; and one or more non-transitory computer-readable media that store: one or more applications (120a-c) implemented by the one or more processors; one or more machine-learned models (132a-c); and instructions that, when executed by the one or more processors, cause the computing device to implement an on-device machine learning platform that performs operations, the operations comprising: receiving a prediction plan and model parameters from a cloud server, wherein a prediction plan comprises instructions for running the model to obtain inferences; receiving input data from a first application of the one or more applications (120a-c) via a prediction application programming interface; employing at least a first machine-learned model of the one or more machine-learned models to generate at least one inference based at least in part on the input data and according to the prediction plan and model parameters; and providing the at least one inference generated by the first machine-learned model to the first application via the prediction application programming interface. one or more processors; and one or more non-transitory computer-readable media that store: one or more applications (120a-c) implemented by the one or more processors; one or more machine-learned models (132a-c); and instructions that, when executed by the one or more processors, cause the computing device to implement an on-device machine learning platform that performs operations, the operations comprising: receiving a prediction plan and model parameters from a cloud server, wherein a prediction plan comprises instructions for running the model to obtain inferences; receiving input data from a first application of the one or more applications (120a-c) via a prediction application programming interface; employing at least a first machine-learned model of the one or more machine-learned models to generate at least one inference based at least in part on the input data and according to the prediction plan and model parameters; and providing the at least one inference generated by the first machine-learned model to the first application via the prediction application programming interface. one or more applications (120a-c) implemented by the one or more processors; one or more machine-learned models (132a-c); and instructions that, when executed by the one or more processors, cause the computing device to implement an on-device machine learning platform that performs operations, the operations comprising: receiving a prediction plan and model parameters from a cloud server, wherein a prediction plan comprises instructions for running the model to obtain inferences; receiving input data from a first application of the one or more applications (120a-c) via a prediction application programming interface; employing at least a first machine-learned model of the one or more machine-learned models to generate at least one inference based at least in part on the input data and according to the prediction plan and model parameters; and providing the at least one inference generated by the first machine-learned model to the first application via the prediction application programming interface. receiving a prediction plan and model parameters from a cloud server, wherein a prediction plan comprises instructions for running the model to obtain inferences; receiving input data from a first application of the one or more applications (120a-c) via a prediction application programming interface; employing at least a first machine-learned model of the one or more machine-learned models to generate at least one inference based at least in part on the input data and according to the prediction plan and model parameters; and providing the at least one inference generated by the first machine-learned model to the first application via the prediction application programming interface.