A method implemented by one or more processors, the method comprising: causing a client device to generate a plurality of performance measures for a machine learning model based on a plurality of testing instances for the ML model, wherein the client device has a first set of device characteristics and on-device memory storing the ML model and the plurality of testing instances for the ML model, and wherein causing the client device to generate the plurality of performance measures based on a given testing instance, of the plurality of testing instances, comprises: causing the client device to process, using the ML model, testing instance input for the given testing instance to generate output; and causing the client device to generate the plurality of performance measures based on the processing of the testing instance input of the given testing instance; determining, based on the plurality of performance measures, whether to activate or keep active the ML model at the client device; and in response to determining to activate or keep active the ML model at the client device: causing the ML model to be activated or kept active for use locally at the client device; causing one or more additional client devices that also have the first set of device characteristics to activate or keep active a corresponding instance of the ML model. causing a client device to generate a plurality of performance measures for a machine learning model based on a plurality of testing instances for the ML model, wherein the client device has a first set of device characteristics and on-device memory storing the ML model and the plurality of testing instances for the ML model, and wherein causing the client device to generate the plurality of performance measures based on a given testing instance, of the plurality of testing instances, comprises: causing the client device to process, using the ML model, testing instance input for the given testing instance to generate output; and causing the client device to generate the plurality of performance measures based on the processing of the testing instance input of the given testing instance; wherein the client device has a first set of device characteristics and on-device memory storing the ML model and the plurality of testing instances for the ML model, and wherein causing the client device to generate the plurality of performance measures based on a given testing instance, of the plurality of testing instances, comprises: causing the client device to process, using the ML model, testing instance input for the given testing instance to generate output; and causing the client device to generate the plurality of performance measures based on the processing of the testing instance input of the given testing instance; causing the client device to process, using the ML model, testing instance input for the given testing instance to generate output; and causing the client device to generate the plurality of performance measures based on the processing of the testing instance input of the given testing instance; determining, based on the plurality of performance measures, whether to activate or keep active the ML model at the client device; and in response to determining to activate or keep active the ML model at the client device: causing the ML model to be activated or kept active for use locally at the client device; causing one or more additional client devices that also have the first set of device characteristics to activate or keep active a corresponding instance of the ML model. causing the ML model to be activated or kept active for use locally at the client device; causing one or more additional client devices that also have the first set of device characteristics to activate or keep active a corresponding instance of the ML model.