A computer-implemented method of providing machine-learned models for detecting a state of an environment external to a vehicle, the method comprising: providing, at one or more computing devices of a vehicle map service system, a remote map service system or a remote vehicle service system, a machine-learned model library comprising a plurality of machine-learned models trained to generate semantic observations based on sensor data associated with the vehicle, wherein the semantic observations comprise information associated with detection or recognition of one or more entities comprising one or more objects, one or more scenes, or one or more events, wherein each machine-learned model of the plurality of machine-learned models is associated with one or more configurations supported by each machine-learned model; receiving, at the one or more computing devices, from a remote computing device, one or more requests for a machine-learned model from the machine-learned model library; providing, from the one or more computing devices, based at least in part on the one or more requests, the machine-learned model to the remote computing device; receiving, at the one or more computing devices and from the remote computing device, an updated machine-learned model generated by training the machine-learned model using at least one of the remote computing device or one or more additional computing devices; and storing, at the one or more computing devices, the updated machine-learned model in the machine-learned model library, wherein the one or more configurations supported by each machine-learned model of the plurality of machine-learned models comprises at least one of a vehicle configuration or a sensor configuration supported by each machine-learned model, wherein the sensor configuration comprises data associated with one or more types of sensors, one or more locations of the sensors on the vehicle, or a range of the sensors, and wherein the vehicle configuration comprises data associated with one or more types of the vehicle, one or more physical dimensions of the vehicle, or compatibility of one or more sensor configurations with the vehicle. providing, at one or more computing devices of a vehicle map service system, a remote map service system or a remote vehicle service system, a machine-learned model library comprising a plurality of machine-learned models trained to generate semantic observations based on sensor data associated with the vehicle, wherein the semantic observations comprise information associated with detection or recognition of one or more entities comprising one or more objects, one or more scenes, or one or more events, wherein each machine-learned model of the plurality of machine-learned models is associated with one or more configurations supported by each machine-learned model; receiving, at the one or more computing devices, from a remote computing device, one or more requests for a machine-learned model from the machine-learned model library; providing, from the one or more computing devices, based at least in part on the one or more requests, the machine-learned model to the remote computing device; receiving, at the one or more computing devices and from the remote computing device, an updated machine-learned model generated by training the machine-learned model using at least one of the remote computing device or one or more additional computing devices; and storing, at the one or more computing devices, the updated machine-learned model in the machine-learned model library, wherein the one or more configurations supported by each machine-learned model of the plurality of machine-learned models comprises at least one of a vehicle configuration or a sensor configuration supported by each machine-learned model, wherein the sensor configuration comprises data associated with one or more types of sensors, one or more locations of the sensors on the vehicle, or a range of the sensors, and wherein the vehicle configuration comprises data associated with one or more types of the vehicle, one or more physical dimensions of the vehicle, or compatibility of one or more sensor configurations with the vehicle.