A method implemented by one or more processors, the method comprising: receiving, from a client device and via a network, an automated assistant request that includes: a text independent, TI, speaker embedding for a particular user of the client device, and audio data that captures spoken input of the particular user, wherein the audio data is captured via one or more microphones of the client device; determining that the TI speaker embedding was generated using an outdated version of a TI speaker recognition model; in response to determining that the speaker embedding was generated using the outdated version of the TI speaker recognition model: processing at least a portion of the audio data using the outdated version of the TI speaker recognition model to generate TI output; determining whether the particular user spoke the spoken input by comparing the TI output with the speaker embedding of the particular user; in response to determining the particular user spoke the spoken input: performing one or more actions that are based on the audio data; processing previous audio data, that captures previous spoken input of the particular user, using an updated version of the TI speaker recognition model to generate an updated speaker embedding; and transmitting, to the client device, the updated speaker embedding for the particular user to cause the client device to locally store the updated speaker embedding for transmission with future automated assistant requests. receiving, from a client device and via a network, an automated assistant request that includes: a text independent, TI, speaker embedding for a particular user of the client device, and audio data that captures spoken input of the particular user, wherein the audio data is captured via one or more microphones of the client device; a text independent, TI, speaker embedding for a particular user of the client device, and audio data that captures spoken input of the particular user, wherein the audio data is captured via one or more microphones of the client device; determining that the TI speaker embedding was generated using an outdated version of a TI speaker recognition model; in response to determining that the speaker embedding was generated using the outdated version of the TI speaker recognition model: processing at least a portion of the audio data using the outdated version of the TI speaker recognition model to generate TI output; determining whether the particular user spoke the spoken input by comparing the TI output with the speaker embedding of the particular user; processing at least a portion of the audio data using the outdated version of the TI speaker recognition model to generate TI output; determining whether the particular user spoke the spoken input by comparing the TI output with the speaker embedding of the particular user; in response to determining the particular user spoke the spoken input: performing one or more actions that are based on the audio data; processing previous audio data, that captures previous spoken input of the particular user, using an updated version of the TI speaker recognition model to generate an updated speaker embedding; and transmitting, to the client device, the updated speaker embedding for the particular user to cause the client device to locally store the updated speaker embedding for transmission with future automated assistant requests. performing one or more actions that are based on the audio data; processing previous audio data, that captures previous spoken input of the particular user, using an updated version of the TI speaker recognition model to generate an updated speaker embedding; and transmitting, to the client device, the updated speaker embedding for the particular user to cause the client device to locally store the updated speaker embedding for transmission with future automated assistant requests.