A method of detecting an invocation phrase for an automated assistant, the method implemented by one or more processors of a client device and comprising: receiving a stream of audio data frames that are based on output from one or more microphones of the client device; processing each of the audio data frames of the stream using a trained machine learning model to generate respective output indicating one or more corresponding probabilities of the presence of one or more corresponding invocation phonemes; determining, at a first instance, that the respective output, generated for a given audio data frame of the audio data frames, indicates that the given audio data frame fails to include any of the one or more corresponding invocation phonemes; in response to the determination at the first instance: using the given audio data frame to adapt a noise reduction filter; determining, at a second instance after the first instance, that at least one of one or more of the audio data frames received after the given audio data frame potentially includes at least one of the one or more corresponding invocation phonemes; in response to the determination at the second instance: generating filtered data frames based on processing of the one or more of the audio data frames using the noise reduction filter as adapted at least in part in response to the determination at the first instance; and determining whether the filtered data frames indicate presence of the invocation phrase based on processing the filtered data frames using the trained machine learning model, or an additional trained machine learning model; and in response to determining that the filtered data frames indicate presence of the invocation phrase: causing at least one function of the automated assistant to be activated. receiving a stream of audio data frames that are based on output from one or more microphones of the client device; processing each of the audio data frames of the stream using a trained machine learning model to generate respective output indicating one or more corresponding probabilities of the presence of one or more corresponding invocation phonemes; determining, at a first instance, that the respective output, generated for a given audio data frame of the audio data frames, indicates that the given audio data frame fails to include any of the one or more corresponding invocation phonemes; in response to the determination at the first instance: using the given audio data frame to adapt a noise reduction filter; determining, at a second instance after the first instance, that at least one of one or more of the audio data frames received after the given audio data frame potentially includes at least one of the one or more corresponding invocation phonemes; in response to the determination at the second instance: generating filtered data frames based on processing of the one or more of the audio data frames using the noise reduction filter as adapted at least in part in response to the determination at the first instance; and determining whether the filtered data frames indicate presence of the invocation phrase based on processing the filtered data frames using the trained machine learning model, or an additional trained machine learning model; and generating filtered data frames based on processing of the one or more of the audio data frames using the noise reduction filter as adapted at least in part in response to the determination at the first instance; and determining whether the filtered data frames indicate presence of the invocation phrase based on processing the filtered data frames using the trained machine learning model, or an additional trained machine learning model; and in response to determining that the filtered data frames indicate presence of the invocation phrase: causing at least one function of the automated assistant to be activated.