Patent Description:
Advances in technology have resulted in smaller and more powerful computing devices. For example, there currently exist a variety of portable personal computing devices, including wireless telephones such as mobile and smart phones, tablets and laptop computers that are small, lightweight, and easily carried by users. These devices can communicate voice and data packets over wireless networks. Further, many such devices incorporate additional functionality such as a digital still camera, a digital video camera, a digital recorder, and an audio file player. Also, such devices can process executable instructions, including software applications, such as a web browser application, that can be used to access the Internet. As such, these devices can include significant computing capabilities.

Such computing devices often incorporate functionality to receive an audio signal from one or more microphones. For example, the audio signal may represent user speech captured by the microphones, external sounds captured by the microphones, or a combination thereof. To illustrate, a headset device may include self-voice activity detection in an effort to distinguish between the user's speech (e.g., speech spoken by the person wearing the headset) and speech originating from other sources. For example, when a system including a headset device supports keyword activation, self-voice activity detection can reduce "false alarms" in which activation of one or more components or operations is initiated based on speech originating from nearby people (referred to as "non-user speech"). Reducing such false alarms improves power consumption efficiency of the device. However, performing audio signal processing to distinguish between user speech and non-user speech also consumes power, and conventional techniques to improve the accuracy of the device in distinguishing between user speech and non-user speech also tend to increase the power consumption and processing resource requirements of the device. <CIT> discloses a low-power voice command detection method, in which captured sound is analyzed to determine whether it fulfills a number of criteria. The processing included a number of steps, each of which is more complex and power demanding. A dynamically adjusted threshold is applied to determine whether to proceed to a subsequent step. In one example, the sound may be captured by a first and a second microphone, and the sound captured by each microphone is compared to determine whether the sound includes user speech or is simply ambient noise. For example, if the sound pressure and phase difference between the sound captured by both microphones is less than a prescribed threshold, it can be determined that the sound is not a valid command, and therefore no further processing is performed. <CIT> relates to a noise cancelling headset for voice communications that includes a microphone at each of the user's ears and a voice microphone. In one example, a voice activity detector produces a voice activation detection signal based on the levels of first and second audio signals captured by the microphones. If the level of one or both signals have a level above a threshold, which indicates that the sound originates from close to the microphone, and the sound level from the two microphones is substantially the same, voice detection is indicated. <CIT> relates to a headset and associated method for determining that a headset user is speaking, in which signals are received from first and second microphones. The two signals are summed to generate a primary signal, and the difference between the signals forms a reference signal. A determination that the user is speaking is made on the basis of a comparison between the primary signal and the reference signal.

The scope of the present invention is defined by the scope of the appended claims. Any embodiments that do not fall under the scope of the claims are examples which are useful for understanding the invention, but do not form a part of the invention.

Other aspects, advantages, and features of the present disclosure will become apparent after review of the entire application, including the following sections: Brief Description of the Drawings, Detailed Description, and the Claims.

Self-voice activity detection ("SVAD") that reduces "false alarms," in which activation of one or more components or operations results from non-user speech, can improve power consumption efficiency of the device by preventing activation of such components or operations when false alarms are detected. However, conventional audio signal processing techniques to improve SVAD accuracy also increase power consumption and processing resources of the device while performing the improved-accuracy techniques. Since SVAD processing is typically continually operating, even while the device is in a low-power or sleep mode, the reduction in power consumption due to reducing false alarms using conventional SVAD techniques can be partially or fully offset by increased power consumption associated with the SVAD processing itself.

Systems and methods of self-voice activity detection using a dynamic classifier are disclosed. For example, in a headset implementation, audio signals may be received from a first microphone that is positioned to capture the user's voice and from a second microphone that is positioned to capture external sounds, such as to perform noise reduction and echo cancellation. The audio signals may be processed to extract frequency domain feature sets including interaural phase differences ("IPDs") and interaural intensity differences ("IIDs").

The dynamic classifier processes the extracted frequency domain feature sets and generates an output indicating classification of the feature sets. The dynamic classifier may perform adaptive clustering of the feature data and adjustment of a decision boundary between the two most discriminative categories of the feature data space to distinguish between feature sets corresponding to user voice activity and feature sets corresponding to other audio activity. In an illustrative example, the dynamic classifier is implemented using self-organizing maps.

The dynamic classifier enables discrimination using the extracted feature sets to actively respond and adapt to various conditions, such as: environmental conditions in highly nonstationary situations; mismatched microphones; changes in user headset fitting; different user head-related transfer functions ("HRTFs"); direction-of-arrival ("DOA") tracking of non-user signals; noise floor, bias, and sensitivities of microphones across the frequency spectrum; or a combination thereof. In some implementations, the dynamic classifier enables adaptive feature mapping capable of responding to such variations and reducing or minimizing a number of thresholding parameters used and an amount of headset tuning by customers. In some implementations, the dynamic classifier enables effective discrimination between user voice activity and other audio activity with high accuracy under varying conditions and with relatively low power consumption as compared to conventional SVAD systems that provide comparable accuracy.

Particular aspects of the present disclosure are described below with reference to the drawings. In the description, common features are designated by common reference numbers. As used herein, various terminology is used for the purpose of describing particular implementations only and is not intended to be limiting of implementations. For example, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, some features described herein are singular in some implementations and plural in other implementations. To illustrate, <FIG> depicts a device <NUM> including one or more processors ("processor(s)" <NUM> of <FIG>), which indicates that in some implementations the device <NUM> includes a single processor <NUM> and in other implementations the device <NUM> includes multiple processors <NUM>. For ease of reference herein, such features are generally introduced as "one or more" features and are subsequently referred to in the singular unless aspects related to multiple of the features are being described.

It may be further understood that the terms "comprise," "comprises," and "comprising" may be used interchangeably with "include," "includes," or "including. " Additionally, it will be understood that the term "wherein" may be used interchangeably with "where. " As used herein, "exemplary" may indicate an example, an implementation, and/or an aspect, and should not be construed as limiting or as indicating a preference or a preferred implementation. As used herein, an ordinal term (e.g., "first," "second," "third," etc.) used to modify an element, such as a structure, a component, an operation, etc., does not by itself indicate any priority or order of the element with respect to another element, but rather merely distinguishes the element from another element having a same name (but for use of the ordinal term). As used herein, the term "set" refers to one or more of a particular element, and the term "plurality" refers to multiple (e.g., two or more) of a particular element.

As used herein, "coupled" may include "communicatively coupled," "electrically coupled," or "physically coupled," and may also (or alternatively) include any combinations thereof. Two devices (or components) may be coupled (e.g., communicatively coupled, electrically coupled, or physically coupled) directly or indirectly via one or more other devices, components, wires, buses, networks (e.g., a wired network, a wireless network, or a combination thereof), etc. Two devices (or components) that are electrically coupled may be included in the same device or in different devices and may be connected via electronics, one or more connectors, or inductive coupling, as illustrative, non-limiting examples. In some implementations, two devices (or components) that are communicatively coupled, such as in electrical communication, may send and receive signals (e.g., digital signals or analog signals) directly or indirectly, via one or more wires, buses, networks, etc. As used herein, "directly coupled" may include two devices that are coupled (e.g., communicatively coupled, electrically coupled, or physically coupled) without intervening components.

In the present disclosure, terms such as "determining," "calculating," "estimating," "shifting," "adjusting," etc. may be used to describe how one or more operations are performed. It should be noted that such terms are not to be construed as limiting and other techniques may be utilized to perform similar operations. Additionally, as referred to herein, "generating," "calculating," "estimating," "using," "selecting," "accessing," and "determining" may be used interchangeably. For example, "generating," "calculating," "estimating," or "determining" a parameter (or a signal) may refer to actively generating, estimating, calculating, or determining the parameter (or the signal) or may refer to using, selecting, or accessing the parameter (or signal) that is already generated, such as by another component or device.

Referring to <FIG>, a particular illustrative aspect of a system configured to perform self-voice activity detection using a dynamic classifier is disclosed and generally designated <NUM>. The system <NUM> includes a device <NUM> that is coupled to a first microphone <NUM>, a second microphone <NUM>, and a second device <NUM>. The device <NUM> is configured to perform self-voice activity detection of sounds captured by the microphones <NUM>, <NUM> using a dynamic classifier <NUM>. To illustrate, in an implementation in which the device <NUM> corresponds to a headset, the first microphone <NUM> (e.g., a "primary" microphone) may be configured to primarily capture utterances of a user of the device <NUM>, such as microphone positioned proximate to the mouth of a wearer of the device <NUM>, and the second microphone <NUM> (e.g., a "secondary" microphone) may be configured to primarily capture ambient sound, such as positioned proximate to an ear of the wearer. In other implementations, such as when the device <NUM> corresponds to a standalone voice assistant (e.g., including a loudspeaker with microphones, as described further with reference to <FIG>) that may be in the vicinity of multiple people, the device <NUM> may be configured to detect speech from the person closest to the primary microphone as self-voice activity, even though the person may be relatively remote from the primary microphone as compared to in a headset implementation. As used herein, the term "self-voice activity detection" is used interchangeably with "user voice activity detection" to indicate distinguishing between speech (e.g., voice or utterance) of a user of the device <NUM> (e.g., "user voice activity") as compared to sounds that do not originate from a user of the device (e.g., "other audio activity").

The device <NUM> includes a first input interface <NUM>, a second input interface <NUM>, one or more processors <NUM>, and a modem <NUM>. The first input interface <NUM> is coupled to the processor <NUM> and configured to be coupled to the first microphone <NUM>. The first input interface <NUM> is configured to receive a first microphone output <NUM> from the first microphone <NUM> and to provide the first microphone output <NUM> to the processor <NUM> as first audio data <NUM>.

The second input interface <NUM> is coupled to the processor <NUM> and configured to be coupled to the second microphone <NUM>. The second input interface <NUM> is configured to receive a second microphone output <NUM> from the second microphone <NUM> and to provide the second microphone output <NUM> to the processor <NUM> as second audio data <NUM>.

The processor <NUM> is coupled to the modem <NUM> and includes a feature extractor <NUM> and the dynamic classifier <NUM>. The processor is configured to receive audio data <NUM> including the first audio data <NUM> corresponding to the first output <NUM> of the first microphone <NUM> and the second audio data <NUM> corresponding to the second output <NUM> of the second microphone <NUM>. The processor <NUM> is configured to process the audio data <NUM> at the feature extractor <NUM> to generate feature data <NUM>.

In some implementations, the processor <NUM> is configured to process the first audio data <NUM> and the second audio data <NUM> prior to generating feature data <NUM>. In an example, the processor <NUM> is configured to perform echo-cancellation, noise suppression, or both, on the first audio data <NUM> and the second audio data <NUM>. In some implementations, the processor <NUM> is configured to transform the first audio data <NUM> and the second audio data <NUM> (e.g., a Fourier transform) to a transform domain prior to generating the feature data <NUM>.

The processor <NUM> is configured to generate feature data <NUM> based on the first audio data <NUM> and the second audio data <NUM>. In accordance with some aspects, the feature data <NUM> includes at least one interaural phase difference <NUM> between the first audio data <NUM> and the second audio data <NUM> and at least one interaural intensity difference <NUM> between the first audio data <NUM> and the second audio data <NUM>. In a particular example, the feature data <NUM> includes interaural phase differences (IPDs) <NUM> for multiple frequencies and interaural intensity differences (IIDs) <NUM> for multiple frequencies.

The processor <NUM> is configured to process the feature data <NUM> at the dynamic classifier <NUM> to generate a classification output <NUM> of the feature data <NUM>. In some implementations, the dynamic classifier <NUM> is configured to adaptively cluster sets (e.g., samples) of the feature data <NUM> based on whether a sound represented in the audio data <NUM> originates from a source that is closer to the first microphone <NUM> than to the second microphone <NUM>. For example, the dynamic classifier <NUM> may be configured to receive a sequence of samples of the feature data <NUM> and adaptively cluster the samples in a feature space containing IID and IPD frequency values.

The dynamic classifier <NUM> may also be configured to adjust a decision boundary between the two most discriminative categories of the feature space to distinguish between sets of feature data corresponding to user voice activity (e.g., an utterance <NUM> of a user <NUM>) and sets of feature data corresponding to other audio activity. To illustrate, the dynamic classifier <NUM> may be configured to classify incoming feature data into one of two classes (e.g., class <NUM> or class <NUM>), where one of the two classes corresponds to user voice activity, and the other of the two classes corresponds to other audio activity. The classification output <NUM> may include a single bit or flag that has one of two values: a first value (e.g., "<NUM>") to indicate that the feature data <NUM> corresponds to one of the two classes; or a second value (e.g., "<NUM>") to indicate that the feature data <NUM> corresponds to the other of the two classes.

In some implementations, the dynamic classifier <NUM> performs clustering and vector quantization. For example, clustering includes reducing (e.g., minimizing) the within-cluster sum of squares, defined as <MAT>, where Ci represents cluster i, pi represents a weight assigned to cluster i, xj represents a node j in the feature space, and µi represents the centroid of cluster i. The cluster weight pi may be probabilistic, such as a prior cluster distribution; possibilistic, such as a confidence measure assigned to possibility of each cluster; or determined by any other factor that would enforce some form of non-uniform bias towards different clusters. Vector quantization includes reducing (e.g., minimizing) error by quantizing an input vector into a quantization weight vector defined by <MAT>, where wi represents quantization weight vector i.

In some implementations, the dynamic classifier <NUM> is configured to perform competitive learning in which units of quantization compete to absorb new samples of the feature data <NUM>. The winning unit is then adjusted in the direction of the new sample. For example, each unit's weight vector may be initialized for separation or randomly. For each new sample of feature data that is received, a determination is made as to which weight vector is closest to the new sample, such as based on Euclidean distance or inner product similarity, as non-limiting examples. The weight vector closest to the new sample (the "winner" or best matching unit) may then be moved in the direction of the new sample. For example, in Hebbian learning, the winners strengthen their correlations with the input, such by adjusting the weights between two nodes in proportion to the product of the inputs to the two nodes.

In some implementations, the dynamic classifier <NUM> includes local clusters in a presynaptic sheet that are connected to local clusters in a postsynaptic sheet, and interconnections among neighboring neurons are reinforced through Hebbian learning to strengthen connections between correlating stimulations. The dynamic classifier <NUM> may include a Kohonen self-organizing map in which the input is connected to every neuron in the postsynaptic sheet or the map. Learning causes the map to be localized in that different fields of absorption respond to different regions of input space (e.g., the feature data space).

In a particular implementation, the dynamic classifier <NUM> includes a self-organizing map <NUM>. The self-organizing map <NUM> may operate by initializing weight vectors, and then for each input t (e.g., each received set of the feature data <NUM>), determining the winning unit (or cell or neuron) according to <MAT><MAT>, to find the winner v(t) as the unit that has the smallest distance (e.g., Euclidean distance) to the input x(t). The weights of the winning unit and its neighbors are updated, such as according to Δwi(t) = α(t)l(v, i, t)[x(t) - wv(t)], where Δwi(t) represents the change for uniti, α(t) represents a learning parameter, and l(v, i, t) represents a neighborhood function around the winning unit, such as a Gaussian radial basis function. In some implementations, inner products or another metric can be used as the similarity measure instead of Euclidean distance.

In some implementations, the dynamic classifier <NUM> includes a variant of a Kohonen self-organizing map to accommodate sequences of speech samples, such as described further with reference to <FIG>. In an example, the dynamic classifier <NUM> may implement temporal sequence processing, such as according to a temporal Kohonen map in which an activation function with a time constant modeling decay ("D") is defined for each unit and updated as <MAT> <MAT> , and the winning unit is the unit having the largest activity. As another example, the dynamic classifier <NUM> may implement a recurrent network, such as according to a recurrent self-organizing map which uses a difference vector y instead of a squared norm: yi(t, γ) = (<NUM> - γ)yi(t - <NUM>,γ) + y(x(t) - wi(t)), where γ represents a forgetting factor having a value between <NUM> and <NUM>, the winning unit is determined as the unit with the smallest difference vector <MAT><MAT>, and the weights are updated as Δwi(t) = α(t)l(v, i, t)[x(t) - yv(t, γ)].

In some implementations, the processor <NUM> is configured to update a clustering operation <NUM> of the dynamic classifier <NUM> based on the feature data <NUM> and to update a classification decision criterion <NUM> of the dynamic classifier <NUM>. For example, as explained above, the processor <NUM> is configured to adapt the clustering and the decision boundary between user voice activity and other audio activity based on incoming samples of the audio data <NUM>, enabling the dynamic classifier <NUM> to adjust operation based on changing conditions of the user <NUM>, the environment, other conditions (e.g., microphone placement or adjustment), or any combination thereof.

Although the dynamic classifier <NUM> is illustrated as including the self-organizing map <NUM>, in other implementations the dynamic classifier <NUM> may incorporate one or more other techniques to generate the classification output <NUM> instead of, or in addition to, the self-organizing map <NUM>. As non-limiting examples, the dynamic classifier <NUM> may include a restricted Boltzmann machine having an unsupervised configuration, an unsupervised autoencoder, an online variation of Hopfield networks, online clustering, or a combination thereof. As another non-limiting example, the dynamic classifier <NUM> may be configured to perform a principal component analysis (e.g., sequentially fitting a set orthogonal direction vectors to the feature vector samples in the feature space, where each direction vector is selected as maximizing the variance of the feature vector samples projected onto the direction vector in feature space). As another non-limiting example, the dynamic classifier <NUM> may be configured to perform an independent component analysis (e.g., determining a set of additive subcomponents of the feature vector samples in the feature space, with the assumption that the subcomponents are non-Gaussian signals that are statistically independent from each other).

The processor <NUM> is configured to determine, at least partially based on the classification output <NUM>, whether the audio data <NUM> corresponds to user voice activity. and to generate a user voice activity indicator <NUM> that indicates whether user voice activity is detected. For example, although the classification output <NUM> may indicate whether the feature data <NUM> is classified as one of two classes (e.g., class "<NUM>" or class "<NUM>"), the classification output <NUM> may not indicate which class corresponds to user voice activity and which class corresponds to other audio activity. For example, based on how the dynamic classifier <NUM> is initialized and the feature data that has been used to update the dynamic classifier <NUM>, in some cases the classification output <NUM> having the value "<NUM>" indicates user voice activity, while in other cases the classification output having the value "<NUM>" indicates other audio activity. The processor <NUM> may determine which of the two classes indicates user voice activity and which of the two classes indicates other audio activity, further based on at least one of a sign or a magnitude of at least one value of the feature data <NUM>, as described further with reference to <FIG>.

To illustrate, sound propagation of the utterance <NUM> from the mouth of the user <NUM> to the first microphone <NUM> and to the second microphone <NUM> results in a phase difference (due to the utterance <NUM> arriving at the first microphone <NUM> before the second microphone <NUM>) and a signal strength difference that may be detected in the feature data <NUM> and that may be distinguishable from phase and signal strength differences of sound from other audio sources. The phase and signal strength differences may be determined from the IPDs <NUM> and the IIDs <NUM> in the feature data <NUM> and used to map the classification output <NUM> to user voice activity or other audio activity. The processor <NUM> may generate a user voice activity indicator <NUM> that indicates whether the audio data <NUM> corresponds to user voice activity.

In some implementations, the processor <NUM> is configured to initiate a voice command processing operation <NUM> in response to a determination that the audio data <NUM> corresponds to user voice activity. In an illustrative example, the voice command processing operation <NUM> includes a voice activation operation, such as keyword or key phrase detection, voice print authentication, natural language processing, one or more other operations, or any combination thereof. As another example, the processor <NUM> may process the audio data <NUM> to perform a first stage of keyword detection and may use the user voice activity indicator <NUM> to confirm that a detected keyword was spoken by the user <NUM> of the device <NUM>, rather than by a nearby person, prior to initiating further processing of the audio data <NUM> via the voice command processing operation <NUM> (e.g., at a second stage of detection that includes more powerful voice activity recognition and speech recognition operations).

The modem <NUM> is coupled to the processor <NUM> and is configured to enable communication with the second device <NUM>, such as via wireless transmission. In some examples, the modem <NUM> is configured to transmit the audio data <NUM> to the second device <NUM> in response to a determination that the audio data <NUM> corresponds to user voice activity based on the dynamic classifier <NUM>. For example, in an implementation in which the device <NUM> corresponds to a headset device that is wirelessly coupled to the second device <NUM>, (e.g., a Bluetooth connection to a mobile phone or computer), the device <NUM> may send the audio data <NUM> to the second device <NUM> to perform the voice command processing operation <NUM> at a voice activation system <NUM> of the second device <NUM>. In this example, the device <NUM> offloads more computationally expensive processing (e.g., the voice command processing operation <NUM>) to be performed using the greater processing resources and power resources of the second device <NUM>.

In some implementations, the device <NUM> corresponds to or is included in one or various types of devices. In an illustrative example, the processor <NUM> is integrated in a headset device that includes the first microphone <NUM> and the second microphone <NUM>. The headset device is configured, when worn by the user <NUM>, to position the first microphone <NUM> closer than the second microphone <NUM> to the user's mouth to capture utterances <NUM> of the user <NUM> at the first microphone <NUM> with greater intensity and less delay as compared to at the second microphone <NUM>, such as described further with reference to <FIG>. In other examples, the processor <NUM> is integrated in at least one of a mobile phone or a tablet computer device, as described with reference to <FIG>, a wearable electronic device, as described with reference to <FIG>, a voice-controlled speaker system, as described with reference to <FIG>, a camera device, as described with reference to <FIG>, or a virtual reality headset, mixed reality headset, or an augmented reality headset, as described with reference to <FIG>. In another illustrative example, the processor <NUM> is integrated into a vehicle that also includes the first microphone <NUM> and the second microphone <NUM>, such as described further with reference to <FIG> and <FIG>.

During operation, the first microphone <NUM> is configured to capture utterances <NUM> of a user <NUM>, and the second microphone <NUM> is configured to capture ambient sound <NUM>. In one example, an utterance <NUM> from a user <NUM> of the device <NUM> is captured by the first microphone <NUM> and by the second microphone <NUM>. Because the first microphone <NUM> is nearer the mouth of the user <NUM>, the speech of the user <NUM> is captured by the first microphone <NUM> with higher signal strength and less delay as compared to the second microphone <NUM>. In another example, ambient sound <NUM> from one or more sound sources <NUM> (e.g., a conversation between two nearby people) may be captured by the first microphone <NUM> and by the second microphone <NUM>. Based on the position and distance of the sound sources <NUM> relative to the first microphone <NUM> and the second microphone <NUM>, a signal strength difference and relative delay between capturing the ambient sound <NUM> at the first microphone <NUM> and the second microphone <NUM> will vary from that for the utterance <NUM> from the user <NUM>.

The first audio data <NUM> and the second audio data <NUM> are processed at the processor <NUM>, such as by performing echo cancellation, noise suppression, frequency domain transform etc. The resulting audio data is processed at the feature extractor <NUM> to generate the feature data <NUM> including the IPDs <NUM> and the IIDs <NUM>. The feature data <NUM> is input to the dynamic classifier <NUM> to generate the classification output <NUM>, which is interpreted by the processor <NUM> as either user voice activity or other sound activity. The processor <NUM> generates the user voice activity indicator <NUM>, such as a "<NUM>" value to indicate the audio data <NUM> corresponds to user voice activity, or a "<NUM>" value to indicate the audio data <NUM> corresponds to other audio activity (or vice versa).

The user voice activity indicator <NUM> can be used to determine whether to initiate the voice command processing operation <NUM> at the device <NUM>. Alternatively, or in addition, the user voice activity indicator <NUM> can be used to determine whether to initiate generation of an output signal <NUM> (e.g., the audio data <NUM>) to the second device <NUM> for further processing at the voice activation system <NUM>.

In addition, in conjunction with generating the classification output <NUM>, the dynamic classifier <NUM> is updated based on the feature data <NUM>, such as by adjusting weights of the winning unit and its neighbors to be more similar to the feature data <NUM>, updating the clustering operation <NUM>, the classification criterion <NUM>, or a combination thereof. In this manner, the dynamic classifier <NUM> automatically adapts to changes in the user speech, changes in the environment, changes in the characteristics of the device <NUM> or the microphones <NUM>, <NUM>, or a combination thereof.

The system <NUM> thus improves performance of self-voice activity detection by using the dynamic classifier <NUM> to discriminate between user voice activity and other audio activity with relatively low complexity, low power consumption, and high accuracy as compared to conventional self-voice activity detection techniques. Automatically adapting to user and environment changes provides improved benefit by reducing or eliminating calibration to be performed by the user and enhancing the user's experience.

Although in some implementations the processor <NUM> provides the audio data <NUM> to the dynamic classifier <NUM> in the form of the feature data <NUM> (e.g., frequency domain data) that is generated by the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted. In an example, the processor <NUM> provides the audio data <NUM> to the dynamic classifier <NUM> as a time series of audio samples, and the dynamic classifier <NUM> processes the audio data <NUM> to generate the classification output <NUM>. In an illustrative implementation, the dynamic classifier <NUM> is configured to determine frequency domain data from the audio data <NUM> (e.g., generate the feature data <NUM>) and use the extracted frequency domain data to generate the classification output <NUM>.

Although the first microphone <NUM> and the second microphone <NUM> are illustrated as being coupled to the device <NUM>, in other implementations one or both of the first microphone <NUM> or the second microphone <NUM> may be integrated in the device <NUM>. Although the two microphones <NUM>, <NUM> are illustrated, in other implementations one or more additional microphones configured to capture user speech, one or more microphones configured to capture environmental sounds, or both, may be included. Although the system <NUM> is illustrated as including the second device <NUM>, in other implementations the second device <NUM> may be omitted, and the device <NUM> may perform operations described as being performed at the second device <NUM>.

<FIG> is a diagram of an illustrative aspect of operations <NUM> associated with self-voice activity detection that may be performed by the device <NUM> of <FIG> (e.g., the processor <NUM>). Feature extraction <NUM> is performed on an input <NUM> to generate feature data <NUM>. In an example, the input <NUM> corresponds to the audio data <NUM>, the feature extraction <NUM> is performed by the feature extractor <NUM>, and the feature data <NUM> corresponds to the feature data <NUM>.

A dynamic classifier <NUM> operates on the feature data <NUM> to generate a classification output <NUM>. In an example, the dynamic classifier <NUM> corresponds to the dynamic classifier <NUM> and is configured to perform unsupervised real-time clustering based on the feature data <NUM> with highly dynamic decision boundaries for "self" vs "other" labeling for voice activation classes in a classification output <NUM>. For example, the dynamic classifier <NUM> may divide the feature space into two classes, one class associated with user voice activity and the other class associated with other sound activity. The classification output <NUM> may include a binary indicator of which class is associated with the feature data <NUM>. In an example, the classification output <NUM> corresponds to the classification output <NUM>.

A self/other association operation <NUM> generates a self/other indicator <NUM> based on the classification output <NUM> and a verification input <NUM>. The verification input <NUM> may provide information that associates each of the classes of the classification output <NUM> with user voice activity (e.g., "self") or other sound activity (e.g., "other"). For example, the verification input <NUM> may be generated based on at least one prior verification criterion <NUM>, such as comparing a sign <NUM> of a phase difference (e.g., a value of one or more of the IPDs <NUM> over one or more particular frequency ranges, indicating which microphone is closer to the source of the audio represented by the input <NUM>), comparing a magnitude <NUM> of an intensity difference (e.g., a value of one or more of the IIDs <NUM> over one or more particular frequency ranges, indicating relative distances of the source of the audio to the separate microphones), or a combination thereof. For example, the self/other association may determine that a classification output <NUM> value of "<NUM>" corresponds to feature data <NUM> exhibiting a negative sign <NUM> in one or more pertinent frequency ranges, or exhibiting a magnitude <NUM> less than a threshold amount in one or more pertinent frequency ranges, or both, and as a result may populate a table such that "<NUM>" corresponds to "other" and "<NUM>" corresponds to "self.

The self/other association operation <NUM> results in generation of a self/other indicator <NUM> (e.g., a binary indicator having a first value (e.g., "<NUM>") to indicate user voice activity or having a second value (e.g., "<NUM>") to indicate other sound activity, or vice-versa). A wakeup/barge-in control operation <NUM> is responsive to the self/other indicator <NUM> to generate a signal <NUM> to a voice command process <NUM>. For example, the signal <NUM> may have a first value (e.g., "<NUM>") to indicate that the voice command process <NUM> is to be executed on the input <NUM>, the feature data <NUM>, or both, to perform further voice command processing (e.g., to perform keyword detection, voice authentication, or both) when the input <NUM> corresponds to user voice activity, or may have a second value (e.g., "<NUM>") to indicate that the voice command process <NUM> is not to perform the voice command processing when the input <NUM> corresponds to other sound activity.

Dynamic classification, such as described with reference to the dynamic classifier <NUM> of <FIG> and the dynamic classifier <NUM> of <FIG>, assists with improving SVAD accuracy, with the objectives of responding only when the user speaks and always suppressing responses when other interferences (e.g., external speech) arrive, and maximizing the self-keyword acceptance rate ("SKAR") and the other keyword rejection rate ("OKRR"). By using dynamic classification, various challenges associated with conventional SVAD processing are circumvented or otherwise reduced. For example, conventional SVAD processing challenges that are circumvented or reduced via implementation of dynamic classification include noise and echo conditions (which may cause erroneous wakeup and barge-in under severe conditions), microphone mismatch and sensitivity, voice activation engine dependence, different user head-related transfer functions (HRTFs), different headset hardware effects, user's behavior driven variations of occlusion and isolation levels, user's feature resemblance of other voice activity, final negative effects on voice activation, and response delay of onset of user speech. To illustrate, conventional SVAD is highly dependent on internal/external microphone calibration and sensitivities, directions of arrival of interfering speech, variations of headset fitting and isolation, and non stationary statistics of features, which may be accommodated by operation of dynamic classification.

Use of dynamic classification enables discrimination using the extracted feature data <NUM> to actively respond and adapt to various conditions, such as: environmental conditions in highly nonstationary situations; mismatched microphones; changes in user headset fitting; different user head-related transfer functions; direction-of-arrival ("DOA") tracking of non-user signals; and noise floor, bias, and sensitivities of microphones across the frequency spectrum. Dynamic classification enables adaptive feature mapping capable of responding to such variations and reducing or minimizing a number of thresholding parameters used and an amount of headset tuning by customers.

<FIG> is a block diagram of an illustrative aspect of a system operable to perform self-voice activity detection, in accordance with some examples of the present disclosure, in which the processor <NUM> includes an always-on power domain <NUM> and a second power domain <NUM>, such as an on-demand power domain. In some implementations, a first stage <NUM> of a self-voice activity detector <NUM> and a buffer <NUM> are configured to operate in an always-on mode, and a second stage <NUM> of the self-voice activity detector <NUM> is configured to operate in an on-demand mode.

The always-on power domain <NUM> includes the buffer <NUM>, the feature extractor <NUM>, and the dynamic classifier <NUM>. The buffer <NUM> is configured to store the first audio data <NUM> and the second audio data <NUM> to be accessible for processing by components of the self-voice activity detector <NUM>.

The second power domain <NUM> includes a voice command processing unit <NUM> in the second stage <NUM> of the self-voice activity detector <NUM> and also includes activation circuitry <NUM>. In some implementations, the voice command processing unit <NUM> is configured to perform the voice command processing operation <NUM> of <FIG> or the voice command process <NUM> of <FIG>.

The first stage <NUM> of the self-voice activity detector <NUM> is configured to generate at least one of a wakeup signal <NUM> or an interrupt <NUM> to initiate the voice command processing operation <NUM> (or the voice command process <NUM>) at the voice command processing unit <NUM>. In an example, the wakeup signal <NUM> is configured to transition the second power domain <NUM> from a low-power mode <NUM> to an active mode <NUM> to activate the voice command processing unit <NUM>. In some implementations, the wakeup signal <NUM>, the interrupt <NUM>, or both, correspond to the signal <NUM> of <FIG>.

For example, the activation circuitry <NUM> may include or be coupled to power management circuitry, clock circuitry, head switch or foot switch circuitry, buffer control circuitry, or any combination thereof. The activation circuitry <NUM> may be configured to initiate powering-on of the second stage <NUM>, such as by selectively applying or raising a voltage of a power supply of the second stage <NUM>, of the second power domain <NUM>, or both. As another example, the activation circuitry <NUM> may be configured to selectively gate or un-gate a clock signal to the second stage <NUM>, such as to prevent or enable circuit operation without removing a power supply.

A detector output <NUM> generated by the second stage <NUM> of the self-voice activity detector <NUM> is provided to an application <NUM>. The application <NUM> may be configured to perform one or more operations based on detected user speech. To illustrate, the application <NUM> may correspond to a voice interface application, an integrated assistant application, a vehicle navigation and entertainment application, or a home automation system, as illustrative, non-limiting examples.

By selectively activating the second stage <NUM> based on a result of processing audio data at the first stage <NUM> of the self-voice activity detector <NUM>, overall power consumption associated with self-voice activity detection, voice command processing, or both, may be reduced.

<FIG> is a diagram of an illustrative aspect of operation of components of the system of <FIG>, in accordance with some examples of the present disclosure. The feature extractor <NUM> is configured to receive a sequence <NUM> of audio data samples, such as a sequence of successively captured frames of the audio data <NUM>, illustrated as a first frame (F1) <NUM>, a second frame (F2) <NUM>, and one or more additional frames including an Nth frame (FN) <NUM> (where N is an integer greater than two). The feature extractor <NUM> is configured to output a sequence <NUM> of sets of feature data including a first set <NUM>, a second set <NUM>, and one or more additional sets including an Nth set <NUM>.

The dynamic classifier <NUM> is configured to receive the sequence <NUM> of sets of feature data and to adaptively cluster a set (e.g., the second set <NUM>) of the sequence <NUM> at least partially based on a prior set (e.g., the first set <NUM>) of feature data in the sequence <NUM>. As illustrative, non-limiting examples, the dynamic classifier <NUM> may be implemented as a temporal Kohonen map or a recurrent self-organizing map.

During operation, the feature extractor <NUM> processes the first frame <NUM> to generate the first set <NUM> of feature data, and the dynamic classifier <NUM> processes the first set <NUM> of feature data to generate a first classification output (C1) <NUM> of a sequence <NUM> of classification outputs. The feature extractor <NUM> processes the second frame <NUM> to generate the second set <NUM> of feature data, and the dynamic classifier <NUM> processes the second set <NUM> of feature data to generate a second classification output (C2) <NUM> based on the second set <NUM> of feature data and at least partially based on the first set <NUM> of feature data. Such processing continues, including the feature extractor <NUM> processing the Nth frame <NUM> to generate the Nth set <NUM> of feature data, and the dynamic classifier <NUM> processes the Nth set <NUM> of feature data to generate an Nth classification output (CN) <NUM>. The Nth classification output <NUM> is based on the Nth set <NUM> of feature data and at least partially based on one or more of the previous sets of feature data of the sequence <NUM>.

By dynamically classifying based on one or more prior sets of feature data, accuracy of classification by the dynamic classifier <NUM> may be improved for speech signals that may span multiple frames of audio data.

<FIG> depicts an implementation <NUM> of the device <NUM> as an integrated circuit <NUM> that includes the one or more processors <NUM>. The integrated circuit <NUM> also includes an audio input <NUM>, such as one or more bus interfaces, to enable the audio data <NUM> to be received for processing. The integrated circuit <NUM> also includes a signal output <NUM>, such as a bus interface, to enable sending of an output signal, such as the user voice activity indicator <NUM>. The integrated circuit <NUM> enables implementation of self-voice activity detection as a component in a system that includes microphones, such as a mobile phone or tablet as depicted in <FIG>, a headset as depicted in <FIG>, a wearable electronic device as depicted in <FIG>, a voice-controlled speaker system as depicted in <FIG>, a camera as depicted in <FIG>, a virtual reality headset, mixed reality headset, or an augmented reality headset as depicted in <FIG>, or a vehicle as depicted in <FIG> or <FIG>.

<FIG> depicts an implementation <NUM> in which the device <NUM> is a mobile device <NUM>, such as a phone or tablet, as illustrative, non-limiting examples. The mobile device <NUM> includes the first microphone <NUM> positioned to primarily capture speech of a user, multiple second microphones <NUM> positioned to primarily capture environmental sounds, and a display screen <NUM>. Components of the processor <NUM>, including the feature extractor <NUM> and the dynamic classifier <NUM>, are integrated in the mobile device <NUM> and are illustrated using dashed lines to indicate internal components that are not generally visible to a user of the mobile device <NUM>. Although the processor <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>. In a particular example, the dynamic classifier <NUM> operates to detect user voice activity, which is then processed to perform one or more operations at the mobile device <NUM>, such as to launch a graphical user interface or otherwise display other information associated with the user's speech at the display screen <NUM> (e.g., via an integrated "smart assistant" application).

<FIG> depicts an implementation <NUM> in which the device <NUM> is a headset device <NUM>. The headset device <NUM> includes the first microphone <NUM> positioned to primarily capture speech of a user and the second microphone <NUM> positioned to primarily capture environmental sounds. Components of the processor <NUM>, including the feature extractor <NUM> and the dynamic classifier <NUM>, are integrated in the headset device <NUM>. In a particular example, the dynamic classifier <NUM> operates to detect user voice activity, which may cause the headset device <NUM> to perform one or more operations at the headset device <NUM>, to transmit audio data corresponding to the user voice activity to a second device (not shown), such as the second device <NUM> of <FIG>, for further processing, or a combination thereof. Although the processor <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>.

<FIG> depicts an implementation <NUM> in which the device <NUM> is a wearable electronic device <NUM>, illustrated as a "smart watch. " The feature extractor <NUM>, the dynamic classifier <NUM>, the first microphone <NUM>, and the second microphone <NUM> are integrated into the wearable electronic device <NUM>. In a particular example, the dynamic classifier <NUM> operates to detect user voice activity, which is then processed to perform one or more operations at the wearable electronic device <NUM>, such as to launch a graphical user interface or otherwise display other information associated with the user's speech at a display screen <NUM> of the wearable electronic device <NUM>. To illustrate, the wearable electronic device <NUM> may include a display screen that is configured to display a notification based on user speech detected by the wearable electronic device <NUM>. In a particular example, the wearable electronic device <NUM> includes a haptic device that provides a haptic notification (e.g., vibrates) in response to detection of user voice activity. For example, the haptic notification can cause a user to look at the wearable electronic device <NUM> to see a displayed notification indicating detection of a keyword spoken by the user. The wearable electronic device <NUM> can thus alert a user with a hearing impairment or a user wearing a headset that the user's voice activity is detected. Although the wearable electronic device <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>.

<FIG> is an implementation <NUM> in which the device <NUM> is a wireless speaker and voice activated device <NUM>. The wireless speaker and voice activated device <NUM> can have wireless network connectivity and is configured to execute an assistant operation. The processor <NUM> including the feature extractor <NUM> and the dynamic classifier <NUM>, the first microphone <NUM>, the second microphone <NUM>, or a combination thereof, are included in the wireless speaker and voice activated device <NUM>. Although the processor <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>. The wireless speaker and voice activated device <NUM> also includes a speaker <NUM>. During operation, in response to receiving a verbal command identified as user speech via operation of the dynamic classifier <NUM>, the wireless speaker and voice activated device <NUM> can execute assistant operations, such as via execution of the voice activation system <NUM> (e.g., an integrated assistant application). The assistant operations can include adjusting a temperature, playing music, turning on lights, etc. For example, the assistant operations are performed responsive to receiving a command after a keyword or key phrase (e.g., "hello assistant").

In an illustrative example, when the wireless speaker and voice activated device <NUM> is near a wall of a room (e.g., next to a window) and arranged such that the first microphone <NUM> is arranged closer to an interior of the room as compared to the second microphone <NUM> (e.g., the second microphone may be positioned closer to the wall or window than the first microphone <NUM>), speech originating from the interior of the room can be identified as user voice activity, while sound originating from outside the room (e.g., a speech of a person on the other side of the wall or window) can be identified as other audio activity. Because multiple people may be in the room, the wireless speaker and voice activated device <NUM> may be configured to identify speech from any of the multiple people as user voice activity (e.g., there may be multiple "users" of the wireless speaker and voice activated device <NUM>). To illustrate, the dynamic classifier <NUM> may be configured to recognize feature data corresponding to speech originating from within the room as "self-voice" even when the person speaking may be relatively distant (e.g., several meters) from the wireless speaker and voice activated device <NUM> and is closer to the first microphone <NUM> than to the second microphone <NUM>. In some implementations in which speech is detected from multiple people in the room, the wireless speaker and voice activated device <NUM> (e.g., the dynamic classifier <NUM>) may be configured to identify the speech from the person closest to the first microphone <NUM> as user voice activity (e.g., self-voice with the closest user).

<FIG> depicts an implementation <NUM> in which the device <NUM> is a portable electronic device that corresponds to a camera device <NUM>. The feature extractor <NUM> and the dynamic classifier <NUM>, the first microphone <NUM>, the second microphone <NUM>, or a combination thereof, are included in the camera device <NUM>. During operation, in response to receiving a verbal command identified as user speech via operation of the dynamic classifier <NUM>, the camera device <NUM> can execute operations responsive to spoken user commands, such as to adjust image or video capture settings, image or video playback settings, or image or video capture instructions, as illustrative examples. Although the camera device <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>.

<FIG> depicts an implementation <NUM> in which the device <NUM> includes a portable electronic device that corresponds to an extended reality ("XR") headset <NUM>, such as a virtual reality ("VR"), augmented reality ("AR"), or mixed reality ("MR") headset device. The feature extractor <NUM>, the dynamic classifier <NUM>, the first microphone <NUM>, the second microphone <NUM>, or a combination thereof, are integrated into the headset <NUM>. In a particular aspect, the headset <NUM> includes the first microphone <NUM> positioned to primarily capture speech of a user and the second microphone <NUM> positioned to primarily capture environmental sounds. User voice activity detection can be performed based on audio signals received from the first microphone <NUM> and the second microphone <NUM> of the headset <NUM>. A visual interface device is positioned in front of the user's eyes to enable display of augmented reality or virtual reality images or scenes to the user while the headset <NUM> is worn. In a particular example, the visual interface device is configured to display a notification indicating user speech detected in the audio signal. Although the headset <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>.

<FIG> depicts an implementation <NUM> in which the device <NUM> corresponds to or is integrated within a vehicle <NUM>, illustrated as a manned or unmanned aerial device (e.g., a package delivery drone). The feature extractor <NUM>, the dynamic classifier <NUM>, the first microphone <NUM>, the second microphone <NUM>, or a combination thereof, are integrated into the vehicle <NUM>. User voice activity detection can be performed based on audio signals received from the first microphone <NUM> and the second microphone <NUM> of the vehicle <NUM>, such as for delivery instructions from an authorized user of the vehicle <NUM>. Although the vehicle <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>.

<FIG> depicts another implementation <NUM> in which the device <NUM> corresponds to, or is integrated within, a vehicle <NUM>, illustrated as a car. The vehicle <NUM> includes the processor <NUM> including the feature extractor <NUM> and the dynamic classifier <NUM>. Although the vehicle <NUM> is illustrated as including the feature extractor <NUM>, in other implementations the feature extractor <NUM> is omitted, such as when the dynamic classifier <NUM> is configured to extract feature data during processing of the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>. The vehicle <NUM> also includes the first microphone <NUM> and the second microphone <NUM>. The first microphone <NUM> is positioned to capture utterances of an operator of the vehicle <NUM>. User voice activity detection can be performed based on audio signals received from the first microphone <NUM> and the second microphone <NUM> of the vehicle <NUM>. In some implementations, user voice activity detection can be performed based on an audio signal received from interior microphones (e.g., the first microphone <NUM> and the second microphone <NUM>), such as for a voice command from an authorized passenger. For example, the user voice activity detection can be used to detect a voice command from an operator of the vehicle <NUM> (e.g., from a parent to set a volume to <NUM> or to set a destination for a self-driving vehicle) and to disregard the voice of another passenger (e.g., a voice command from a child to set the volume to <NUM> or other passengers discussing another location). In some implementations, user voice activity detection can be performed based on an audio signal received from external microphones (e.g., the first microphone <NUM> and the second microphone <NUM>), such as an authorized user of the vehicle. In a particular implementation, in response to receiving a verbal command identified as user speech via operation of the dynamic classifier <NUM>, the voice activation system <NUM> initiates one or more operations of the vehicle <NUM> based on one or more keywords (e.g., "unlock", "start engine", "play music", "display weather forecast", or another voice command) detected in the output signal <NUM>, such as by providing feedback or information via a display <NUM> or one or more speakers (e.g., a speaker <NUM>).

Referring to <FIG>, a particular implementation of a method <NUM> of user voice activity detection is shown. In a particular aspect, one or more operations of the method <NUM> are performed by at least one of the feature extractor <NUM>, the dynamic classifier <NUM>, the processor <NUM>, the device <NUM>, the system <NUM> of <FIG>, or a combination thereof.

The method <NUM> includes receiving, at one or more processors, audio data including first audio data corresponding to a first output of a first microphone and second audio data corresponding to a second output of a second microphone, at <NUM>. For example, the feature extractor <NUM> of <FIG> receives the audio data <NUM> including the first audio data <NUM> corresponding to a first output of the first microphone <NUM> and the second audio data <NUM> corresponding to a second output of the second microphone <NUM>, as described with reference to <FIG>.

The method <NUM> includes generating, at the one or more processors, feature data based on the first audio data and the second audio data, at <NUM>. For example, the feature extractor <NUM> of <FIG> generates the feature data <NUM> based on the first audio data <NUM> and the second audio data <NUM>, as described with reference to <FIG>. In another example, a dynamic classifier, such as the dynamic classifier <NUM> of <FIG>, is configured to receive the first audio data <NUM> and the second audio data <NUM> and to extract the feature data <NUM> during processing of the first audio data <NUM> and the second audio data <NUM>.

The method <NUM> includes generating, at a dynamic classifier of the one or more processors, a classification output of the feature data, at <NUM>. For example, the dynamic classifier <NUM> of <FIG> generates the classification output <NUM> of the feature data <NUM>, as described with reference to <FIG>.

The method <NUM> includes determining, at the one or more processors and at least partially based on the classification output, whether the audio data corresponds to user voice activity, at <NUM>. For example, the processor <NUM> of <FIG> determines, at least partially based on the classification output <NUM>, whether the audio data <NUM> corresponds to user voice activity, as described with reference to <FIG>.

The method <NUM> improves performance of self-voice activity detection by using the dynamic classifier <NUM> to discriminate between user voice activity and other audio activity with relatively low complexity, low power consumption, and high accuracy as compared to conventional self-voice activity detection techniques. Automatically adapting to user and environment changes provides improved benefit by reducing or eliminating calibration to be performed by the user and enhancing the user's experience.

Referring to <FIG>, a particular implementation of a method <NUM> of user voice activity detection is shown. In a particular aspect, one or more operations of the method <NUM> are performed by at least one of the dynamic classifier <NUM>, the processor <NUM>, the device <NUM>, the system <NUM> of <FIG>, or a combination thereof.

The method <NUM> includes receiving, at one or more processors, audio data including first audio data corresponding to a first output of a first microphone and second audio data corresponding to a second output of a second microphone, at <NUM>. In an example, the feature extractor <NUM> of <FIG> receives the audio data <NUM>, including the first audio data <NUM> and the second audio data <NUM> corresponding to a second output of the second microphone <NUM>, as described with reference to <FIG>.

The method <NUM> includes providing, at the one or more processors, the audio data to a dynamic classifier to generate a classification output corresponding to the audio data, at <NUM>. In an example, the feature extractor <NUM> of <FIG> generates the feature data <NUM> based on the first audio data <NUM> and the second audio data <NUM>, and the feature data <NUM> is processed by the dynamic classifier <NUM> to generate the classification output <NUM>, such as described in <FIG> and in accordance with the method <NUM> of <FIG>. In another example, the processor <NUM> provides the first audio data <NUM> and the second audio data <NUM> to the dynamic classifier <NUM>, and the dynamic classifier <NUM> processes the first audio data <NUM> and the second audio data <NUM> to generate the classification output <NUM>. In an illustrative implementation, the dynamic classifier <NUM> processes the first audio data <NUM> and the second audio data <NUM> to extract the feature data <NUM>, and determines the classification output <NUM> based on the feature data <NUM>.

The method <NUM> of <FIG>, the method <NUM> of <FIG>, or a combination thereof, may be implemented by a field-programmable gate array (FPGA) device, an application-specific integrated circuit (ASIC), a processing unit such as a central processing unit (CPU), a DSP, a controller, another hardware device, firmware device, or any combination thereof. As an example, the method <NUM> of <FIG>, the method <NUM> of <FIG>, or a combination thereof, may be performed by a processor that executes instructions, such as described with reference to <FIG>.

Referring to <FIG>, a block diagram of a particular illustrative implementation of a device is depicted and generally designated <NUM>. In various implementations, the device <NUM> may have more or fewer components than illustrated in <FIG>. In an illustrative implementation, the device <NUM> may correspond to the device <NUM>. In an illustrative implementation, the device <NUM> may perform one or more operations described with reference to <FIG>.

In a particular implementation, the device <NUM> includes a processor <NUM> (e.g., a central processing unit (CPU)). The device <NUM> may include one or more additional processors <NUM> (e.g., one or more DSPs). In a particular aspect, the processor <NUM> of <FIG> corresponds to the processor <NUM>, the processors <NUM>, or a combination thereof. The processors <NUM> may include a speech and music coder-decoder (CODEC) <NUM> that includes a voice coder ("vocoder") encoder <NUM>, a vocoder decoder <NUM>, the feature extractor <NUM>, the dynamic classifier <NUM>, or a combination thereof.

The device <NUM> may include a memory <NUM> and a CODEC <NUM>. The memory <NUM> may include instructions <NUM>, that are executable by the one or more additional processors <NUM> (or the processor <NUM>) to implement the functionality described with reference to the feature extractor <NUM>, the dynamic classifier <NUM>, or both. The device <NUM> may include the modem <NUM> coupled, via a transceiver <NUM>, to an antenna <NUM>.

The device <NUM> may include a display <NUM> coupled to a display controller <NUM>. A speaker <NUM>, the first microphone <NUM>, and the second microphone <NUM> may be coupled to the CODEC <NUM>. The CODEC <NUM> may include a digital-to-analog converter (DAC) <NUM>, an analog-to-digital converter (ADC) <NUM>, or both. In a particular implementation, the CODEC <NUM> may receive analog signals from the first microphone <NUM> and the second microphone <NUM>, convert the analog signals to digital signals using the analog-to-digital converter <NUM>, and provide the digital signals to the speech and music codec <NUM>. The speech and music codec <NUM> may process the digital signals, and the digital signals may further be processed by the feature extractor <NUM> and the dynamic classifier <NUM>. In a particular implementation, the speech and music codec <NUM> may provide digital signals to the CODEC <NUM>. The CODEC <NUM> may convert the digital signals to analog signals using the digital-to-analog converter <NUM> and may provide the analog signals to the speaker <NUM>.

In a particular implementation, the device <NUM> may be included in a system-in-package or system-on-chip device <NUM>. In a particular implementation, the memory <NUM>, the processor <NUM>, the processors <NUM>, the display controller <NUM>, the CODEC <NUM>, and the modem <NUM> are included in a system-in-package or system-on-chip device <NUM>. In a particular implementation, an input device <NUM> and a power supply <NUM> are coupled to the system-on-chip device <NUM>. Moreover, in a particular implementation, as illustrated in <FIG>, the display <NUM>, the input device <NUM>, the speaker <NUM>, the first microphone <NUM>, the second microphone <NUM>, the antenna <NUM>, and the power supply <NUM> are external to the system-on-chip device <NUM>. In a particular implementation, each of the display <NUM>, the input device <NUM>, the speaker <NUM>, the first microphone <NUM>, the second microphone <NUM>, the antenna <NUM>, and the power supply <NUM> may be coupled to a component of the system-on-chip device <NUM>, such as an interface (e.g., the first input interface <NUM> or the second input interface <NUM>) or a controller.

The device <NUM> may include a smart speaker, a speaker bar, a mobile communication device, a smart phone, a cellular phone, a laptop computer, a computer, a tablet, a personal digital assistant, a display device, a television, a gaming console, a music player, a radio, a digital video player, a digital video disc (DVD) player, a tuner, a camera, a navigation device, a vehicle, a headset, an augmented reality headset, a virtual reality headset, an aerial vehicle, a home automation system, a voice-activated device, a wireless speaker and voice activated device, a portable electronic device, a car, a vehicle, a computing device, a communication device, an internet-of-things (IoT) device, a virtual reality (VR) device, a base station, a mobile device, or any combination thereof.

In conjunction with the described implementations, an apparatus includes means for receiving audio data including first audio data corresponding to a first output of a first microphone and second audio data corresponding to a second output of a second microphone. For example, the means for receiving can correspond to the first input interface <NUM>, the second input interface <NUM>, the feature extractor <NUM>, the dynamic classifier <NUM>, the processor <NUM>, the one or more processors <NUM>, one or more other circuits or components configured to receive audio data including first audio data corresponding to a first output of a first microphone and second audio data corresponding to a second output of a second microphone, or any combination thereof.

The apparatus also includes means for generating feature data based on the first audio data and the second audio data. For example, the means for generating the feature data can correspond to the feature extractor <NUM>, the dynamic classifier <NUM>, the processor <NUM>, the one or more processors <NUM>, one or more other circuits or components configured to generate feature data, or any combination thereof.

The apparatus further includes means for generating, at a dynamic classifier, a classification output of the feature data. For example, the means for generating the classification output can correspond to the dynamic classifier <NUM>, the processor <NUM>, the one or more processors <NUM>, one or more other circuits or components configured to generate classification output at a dynamic classifier, or any combination thereof.

The apparatus also includes means for determining, at least partially based on the classification output, whether the audio data corresponds to user voice activity. For example, the means for determining can correspond to the dynamic classifier <NUM>, the processor <NUM>, the one or more processors <NUM>, one or more other circuits or components configured to determine, at least partially based on the classification output, whether the audio data corresponds to user voice activity, or any combination thereof.

The apparatus further includes means for generating, at a dynamic classifier, a classification output corresponding to the audio data. For example, the means for generating the classification output can correspond to the feature extractor <NUM>, the dynamic classifier <NUM>, the processor <NUM>, the one or more processors <NUM>, one or more other circuits or components configured to generate classification output at a dynamic classifier, or any combination thereof.

The apparatus also includes means for determining, at least partially based on the classification output, whether the audio data corresponds to user voice activity. For example, the means for determining can correspond to the dynamic classifier <NUM>, the processor <NUM>, the one or more processors <NUM>, one or more other circuits or components configured to determine, at least partially based on the classification output, whether the audio data corresponds to user voice activity, or any combination thereof.

In some implementations, a non-transitory computer-readable medium (e.g., a computer-readable storage device, such as the memory <NUM>) includes instructions (e.g., the instructions <NUM>) that, when executed by one or more processors (e.g., the one or more processors <NUM> or the processor <NUM>), cause the one or more processors to receive audio data (e.g., the audio data <NUM>) including first audio data (e.g., the first audio data <NUM>) corresponding to a first output of a first microphone (e.g., the first microphone <NUM>) and second audio data (e.g., the second audio data <NUM>) corresponding to a second output of a second microphone (e.g., the second microphone <NUM>). The instructions, when executed by the one or more processors, also cause the one or more processors to provide the audio data to a dynamic classifier (e.g., the dynamic classifier <NUM>) to generate a classification output (e.g., the classification output <NUM>) corresponding to the audio data. In an example, the instructions, when executed by the one or more processors, cause the one or more processors to generate feature data (e.g., the feature data <NUM>) based on the first audio data and the second audio data and to process the feature data at the dynamic classifier. The instructions, when executed by the one or more processors, also cause the one or more processors to determine, at least partially based on the classification output, whether the audio data corresponds to user voice activity.

Those of skill would further appreciate that the various illustrative logical blocks, configurations, modules, circuits, and algorithm steps described in connection with the implementations disclosed herein may be implemented as electronic hardware, computer software executed by a processor, or combinations of both. Various illustrative components, blocks, configurations, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or processor executable instructions depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, such implementation decisions are not to be interpreted as causing a departure from the scope of the present disclosure.

The steps of a method or algorithm described in connection with the implementations disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of non-transient storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor may read information from, and write information to, the storage medium. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). The ASIC may reside in a computing device or a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a computing device or user terminal.

Claim 1:
A device (<NUM>; <NUM>) comprising:
a memory configured to store instructions; and
one or more processors (<NUM>) configured to execute the instructions to:
receive audio data (<NUM>; <NUM>) including first audio data (<NUM>) corresponding to a first output (<NUM>) of a first microphone (<NUM>) and second audio data (<NUM>) corresponding to a second output (<NUM>) of a second microphone (<NUM>);
provide the audio data (<NUM>; <NUM>) to a dynamic classifier (<NUM>), the dynamic classifier (<NUM>) configured to generate a classification output (<NUM>) corresponding to the audio data (<NUM>; <NUM>);
provide the audio data (<NUM>; <NUM>) to a verifier to generate a verification output (<NUM>) that associates each class of classification output with a self-voice activity or other sound activity; and
determine, at least partially based on the classification output (<NUM>) and the verification output (<NUM>) , whether the audio data corresponds to self-voice activity.