Patent Description:
<CIT> relates to hearing device comprising a microphone and a processor, wherein own voice content representative of a voice of the user is determined in the audio signal captured by the microphone and a biomarker feature analysis is applied to the own voice content so as to provide for an alert if a certain medical condition, such as a stroke, related to the own voice content is detected; the hearing device also is able to determine an environmental noise level within the captured audio signal.

The invention relates to a hearing device as defined in claim <NUM> and to a method as defined in claim <NUM>.

The systems and methods described herein may advantageously provide many benefits to users of hearing devices. For example, the hearing devices described herein may analyze biomarker features extracted from the user's own voice to provide early detection of potential neurodegenerative disorders and/or other diseases of a user during a course of general usage by the user of the hearing device. This, in turn, may lead to earlier diagnosis and treatment of the neurodegenerative disorders and/or other diseases than conventional diagnosis methods. For at least these reasons, the systems and methods described herein advantageously provide additional functionality and/or features for hearing device users compared to conventional hearing devices. These and other benefits of the systems and methods described herein will be made apparent herein.

<FIG> illustrates an exemplary hearing device <NUM>. Hearing device <NUM> may be implemented by any type of hearing device configured to enable or enhance hearing by a user wearing hearing device <NUM>. For example, hearing device <NUM> may be implemented by a hearing aid configured to provide an amplified version of audio content to a user, a sound processor included in a cochlear implant system configured to provide electrical stimulation representative of audio content to a user, a sound processor included in a bimodal hearing system configured to provide both amplification and electrical stimulation representative of audio content to a user, a head-worn headset, an ear-worn ear-bud, or any other suitable hearing prosthesis.

As shown, hearing device <NUM> includes a processor <NUM> communicatively coupled to a memory <NUM>, a microphone <NUM>, and an output transducer <NUM>. Hearing device <NUM> may include additional or alternative components as may serve a particular implementation.

Microphone <NUM> may be implemented by any suitable audio detection device and is configured to detect an audio signal presented to a user of hearing device <NUM>. The audio signal may include, for example, audio content (e.g., music, speech, noise, etc.) generated by one or more audio sources included in an environment of the user, including the user. Microphone <NUM> may be included in or communicatively coupled to hearing device <NUM> in any suitable manner. Output transducer <NUM> may be implemented by any suitable audio output device, for instance a loudspeaker of a hearing device or an output electrode of a cochlear implant system.

Memory <NUM> may be implemented by any suitable type of storage medium and may be configured to maintain (e.g., store) data generated, accessed, or otherwise used by processor <NUM>. For example, memory <NUM> may maintain data representative of a plurality of sound processing programs that specify how processor <NUM> processes audio content (e.g., audio content included in the audio signal detected by microphone <NUM>) to present the audio content to a user. To illustrate, if hearing device <NUM> is a hearing aid, memory <NUM> may maintain data representative of sound processing programs that specify audio amplification schemes (e.g., amplification levels, etc.) used by processor <NUM> to provide an amplified version of the audio content to the user. As another example, if hearing device <NUM> is a sound processor included in a cochlear implant system, memory <NUM> may maintain data representative of sound processing programs that specify stimulation schemes used by processor <NUM> to direct a cochlear implant to provide electrical stimulation representative of the audio content to the user.

Processor <NUM> may be configured to perform various processing operations with respect to an audio signal detected by microphone <NUM>. Processor <NUM> is configured to receive the audio signal (e.g., a digitized version of the audio signal) from microphone <NUM> and process the audio content contained in the audio signal in accordance with a biomarker feature analysis , for example, to detect early signs of neurodegenerative disease. These and other operations that may be performed by processor <NUM> are described in more detail herein. In the description that follows, any references to operations performed by hearing device <NUM> may be understood to be performed by processor <NUM> of hearing device <NUM>. Processor <NUM> may be implemented by any suitable combination of hardware and software.

<FIG> illustrates an exemplary configuration <NUM> for biomarker analysis on a hearing device. Configuration <NUM> includes an environmental noise detector <NUM>, an own voice detector <NUM>, and a biomarker feature analyzer <NUM>, each of which may be included in and/or communicatively coupled to a hearing device as described herein.

As shown, environmental noise detector <NUM> and own voice detector <NUM> may receive an audio signal <NUM> (e.g., a signal detected by a microphone of the hearing device, a signal transmitted to the hearing device by an audio source, etc.). Environmental noise detector <NUM> may detect an environmental noise component of audio signal <NUM>. The environmental noise component may include data representative of environmental noise, which may include any audio present in an environment of a user of the hearing device, any audio detected by the hearing device from an environment of the user, and/or any audio excluding audio generated by the user. Environmental noise detector <NUM> may detect the environmental noise component in any suitable manner, examples of which are described herein. Environmental noise detector <NUM> may detect the environmental noise component and determines a level of the environmental noise. The level may be any suitable measurement of the environmental noise, such as a volume level, a relative volume level, a decibel level, an amplitude, a noise spectrum, a loudness level, etc. Environmental noise detector <NUM> provides environmental noise level data <NUM> representative of the environmental noise level to biomarker feature analyzer <NUM>.

Own voice detector <NUM> may receive audio signal <NUM> and detect an own voice component of audio signal <NUM>. The own voice component may include data representative of own voice content <NUM> generated by a voice of the user. Own voice detector <NUM> may detect the own voice component in any suitable manner, examples of which are described herein. Own voice detector <NUM> provides own voice content <NUM> to biomarker feature analyzer <NUM>.

Biomarker feature analyzer <NUM> receives environmental noise level data <NUM> and own voice content <NUM>. Biomarker feature analyzer <NUM> analyzes own voice content <NUM> for biomarkers. Such analysis is based on environmental noise level data <NUM>. Biomarker feature analyzer <NUM> determines whether environmental noise level data <NUM> indicates that the environmental noise level is below a threshold level so that own voice content <NUM> received may be clear enough for biomarker feature analysis. If the environmental noise level is too high (e.g., at or above the threshold level), own voice content <NUM> may include too much environmental noise for biomarker feature analyzer <NUM> to be able to accurately analyze for biomarkers.

Based on the environmental noise level being below the threshold, biomarker feature analyzer <NUM> analyzes own voice content <NUM> to extract biomarker features and analyze the biomarker features. Biomarker feature analyzer <NUM> may extract any suitable biomarker features and analyze the biomarker features for any suitable biological conditions of the user. Example biomarker features may include characteristics of respiration (e.g., relative loudness of respiration, latency of respiratory exchange, pause intervals per respiration, rate of speech respiration, etc.), characteristics of articulation (e.g., duration of unvoiced stops, decay of unvoiced fricatives, etc.), characteristics of timing (e.g., rate of speech timing, acceleration of speech timing, duration of pause intervals, entropy of speech timing, etc.), characteristics of phonation (e.g., gaping in-between voiced intervals, duration of voiced intervals, etc.), semantic content (e.g., a repetition of words by the user), and any other suitable characteristics. Such features may be analyzed for biological conditions such as neurodegenerative disorders (e.g., Parkinson's, Alzheimer's, etc.), or any other conditions that may be marked by speech and/or voice related biomarkers. Such analysis may also be performed using artificial intelligence techniques (e.g., deep learning, neural networks, etc.). Based on analysis of biomarker features, biomarker feature analyzer <NUM> may provide an output <NUM> indicating results of the biomarker feature analysis. As used herein, biomarker feature analysis heuristics may include extraction of biomarkers, biomarker features, and/or analysis of biomarker features for specific biological conditions.

<FIG> illustrates an exemplary hearing device <NUM> configured to perform biomarker analysis. Hearing device <NUM> may be an implementation of hearing device <NUM> that includes an implementation of configuration <NUM>. For instance, hearing device <NUM> includes a microphone <NUM> (e.g., an implementation of microphone <NUM>) along with additional own voice sensors <NUM>. Hearing device <NUM> further includes a classifier <NUM>, an environmental noise analysis module <NUM>, and a biomarker feature analysis module <NUM>.

Own voice sensors <NUM> may be implemented in any suitable manner, using any sensors and/or devices capable of detecting a user's own voice content (e.g., detect audio content of the user's own voice) and/or providing information indicative of a presence of the user's own voice (e.g., an indication that the user is speaking). For example, own voice sensors <NUM> may include one or more of any combination of voice pickup sensors, microphones, bone conduction microphones, canal microphones, etc. In some alternative configurations, hearing device <NUM> does not include own voice sensors <NUM>.

Own voice detection module <NUM> receives data or any other suitable signal from microphone <NUM> and/or own voice sensors <NUM>. Own voice detection module <NUM> determines, based on the data received from microphone <NUM> and own voice sensors <NUM> whether own voice content is present in an audio signal detected by microphone <NUM>. For example, own voice sensors <NUM> may provide information specifying portions (e.g., temporal portions) of the audio signal and/or times during which the user is speaking, which may indicate that the audio signal includes own voice content during those portions and/or times. Additionally or alternatively, own voice sensors <NUM> may provide data representative of the own voice content, such as from a localized and/or directional microphone configured to detect what the user is saying with minimal other audio content (e.g., environmental noise content).

In some examples, own voice detection module <NUM> may perform one or more algorithms to determine from the audio signal provided by microphone <NUM> whether the audio signal includes own voice content. For example, own voice detection module <NUM> may analyze a directionality and/or a volume level of the audio signal, store audio samples of the user's voice and perform voice and/or speech recognition algorithms on the audio signal, and/or use any other suitable techniques and algorithms for detecting own voice content in the audio signal. In some examples, hearing device <NUM> may include no own voice sensors <NUM> and own voice detection module <NUM> may detect own voice content based on such algorithms.

Classifier <NUM> may also receive the audio signal from microphone <NUM>. Classifier <NUM> may be configured to classify a type of the audio content represented by the audio signal. For instance, classifier <NUM> may classify the audio content as music, speech, background noise, etc. Such classifications may provide additional information for analyzing an environmental noise component of the audio signal. In some examples, classifier <NUM> may analyze the audio signal to extract the own voice component and/or the environmental noise component from the audio signal. Classifier <NUM> may be implemented in any suitable manner. In some alternative implementations, hearing device <NUM> does not include classifier <NUM>. In these alternative implementations, environmental noise analysis module <NUM> is configured to analyze the output of microphone <NUM> directly.

Environmental noise analysis module <NUM> may receive data from classifier <NUM>, such as an environmental noise component of the audio signal. Additionally or alternatively, environmental noise analysis module <NUM> may receive the audio signal from classifier <NUM> (or from microphone <NUM>) and analyze the audio signal for the environmental noise component of the audio signal. Based on the environmental noise component, environmental noise analysis module <NUM> determines an environmental noise level. Environmental noise analysis module <NUM> further determines whether the environmental noise level is above or below a threshold level that may allow for accurate biomarker feature analysis.

In some examples, environmental noise analysis module <NUM> may analyze the environmental noise component of the audio signal by analyzing one or more portions of the audio signal during which own voice content is absent (e.g., as determined by own voice detection module <NUM>). Portions of the audio signal in which own voice content is determined to be absent may indicate that an entirety of the audio content represented by the audio signal during those portions is environmental noise. Thus, environmental noise analysis module <NUM> may analyze the environmental noise level during those portions of the audio signal.

Additionally or alternatively, environmental noise analysis module <NUM> may sample portions of the audio signal during which own voice content is absent. Environmental noise analysis module <NUM> may analyze such samples to determine whether the environmental noise is suitable for performing noise canceling techniques and/or algorithms. For instance, if the environmental noise is of a known type, the environmental noise is substantially periodic, the environmental noise has an unchanging or periodic average characteristic, and/or the environmental noise has other such suitable characteristics for canceling, environmental noise analysis module <NUM> may cancel the environmental noise. Such canceling may be to an extent that the canceled environmental noise is below the threshold for biomarker feature analysis.

Biomarker feature analysis module <NUM> may receive own voice content from own voice detection module <NUM> and environmental noise data from environmental noise analysis module <NUM>. The own voice content may be included in portions of the audio signal during which own voice detection module <NUM> indicates that the user is speaking. Based on the environmental noise level being below a threshold, an entirety of the audio signal during those portions may be considered own voice content. Additionally or alternatively, the own voice content may be extracted from the audio signal based on the environmental noise data, such as by canceling the environmental noise data and/or filtering the environmental noise data. Additionally or alternatively, biomarker feature analysis module <NUM> may receive own voice content extracted from the audio signal (e.g., by classifier <NUM>).

Biomarker feature analysis module <NUM> may apply any suitable biomarker feature analysis to the own voice content. For example, biomarker feature analysis module <NUM> may extract biomarker features from the own voice content, analyze the biomarker features for early signs of neurodegenerative disorders, and output diagnosis data <NUM>. Diagnosis data <NUM> may include any data indicative of whether the user is presenting any such early signs of neurodegenerative disorders.

While hearing device <NUM> is shown to include specific modules, other example embodiments may omit modules, combine modules and/or include different modules may perform different portions of described functionality.

<FIG> illustrates another exemplary hearing device <NUM> including biomarker analysis. Hearing device <NUM> may be an implementation of hearing device <NUM> that includes an implementation of configuration <NUM>. For instance, hearing device <NUM> includes a microphone <NUM> (e.g., an implementation of microphone <NUM>). Hearing device <NUM> further includes a wireless interface <NUM>, an audio processing module <NUM>, a loudspeaker <NUM>, a voice activity detection module <NUM>, an environmental noise analysis module <NUM>, and a biomarker feature analysis module <NUM>.

Wireless interface <NUM> may be configured to wirelessly communicate with other devices, such as a mobile phone, a tablet, a computer, or any other device including a processor and a wireless interface. Such communication with other devices may provide additional information and/or context to hearing device <NUM> for determining own voice content and environmental noise content. Wireless interface <NUM> may be implemented in any suitable manner, such as a Bluetooth interface, a near field communication interface, or any other suitable interface configured to operate in accordance with any suitable wireless protocol.

For example, hearing device <NUM> may communicate with a smartphone to enable a user of hearing device <NUM> to have phone conversations via hearing device <NUM>. Generally during a phone conversation, the user may alternate dialogue with a person on a far end of the phone conversation. Audio received by the smartphone from the person on the far end may be transmitted to hearing device <NUM> via wireless interface <NUM>. Hearing device <NUM> may receive such far end voice content and process the far end voice content with audio processing module <NUM> (e.g., implemented in any suitable manner). Audio processing module <NUM> may provide processed audio to loudspeaker <NUM> (e.g., an implementation of output transducer <NUM>) to provide the audio content to the user.

As far end voice content is received directly via wireless interface <NUM>, speech content in an audio signal detected by microphone <NUM> may be considered likely to be own voice content. Hearing device <NUM> may detect a voice in the audio signal via voice activity detection module <NUM> (e.g., implemented by any combination of own voice sensors <NUM>, own voice detection module <NUM>, classifier <NUM>, or any other suitable manner). Further, a volume of the voice detected and/or a relative volume of the voice to an environmental noise level may provide additional indication whether the voice is own voice content. Hearing device may provide the own voice content to wireless interface <NUM> to transmit to the person on the other end of the phone conversation. Additionally or alternatively, hearing device <NUM> may receive own voice content from the smartphone (e.g., detected by a microphone on the smartphone) via wireless interface <NUM>. Hearing device <NUM> may compare own voice content received from the smartphone with own voice content detected in the audio signal and/or use the own voice content received from the smartphone to enhance or augment the own voice content detected in the audio signal.

Environmental noise analysis module <NUM> may analyze environmental noise in a manner similar to environmental noise analysis module <NUM>. Additionally, however, environmental noise analysis module <NUM> may receive information via wireless interface <NUM> that indicates portions of the audio signal detected by microphone <NUM> in which own voice content is absent (e.g., when wireless interface <NUM> is receiving incoming voice content in a phone conversation environment). Alternatively, environmental noise analysis module <NUM> may directly receive such portions of the audio signal in which own voice content is absent, as such portions of the audio signal may be considered entirely or substantially environmental noise content.

Biomarker feature analysis module <NUM> (e.g., an implementation of biomarker feature analysis module <NUM>) may receive own voice content from voice activity detection module <NUM> and environmental noise data from environmental noise analysis module <NUM> to apply biomarker feature analysis and output diagnosis data <NUM>, which may include any data indicative of whether the user presents any early signs of neurodegenerative disorders.

Hearing device <NUM> may receive other any other suitable information via wireless interface <NUM> that provides additional context for own voice content and/or environmental noise content. As another example, hearing device <NUM> may be communicatively coupled via wireless interface <NUM> with a device (e.g., a mobile phone) that includes an application that enables a user to record and/or present the user's voice for biomarker feature analysis. For instance, hearing device <NUM> may include a biomarker feature analysis mode, in which hearing device <NUM> presents to the user via the application on the device instructions to provide specific types of audio content. For example, hearing device <NUM> may instruct the user to present a sample of environmental noise content for a first portion of time. Hearing device <NUM> may determine based on the environmental noise content whether a level of the environmental noise is below a threshold. Based on such a determination, hearing device <NUM> may then instruct the user to provide own voice content. For instance, hearing device <NUM> may provide, via the application on the device, a set of words for the user to read aloud. The own voice content may be detected by microphone <NUM> and/or received via a microphone on the device. Hearing device <NUM> may then apply biomarker feature analysis heuristics to the own voice content as described herein. As an additional example, hearing device <NUM> may be communicatively coupled to a device (e.g., a mobile phone) that includes a push-to-talk over cellular feature. The push-to-talk feature may enable the device to operate in a fashion similar to a walkie-talkie, allowing the user to push (and/or push and hold) a button to configure the device between a voice reception mode and a transmit mode. The push-to-talk feature may provide additional context, as the user is likely providing own voice content during the transmit mode. Further, environmental noise may be analyzed during the voice reception mode. Based on such information, hearing device <NUM> may detect own voice content and apply biomarker feature analysis heuristics as described herein.

<FIG> illustrates a method <NUM> which may be performed by any of the hearing devices described herein. While <FIG> illustrates operations according to one embodiment, other embodiments may add to the operations shown in <FIG>.

In operation <NUM>, a hearing device configured to be worn by a user determines that an audio signal received by the hearing device includes own voice content representative of a voice of the user. Operation <NUM> may be performed in any of the ways described herein.

In operation <NUM>, the hearing device determines that an environmental noise level within the audio signal is below a threshold. Operation <NUM> may be performed in any of the ways described herein.

In operation <NUM>, the hearing device applies, based on the environmental noise level being below the threshold, a biomarker feature analysis heuristic to the own voice content. Operation <NUM> may be performed in any of the ways described herein.

Claim 1:
A hearing device configured to be worn by a user, the hearing device (<NUM>, <NUM>, <NUM>) comprising:
a microphone (<NUM>, <NUM>, <NUM>) configured to detect an audio signal (<NUM>);
a processor (<NUM>) communicatively coupled to the microphone and configured to:
determine that the audio signal includes own voice content (<NUM>) representative of a voice of the user;
the processor being characterised by further being configured to: determine that an environmental noise level within the audio signal is below a threshold such that the own voice content is clear enough for applying a biomarker feature analysis to the own voice content; and
apply, based on the environmental noise level within the audio signal being below the threshold, the biomarker feature analysis to the own voice content so as to extract biomarker features from the user's own voice and to analyze the extracted biomarker features for any suitable biological conditions of the user.