Patent Description:
In some speech recognition systems, a single audio capture device, such as a microphone or microphone panel, captures the speech emitted by multiple audio sources. In these systems, the audio capture device generally comingles the multiple audio signals emitted by the multiple audio sources and produces therefrom a collective audio signal provided to a system downstream.

Depending on the application, the system receiving the collective audio signal may assign varying priorities to the varying sources; in addition, the system may dynamically vary priorities for audio sources. Accordingly, effective use of the collective audio signal may require separating out the contributing audio signals, according to their respective audio sources. In one example, the collective audio signal may include words that are system commands, and it may be imperative for the receiving system to only respond to the commands from one audio source. In another example, the collective audio signal again may include words that are system commands, and the receiving system may be required to dynamically switch among audio sources that it responds to. Consequently, distinguishing sources in a multiple source environment is desired. The desired audio system distinguishes among audio sources contributing to a collective audio signal.

Patent documents <CIT> and <CIT> disclose examples of filtering out sources in a multiple source environment.

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description section.

An audio system according to claim <NUM> is provided.

Also provided is an audio processing method according to claim <NUM>.

Other desired features will become apparent from the following detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background.

A more complete understanding of the subject matter may be derived by referring to the following Detailed Description and Claims when considered in conjunction with the following figures, wherein like reference numerals refer to similar elements throughout the figures, and wherein:.

The following Detailed Description is merely exemplary in nature and is not intended to limit the embodiments of the subject matter or the application and uses of such embodiments. " Any implementation described herein as exemplary is not necessarily to be construed as preferred or advantageous over any other implementations. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding Technical Field, Background, Brief Summary or the following Detailed Description.

Techniques and technologies may be described herein in terms of functional and/or logical block components and with reference to symbolic representations of operations, processing tasks, and functions that may be performed by various computing components or devices. Operations, tasks, and functions are sometimes referred to as being processor-executed, computer-executed, computerized, software-implemented, or computer-implemented.

In practice, one or more processor devices can carry out the described operations, tasks, and functions by manipulating electrical signals representing data bits at memory locations in the processor electronics of the display system, as well as other processing of signals. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to the data bits. It should be appreciated that the various block components shown in the figures may be realized by any number of hardware, software, and/or firmware components configured to perform the specified functions.

The following descriptions may refer to elements or nodes or features being "coupled" together. As used herein, unless expressly stated otherwise, "coupled" means that one element/node/feature is directly or indirectly joined to (or directly or indirectly communicates with) another element/node/feature, and not necessarily mechanically. Thus, although the drawings may depict one exemplary arrangement of elements, additional intervening elements, devices, features, or components may be present in an embodiment of the depicted subject matter. It should be noted that many alternative or additional functional relationships or physical connections may be present in an embodiment of the subject matter.

The embodiment described herein is merely an example and serves as a guide for implementing the novel systems and methods herein in any audio communication application having multiple audio sources. It is readily appreciated that audio communication systems may be designed to meet a plurality of application-specific standards beyond the scope of the examples presented below. For the sake of brevity, conventional techniques related to audio processing, speech processing, sensors, and other functional aspects of certain systems and subsystems (and the individual operating components thereof) may not be described in detail herein. In addition, certain terminology may also be used in the following description for the purpose of reference only. Accordingly, the examples presented herein are intended as non-limiting.

As an overview, an audio communication system capable of distinguishing among multiple sources is described below. The audio communication system provided receives a collective audio signal comprising audio signals transmitted from multiple sources; the multiple sources may have transmitted respective audio signals in concurrent, overlapping, or sequential order. Audio tags are employed to identify audio signal sources. In an encoding and decoding approach, the collective audio signal is processed to identify and eliminate all undesired audio signals prior to performing speech recognition on the audio signal remaining (i.e., the desired audio signal), advantageously ensuring that speech recognition or further processing may be performed on only the audio signal from an intended or desired audio source.

In the following example, speaker devices are the sources of undesired audio signals a person is the source of a desired audio signal, and commands are generated for an external device based only upon audio signals (speech) from the person. However, one with skill in the art will readily appreciate that desired audio signals and/or undesired audio signals may be transmitted by a variety of combinations of devices and/or people. Additionally, the below described system and method are operable for communication environments with sources of audio signals placed in a variety of locations, such as, in enclosed spaces, unenclosed spaces, throughout a room, and in a small enclosed space, such as a helmet.

<FIG> provides an exemplary block diagram and <FIG> provides an exemplary flow chart for a system to distinguish sources in a multiple source environment. With reference to <FIG>, the system to distinguish sources <NUM> comprises a processor <NUM>, source of externally created audio tags <NUM>, and memory device <NUM>. In some embodiments, the system to distinguish sources <NUM> additionally comprises sensors <NUM> and circuitry <NUM>. One or more sources of undesired audio signals <NUM> and one source of a desired audio signal <NUM> each transmit audio signals. An audio capture device, such as one or more microphones <NUM>, is operatively coupled to the system to distinguish sources <NUM>; the audio capture device detects or captures separately-sourced audio transmissions and converts the audio transmissions into a collective audio signal suitably formatted for processing by the system to distinguish sources <NUM>. The system to distinguish sources <NUM> is operatively coupled to one or more devices under command <NUM>. The components of the system to distinguish sources <NUM> are described in detail below.

In the embodiment shown in <FIG>, the source of undesired audio signal <NUM> and the source of desired audio signal <NUM> are coupled to the system to distinguish sources <NUM>. As mentioned above, the sources of audio signals may comprise any combination of audio signal generating sources, such as speaker devices and humans. It will be appreciated that the speaker devices (one to a plurality) may be implemented using any one of numerous known speaker devices or transducer devices suitable for generating audio signals that are detectable by an audio capture device such as a microphone <NUM>. Although shown coupled to the system to distinguish sources <NUM> in <FIG>, in some embodiments the sources of audio signals are not coupled to the system to distinguish sources <NUM>. Functionality of the provided embodiments is described in more detail below.

The processor <NUM> may be implemented or realized with a variety of components, such as, a general purpose processor device, a content addressable memory, a digital signal processor, an application specific integrated circuit, a field programmable gate array, any suitable programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination designed to perform the functions described herein. A processor device may be realized as a microprocessor, a controller, a microcontroller, or a state machine. Moreover, a processor device may be implemented as a combination of computing devices, e.g., a combination of a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other such configuration.

In practice, processor <NUM> may further include or cooperate with system control computers, navigational equipment, memory devices (such as memory device <NUM>), power supplies, storage devices (such as databases), interface cards, and other standard components known in the art. In this respect, the functionality attributed to processor <NUM> may be distributed among processor components. In addition, processor <NUM> may include or cooperate with any number of software models, software programs (e.g., audio processing programs, speech recognition programs, gesture interpretation programs, display programs to generate visual feedback on a display unit, etc.) or instructions designed to carry out the various methods, process tasks, and calculations described.

Memory device <NUM> is coupled to the processor <NUM> such that the processor <NUM> can read information from, and write information to, the memory device <NUM>. Memory device <NUM> can be realized as RAM memory, flash memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. In the alternative, the memory device <NUM> may be integral to processor <NUM>. Memory device <NUM> may store non-transitory computer readable instructions for operating the system to distinguish sources <NUM>. In addition, the memory device may maintain program code associated with various functional or logical module/components of the system described herein.

The system to distinguish sources <NUM> employs encoding/decoding techniques using audio tags to identify and eliminate undesired audio signals from the collective audio signal. Audio tags may be audible, sub audible, and/or super audible carrier signals that are overlaid on the audio signals generated by a respective source. In an alternative, audio tags may be a separate radio frequency signal or data packet associated with a respective source of audio signals. Audio tags comprise at least one signal characteristic from the set of signal characteristics including: analog, digital, continuous, pulsed, patterned, audible, sub audible and super audible. In operation, the system to distinguish sources <NUM> creates or identifies, for each audio signal source of a plurality of audio signal sources, a unique audio tag, and an association between each unique audio tag and respective audio signal source. The association is then stored for reference during operation of the system to distinguish sources <NUM>.

Audio tags may be externally created or naturally occurring. The source of externally created audio tags <NUM> is a source of application specific audio tags used for identifying sources of audio signals. Depending upon the application, the unique externally created audio tags may be added to respective sources of audio signals at a prior location and/or prior time, or may be added real-time, as the system to distinguish sources <NUM> operates.

Naturally occurring audio tags also may be identified a priori or during operation of the system to distinguish sources <NUM>. Various calibration techniques, employing, for example, sensors <NUM> and circuitry <NUM>, may be used to identify naturally occurring audio tags. Naturally occurring audio tags may comprise many of the same audio features as the externally created audio tags, described above, and may further vary depending on source type. Moreover, naturally occurring audio tags associated with device audio sources (such as audio speakers, as opposed to human audio sources), comprise audio in frequencies within a range of about <NUM>-<NUM>, lower audio quality than that of human speech, and digital audio artifacts like humming sounds (as used herein, audio quality comprises harmonic content of a sound and associated dynamic characteristics of the sound). In contrast, naturally occurring audio tags in human speech may comprise frequency content outside of the <NUM>-<NUM> range, higher audio quality than audio speakers provide, and digital and/or analog artifacts caused by, for example, increased air pressure on a microphone <NUM>.

As described above, unique audio tags are used to distinguish individual sources of audio signals; therefore, the association between an audio tag and its audio signal source is a unique data pairing useful in the encoding and decoding process of identifying audio sources. Once an audio tag is added to an audio signal source, this pairing is referred to as "audio tag data" and stored for reference. The storage format of the audio tag data may be, for example, a data structure or a lookup table. In an embodiment, audio tag data is arranged and stored in the memory device <NUM> as a data structure, comprising [audio tag: source].

In some embodiments, the system to distinguish sources <NUM> has access to at least one of: the source of undesired audio signal and the source of desired audio signal. In these embodiments, prior to operation, the system to distinguish sources <NUM> either (i) adds unique externally created audio tags to the one or more sources of the undesired audio signal or (ii) identifies a naturally occurring audio tag associated with the source of the desired audio signal. In operation of these embodiments, the system to distinguish sources <NUM> is either (i) filtering out undesired audio signals by recognizing undesired sources based on their audio tags, or (ii) filtering out undesired audio signals by recognizing a desired audio signal source by its' "desired" audio tag, and eliminating all audio signals from sources not having the desired audio tag. In embodiments in which the system to distinguish sources <NUM> has access to the source of desired audio signal; audio signals transmitted from the source of desired audio signal may be processed to identify a naturally occurring audio tag associated with the source of desired audio signal. For example, a person speaks into the system until the system to distinguish sources <NUM> identifies a voice pattern or other audio quality characteristic to use as the naturally occurring audio tag. The process then creates an association between the naturally occurring tag and the source of desired audio signal, and stores the association in memory device <NUM>.

In other embodiments, the system to distinguish sources <NUM> does not have access to any sources of the audio signal (neither the source(s) of undesired audio signal <NUM>, or the source of desired audio signal <NUM>); in such embodiments, the system to distinguish sources <NUM> audio processes the collective audio signal to identify naturally occurring audio tags therein, and then sorts or separates the audio signal in accordance with the identified naturally occurring audio tags. The identified naturally occurring audio tags are associated with audio sources and the associations are stored. In these embodiments, calibration techniques using various combinations of sensors <NUM> and circuity <NUM> may be employed to assist the processor <NUM> in identifying sources of audio signals. A first non-limiting example includes placing sensors <NUM> and circuity <NUM> on a power line of a speaker to detect when the speaker is receiving power. A second non-limiting example includes employing multiple sensors <NUM> coupled to the processor <NUM>; the sensors <NUM> spatially arranged, such that each incoming audio signal is detected by multiple sensors <NUM>. The audio signals have different travel times to each of the multiple sensors. The resultant sensor data is triangulated by the processor <NUM> and processed with the collective audio signal to locate the source and then use the source location to determine whether the source is likely a person (source of desired audio signals) or a speaker (source of undesired audio signals). Therefrom, undesired audio signal may be identified and eliminated from the audio signal prior to speech processing and command generation.

As used herein, audio processing comprises audio processing and speech recognition. The required audio processing software and speech recognition software may be stored in memory device <NUM>. A variety of currently available audio processing and speech recognition software products may be used to process the audio signal, parse it to eliminate undesired audio signal and generate a text stream based on only desired audio signal. Non-limiting examples of speech recognition algorithms include hidden Markov models, dynamic time warping (DTW), neural networks, deep neural networks, or the like. The processor <NUM> and memory device <NUM> cooperate to further process the text stream with the undesired audio signal removed (i.e., the text stream based on desired audio signal) to identify (i) an intended command, and (ii) an intended device under command <NUM>. In this manner, processor <NUM> may generate a command from the audio signal that is responsive to the desired audio signal and exclusive of the undesired audio signal.

A user interface <NUM> may optionally be coupled to the processor <NUM> and memory device <NUM>. User interface <NUM> is configured to receive input from a user and, in response to the user input, supply command signals to the processor <NUM>. The user interface <NUM> may comprise any combination of various known user interface devices, such as: a voice recognition device, a gesture recognition device, a keyboard, a touch sensitive screen, and a cursor control device, such as a mouse, trackball, joystick, or combination of buttons, switches, or knobs configured to receive user input. In some embodiments the user interface <NUM> and a display device (not shown) may be combined, for example, as a touch sensitive screen.

In some embodiments, the system to distinguish sources <NUM> employs enable signals provided via user interface <NUM>. For example, a first enable signal may be required by the processor <NUM> to enable audio processing of the incoming audio stream audio signal, and/or, a second enable signal may be required by the processor <NUM> to generate (based on an intended command) the command for the external device under command <NUM>. A variety of enable techniques may be employed. As a non-limiting example, the first enable signal may comprise toggling a push-to-talk switch, and the second enable may comprise toggling another switch, or a gesture interpretation. In other embodiments, the system to distinguish sources <NUM> employs enable signals provided by the audio tags described herein; for example, an audio tag associated with a source of desired audio signals may serve as an enable signal.

In <FIG>, an exemplary process for distinguishing sources in a multiple source environment is described. It is to be understood that the provided process steps may be differently arranged, and process steps may be added or combined without straying from the inventive concept provided herein. The process begins by assessing the communication environment at STEP <NUM> to determine whether the system to distinguish sources <NUM> (hereinafter referred to as "the system") has access to the source of undesired audio signal (STEP <NUM>) and to determine whether the system has access to the source of a desired audio signal (STEP <NUM>). When the system has access to a source of the undesired audio signal, the processor commands a source of externally created audio tags <NUM> to add a unique externally created audio tag (ECT) to the source of undesired audio signal, and an association is created between the ECT and the respective undesired audio signal (STEP <NUM>). It is to be understood that, when the communication environment contains a plurality of sources of undesired audio signals, each source of undesired audio signals of the plurality of sources of undesired audio signals is assessed for system control, each source of undesired audio signals of the plurality of sources of undesired audio signals has a unique ECT added and association created thereto, and each association is stored in memory device <NUM> for reference.

In STEP <NUM>, when a source of audio signals is a person, and the system has a-priori access to the person, the system may calibrate the person's speech (audio signals). In the present example, the source of the desired audio signals is a person. The system calibrates the person's speech by audio processing samples of audio signals from the person to identify a naturally occurring tag. As described above, this process may employ calibration techniques using sensors <NUM> and circuity <NUM>. Upon identifying a naturally occurring tag, an association is created and between the naturally occurring tag and the source of desired audio signal; the association is stored in memory device <NUM> for reference.

At STEP <NUM>, a collective audio signal is received from an audio capture device, such as one or more microphones <NUM>. The audio signal may comprise any combination of: an audio tag, a desired audio signal, and one or more undesired audio signals. At STEP <NUM>, the audio signal is processed to check for audio tags present. Checking for audio tags present comprises referencing the lookup table of associations between audio tags and sources that has been stored in memory device <NUM>, to identify audio tags. When an audio tag in the audio signal is located in the memory device <NUM>, the stored association provides the respective audio source. As mentioned above, in some embodiments, at STEP <NUM>, a first enable signal may be required to be asserted to initiate audio processing to check for audio tags.

If one or more audio tags are present in the audio signal at STEP <NUM>, the audio signal is audio processed to (i) determine whether a desired audio signal is present and (ii) to eliminate any undesired audio signals in STEP <NUM>. The processor <NUM> may reference audio processing algorithms (including speech recognition algorithms) stored in memory device <NUM> to perform audio processing steps. As mentioned above, eliminating undesired audio signals may be performed in at least two methodologies. First, eliminating undesired audio signals may comprise using audio tags associated with sources of undesired audio signal to identify and filter respective undesired audio signals out of the audio signal, leaving behind only a remaining desired audio signal. And, second, eliminating undesired audio signals may comprise using a desired audio tag associated with a source of desired audio signals as an exclusive filter such that audio signals without the desired audio tag are excluded or filtered out of the audio signal. Regardless of the methodology performed, any undesired audio signals are eliminated from the collective audio signal using audio tags, and what remains is the desired audio signal. The desired audio signal may be further audio processed (using speech recognition software) to identify an intended command and an intended external device under command <NUM> for the command in STEP <NUM>.

In STEP <NUM>, the results of STEP <NUM> are processed, and an identified "intended command" may be used to generate a command for an external device (device under command <NUM>), wherein the command is exclusive of undesired audio signals and responsive to only a desired audio signal. As mentioned above, in some embodiments, at STEP <NUM>, a second enable signal may be required to be asserted to generate a command in STEP <NUM>. Where a second optional enable is employed, the second enable must be asserted for the process <NUM> to generate a command based on the intended command from step <NUM>. The device under command <NUM> may be any external device; a non-limiting example is a display unit displaying a menu system. In which case, the generated command may be to select an item on the menu system.

If audio tags are not present in the audio signal at STEP <NUM>, the audio signal is processed in STEP <NUM> to identify a naturally occurring audio tag and associate it with a source. A naturally occurring audio tag may be present in the undesired audio signal and/or in the desired audio signal. Processing continues in STEP <NUM> until either (i) a naturally occurring audio tag is associated with a source determined to be a source of desired audio signals, or (ii) for all undesired audio signals determined present in the audio signal, naturally occurring audio tags are associated with their respective sources. Associations are created and stored as described herein. The process of identifying naturally occurring audio tags in this step is similar to the process of identifying the naturally occurring audio tag when the system has access to a source of desired audio signal described in STEP <NUM> above; various combinations of sensors <NUM> and circuity <NUM> may be employed. In addition, algorithms for locating sources may be performed. For example, to triangulate signals and identify source locations, the system may measure frequency content with respect to the range of about <NUM>-<NUM>, and/or detect digital and/or analog artifacts caused by, for example, increased air pressure on a microphone. At the completion of STEP <NUM>, audio processing may be performed at STEP <NUM>.

Claim 1:
An audio system (<NUM>), comprising:
a memory device (<NUM>) for storage of audio processing algorithms and audio tags;
an audio capture device (<NUM>);
a processor (<NUM>) coupled to the memory device, the audio capture device, and to a source of undesired audio signal (<NUM>) or a source of desired audio signal (<NUM>), the source of the undesired audio signal and the source of the desired audio signal being separate, the processor configured to, prior to the operation of processing a collective audio signal:
determine whether it has access to the source of undesired audio signals;
determine whether it has access to the source of desired audio signals; and
when the processor has access to the source of desired audio signals, process audio signals generated from the source of desired audio signals to identify a naturally occurring audio tag, NT, associated with the source of desired audio signal, wherein the NT comprises detected frequency content outside of the <NUM>-<NUM> range, higher audio quality than audio speakers are capable of providing, or an audio artefact caused by increased air pressure on a microphone; and
when the processor has access to the source of undesired audio signals, add a unique externally created audio tag, ECT, to the audio signals generated from the source of undesired audio signals;
the audio capture device (<NUM>) configured to receive a plurality of separately-sourced audio transmissions and convert the audio transmissions into one collective audio signal, the separately sourced audio transmissions including transmissions from the source of desired audio and the source of undesired audio;
the processor further configured to, after identifying the naturally occurring audio tag associated with the source of desired audio signals or adding a unique externally created audio tag to the audio signal generated by the source of undesired audio signals: receive (<NUM>), from the audio capture device (<NUM>), the collective audio signal, and, eliminate the undesired audio signals by obtaining a desired audio signal (<NUM>) by either (i) filtering out undesired audio signals in the collective audio signal by recognizing the undesired audio source based on its ECT, or (ii) recognizing the desired audio source by its NT, and eliminating all audio signals from sources not having the desired audio tag, prior to using a speech recognition algorithm to identify an intended command in the desired audio signal (<NUM>).