Patent Description:
A telecommunications device such as an audio conferencing system generally includes both a loudspeaker and a microphone. The two parties in a communication may be referred to as the near end party and the far end party. The near end party is proximal to a first telecommunications device, and the far end party is at a different location than the near end party and communicates using a second telecommunications device via a wired or wireless telecommunications network. The microphone of the near end device captures not only the speech of the near end party, but may also capture the speech of the far end party that has been output from the loudspeaker at the near end. The output from the loudspeaker that is captured by the microphone is generally referred to as echo. The near end telecommunications device generally includes an echo management system for reducing the echo prior to transmitting the audio captured at the near end to the far end.

The term double talk is generally used to describe the situation when both parties in a conversation are talking at the same time. Both parties consider double talk to be annoying, and generally one will stop talking. It would be advantageous to have a device that can respond appropriately when double talk occurs in order to improve the quality of the communication, thereby enhancing the user experience.

<CIT> discloses an echo cancellation system, method and apparatus. The system includes a double talk detector configured for detecting a double talk condition by monitoring voice energy in a first frequency band. An adaptive filter is configured for producing an echo signal based on a set of coefficients, and holds the set of coefficients constant when the double talk detector detects the double talk condition.

<CIT> discloses an echo cancelling algorithm in a communication device initializing a step size value used in an adaptive echo filter based on a background noise signal power level relative to a power level of a received signal and a power level of an echo estimate relative to an output of an echo canceller. The algorithm then adjusts the step size value.

When double talk exists, it is desirable to transmit the near end speech to the far end without performing much (or any) echo reduction, in order to provide the audible clue to the far end that double talk is occurring. The telecommunications device at the near end may include a double talk detector to detect double talk, and in turn to control the echo management system not to perform too much attenuation.

One issue with existing double talk detection systems is that the non-stationary nature of voice signals results in a high false positive rate of detecting double talk. Furthermore, for telecommunications devices such as laptop computers where the loudspeaker is in close proximity to the microphone, the echo management system needs to perform more attenuation as a default, so false positive detection of double talk becomes even more undesirable in a conversation. Given the above, there is a need to improve double talk detection, especially for devices where the loudspeaker is in close proximity to the microphone.

According to an embodiment, a computer-implemented method of audio processing includes receiving a first audio signal, wherein the first audio signal has a first sampling frequency. The method further includes up-sampling the first audio signal to generate a second audio signal, wherein the second audio signal has a second sampling frequency that is greater than the first sampling frequency. The method further includes outputting, by a loudspeaker, a loudspeaker output corresponding to the second audio signal. The method further includes capturing, by a microphone, a third audio signal, wherein the third audio signal is sampled at the second sampling frequency. The method further includes determining a signal power of the third audio signal and detecting double talk when there is signal power of the third audio signal determined in a frequency band having frequencies all greater than half the first sampling frequency.

The method may further include selectively generating a control signal when the double talk is detected, and performing echo management on the third audio signal according to the control signal.

Determining the signal power of the third audio signal and detecting the double talk may include measuring the signal power of the third audio signal in the frequency band greater than the first sampling frequency; tracking a background noise power of the third audio signal in the frequency band greater than the first sampling frequency; and detecting the double talk as a result of comparing the signal power of the third audio signal in the frequency band having frequencies all greater than half the first sampling frequency and the background noise power of the third audio signal in the frequency band having frequencies all greater than half the first sampling frequency.

According to another embodiment, an apparatus includes a loudspeaker, a microphone and a processor. The processor is configured to control the apparatus to implement one or more of the methods described herein. The apparatus may additionally include similar details to those of one or more of the methods described herein.

According to another embodiment, a non-transitory computer readable medium stores a computer program that, when executed by a processor, controls an apparatus to execute processing including one or more of the methods described herein.

The following detailed description and accompanying drawings provide a further understanding of the nature and advantages of various implementations.

Described herein are techniques related to double talk detection. In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be evident, however, to one skilled in the art that the present disclosure as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

In the following description, various methods, processes and procedures are detailed. Although particular steps may be described in a certain order, such order is mainly for convenience and clarity. A particular step may be repeated more than once, may occur before or after other steps (even if those steps are otherwise described in another order), and may occur in parallel with other steps. A second step is required to follow a first step only when the first step must be completed before the second step is begun. Such a situation will be specifically pointed out when not clear from the context.

In this document, the terms "and", "or" and "and/or" are used. Such terms are to be read as having an inclusive meaning. For example, "A and B" may mean at least the following: "both A and B", "at least both A and B". As another example, "A or B" may mean at least the following: "at least A", "at least B", "both A and B", "at least both A and B". As another example, "A and/or B" may mean at least the following: "A and B", "A or B". When an exclusive-or is intended, such will be specifically noted (e.g., "either A or B", "at most one of A and B").

This document describes various processing functions that are associated with structures such as blocks, elements, components, circuits, etc. In general, these structures may be implemented by a processor that is controlled by one or more computer programs.

<FIG> is a block diagram of an audio processing system <NUM>. The audio processing system <NUM> may be implemented in various devices, such as laptop computers, mobile telephones, speakerphones, audioconferencing systems, videoconferencing systems, etc. For example, the audio processing system <NUM> may be implemented in a laptop computer, with various components implemented by computer programs that the laptop computer executes. The audio processing system <NUM> includes a communication application <NUM>, an audio driver system <NUM>, an audio codec system <NUM>, a loudspeaker <NUM>, and a microphone <NUM>. The audio processing system <NUM> may include other components that (for brevity) are not discussed in detail.

The communication application <NUM> generally controls the audio inputs and outputs of the device that implements the audio processing system <NUM>. For example, when the implementing device is a laptop computer, the communications application <NUM> may be a computer program such as a Microsoft Skype™ application, a Microsoft Teams™ application, a Zoom™ application, etc. The communication application <NUM> communicates with a network (not shown), to receive audio from remote devices (also referred to as far end devices) for output by the audio processing system <NUM> (also referred to as the near end device), and to transmit audio captured by the audio processing system <NUM> to the remote devices. The audio received from the network for near end output is referred to as the playback audio signal <NUM>, and the audio transmitted to the network for far end output is referred to as the captured audio signal <NUM>.

The audio driver system <NUM> generally performs audio processing on the signals it receives and generates processed audio signals. The audio driver system <NUM> receives the playback audio signal <NUM> and generates a playback audio signal <NUM>; and receives a captured audio signal <NUM> and generates the captured audio signal <NUM>. The communications application <NUM> may offload various audio processing processes to the audio driver system <NUM>, and the audio driver system <NUM> may be a component of the communications application <NUM>. The audio driver system <NUM> may be referred to as a playback/capture stack, an audio processing object (APO), etc. An example of the audio driver system <NUM> is the Dolby Voice™ communications system. The audio driver system <NUM> provides the playback audio signal <NUM> to the audio codec system <NUM> and receives the captured audio signal <NUM> from the audio codec system <NUM>.

The audio driver system <NUM> includes various processing modules, including an echo management system <NUM>. The echo management system <NUM> generally attenuates the echo of the far end voice output from the loudspeaker <NUM> and captured by the microphone <NUM>, while preserving the near end voice captured by the microphone <NUM>. The echo management system <NUM> includes an echo canceller <NUM>, an echo suppressor <NUM>, and a double talk detector <NUM>.

The echo canceller <NUM> generally performs echo cancellation on the captured audio signal <NUM>. Echo cancellation may also be referred to as acoustic echo cancellation. In general, echo cancellation applies a linear attenuation to the signal. The echo canceller may be implemented with an adaptive filter. The adaptive filter models the room response of the combined system of the loudspeaker <NUM> and the microphone <NUM>. The echo canceller <NUM> typically may apply up to between <NUM> and <NUM> dB of attenuation to the captured audio signal <NUM>.

The echo suppressor <NUM> generally performs echo suppression on the captured audio signal <NUM>. In general, echo suppression applies a non-linear attenuation to the signal. The non-linear attenuation may be performed on the basis of power bands, and the echo suppressor <NUM> may apply different suppressions to different bands. If the echo suppressor <NUM> detects echo in particular bands, the echo suppressor <NUM> applies suppression to those particular bands. The echo suppressor <NUM> typically may apply up to between <NUM> and <NUM> dB of attenuation to the captured audio signal <NUM>.

The double talk detector <NUM> generally generates a control signal <NUM> for controlling the echo canceller <NUM> and the echo suppressor <NUM>. Double talk generally refers to the microphone <NUM> capturing audio (at the near end) concurrently with the loudspeaker <NUM> outputting audio (received from the far end). When there is no double talk, the captured audio signal <NUM> includes only echo of the far end speech output by the loudspeaker <NUM>, and the control signal <NUM> controls the echo management system <NUM> to perform attenuation to reduce the amount of echo in the captured audio signal <NUM> transmitted to the far end. When there is double talk, the captured audio signal <NUM> includes both the echo of the far end and near end speech captured by the microphone <NUM>, and the control signal <NUM> controls the echo suppressor <NUM> to perform little (or no) attenuation when generating the captured audio signal <NUM> transmitted to the far end; the control signal <NUM> may also control the echo canceller <NUM> to stop the adaptive filter from updating, in order to reduce mis-adaptation due to double talk. Additional details of the double talk detector <NUM> are provided below with reference to <FIG>.

The audio codec system <NUM> generally performs analog-to-digital and digital-to-analog conversion on the signals it receives. The audio codec system <NUM> also performs up-sampling and down-sampling, as further detailed below with reference to <FIG>. The audio codec system <NUM> receives the playback audio signal <NUM>, performs digital-to-analog conversion, and generates a playback audio signal <NUM>. The audio codec system <NUM> receives a captured audio signal <NUM>, performs analog-to-digital conversion, and generates the captured audio signal <NUM>. The audio codec system <NUM> provides the playback audio signal <NUM> to the loudspeaker <NUM>, and receives the captured audio signal <NUM> from the microphone <NUM>. Additional details of the audio codec system <NUM> are provided below with reference to <FIG>.

The loudspeaker <NUM> generally outputs sound corresponding to the playback audio signal <NUM>.

The microphone <NUM> generally captures sound in the environment where the device that implements the audio processing system <NUM> is present, and generates the captured audio signal <NUM>. The captured sound not only includes the desired sound (e.g., the speech of persons speaking in the near end environment), but also sound output from the loudspeaker <NUM>, which is referred to as "echo". One goal of the echo management system <NUM> is generally to reduce (or attenuate or remove) the echo from the captured audio signal <NUM>, in appropriate circumstances.

The echo management system <NUM> is generally operable in three situations, based on the combinations of either or both of far end speech and near end speech being present. (The term "speech" is used because speech is generally the signal of interest; however, the signals captured at the near end and the far end will generally include both speech and other non-speech audio such as music, environmental noise, etc., with the term "speech" not meant to exclude non-speech audio. ) When there is far end speech output by the loudspeaker <NUM> and no near end speech, the microphone <NUM> captures only the echo of the far end speech, so the echo management system <NUM> performs echo management to cancel the far end speech from the captured audio signal <NUM> when generating the captured audio signal <NUM> (e.g., a large amount of attenuation). When there is both far end speech output by the loudspeaker <NUM> and near end speech, the microphone <NUM> captures both the echo of the far end speech and the near end speech ("double talk"), so the echo management system <NUM> operates in accordance with the control signal <NUM>. When there is no far end speech, the microphone <NUM> captures only near end speech, so the echo management system <NUM> performs minimal (or no) attenuation. In this manner, the control signal <NUM> helps the echo management system <NUM> to differentiate between the three situations.

In summary, the echo management system <NUM> generally operates to cancel the far end speech from the captured audio signal <NUM>, leaving the near end speech when generating the captured audio signal <NUM>. The double talk detector <NUM> generally controls the echo management system <NUM> to avoid applying aggressive attenuation when there is near end speech and no far end speech. For example, in an ideal situation the echo suppressor <NUM> performs minimal (or no) attenuation when there is near end speech.

The amount of echo present in the captured audio signal <NUM> may vary depending upon the physical attributes of the device that implements the audio processing system <NUM>. For example, for a laptop device, the physical separation between the loudspeaker and the microphone <NUM> may only provide approximately <NUM> dB of signal attenuation. In telecommunications systems, the user experience generally prefers between <NUM> and <NUM> dB of echo cancellation and echo suppression, so the echo management system <NUM> generally operates to provide the other <NUM> to <NUM> dB of echo cancellation and echo suppression.

<FIG> is a block diagram showing additional details of the audio codec system <NUM> (see <FIG>). The audio codec system <NUM> includes an up-sampler <NUM>, a signal converter <NUM>, and a down-sampler <NUM>. Other components shown in <FIG> are similar to those described above with reference to <FIG> (e.g., the audio driver system <NUM>, the loudspeaker <NUM>, the microphone <NUM>, the double talk detector <NUM>, etc.) that have similar reference numerals. The audio codec system <NUM> may include additional components that (for brevity) are not discussed in detail.

The up-sampler <NUM> receives a playback audio signal <NUM>, performs up-sampling, and generates an up-sampled signal <NUM>. The playback audio signal <NUM> generally corresponds to the playback audio signal <NUM> provided by the audio driver system <NUM> (see <FIG>). Up-sampling generally refers to converting a signal at a given sampling frequency to a higher sampling frequency. For example, the playback audio signal <NUM> may have a sampling frequency of <NUM> (e.g., for a telephone connection), <NUM> (e.g., a Microsoft Teams™ audio signal), <NUM> (e.g., a Zoom™ audio signal), etc.; and the up-sampled signal <NUM> may have a sampling frequency of <NUM> (e.g., 2x the <NUM> signal, etc.), <NUM> (e.g., 4x the <NUM> signal, 2x the <NUM> signal, <NUM>. 333x the <NUM> signal, etc.), <NUM> (6x the <NUM> signal, 4x the <NUM> signal, 2x the <NUM> signal, etc.), etc. The lower sampling frequency may be referred to as fs0, and the higher sampling frequency may be referred to as fs1.

The signal converter <NUM> generally performs analog-to-digital and digital-to-analog conversion on signals. The signal converter <NUM> receives the up-sampled signal <NUM>, performs digital-to-analog conversion, and generates the playback audio signal <NUM> for output by the loudspeaker <NUM>. The signal converter <NUM> receives the captured audio signal <NUM> captured by the microphone <NUM>, performs analog-to-digital conversion, and generates a captured audio signal <NUM>. The signal converter <NUM> generally performs conversion at the higher sampling frequency (e.g. <NUM>, corresponding to fs1 that is higher than the lower sampling frequency fs0 of the playback audio signal <NUM>), so the captured audio signal <NUM> also has the higher sampling frequency (e.g. <NUM>).

The down-sampler <NUM> receives the captured audio signal <NUM>, performs down-sampling, and generates a down-sampled signal <NUM>. The down-sampled signal <NUM> generally corresponds to the captured audio signal <NUM> provided to the audio driver system <NUM> (see <FIG>). Down-sampling generally refers to converting a signal at a given sampling frequency to a lower sampling frequency. For example, the captured audio signal <NUM> may have a sampling frequency of <NUM> (e.g., for a telephone connection), <NUM> (e.g., a Microsoft Teams™ audio signal), <NUM> (e.g., a Zoom™ audio signal), etc. In general, the down-sampled signal <NUM> and the playback audio signal <NUM> will have the same sampling frequency.

The double talk detector <NUM> receives the captured audio signal <NUM> from the audio codec system <NUM>. Consequently, the captured audio signal <NUM> of <FIG> corresponds to both the captured audio signal <NUM> and the down-sampled signal <NUM>.

Optionally, the double talk detector <NUM> may also receive the playback audio signal <NUM> that the audio driver system <NUM> provides to the audio codec system <NUM>. This optional arrangement is discussed in more detail below with reference to <FIG>.

Because the up-sampled signal <NUM> that is provided to the loudspeaker <NUM> for output results from up-sampling the playback audio signal <NUM>, the echo of the up-sampled signal <NUM> captured by the microphone <NUM> will lack signal energy at frequencies above half of the sampling frequency of the playback audio signal <NUM>, as further detailed with reference to <FIG>.

<FIG> are graphs showing the power spectrum of the up-sampled signal <NUM> and the captured audio signal <NUM> in two situations. <FIG> shows the power spectrum when the microphone <NUM> captures only far-end speech that has been output by the loudspeaker <NUM> (see <FIG>) without any captured near-end speech. <FIG> shows the power spectrum when the microphone <NUM> captures both far-end speech that has been output by the loudspeaker <NUM> (see <FIG>) and near-end speech. These two situations illustrate that when the microphone <NUM> has captured a signal, the audio processing system <NUM> needs to determine whether double talk exists (in which case it needs to apply little or no attenuation) or whether double talk does not exist (in which case it needs to apply a relatively large amount of attenuation due to the echo of the far end signal).

In <FIG>, the y-axis is signal power and the x-axis is frequency. The frequencies shown are ½ fs0 and ½ fs1 because as per the Nyquist-Shannon sampling theorem, a given sample rate S allows accurate reconstruction of a signal with a maximum frequency present in the signal of ½ S. For example, fs0 may be <NUM> and fs1 may be <NUM>, in which case ½ fs0 is <NUM> and ½ fs1 is <NUM>. When the microphone <NUM> captures only far-end speech that has been output by the loudspeaker <NUM> without any captured near-end speech, the up-sampled signal <NUM> and the captured audio signal <NUM> both have signal power only below ½ fs0. This is because the playback audio signal <NUM> has a sampling frequency of fs0 and thus has no signal energy above ½ fs0, so performing up-sampling likewise results in the up-sampled signal <NUM> also having no signal energy above ½ fs0. Thus, in the situation of <FIG>, the absence of signal power above ½ fs0 indicates the absence of double talk.

In <FIG>, when the microphone <NUM> captures both far-end speech that has been output by the loudspeaker <NUM> (see <FIG>) and near-end speech, the up-sampled signal <NUM> has signal power only below ½ fs0, but the captured audio signal <NUM> has signal power above ½ fs0 (both below ½ fs0 and between ½ fs0 and ½ fs1). This is because the captured audio signal <NUM> has a sampling frequency of fs1 and thus the near-end speech has energy that is captured up to ½ fs1, but the playback audio signal <NUM> still has no signal energy above ½ fs0. Thus, in the situation of <FIG>, the presence of signal power above ½ fs0 (e.g., between ½ fs0 and ½ fs1) indicates the presence of double talk.

<FIG> is a block diagram showing additional details of the double talk detector <NUM> (see also <FIG>). The double talk detector <NUM> includes a power meter <NUM>, a minimum follower <NUM>, and a decision maker <NUM> The double talk detector <NUM> may include other components that (for brevity) are not discussed in detail.

The power meter <NUM> generally receives the captured audio signal <NUM> (see <FIG>), measures the power between ½ fs0 and ½ fs1, and generates a power signal <NUM>. The power signal <NUM> generally corresponds to the root mean square (rms) power in the band between ½ fs0 and ½ fs1; it may also be referred to as the instant power or the smoothed power of the captured audio signal <NUM>.

The minimum follower <NUM> generally receives the power signal <NUM>, tracks the background noise power, and generates a background noise power signal <NUM>. The background noise power signal <NUM> generally corresponds to the background noise power between ½ fs0 and ½ fs1 of the power signal <NUM>.

The decision maker <NUM> generally receives the power signal <NUM> and the background noise power signal <NUM>, compares their levels, and generates the control signal <NUM>. The decision maker <NUM> may operate according to a hysteresis decision process, e.g., to filter the inputs so that the output reacts less rapidly than it otherwise would by taking recent system history into account. When there is no double talk and the level of the power signal <NUM> exceeds the level of the background noise power signal <NUM> by a first threshold amount, the decision maker <NUM> fires. When there is double talk (that is, the decision maker <NUM> is in the fire state), the decision maker <NUM> changes to the off state only when the power signal <NUM> falls below a second threshold amount.

<FIG> is a block diagram of a double talk detector <NUM>. The double talk detector <NUM> is similar to the double talk detector <NUM> (see <FIG>), with additional components, and that also receives the playback audio signal <NUM> (see <FIG>). Devices such as laptops and mobile telephones often implement the loudspeaker <NUM> (see <FIG>) using micro-speakers. For micro-speakers, the transducer components and general mechanical distortions of the device may create additional power in the frequency range of [½ fs0, ½ fs1]. The double talk detector <NUM> may be used in such a case to reduce the false alarm rate (e.g., a false alarm due to detecting that near end voice is captured when in actuality there is no near end voice).

The double talk detector <NUM> includes a band pass filter <NUM>, a power meter <NUM>, and a non-linear regulator <NUM>. The double talk detector <NUM> also includes a power meter <NUM>, a minimum follower <NUM>, and a decision maker <NUM> (which are similar to the power meter <NUM>, the minimum follower <NUM> and the decision maker <NUM> of <FIG>).

The power meter <NUM> generally receives the captured audio signal <NUM> (see <FIG>) and generates a power signal <NUM>, in a manner similar to that of the power meter <NUM>. The minimum follower <NUM> generally receives the power signal <NUM> and generates a background noise power signal <NUM>, in a manner similar to that of the minimum follower <NUM>.

The band pass filter <NUM> generally receives the playback audio signal <NUM>, performs band pass filtering, and generates a filtered signal <NUM>. The pass band of the band pass filter <NUM> may be a band B around a resonant frequency fres. The resonant frequency fres generally corresponds to the specific components used to implement the loudspeaker <NUM> and the other components of the device implementing the audio processing system <NUM>, and may be measured empirically. The band B may also be determined empirically based on the other components of the device implementing the audio processing system <NUM>. An example range of the band B is <NUM>, resulting in the band pass filter <NUM> having a pass band of [fres - <NUM>, fres + <NUM>].

The power meter <NUM> generally receives the filtered signal <NUM>, measures the signal power, and generates a resonant power signal <NUM>. The resonant power signal (Pres) <NUM> corresponds to the signal power of the filtered signal <NUM> (e.g., the power of the mechanical resonance of the loudspeaker <NUM>).

The non-linear regulator <NUM> generally receives the resonant power signal <NUM>, performs non-linear regulation, and generates a distortion power signal (Pdist) <NUM>. The distortion power signal <NUM> corresponds to the distortion power in the frequency range [½ fs0, ½ fs1]. The non-linear regulator <NUM> may perform non-linear regulation to generate the distortion power signal Pdist as follows: <MAT>.

In the above equation, th<NUM> is a threshold parameter, and k is a tuning parameter; these parameters may be adjusted as desired according to empirical measurements. The regulation is referred to as non-linear due to the two functions of Pdist that depend upon the relation between Pres and th<NUM>. The slope of Pdist is controlled by the tuning parameter k applied to the difference between Pres and th<NUM>, and the starting point where Pdist starts increasing from zero is controlled by the relation between Pres and th<NUM>.

The decision maker <NUM> generally receives the power signal <NUM>, the background noise power signal <NUM> and the distortion power signal <NUM>, compares their levels, and generates the control signal <NUM>. In general, the decision maker <NUM> uses the distortion power signal <NUM> as part of determining whether the energy is mainly from the captured near end voice or from device distortions. More specifically, the decision maker <NUM> uses the distortion power signal <NUM> to increase the threshold of the hysteresis applied to the power signal <NUM> and the background noise power signal <NUM> (e.g., the first threshold discussed above regarding the decision maker <NUM>). The decision maker <NUM> is otherwise similar to the decision maker <NUM>.

<FIG> is a mobile device architecture <NUM> for implementing the features and processes described herein, according to an embodiment. The architecture <NUM> may be implemented in any electronic device, including but not limited to: a desktop computer, consumer audio/visual (AV) equipment, radio broadcast equipment, mobile devices (e.g., smartphone, tablet computer, laptop computer, wearable device), etc.. In the example embodiment shown, the architecture <NUM> is for a laptop computer and includes processor(s) <NUM>, peripherals interface <NUM>, audio subsystem <NUM>, loudspeakers <NUM>, microphone <NUM>, sensors <NUM> (e.g., accelerometers, gyros, barometer, magnetometer, camera), location processor <NUM> (e.g., GNSS receiver), wireless communications subsystems <NUM> (e.g., Wi-Fi, Bluetooth, cellular) and I/O subsystem(s) <NUM>, which includes touch controller <NUM> and other input controllers <NUM>, touch surface <NUM> and other input/control devices <NUM>. Other architectures with more or fewer components can also be used to implement the disclosed embodiments.

Memory interface <NUM> is coupled to processors <NUM>, peripherals interface <NUM> and memory <NUM> (e.g., flash, RAM, ROM). Memory <NUM> stores computer program instructions and data, including but not limited to: operating system instructions <NUM>, communication instructions <NUM>, GUI instructions <NUM>, sensor processing instructions <NUM>, phone instructions <NUM>, electronic messaging instructions <NUM>, web browsing instructions <NUM>, audio processing instructions <NUM>, GNSS/navigation instructions <NUM> and applications/data <NUM>. Audio processing instructions <NUM> include instructions for performing the audio processing described herein.

<FIG> is a flowchart of a method <NUM> of audio processing. The method <NUM> may be performed by a device (e.g., a laptop computer, a mobile telephone, etc.) with the components of the architecture <NUM> of <FIG>, to implement the functionality of the audio processing system <NUM> (see <FIG>), the audio codec system <NUM> (see <FIG>), the double talk detector <NUM> (see <FIG>), the double talk detector <NUM> (see <FIG>), etc., for example by executing one or more computer programs.

At <NUM>, a first audio signal is received. The first audio signal has a first sampling frequency. For example, the audio codec system <NUM> (see <FIG>) may receive the playback audio signal <NUM> that has the sampling frequency fs0.

At <NUM>, the first audio signal is up-sampled to generate a second audio signal. The second audio signal has a second sampling frequency that is greater than the first sampling frequency. For example, the up-sampler <NUM> (see <FIG>) may up-sample the playback audio signal <NUM> to generate the up-sampled signal <NUM> having the sampling frequency fs1. As a specific example, fs0 may be <NUM> and fs1 may be <NUM>.

At <NUM>, a loudspeaker output corresponding to the second audio signal may be outputted by a loudspeaker. For example, the loudspeaker <NUM> (see <FIG>) may output an audio output corresponding to the up-sampled signal <NUM>.

At <NUM>, a third audio signal is captured by a microphone. The third audio signal has a third sampling frequency that is greater than the first sampling frequency. The third sampling frequency may be the same as the second sampling frequency. For example, the microphone <NUM> (see <FIG>) may capture the captured audio signal <NUM> having the sampling frequency fs1. The captured audio signal <NUM> may include echo (e.g., of the loudspeaker output corresponding to the second audio signal), captured near end speech (e.g., local talk), mechanical distortion of the device that is performing the method <NUM> (e.g., other local audio), etc..

At <NUM>, a signal power of the third audio signal is determined. For example, the double talk detector <NUM> (see <FIG>) may determine the signal power of the captured audio signal <NUM>. As another example, the double talk detector <NUM> (see <FIG>) may determine the signal power of the captured audio signal <NUM>.

At <NUM>, double talk is detected when there is signal power of the third audio signal determined in a frequency band greater than the first sampling frequency. For example, the double talk detector <NUM> may detect double talk based on the signal power in the frequency band [½ fs0, ½ fs1]; when there is no signal power (e.g., as shown in <FIG>), no double talk is detected, and when there is signal power (e.g., as shown in <FIG>), double talk is detected.

At <NUM>, a control signal is selectively generated when the double talk is detected. For example, the double talk detector <NUM> (see <FIG>) may generate the control signal <NUM> when double talk is detected. As another example, the double talk detector <NUM> (see <FIG>) may generate the control signal <NUM> when double talk is detected.

At <NUM>, echo management is performed on the third audio signal according to the control signal. For example, the echo management system <NUM> (see <FIG>) may perform echo cancellation, echo suppression, etc. on the captured audio signal <NUM>, based on the control signal <NUM>, to generate the captured audio signal <NUM>.

The method <NUM> may include additional steps corresponding to the other functionalities of the audio processing system <NUM> described herein.

As discussed above, the audio processing system <NUM> is able to detect double talk as part of the echo management process. In addition, the audio processing system <NUM> is able to detect other audio distortions, for example due to moving the device, or otherwise when the device is subjected to tactile interactions. In such a case, the echo management system <NUM> may adapt the echo canceller <NUM> to perform echo cancellation, even in the absence of near end speech or captured far end speech.

An embodiment may be implemented in hardware, executable modules stored on a computer readable medium, or a combination of both (e.g., programmable logic arrays). Unless otherwise specified, the steps executed by embodiments need not inherently be related to any particular computer or other apparatus, although they may be in certain embodiments. In particular, various general-purpose machines may be used with programs written in accordance with the teachings herein, or it may be more convenient to construct more specialized apparatus (e.g., integrated circuits) to perform the required method steps. Thus, embodiments may be implemented in one or more computer programs executing on one or more programmable computer systems each comprising at least one processor, at least one data storage system (including volatile and non-volatile memory and/or storage elements), at least one input device or port, and at least one output device or port. Program code is applied to input data to perform the functions described herein and generate output information. The output information is applied to one or more output devices, in known fashion.

Each such computer program is preferably stored on or downloaded to a storage media or device (e.g., solid state memory or media, or magnetic or optical media) readable by a general or special purpose programmable computer, for configuring and operating the computer when the storage media or device is read by the computer system to perform the procedures described herein. The inventive system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer system to operate in a specific and predefined manner to perform the functions described herein. (Software per se and intangible or transitory signals are excluded to the extent that they are unpatentable subject matter.

Aspects of the systems described herein may be implemented in an appropriate computer-based sound processing network environment for processing digital or digitized audio files. Portions of the adaptive audio system may include one or more networks that comprise any desired number of individual machines, including one or more routers (not shown) that serve to buffer and route the data transmitted among the computers. Such a network may be built on various different network protocols, and may be the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), or any combination thereof.

One or more of the components, blocks, processes or other functional components may be implemented through a computer program that controls execution of a processor-based computing device of the system. It should also be noted that the various functions disclosed herein may be described using any number of combinations of hardware, firmware, and/or as data and/or instructions embodied in various machine-readable or computer-readable media, in terms of their behavioral, register transfer, logic component, and/or other characteristics. Computer-readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, physical (non-transitory), non-volatile storage media in various forms, such as optical, magnetic or semiconductor storage media.

Claim 1:
A computer-implemented method of audio processing, the method comprising:
receiving (<NUM>) a first audio signal, wherein the first audio signal has a first sampling frequency;
up-sampling (<NUM>) the first audio signal to generate a second audio signal, wherein the second audio signal has a second sampling frequency that is greater than the first sampling frequency;
outputting (<NUM>), by a loudspeaker, a loudspeaker output corresponding to the second audio signal;
capturing (<NUM>), by a microphone, a third audio signal, wherein the third audio signal is sampled at the second sampling frequency;
determining (<NUM>) a signal power of the third audio signal; and
detecting (<NUM>) double talk when there is signal power of the third audio signal determined in a frequency band having frequencies all greater than half the first sampling frequency.