Source: https://patents.google.com/patent/KR101275442B1/en
Timestamp: 2019-12-09 18:34:50
Document Index: 3014866

Matched Legal Cases: ['§119', 'Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 12', 'art 14']

KR101275442B1 - Systems, methods, apparatus, and computer-readable media for phase-based processing of multichannel signal - Google Patents
KR101275442B1
KR101275442B1 KR1020127000692A KR20127000692A KR101275442B1 KR 101275442 B1 KR101275442 B1 KR 101275442B1 KR 1020127000692 A KR1020127000692 A KR 1020127000692A KR 20127000692 A KR20127000692 A KR 20127000692A KR 101275442 B1 KR101275442 B1 KR 101275442B1
KR1020127000692A
KR20120027510A (en
에릭 피써
어난 류
2009-06-09 Priority to US18551809P priority Critical
2009-06-09 Priority to US61/185,518 priority
2009-07-20 Priority to US22703709P priority
2009-07-20 Priority to US61/227,037 priority
2009-09-08 Priority to US24032009P priority
2009-09-08 Priority to US61/240,318 priority
2009-09-08 Priority to US61/240,320 priority
2010-06-08 Priority to US12/796,566 priority
2010-06-08 Priority to US12/796,566 priority patent/US8620672B2/en
2010-06-09 Application filed by 퀄컴 인코포레이티드 filed Critical 퀄컴 인코포레이티드
2010-06-09 Priority to PCT/US2010/037973 priority patent/WO2010144577A1/en
2012-03-21 Publication of KR20120027510A publication Critical patent/KR20120027510A/en
2013-06-17 Publication of KR101275442B1 publication Critical patent/KR101275442B1/en
Applications are disclosed that include phase-based processing of multichannel signals, and proximity detection.
Systems, methods, apparatuses, and computer readable media for phase based processing of multichannel signals {SYSTEMS, METHODS, APPARATUS, AND COMPUTER-READABLE MEDIA FOR PHASE-BASED PROCESSING OF MULTICHANNEL SIGNAL}
Claims of Priority under 35 U.S.C.§119
This patent application is filed on June 9, 2009 and is assigned to U.S. Provisional Patent Application No. 61 / 185,518 entitled "Systems, methods, apparatus, and computer-readable media for coherence detection" assigned to the assignee of the present application. Insist on priority. This patent application is also filed on September 8, 2009, assigned to the assignee of the present application, US Provisional Patent Application No. 61 / 240,318 entitled "Systems, methods, apparatus, and computer-readable media for coherence detection". Insist on priority.
This patent application is also entitled Agent Case No. 091561P1, filed July 20, 2009, and assigned to the assignee of the present application, "Systems, methods, apparatus, and computer-readable media for phase-based processing of multichannel signal." It claims priority to US Provisional Patent Application No. 61 / 227,037. This patent application is also filed on September 8, 2009 and assigned to the assignee of the present application, entitled “Systems, methods, apparatus, and computer-readable media for phase-based processing of multichannel signal”. Claim priority to number 61 / 240,320.
Many of the activities previously performed in quiet office or home environments are now being performed in acoustically changing situations such as cars, streets, or cafes. For example, a person may want to communicate with another person using a voice communication channel. This channel may be provided by, for example, a mobile wireless handset or headset, walkie-talkie, two-way radio, car-kit, or other communication device. As a result, the kind of noise content typically encountered where people tend to gather, such as mobile devices (eg, smartphones, handsets, and / or headsets) in environments where users are surrounded by others. ), A significant amount of voice communication is taking place. This noise tends to distract and bother the user at the far end of the telephone conversation. In addition, many standard automated business transactions (eg, balance or stock confirmations) use voice recognition based data queries, and the accuracy of these systems may be significantly hampered by interference noise.
For applications where communication takes place in noisy environments, it may be desirable to separate the desired speech signal from background noise. Noise may be defined as a combination of all signals that interfere with or otherwise degrade the desired signal. Background noise may include a number of noise signals generated within an acoustic environment, such as background conversations of others, as well as reflections and reverberations generated from a desired signal and / or any other signals. . If the desired speech signal is not separated from the background noise, it may be difficult to reliably and efficiently use the signal. In one particular example, the speech signal is generated in a noisy environment, and speech processing methods are used to separate the speech signal from environmental noise.
Noise encountered in a mobile environment may include various other components such as competing talkers, music, murmurs, street noise, and / or airport noise. Since this signature of noise is typically unusual and close to the user's own frequency signature, the noise may be difficult to model using conventional single microphone or fixed beamforming type methods. Single microphone noise reduction techniques typically require significant parameter tuning to achieve optimal performance. For example, a suitable noise reference may not be available directly in this case, and it may be necessary to indirectly derive the noise reference. Thus, multiple microphone based advanced signal processing may be desirable to support the use of mobile devices for voice communication in noisy environments.
According to a general configuration, a method for processing a multichannel signal includes, for each of a plurality of different frequency components of the multichannel signal, a phase of a frequency component in a first channel of the multichannel signal and a frequency in a second channel of the multichannel signal. Calculating a difference between the phases of the components to obtain a plurality of calculated phase differences. The method includes calculating a level of the first channel and a corresponding level of the second channel. The method includes calculating an updated value of the gain factor based on at least one of the calculated level of the first channel, the calculated level of the second channel, and the plurality of calculated phase differences, and in accordance with the updated value: Generating a processed multichannel signal by changing the amplitude of the second channel relative to the corresponding amplitude of one channel. Also disclosed herein is a device comprising means for performing each of these actions. Also disclosed herein is a computer readable medium having tangible features for storing machine executable instructions for performing this method.
According to a general configuration, an apparatus for processing a multichannel signal includes, for each of a plurality of different frequency components of the multichannel signal, a phase of a frequency component in a first channel of the multichannel signal and a frequency in a second channel of the multichannel signal. And a first calculator configured to obtain a plurality of calculated phase differences by calculating a difference between the phases of the components. Such an apparatus includes a second calculator configured to calculate a level of the first channel and a corresponding level of the second channel, and at least one of the calculated level of the first channel, the calculated level of the second channel, and the plurality of calculated phase differences. Based on one, a third calculator configured to calculate the updated value of the gain factor. Such an apparatus includes a gain control element configured to generate a processed multichannel signal by changing the amplitude of the second channel relative to the corresponding amplitude of the first channel in accordance with the updated value.
1 shows a side view of a headset D100 in use.
2 shows a top view of a headset D100 mounted to a user's ear.
3A shows a side view of handset D300 in use.
3B shows examples of broadside and endfire for a microphone array.
4A shows a flowchart for a method M100 of processing a multichannel signal in accordance with a general configuration.
4B shows a flowchart of an implementation T102 of task T100.
4C shows a flowchart of an implementation T112 of task T110.
5A shows a flowchart of an implementation T302 of task T300.
5B shows a flowchart of an alternative implementation T304 of task T300.
5C shows a flowchart of an implementation M200 of method M100.
6A shows an example of geographic approximation that illustrates an approach for estimating the direction of arrival.
FIG. 6B shows an example of using the approximation of FIG. 6A for two and three quadrant values.
7 shows an example of a model that assumes a spherical wavefront.
8A shows an example of a masking function with relatively sudden transitions between passband and stopband.
8B shows an example of a linear rolloff for a masking function.
8C shows an example of nonlinear rolloff for a masking function.
9A-9C show examples of nonlinear functions for different parameter values.
10 shows forward and reverse lobes of the directivity pattern of the masking function.
11A shows a flowchart of an implementation M110 of method M100.
11B shows a flowchart of an implementation T362 of task T360.
11C shows a flowchart of an implementation T364 of task T360.
12A shows a flowchart of an implementation M120 of method M100.
12B shows a flowchart of an implementation M130 of method M100.
13A shows a flowchart of an implementation M140 of method M100.
13B shows a flowchart of an implementation M150 of method M100.
14A shows an example of boundaries of proximity detection regions corresponding to three different thresholds.
14B shows an example of range intersection of the proximity bubble and allowed direction to obtain a cone of speaker coverage.
15 and 16 show top and side views of the source selection region boundary as shown in FIG. 14B.
17A shows a flowchart of an implementation M160 of method M100.
17B shows a flowchart of an implementation M170 of method M100.
18 shows a flowchart of an implementation M180 of method M170.
19A shows a flowchart of a method M300 in accordance with a general configuration.
19B shows a flowchart of an implementation M310 of method M300.
20A shows a flowchart of an implementation M320 of method M310.
20B shows a block diagram of an apparatus G100 according to a general configuration.
21A shows a block diagram of an apparatus A100 according to a general configuration.
21B shows a block diagram of the apparatus A110.
22 shows a block diagram of apparatus A120.
23A shows a block diagram of an implementation R200 of array R100.
23B shows a block diagram of an implementation R210 of array R200.
24A shows a block diagram of the device D10 according to the general configuration.
24B shows a block diagram of an implementation D20 of device D10.
25A-25D show various views of the multi-microphone wireless headset D100.
26A-26D show various views of the multi-microphone wireless headset D200.
27A shows a cross-sectional view (along the center axis) of a multi-microphone communication handset D300.
27B shows a cross-sectional view of an implementation D310 of device D300.
28A shows a diagram of a multi-microphone media player D400.
28B shows another implementation D410 of device D400.
28C shows another implementation D420 of device D400.
29 shows a diagram of a multi-microphone handsfree car kit D500.
30 shows a diagram of a multi-microphone portable audio sensing implementation D600 of device D10.
The real world has many sources of noise, including single point noise sources that often deviate into multiple sounds that produce reverberation. Background acoustic noise is echo signals generated from a number of noise signals generated by the general environment and interference signals generated by background conversations of others, as well as other desired sound signals and / or any other signals. And reverberation sound.
Environmental noise may affect the clarity of sensed audio signals, such as near-end speech signals. It may be desirable to use signal processing to distinguish the desired audio signal from background noise. For example, for applications where communication may occur in a noisy environment, it may be desirable to use a speech processing method to distinguish the speech signal from background noise and enhance its intelligibility. Such processing may be important in many areas of everyday communication because noise is almost always present in realistic world conditions.
It may be desirable to fabricate a portable audio sensing device having an array of two or more microphones R100 configured to receive acoustic signals. Examples of portable audio sensing devices that may be implemented to include such an array and that may be used for audio recording and / or voice communications applications include, but are not limited to, telephone handsets (eg, cellular telephone handsets or smartphones); Wired or wireless headsets (eg, Bluetooth headsets); Handheld audio and / or video recorders; A personal media player configured to record audio and / or video content; A portable assistant (PDA) or other handheld computing device; And notebook computers, laptop computers, netbook computers, or other portable computing devices.
During normal use, the portable audio sensing device may operate in any range of standard orientations for the desired sound source. For example, different users may wear or hold the device differently, and different users may wear or hold the device differently at different times, even if the same user is in the same period of use (eg, during a single phone call). 1 shows a side view of a quadruple headset D100 that includes two examples in the range of the device's standard orientation to the user's mouth. The headset D100 includes a primary microphone MC10 positioned to receive a user's voice more directly during normal use of the device, and a secondary microphone positioned to receive a voice of the user less directly during normal use of the device ( Has an example of an array R100 that includes MC20. 2 shows a top view of a headset D100 mounted to a user's ear in a standard orientation relative to the user's mouth. 3A shows a side view of the handset D300 in use including two examples in the range of the standard orientation of the device relative to the mouth of the user.
Unless expressly limited by context, the term “signal” is intended to refer to any ordinary meaning, including the state of a memory location (or set of memory locations) as represented on a wired, bus, or other transmission medium. Used here. Unless expressly limited by context, the term “generating” is used herein to refer to any common meaning, such as computing or otherwise generating. Unless expressly limited by context, the term “calculating” is used herein to denote any ordinary meaning, such as computing, evaluating, smoothing, and / or selecting from a plurality of values. Unless expressly limited by the context, the term “acquiring” means any ordinary meaning, such as calculating, deriving, receiving (eg, from an external device) and / or searching (from an array of storage elements). It is used to indicate. Unless expressly limited by context, the term “selecting” is intended to indicate any ordinary meaning, such as identifying, indicating, applying, and / or using, identifying some rather than at least one and all of the two or more sets. Used. When the term "comprising" is used in the present description and claims, it does not exclude other elements or operations. The term "based" (as in "A is based on B") is derived from cases (i) "for example" (eg, "B is a precursor of A"), (ii) "at least Based "(eg," A is based on at least B "), and (i) is" equal "(eg," A is equal to B "), as appropriate in the particular context. Is used to indicate any common meaning. Similarly, the term "responding" is used to denote any ordinary meaning including "responding at least."
The reference to the "position" of the microphone of the multi-microphone audio sensing device indicates the position of the center of the acoustic sensing surface of the microphone unless otherwise indicated by the context. The term "channel" is sometimes used to denote a signal path, and in other cases, a signal carried by such a case, depending on the particular context. Unless indicated otherwise, the term “series” is used to denote a sequence of two or more items. The term "log" is used to denote a base-10 log although the extension of this operation to other bases is within the scope of this disclosure. The term “frequency component” means a sample (or “empty” of a subband of a signal (eg, a bark scale subband) or a frequency domain representation of a signal (eg, as generated by a fast Fourier transform). Is used to indicate one of the frequency bands or set of frequencies of the signal, such as ").
Unless indicated otherwise, any disclosure of the operation of a device having a particular feature is also expressly intended to disclose a method having a similar feature (or vice versa), and any disclosure of the operation of the device according to a particular configuration. It is also clearly intended to disclose a method according to this similar arrangement (or vice versa). The term “configuration” may be used in a reference to a method, apparatus, and / or system as indicated by the specific context. The terms "method", "process", "procedure", and "method" are used generically and interchangeably unless otherwise indicated by a particular context. The terms "apparatus" and "device" are also used generally and interchangeably unless the context indicates otherwise. The terms “element” and module ”are commonly used to refer to some of the larger configurations, and unless explicitly limited by context, the term“ system ”includes“ a group of elements that interact to serve a common purpose ”. Used herein to refer to any common meaning that a reference to a portion of a document should also be understood to incorporate the definitions of terms or variables referenced within that portion, where such Definitions appear anywhere in the document as well as in any of the drawings referenced in the integrated section.
The near field may be defined as an area of space spaced less than one wavelength from a sound receiver (eg, a microphone array). Under this definition, the distance to the boundary of the region changes inversely with frequency. For example, at frequencies of 200, 700, and 2000 hertz, the distance to the 1-wavelength boundary is about 170, 49, and 17 centimeters, respectively. Instead, the near / far field boundary is considered to be at a specific distance from the microphone array (eg, 50 centimeters from the microphone or the center of the array, or 1 meter or 1.5 meters from the microphone or the center of the array). It may be useful.
The microphone array generates a multichannel signal where each channel is based on a corresponding one of the microphones for the acoustic environment. It may be desirable to perform a spatial selection processing (SSP) operation on a multichannel signal to distinguish between components of a signal received from different sources. For example, sound components from a desired source of directional sound (eg, a user's mouth) and sound from one or more sources (eg, competing talkers) of diffuse background noise and / or directional interference noise. It may be desirable to distinguish between the components. Examples of SSP operation include beamforming approaches (eg, generalized sidelobe cancellation (GSC), minimum variance distortionless response (MVDR), and / or linearly constrained minimum variance beamformers), blind source separation (BSS) and Other adaptive learning approaches, and gain based proximity detection. Typical applications of SSP operation include multi-microphone noise reduction schemes for portable audio sensing devices.
The performance of the operation on the multichannel signal generated by the array R100, such as the SSP operation, may depend on how well the response characteristics of the array channels match each other. For example, it is possible that the levels of the channels are different due to the difference in the response characteristics of the respective microphones, the difference in the gain levels of the respective processing stages, and / or the difference in the circuit noise levels of the channels. In such a case, the resulting multichannel signal may not provide an accurate representation of the acoustic environment if the mismatch on the channel response characteristics (also called "channel response imbalance") may not be compensated for.
Without this compensation, SSP operations based on these signals may provide error results. For an operation where gain differences between channels are used to indicate the relative proximity of a directional sound source, an imbalance between the responses of the channels tends to reduce the accuracy of the proximity indication. In another example, amplitude response deviations between channels as small as 1 or 2 decibels at low frequencies (eg, approximately 100 Hz to 1 kHz) may significantly reduce low frequency directivity. The effect of the imbalance in the responses of the channels of array R100 may be particularly harmful for applications that process multichannel signals from the implementation of array R100 having more than two microphones.
Accurate channel calibration may be particularly important for headset applications. For example, it may be desirable to configure a portable audio sensing device to distinguish between sound components arriving from near-field sources and sound components arriving from far-field sources. This differentiation may be performed based on the difference between the gain levels of the two channels of the multichannel signal (i.e., " gain difference between channels "), since the difference is in the endfire direction of the array ( That is, it can be expected to be higher for sound components from near field sources located near the line passing through the centers of the corresponding microphones.
As the distance between the microphones decreases, the interchannel gain level difference for the near field signal also decreases. For handheld applications, the interchannel gain level difference for near field signals is typically about 6 decibels from the interchannel gain level difference for far field signals. However, for headset applications, the interchannel gain level difference for a typical near-field sound component may be within 3 decibels (or less) of the interchannel gain level difference for a typical far-field sound component. In this case, only a few decibels of channel response imbalance may seriously hinder the ability to distinguish between these components, and an imbalance of more than three decibels may destroy it.
An imbalance between the responses of the array channels may arise from the difference between the responses of the microphones themselves. Fluctuations may occur during the manufacture of the microphones of array R100 such that sensitivity may vary significantly from one microphone to another, even among batches of mass produced and seemingly identical microphones. Microphones for use in portable large market audio sensing devices may be manufactured, for example, at a sensitivity tolerance of ± 3 decibels, so that in the implementation of array R100, the sensitivity of these two microphones may differ by as much as 6 decibels. have.
The problem of channel response imbalance may be addressed during the manufacture of a portable audio sensing device by using microphones whose responses have already been matched (eg, via sorting and binning processes). Alternatively or in addition, a channel calibration procedure may be performed on the microphones (or devices comprising the array) of array R100 in a laboratory and / or production facility, such as a factory. This procedure may compensate for the imbalance by calculating one or more gain factors and applying these factors to corresponding channels to generate a balanced multichannel signal. Examples of calibration procedures that may be performed prior to service are described in US Patent Application Nos. 12 / 473,930, entitled “SYSTEMS, METHODS, AND APPARATUS FOR MULTICHANNEL SIGNAL BALANCING,” filed May 28, 2009, and December 2008. US Patent Application No. 12 / 334,246, entitled "SYSTEMS, METHODS, AND APPARATUS FOR MULTI-MICROPHONE BASED SPEECH ENHANCEMENT," filed May 12. Such matching or calibration operations may increase the cost of manufacturing the device, but they may also be ineffective against channel response imbalances that occur during the service life of the device (eg, due to aging).
Alternatively or in addition, channel calibration may be performed in service (eg, as described in US patent application Ser. No. 12 / 473,930). This procedure may be used to correct response imbalance that occurs over time and / or correct initial response imbalance. The initial response imbalance may be due to, for example, microphone mismatch and / or incorrect calibration procedure (eg, the microphone is touched or covered during the procedure). In order to avoid distracting the user at varying channel levels, it may be desirable to apply a reward that gradually changes over time for this procedure. However, for cases where the initial response imbalance is large, this gradual compensation can provide a long convergence period (e.g., from 1 minute to 10 minutes or more) during times when SSP operation on a multichannel signal may be poorly performed. May result in an unsatisfactory user experience.
Phase analysis may be used to classify time-frequency points of a multichannel signal. For example, it may be desirable to configure a system, method, or apparatus to classify time-frequency points of a multichannel signal based on a difference in each of a plurality of different frequencies between estimated phases of the channels of the signal. . This configuration is referred to herein as "phase based".
It may be desirable to use a phase based scheme to identify time-frequency points that exhibit certain phase difference characteristics. For example, a phase-based approach relates to inter-microphone distance and inter-channel phase differences to determine if a particular frequency component of a sensed multichannel signal occurs from within or outside of an acceptable range of angles with respect to the array axis. It may also be configured to apply the information. This determination distinguishes between sound components arriving from different directions (eg, so that sound originating from within an acceptable range is selected and sound originating outside of that range is rejected) and / or near and far field sources. It may be used to distinguish between sound components arrived from.
In a typical application, such a system, method, or apparatus is a pair of microphones for each time-frequency point through at least a portion of a multichannel signal (eg, over a range of specific frequencies and / or over a specific time interval). It is used to calculate the direction of arrival relative to. A directional masking function may be applied to these results to distinguish points with arrival directions within a desired range from points with other arrival directions. The results from the directional masking operation may be used to attenuate sound components from unwanted directions by discarding or attenuating time-frequency points with arrival directions outside the mask.
As mentioned above, many multi-microphone spatial processing operations are inherently dependent on the relative gain responses of the microphone channels, so a correction of the channel gain response may be needed to enable such spatial processing operations. Performing such calibration during manufacturing is typically time consuming and / or otherwise expensive. However, a phase-based approach may be implemented such that it is relatively unaffected by gain imbalance among the input channels so that the degree of gain responses of corresponding channels match each other depends on the accuracy of the calculated phase differences and subsequent operations based on them. It is not a limiting factor for (eg directional masking).
It may be desirable to utilize robustness against phase-based channel imbalance by using this type of classification results to support channel calibration operations (also referred to as " channel balancing operations ") as described herein. have. For example, it may be desirable to use a phase based scheme to identify time intervals and / or frequency components of a recorded multichannel signal that may be useful for channel balancing. This approach may be configured to select time-frequency points whose arrival directions indicate that they are expected to generate a relatively identical response in each channel.
With respect to the range of source directions for the two-microphone array as shown in FIG. 3B, it is desirable to use only sound components arriving from the broadside directions (ie, directions orthogonal to the array axis) for channel calibration. You may. Such a condition may be found, for example, when the near field source is not active and the sound source is distributed (eg, background noise). It may also be acceptable to use sound components resulting from far field endfire sources for calibration, which would be expected to cause negligible inter-channel gain level differences (eg due to dispersion). Because it may. However, near-field sound components arriving from the array's endfire direction (ie, near the array axis) are expected to have gain differences between the channels representing source position information rather than channel imbalance. As a result, using these components for calibration may produce inaccurate results, and it may be desirable to use a directional masking operation to distinguish these components from sound components arriving from the broadside directions. .
This phase based classification scheme may be used to support calibration operations at run time (eg, during use of the device, whether continuous or intermittent). In this manner, a fast and accurate channel correction operation may be achieved that is autoimmunized with channel gain response imbalance. Alternatively, information from selected time-frequency points may accumulate over some time period to later support channel calibration operation.
4A shows a flowchart of a method M100 for processing a multichannel signal in accordance with a general configuration including tasks T100, T200, T300, and T400. Task T100 calculates the phase difference between channels (eg, microphone channels) of the multichannel signal for each of a plurality of different frequency components of the signal. Task T200 calculates the level of the first channel of the multichannel signal and the corresponding level of the second channel of the multichannel signal. Based on at least one of the calculated levels and the calculated phase differences, task T300 updates the gain factor value. Based on the updated gain factor value, task T400 changes the amplitude of the second channel relative to the corresponding amplitude of the first channel to generate a processed (eg, balanced) multichannel signal. The method M100 may also be used to support further operations for a multichannel signal (eg, as described in more detail herein), such as SSP operations.
The method M100 may be configured to process the multichannel signal as a series of segments. Typical segment lengths range from about 5 or 10 milliseconds to about 40 or 50 milliseconds, and the segments may or may not overlap (eg, overlap 25% or 50% with adjacent segments). . In one particular example, the multichannel signal is divided into a series of non-overlapping segments or "frames" each of which is 10 milliseconds in length. Task T100 may be configured to calculate a set of phase differences (eg, a vector) for each of the segments. In some implementations of method M100, task T200 is configured to calculate a level for each of the segments of each channel, and task T300 is configured to update the gain factor value for at least some of the segments. . In another implementation of the method M100, task T200 is configured to calculate a set of subband levels for each of the segments of each channel, and task T300 updates one or more of the set of subband gain factor values. It is configured to. The segment when processed by the method M100 may also be a segment of a larger segment (ie, “subframe”) when processed by a different operation, and vice versa.
4B shows a flowchart of one implementation T102 of task T100. For each microphone channel, task T102 includes each case of subtask T110 that estimates the phase of the channel for each of the different frequency components. 4C shows a flowchart of one implementation T112 of task T110 that includes subtasks T1121 and T1122. Task T1121 calculates the frequency transform of the channel, such as a Fast Fourier Transform (FFT) or Discrete Cosine Transform (DCT). Task T1121 is typically configured to calculate the frequency transform of the channel for each segment. For example, it may be desirable to configure task T1121 to perform a 128-point or 256-point FFT of each segment. An alternative implementation of task T1121 is configured to separate various frequency components of the channel using a bank of subband filters.
Task T1122 calculates (eg, estimates) the phase of the microphone channel for each of the different frequency components (or referred to as “bins”). For example, for each frequency component to be examined, task 1122 may be configured to estimate the phase as the inverse tangent (also called arc tangent) of the ratio of the imaginary term of the corresponding FFT coefficient to the real term of the FFT coefficient. It may be.
Task T102 is also based on the estimated phases for each channel, so that the phase difference for each of the different frequency components (
Subtask T120 that calculates &lt; RTI ID = 0.0 &gt; Task T120 may be configured to calculate the phase difference by subtracting the estimated phase for the frequency component in one channel from the estimated phase for the frequency line segment in the other channel. For example, task T120 may be configured to calculate the phase difference by subtracting the estimated phase for the frequency component in the primary channel from the estimated phase for the frequency component in another (eg, secondary) channel. have. In such a case, the primary channel may be a channel that is expected to have the highest signal-to-noise ratio, such as the channel that corresponds to the microphone that is expected to receive the user's voice most directly during normal use of the device.
It may be desirable to configure the method M100 (or a system or apparatus configured to perform such a method) to estimate the phase difference between the channels of the multichannel signal over frequencies in the wide range. This wideband range may extend from, for example, a low frequency boundary of zero, 50, 100, or 200 Hz to a high frequency boundary of 3, 3.5, or 4 kHz (or higher, such as above 7 or 8 kHz). However, for task T100, it may be unnecessary to calculate phase differences over the entire bandwidth of the signal. For example, for multiple bands in this wide band range, phase estimation may be impractical or unnecessary. Substantial evaluation of the phase relationships of the received waveform at very low frequencies typically requires correspondingly large spaces between the transducers. As a result, the maximum available space between the microphones may establish a low frequency boundary. On the other hand, the distance between the microphones should not exceed 1/2 of the minimum wavelength to avoid spatial aliasing. The 8 kilohertz sampling rate provides, for example, a bandwidth from zero to 4 kilohertz. The wavelength of the 4 kH signal is about 8.5 centimeters, so in this case, the space between adjacent microphones should not exceed about 4 centimeters. The microphone channel may be lowpass filtered to remove frequencies that may cause spatial aliasing.
Accordingly, task T1122 is calculated to calculate a phase estimate for the total less frequency components generated by task T1121 (eg, the total less frequency samples of the FFT performed by task T1121). It may be desirable to configure. For example, task T1122 may be configured to calculate phase estimates for frequencies ranging from about 50, 100, 200 or 300 Hz to about 500 or 1000 Hz (each of these eight combinations may be explicitly Considered and disclosed). It may be expected that this range includes components that are particularly useful for calibration and excludes components that are less useful for calibration.
It may be desirable to configure task T100 to also calculate a phase estimate used for purposes other than channel calibration. For example, task T100 may be configured to also calculate phase estimates used to track and / or enhance a user's voice (eg, as described in greater detail below). In one such example, task T1122 is also configured to calculate phase estimates for the frequency range of 700 Hz to 2000 Hz, which may be expected to include most of the energy of the user's voice. For a 128 point FFT of a 4 kilohertz bandwidth signal, the range of 700 to 2000 Hz corresponds approximately to 23 frequency samples from the tenth sample to the 32nd sample. In other examples, task T1122 may calculate phase estimates over a frequency range that extends from a lower limit of about 50, 100, 200, 300, or 500 Hz to an upper limit of about 700, 1000, 1200, 1500, or 2000 Hz. (Each of these 25 combinations of lower and upper limits are explicitly considered and disclosed).
The level calculation task T200 is configured to calculate a level for each of the first and second channels in the corresponding segment of the multichannel signal. Alternatively, task T200 may be configured to calculate a level for each of the first and second channels in each set of subbands of the corresponding segment of the multichannel signal. In such a case, task T200 may be configured to calculate levels for each set of subbands having the same width (eg, a uniform width of 500, 1000, or 1200 Hz). Alternatively, task T200 may be a non-uniform width, such as a set of subbands (eg, widths according to Bark and Mel scale divisions of the signal spectrum) where at least two (possibly all) of the subbands have different bandwidths. May be configured to calculate levels for each).
Task T200 is a measure of the amplitude or magnitude (also referred to as "absolute amplitude" or "corrected amplitude") of the subbands in the channel over the corresponding time period (eg, via the corresponding segment). It may be configured to calculate the level (L) for each channel of the selected subband in the time domain. Examples of amplitude or magnitude measurements include total size, average size, root mean square (RMS) amplitude, median size, and peak size. In the digital domain, these measurements are
According to an expression such as one of n sample values (
May be calculated through a block (or “frame”).
Task T200 is also in accordance with this expression, the level for each channel of the selected subband in the frequency domain (eg, Fourier transform domain) or other transform domain (eg, discrete cosine transform (DCT) domain). May be configured to calculate (L). Task T200 may also be configured to calculate levels in the analog domain according to a similar expression (eg, using integration instead of summation).
Alternatively, task T200 calculates the level L for each channel of the selected subband in the time domain as a measure of the energy of the subband over the corresponding time period (eg, over the corresponding segment). It may be configured to. Examples of measurements of energy include total energy and average energy. In the digital domain, these measurements are
N sample values according to the expression
May be calculated through a block of
Task T200 may also, according to this expression, for each channel of the selected subband in the frequency domain (eg, Fourier transform domain) or another transform domain (eg, discrete cosine transform (DCT) domain). It may be configured to calculate the level (L). Task T200 may also be configured to calculate levels in the analog domain according to a similar expression (eg, using integration instead of summation). In another alternative, task T200 is configured to calculate the level for each channel of the selected subband as the power spectral density (PSD) of the subband over the corresponding time period (eg, through the corresponding segment). .
Alternatively, task T200 is a level L for each channel i of the selected segment of the multichannel signal in the time domain, frequency domain, or other transform domain as a measure of the amplitude, magnitude, or energy of the segment in the channel. i ) may be configured in a similar manner to calculate. For example, task T200 may determine the level L for the channel of the segment, the sum of the squares of the time domain sample values of the segment in that channel, or the sum of the squares of the frequency domain sample values of the segment in that channel, or It may be configured to calculate as PSD of a segment in that channel. The segment when processed by task T300 may be a larger segment of segment (ie, a “subframe”) when processed by a different operation, and vice versa.
It may be desirable to configure task T200 to perform one or more spectral shaping operations on audio signal channels prior to calculating level values. Such operations may be performed in the analog / digital domains. For example, a lowpass filter (e.g., has a cutoff frequency of 200, 500, or 1000 Hz) or a bandpass filter (e.g., a signal from each channel before calculating the corresponding level value or values). For example, it may be desirable to configure task T200 to apply a passband of 200 Hz to 1 kHz.
The factor gain updating task T300 is configured to update a value for each of the at least one gain factor based on the calculated levels. For example, it is desirable to configure task T300 to update each of the gain factor values based on the observed imbalance between the levels of each channel in the corresponding selected frequency component as calculated by task T200. You may.
This implementation of task T300 is a function of linear level values (eg, a ratio according to an expression such as L 1 / L 2 , where L 1 and L 2 are the levels of each of the first and second channels). May be configured to calculate the observed imbalance. Alternatively, this implementation of task T300 may be configured to calculate the observed imbalance as a function of the level values in the log domain (eg, as a difference according to an expression such as L 2 -L 1 ).
Task T300 may be configured to use the observed imbalance as an updated gain factor value for the corresponding frequency component. Alternatively, task T300 may be configured to use the observed imbalance to update the corresponding previous value of the gain factor. In this case, task T300 is,
May be configured to calculate the updated value according to an expression such as
Denotes a gain factor value corresponding to segment (n) for frequency component (i),
Denotes a gain factor value corresponding to the previous segment (n-1) for the frequency component (i),
Represents the observed imbalance calculated for the frequency component (i) in segment (n),
Denotes a temporal smoothing factor with a value ranging from 0.1 (maximum smoothing) to 1 (no smoothing), such as 0.3, 0.5, or 0.7. Smoothing factor of the same value for each frequency component (
) Is typical for this implementation of task T300, but is not necessarily. It is also possible to configure task T300 to temporarily smooth the values of the observed levels prior to the calculation of the observed imbalance and / or to temporarily smooth the values of the observed channel imbalance before the calculation of the updated gain factor values. .
As described in more detail below, the gain factor updating task T300 is based on information from the plurality of phase differences calculated in task T100 (eg, identification of acoustically balanced portions of the multichannel signal). It is also configured to update the value for each of the at least one gain factor based on that. In any particular segment of the multichannel signal, T300 may update a set of gain factor values less than the total. For example, the presence of a source that causes a frequency component to remain acoustically unbalanced during a calibration operation may prevent task T300 from calculating new gain factor values and observed imbalances for that frequency component. have. As a result, it may be desirable to configure task T300 to smooth the values of observed levels, observed imbalances, and / or gain factors over frequency. For example, task T300 may be configured to calculate an average of the observed levels (or of the observed imbalances or gain factors) of the selected frequency components and assign this calculated average value to the unselected frequency components. . In another example, task T300 is:
And update gain factor values corresponding to frequency components (i) not selected according to an expression such as
Denotes a gain factor value corresponding to segment (n) for neighboring frequency component (i-1),
Is a frequency smoothing factor with a value ranging from zero (no update) to 1 (no smoothing). In another example, the expression (9)
Instead it is changed to use the gain factor value for the closest selected frequency component. Task T300 may be configured to perform smoothing via frequency before, after, or concurrently with temporal smoothing.
Task T400 is based on the at least one gain factor values updated at task T300, with respect to the corresponding response characteristic of another channel of the multichannel signal with respect to the response characteristic of the channel of the multichannel signal (eg, gain). Change the response) to generate a processed multichannel signal (also called a "balanced" or "corrected" signal). Task T400 generates a processed multichannel signal by using each set of subband gain factor values to vary the amplitude of the corresponding frequency component in the second channel relative to the amplitude of the frequency component in the first channel. It may be configured to. Task T400 may be configured to amplify a signal from a small response channel, for example. Alternatively, task T400 may be configured to control (eg, amplify or attenuate) the amplitude of the frequency components in the channel corresponding to the secondary microphone. As mentioned above, in any particular segment of a multichannel signal, it is possible that the set of gain factor values less than the total is updated.
Task T400 may be configured to generate a processed multichannel signal by applying a single gain factor to each segment of the signal, or else applying the gain factor value to more than one frequency component. For example, task T400 is updated to change the amplitude of the secondary microphone channel relative to the corresponding amplitude of the primary microphone channel (eg, to amplify or attenuate the secondary microphone channel for the primary microphone channel). It may be configured to apply the gain factor value.
Task T400 may be configured to perform channel response balancing in the linear domain. For example, task T400 may be configured to control the amplitude of the second channel of the segment by multiplying each of the values of the time domain samples of the segment in the channel by the value of the gain factor corresponding to that segment. For the subband gain factor, task T400 can be used to multiply the amplitude by the value of the gain factor, or use a subband filter to apply the gain factor to the corresponding subband in the time domain. It may also be configured to control the amplitude of the frequency component.
Alternatively, task T400 may be configured to perform channel response balancing in the log domain. For example, task T400 may be configured to control the amplitude of the second channel of the segment by adding the corresponding value of the gain factor to the log gain control value applied to the channel over the duration of the segment. For the subband gain factor, task T400 may be configured to control the amplitude of the frequency component in the second channel by adding the value of the corresponding gain factor to the amplitude. In such cases, task T400 is (eg,
According to an expression such as
Is a linear value,
Is the corresponding log value) amplitude and gain factor values may be configured to receive log values (eg, in decibels) and / or convert linear amplitude or gain factor values to log values.
Task T400 may be combined with another amplitude control of the channel or channels (eg, automatic movement control (AGC) or automatic volume control (AVC) module, user motion volume control, etc.) or upstream of the other amplitude control. Or may be performed downstream.
For an array of more than two microphones, it may be desirable to perform each case of method M100 for each of two or more pairs of channels such that the response of each channel is balanced with the response of at least one other channel. . For example, in the case of method M110 (eg, of method M110), a coherency measure is calculated based on one pair of channels (first and second channels). Another case of the method M100 may be to calculate a coherency measure based on another pair of channels (eg, a first channel and a third channel, or third and fourth channels). To be executed. However, for cases where a common operation is not performed for a pair of channels, the pair's balancing may be omitted.
The gain factor updating task T300 may be referred to as frequency components and / or segments (eg, "acoustically balanced portions" herein) of a multichannel signal that are expected to have the same level in each channel. Frequency components and / or segments expected to result in the same response by respective microphone channels), and from the phase differences calculated to calculate one or more gain factor values based on information from these portions. It may also include the use of information. Sound components received from sources in the broadside directions of array R100 may be expected to result in the same responses by microphones MC10 and MC20. Conversely, sound components received from near field sources in either of the endfire directions of array R100 cause one microphone to have a higher output level than the other microphone (ie, become "acoustically unbalanced"). May be expected. Thus, it may be desirable to configure task T300 to use the phase difference calculated in task T100 to determine whether the corresponding frequency component of the multichannel signal is acoustically balanced or acoustically unbalanced.
Task T300 may be configured to perform a directional masking operation on the phase differences calculated by task T100 to obtain a mask score for each of the corresponding frequency components. According to the above discussion regarding phase estimation by task T100 over a limited frequency range, task T300 is less than the total of the frequency components of the signal (eg, the total of the FFT performed by task T1121). May be configured to obtain mask scores for small frequency samples).
5A shows a flowchart of an implementation T302 of task T300 that includes subtasks T310, T320, and T340. For each of the plurality of calculated phase differences from task T100, task T310 calculates a corresponding direction indicator. Task T320 uses a directional masking function to rate direction indicators (eg, to convert or map values of direction indicators to values of an amplitude or magnitude scale). Based on the ratings generated by task T320, task T340 calculates updated gain factor values (eg, according to expression (8) or (9) above). For example, task T340 selects the frequency components of the signal that the ratings indicate that the frequency components are acoustically balanced and for each of these components based on the observed imbalance between the channels for that component. It may be configured to calculate an updated gain factor value.
Task T310 is configured to correspond to the corresponding frequency component of the multichannel signal (
Direction of arrival of
May be configured to calculate each of the direction indicators. For example, task T310 is a positive inverse cosine (also called arc cosine) (
As the direction of arrival (
), Where c represents the speed of sound (approximately 340 m / sec), d represents the distance between the microphones,
Denotes the difference in radians between the corresponding phase estimates for the two microphones,
Denotes the frequency component (e.g., the frequency of the corresponding FFT samples, or the center or edge frequency of the corresponding subbands) where the phase estimates correspond. Alternatively, task T310 is a positive inverse cosine (
As the angle of arrival (
May be configured to estimate
Is the frequency component (
6A shows the direction of arrival relative to the microphone MC20 of the two microphone arrays MC10 and MC20;
We show an example of geographic approximation that illustrates an approach to estimating. In this example,
Represents a signal that reaches the microphone MC20 from the reference endfire direction (i.e., the direction from the microphone MC10),
The value of represents the signal arriving from the other end fire direction,
The value of represents the signal arriving from the broadside direction. In another example, task T130 may have different reference positions (eg, microphone MC10 or some other point, such as a point in the middle between microphones) and / or different reference directions (eg, another endfire direction). , Broadside direction, etc.)
It may be configured to evaluate.
The geographic approximation shown in FIG. 6A assumes that distance s is equal to distance L, where s is an orthographic projection and microphone of the position of microphone MC10 on the line between sound source and microphone MC20 Is the distance between the positions of MC20, and L is the actual difference between the distances of each microphone with respect to the sound source. The error s-L indicates the direction of arrival relative to the microphone MC20 (
) Becomes smaller when approaching zero. This error also becomes smaller when the relative distance between the sound source and the microphone array increases.
The manner illustrated in FIG. 6A is
Quadrant and quadrant values of (i.e. from zero
May be used for 6b is
Quadrant and quadrant values of (i.e.,
Shows an example using the same approximation. In this case, the inverse cosine has an angle (
May be calculated as described above to evaluate), which is then followed by the direction of arrival (
Subtract from radians. Active duty engineer also reached the direction (
It will be appreciated that) may be expressed in degrees or any other units are suitable for a particular application instead of radians.
(E.g., in the broadside direction of the array)
It may be desirable to configure task T300 to select frequency components with arrival directions close to. As a result, on the one hand,
Between the first and fourth quadrant values of, and on the other hand
The distinction between the 2nd and 3rd quadrant values of is not important for calibration purposes.
In an alternative implementation, task T310 may include the corresponding frequency component of the multichannel signal (
Delay in arrival time
(For example, in seconds), each of the direction indicators. Task T310 is
Using an expression such as, refer to microphone MC10 to delay arrival time at microphone MC20 (
) May be configured to estimate In these examples,
Represents a signal arriving from the broadside direction,
A large positive value of represents the signal arriving from the reference endfire direction,
A large negative value of represents the signal arriving from the other end fire direction. Values (
), The sampling periods (e.g., 125 microseconds for a sampling rate of 8 kHz) or a fraction of a second (e.g., 10 -3 , 10 -4 , 10 -5 or It may be desirable to use time units that are deemed suitable for a particular application, such as 10 -6 seconds. Task T310 may also determine the frequency components of each channel in the time domain (
) By cross-correlating
Note that it may be configured to calculate).
For sound components arriving directly from the same point source,
Is ideally equal to the constant (k) for all frequencies, where the value of k is the direction of arrival (
) And arrival time delay (
) In another alternative implementation, task T310 is estimated phase difference (
To calculate each of the direction indicators (eg,
According to this far-field model (that is, the model assuming a planar wavefront), the direction indicator (
) But expressions
Is determined according to the near field model (ie, the model assuming a spherical wavefront, as illustrated in FIG. 7).
Note that) is calculated. Although the direction indicator based on the near field model may provide more accurate and / or easier to calculate results, the direction indicator based on the far field model may be desirable for some configurations of the method M100. Provides nonlinear mapping between values and phase differences.
Task T302 also includes a subtask T320 rating the direction indicators generated by task T310. Task T320 is configured to rate the direction indicators by converting or mapping the value of the direction indicator for each frequency component to be examined to a corresponding value for amplitude, magnitude, or pass / fail scale (also referred to as a "mask score"). It may be configured. For example, task T320 may use a directional masking function to map the value of each direction indicator to a mask score that indicates whether the indicated direction is within (and / or preferably within) the passband of the masking function. It may be configured to use. (In this context, the term “passband” refers to the range of arrival directions passed by the masking function.) The set of mask scores for the various frequency components may be considered as a vector. Task T320 may be configured to rate various direction indicators in series and / or in parallel.
The passband of the masking function may be selected to include the desired signal direction. The spatial selectivity of the masking function may be controlled by varying the width of the passband. For example, it may be desirable to select a passband width depending on the tradeoff between convergence rate and calibration accuracy. A wider passband may allow faster convergence by allowing more frequency components to contribute to the calibration operation, but this is farther from the broadside of the array (and therefore may be expected to affect microphones differently). It is also expected to be less accurate by allowing components to arrive from directions. In one example, task T300 (e.g., task T320, or task T330, as described below) is a component that arrives from directions within 15 degrees of the broadside axis of the array (ie, 75 to 105). Or, equivalently, components having arrival directions in the range of 5π / 12 to 7π / 12 radians).
8A shows a masking function (also referred to as a “brickwell” profile) and arrival direction with a relatively sharp transition between passband and stopband.
An example of a passband centered at is shown. In one such case, task T320 is a binary mask score having a first value (eg, 1) when the direction indicator indicates a direction within the passband of the function, and the direction indicator is outside the passband of the function. And assign a mask score having a second value (eg, 0) when indicating a direction. One or more factors such as signal-to-noise ratio (SNR), noise floor, etc. (e.g., to use a narrower passband when the SNR is high, indicating the presence of a desired directional signal that may adversely affect calibration accuracy). It may be desirable to change the position of the transition between the stopband and the passband depending on.
Alternatively, it may be desirable to configure task T320 to use a masking function with less abrupt transitions between the passband and stopband (eg, a more gradual rolloff that yields a non-binary mask score). It may be. 8b is the reaching direction
Shows an example of a linear rolloff for a masking function having a passband centered at &lt; RTI ID = 0.0 &gt;
An example of a nonlinear rolloff of the masking function with a passband centered at is shown. Stopband and pass depending on one or more factors, such as SNR, noise floor, etc. (for example, to use a more rapid rolloff when the SNR is high, indicating the presence of a desired directional signal that may adversely affect the calibration accuracy). It may be desirable to change the position and sharpness of transitions between bands. Of course, the masking function (as shown, for example, in FIGS. 8A-8C) is the direction (
Time delay () rather than
Or ratio (r) may also be expressed. For example, reach direction (
) Is zero time delay (
) Or ratio (
One example of a nonlinear masking function
Indicates the direction of target arrival,
Represents the desired width of the mask in radians,
Denotes a sharpness parameter. 9A-9C are
Examples of such a function are shown for. Of course, these functions have a direction (
Or ratio (r) may also be expressed. It is desirable to vary the width and / or sharpness of the mask depending on one or more factors such as SNR, noise floor, etc. (e.g., to use a narrower mask and / or more rapid rolloff when the SNR is high). You may.
5B shows a flowchart of an alternative implementation T304 of task T300. Instead of using the same masking function to rate each of the plurality of direction indicators, task T304 uses the corresponding directional masking function m i to determine each phase difference (
Subtask T330 that uses the calculated phase differences as direction indicators. E.g,
For the case where it is desired to select sound components arriving from directions in the range up to, each masking function m i is
May be configured to have a passband in the range up to
) to be.
For the case where it is desired to select sound components arriving from directions corresponding to the range of arrival time delays up to, each masking function m i is
For the case where it is desired to select sound components arriving from directions corresponding to the range of the ratio of phase differences to frequencies up to, each masking function m i is
to be. As discussed above with reference to task T320, the profile of each masking function may be selected according to one or more factors, such as SNR, noise floor, and the like.
It may be desirable to configure task T300 to generate mask scores for each of one or more (possibly all) of frequency components as temporarily smoothed values. This implementation of task T300 may be configured to calculate this value as an average of mask scores for the frequency component over the most recent m frames, where the m possible values are 5, 10, 20, and 50. It includes. More generally, this implementation of task T300 may be configured to calculate a smoothed value using a temporal smoothing function, such as a finite or infinite impulse response (FIR or IIR) filter. In one such example, task T300 is
The smoothed value of the mask score for the frequency component (i) of frame n according to an expression such as
Denotes the smoothed value of the mask score for the frequency component (i) for the previous frame,
Represents the current value of the mask score for the frequency component (i),
Is a smoothing factor whose value may be selected from the range of zero (no smoothing) to 1 (no updating). Such a first order IIR filter may also be referred to as a "leaky integrator".
Smoothing factor (
Typical values of) include 0.99, 0.09, 0.95, 0.9 and 0.8. For each frequency component in the frame
Using the same value of is common for task T300 but not necessarily. During the initial convergence period (e.g. immediately after power-on or other activity of the audio sensing circuit), the smoothed value is calculated over shorter intervals during the next steady-state operation, or one or more of the smoothing factors (whenever possible Using a smaller value for) may be desirable for task T300.
Task T340 may be configured to use the information from the plurality of mask scores to select the acoustically balanced portions of the signal. Task T340 may be configured to take binary mask scores as direct indicators of acoustic balance. For example, for a mask whose passband is in the broadside direction of array R100, task T340 may be configured to select frequency components having mask scores of 1 and (as shown in FIG. 3B). For a mask whose passbands are in the endfire directions of array R100, task T340 may be configured to select frequency components with zero mask scores.
For the case of a mask score that is not binary, task T340 may be configured to compare the mask score to a threshold. For example, for a mask whose passband is in the broadside direction of array R100, identifying the frequency component as an acoustically balanced portion if its mask score is greater than the threshold (otherwise not small) It may be desirable for task T340. Similarly, for a mask whose passbands are in the endfire directions of array R100, the frequency component is identified as an acoustically balanced portion if its mask score is less than (though not otherwise) a threshold. It may be desirable for task T340 to do.
This implementation of task T340 may be configured to use the same threshold for all frequency components. Alternatively, task T340 may be configured to use different thresholds for each of two or more (possibly all) of the frequency components. Task T340 may be configured to use a fixed threshold (or values), or alternatively, the characteristic of the signal (eg, frame energy) and / or the characteristic of the mask (eg, passband width) May be configured to adapt the threshold (or values) from one segment to another over time based on.
5C illustrates an implementation T205 of task T200; Implementation T305 of task T300 (eg, task T302 or task T304); And a flowchart of an implementation T405 of task T400. Task T205 is configured to calculate a level for each channel in each of (at least) two subbands. Task T305 is configured to update the gain factor value for each of the subbands, and task T405 measures the amplitude of the second channel in the corresponding subband relative to the amplitude of the first channel in the corresponding subband. It is configured to apply each updated gain factor value to change.
When a signal is received without reverberation from an ideal point source, all frequency components must have the same direction of arrival (e.g., ratio (
) Must be constant across all frequencies). The degree to which different frequency components of the signal have the same direction of arrival is also referred to as "coherence". When the microphone array receives sound originating from a far field source (eg background noise source), the resulting multichannel signal is typically received from a near field source (eg user's voice). It will be coherent less oriented than for sound. For example, the phase differences between microphone channels in each of the different frequency components will typically be less correlated with the frequency for the received sound originating from the far field source than for the received sound originating from the near field source. .
Configuring task T300 to use arrival direction as well as directional coherence to indicate whether portions of the multichannel signal (eg segments or subbands) are acoustically balanced or not acoustically balanced. It may be desirable. For example, it may be desirable to configure task T300 to select acoustically balanced portions of the multichannel signal based on the degree to which the frequency components in these portions are directionally coherent. The use of directional coherence can be achieved by, for example, enabling rejection of segments or subbands that include activity by a directional coherent source (eg, a near field source) located in the endfire direction of the array. It may support increased accuracy and / or reliability of channel calibration operation.
10 shows forward and reverse lobes of a directional pattern of a masking function as may be applied by the implementation of task T300 to a multichannel signal from two microphone arrays R100. Sound components received from sources located outside this pattern, such as near field sources in the broadside directions of the array R100 or far field sources in any direction, are acoustically balanced (ie, microphones). May cause the same responses by MC10 and MC20. Similarly, sound components received from sources in the forward or reverse lobes of this pattern (ie, near field sources in either of the endfire directions of array R100) are not acoustically balanced ( That is, one microphone may have a higher output level than the other microphone). Thus, selecting segments or subbands that have no sources within any lobe of this masking function pattern (eg, segments or subbands that are not directionally coherent or coherent only in the broadside direction). It may be desirable to configure the corresponding implementation of task T300 to do so.
As mentioned above, task T300 may be configured to use the information from the phase differences calculated by task T100 to identify acoustically balanced portions of the multichannel signal. Task T300 is based on the signal that the mask scores indicate that the subbands or segments are directional coherent in the broadside direction of the array (or, alternatively, they are not directional coherent in the endfire direction). It may be implemented to identify acoustically balanced portions as subbands or segments, so that updating of the corresponding gain factor value is performed only for these identified subbands or segments.
11A shows a flowchart of an implementation M110 of method M100 that includes an implementation T306 of task T300. Task T306 includes a subtask T360 that calculates a value of the coherency measurement based on the information from the phase differences calculated by task T100. 11B shows a flowchart of an implementation T362 of task T360 that includes the case of subtasks T312 and T322 and subtask T350 as described above. 11C shows a flowchart of an implementation T364 of task T360 that includes the case of subtask T332 and subtask T350 as described above.
Task T350 may be configured to combine mask scores of frequency components in each subband to obtain a coherency measurement for the subband. In one such example, task T350 is configured to calculate a coherency measure based on the number of mask scores having a particular state. In another example, task T350 is configured to calculate the coherency measure as a sum of mask scores. In a further example, task T350 is configured to calculate the coherency measure as an average of the mask scores. In any of these cases, task T350 may equally weight each of the mask scores (eg, weight each mask score by one) or weight one or more mask scores differently from each other (eg, , Weighting the mask score corresponding to the low frequency or high frequency component less severely than the mask score corresponding to the intermediate frequency component.
For a mask whose passband is on the broadside of array R100 (eg, as shown in FIGS. 8A-8C and 9A-9C), task T350 is, for example, a mask. If the sum or average of the scores is not less than (though otherwise) the threshold, or if the frequency components in at least (differently, more) minimum small subbands have a mask score of 1, then the first state (eg, High or " 1 ", otherwise configured to generate a coherency indication with a second state (e.g., low or " 0 "). For a mask whose passband is in the endfire direction of array R100, task T350 may be, for example, if the sum or average of the mask scores is not greater than (though, smaller) or greater than the threshold (or otherwise). Alternatively, if the frequency components in the smaller) maximum number of subbands have a mask score of 1, it may be configured to generate a coherency measurement having a first state, otherwise having a second state.
Task T350 may be configured to use the same threshold for each subband or use a different threshold for each of two or more (possibly all) of the subbands. Each threshold may be heuristically determined and it may be desirable to vary the threshold over time depending on one or more factors such as passband width, one or more characteristics of the signal (eg, SNR, noise floor), and the like. (The same principles apply to the maximum and minimum numbers mentioned in the previous paragraph).
Alternatively, task T350 may be configured to generate a corresponding directional coherency measure for each of the series of segments of the multichannel signal. In this case, task T350 obtains coherency measurements for the segment (eg, based on the number of mask scores having a particular state, or the sum or average of the mask scores, as described above). To combine the mask scores of two or more (possibly all) of the frequency components in each segment. This implementation of task T350 may be configured to change the threshold through time depending on one or more factors as described above, or to use the same threshold for each segment (eg, the same principle). Are applied to the maximum or minimum number of mask scores).
It may be desirable to configure task T350 to calculate a coherency measure for each segment based on mask scores of all frequency components of the segment. Alternatively, it may be desirable to configure task T350 to calculate a coherency measure for each segment based on mask scores of frequency components over a defined frequency range. For example, task T350 may be configured to calculate a coherency measure based on mask scores of frequency components over a frequency range from about 50, 100, 200, or 300 Hz to about 500 or 1000 Hz. (Combinations of each of these eight are explicitly contemplated and disclosed). For example, it may be determined that the differences between the response characteristics of the channels are sufficiently characterized by the difference in the gain responses of the channels over this frequency range.
Task T340 may be configured to calculate an updated value for each of the at least one gain factor based on information from the acoustically balanced portions identified by task T360. For example, the multichannel signal is in the corresponding segment or subband (eg, in response to the selection of the subband or segment in task T360 when indicated by the state of the corresponding coherency indication). It may be desirable to configure task T340 to calculate an updated gain factor value in response to an indication that it is directionally coherent.
Task T400 may be configured to use the updated gain factor value generated by task T300 to control the amplitude of the second channel relative to the amplitude of the first channel. As described herein, it may be desirable to configure task T300 to update the gain factor value based on the observed level imbalance of the acoustically balanced segment. Suppressing the gain factor value may be desirable for task T300, and for the next segments that are not acoustically balanced, continuing to apply the most recently updated gain factor value for task T400. It may be desirable. 12A shows a flowchart of an implementation M120 of method M100 that includes such an implementation T420 of task T400. Task T420 is to change the amplitude of the second channel relative to the amplitude of the first channel in each of a series of consecutive segments of the multichannel signal (eg, each of a series of acoustically unbalanced segments). Configured to use the updated gain factor value. This series may continue until another acoustically balanced segment is identified such that task T300 updates the gain factor value again. (The principles described in this paragraph may also apply to the updating and application of subband gain factor values as described herein).
Implementations of the method M100 may also be spatial selection processing operations that may be calibration dependent (eg, determine the distance between the audio sensing device and a particular sound source, reduce noise, and remove signal components arriving from a particular direction). One or more operations for reinforcing and / or separating one or more sound components from other environmental sounds) and / or various additional operations for the processed multichannel signal. For example, the range of applications for balanced multichannel signals (eg, processed multichannel signals) may include abnormal variance and / or directional noise; Reverberation of sound generated by the desired speaker in the near field; Removal of uncorrelated noise (eg wind and / or sensor noise) between microphone channels; Suppression of sound from unwanted direction; Suppression of far field signals from any direction; Direct path-reverberation signal strength (eg, for substantial cancellation of interference from far-field sources); Reduction of abnormal noise through discrimination between near and far field sources; And gain-based approaches typically include a reduction in sound from the frontal interferer during the desired source activity of the near field as well as during unachievable pauses.
12B shows a flowchart of an implementation M130 of method M100 that includes a task T500 that performs voice activity detection (VAD) operation on a processed multichannel signal. 13A shows a flowchart of an implementation M140 of method M100 that includes a task T600 that may update a noise estimate based on information from the processed multichannel signal and include a voice activity detection operation. do.
It may be desirable to implement a signal processing scheme that distinguishes between near-field and far-field sources (eg, for better noise reduction). One amplitude or gain based example of this approach uses a pressure gradient field between two microphones to determine whether the source is a near or far field. While this technique may be useful for reducing noise from the far-field source during near-field silence, it may not support the distinction between the near-field signal and the far-field signal when both sources are active.
It may be desirable to provide a consistent pickup within a certain angular range. For example, it accepts all near-field signals within a certain range (eg, 60 degrees range with respect to the axis of the microphone array), and the rest (eg, signals from sources above 70 degrees) are all attenuated. It may be desirable to. With beamforming and BSS, each attenuation typically prevents consistent pickup across this range. These methods may also cause negative rejection after a change in orientation of the device (eg, rotation) before the post-processing operation is reconverged. Implementations of the method M100 as described herein may be used to obtain robust noise reduction methods for abrupt rotation of the device, while the direction for the desired speaker is still within the range of acceptable directions, thus converging delay. Voice attenuation due to the noise and / or the voice attenuation due to a previous noise reference may be avoided.
By combining phase differences and phase based directivity information from a balanced multichannel signal, an adjustable spatial region may be selected around the microphone array in which the presence of signals can be monitored. The gain based and / or directional boundaries may be set to define narrow or wide pickup areas for different subtasks. For example, a narrower boundary may be set to detect desired voice activity, while a wider boundary for the selected area may be used for purposes such as noise reduction. The accuracy of phase correlation and gain difference estimates tends to decrease with decreasing SNR, and it may be desirable to adjust thresholds and / or decisions in accordance with false alarm rates.
For applications where the processed multichannel signal is only used to support voice activity detection (VAD) operation, it may be acceptable to operate at a reduced accuracy level for gain correction, thereby reducing efficient and accurate noise reduction operation. It may be done faster with noise reduction convergence time.
As the relative distance between the sound source and the microphone pair increases, it may be expected that the coherence between the arrival directions of the different frequency components decreases (due to an increase in reverberation). Thus, the coherency measurement calculated at task T360 may also serve to some extent as a proximity measurement. Unlike processing operations based only on the direction of arrival, for example, other from interference such as speech of a speaker competing for time and / or frequency dependent amplitude control based on the value of the coherency measurement as described herein. It may be efficient to distinguish the desired near field source or the user's speech from the far field source in the same direction. The rate at which directional coherency decreases with distance may vary from one environment to another. For example, the interior of a motor vehicle is typically very reverberating so that directional coherence over a wide range of frequencies may be maintained at a level that is reliably stable over time in the range of only about 50 centimeters from the source. In such a case, sound from the back seat passenger may be rejected as incoherent even if the speaker is located within the passband of the directional masking function. The range of detectable coherence may also be reduced in this environment for tall speakers (eg, due to reflections from nearby ceilings).
The processed multichannel signal may be used to support BSS, month delay, or other spatial selection processing (SSP) operations such as other directional SSP, or distance SSP such as proximity detection. Proximity detection may be based on gain differences between channels. It may be desirable to calculate the gain difference in the time domain, or in the frequency domain (eg, as a measure of coherence over a limited frequency range and / or at multiple pitch frequencies).
Multi-microphone noise reduction schemes for portable audio sensing devices include a beamforming approach and a blind source separation (BSS) approach. These approaches typically suffer from the inability to suppress noise arriving from the same direction as the desired sound (eg, the nearfield speaker's voice). In particular, in headsets and mid-range or far-field handheld applications (eg, browse-talk and speakerphone modes of a handset or smartphone), the multichannel signal recorded by the microphone array is It may include significant reflections of sound from interfering noise sources and / or speech of the desired near field talker. In particular, for headsets, the large distance to the user's mouth may cause the microphone array to pick up a large amount of noise from the frontal direction, which may be difficult to suppress significantly using only directional information.
Conventional BSS or generalized sidelobe cancellation (GSC) type techniques perform noise reduction by first separating the desired voice into one microphone channel and then performing a post processing operation on the separated voice. This procedure may result in long convergence time in the case of changing acoustic scenarios. For example, noise reduction schemes based on blind source separation, GSC, or similar adaptive learning rules and / or changes in device-user holding patterns (eg, orientation between the device and the mouth of the user). It may exhibit long convergence time during rapid changes in the loudness and / or spectral signature of environmental noise (eg, passing vehicle, announcement). In an echo environment (eg, a car interior), the adaptive learning approach may have problematic convergence. Failure of this approach to convergence may lead to rejecting the desired signal component. In voice communication applications, such rejection may increase voice distortion.
In order to increase the robustness of these approaches to changes in device-user holding patterns and / or increase the convergence time, it would be desirable to define a spatial pickup area around the device to provide a faster initial noise rejection response. It may be. This method may be configured to utilize phase and gain relationships between microphones to define a defined spatial pickup area by discriminating certain angular directions (eg, with respect to a device's reference direction, such as the axis of the microphone array). . By having a selection around the audio device in the desired speaker direction, which always indicates baseline initial noise reduction, a high degree of robustness is achieved against the desired user's spatial changes in the audio device as well as rapid changes to environmental noise. Can be.
The gain differences between the balanced channels may be used for proximity detection, which may support more aggressive near-field / far-field discrimination, such as better frontal noise suppression (eg, suppression of an interfering speaker at the user's front). . Depending on the distance between the microphones, the gain difference between the balanced microphone channels will typically only occur if the source is within 50 centimeters or 1 meter.
13B shows a flowchart of an implementation M150 of method M100. The method M150 includes a task T700 that performs a proximity detection operation on the processed multichannel signal. For example, task T700 may be executed when the difference between the levels of the channels of the processed multichannel signal is greater than the threshold (differently, (A) the level difference between uncorrected channels and (B) task (T300). May be configured to detect that the segment is from a desired source (e.g., to indicate detection of voice activity). The threshold may be determined heuristically, and using different thresholds depending on one or more factors such as signal to noise ratio, noise floor, etc. (e.g., to use a higher threshold when the SNR is low). It may be desirable. 14A shows an example of the boundaries of proximity detection regions corresponding to three different thresholds, which increase smaller as the threshold increases.
It may be desirable to combine the range of allowed detections (e.g., ± 45 degrees) with near / far field proximity bubbles to obtain a cone of speaker coverage and attenuate abnormal noise from sources outside this area. . This method may be used to attenuate sound from the far field sources even when the far field sources are within a range of acceptable directions. For example, it may be desirable to provide good microphone calibration to support aggressive tuning of the near / far field discriminator. FIG. 14B illustrates the intersection of a range of allowed directions (eg, a forward lobe as shown in FIG. 10) with a proximity bubble (as shown in FIG. 14A) to obtain this cone of speaker coverage (bold). Shown). In this case, the plurality of phase difference calculations in task T100 are discussed above with reference to (e.g., tasks T312, T322, and T332) to identify segments resulting from sources in the desired range. Such as) and / or coherency measurements (as discussed above with reference to task T360) may be used to apply a range of allowed directions. The direction and profile of this masking function may be selected according to the desired application (eg, a sharper profile for voice activity detection, or a smoother profile for attenuation of noise components).
As discussed above, FIG. 2 shows a top view of a headset mounted to a user's ear in a standard orientation with respect to the user's mouth. 15 and 16 show top and side views of a source selection region boundary as shown in FIG. 14B applied to such an application.
It may be desirable to use the results of a proximity detection operation (eg, task T700) for voice activity detection (VAD). In one such example, non-binary improved VAD measurements are applied as gain control for one or more of the channels (eg, to attenuate noise frequency components and / or segments). 17A shows a flowchart of an implementation M160 of method M100 that includes a task T800 for performing this gain control operation on a balanced multichannel signal. In another such example, a binary improved VAD can be used to calculate (eg, update) a noise estimate for a noise reduction operation (eg, using frequency components or segments classified as noise by the VAD operation). Is applied to. 17B shows a flowchart of an implementation M170 of method M100 that includes a task T810 that calculates (eg, updates) a noise estimate based on the result of a proximity detection operation. 18 shows a flowchart of an implementation M180 of method M170. The method M180 includes a task T820 that performs a noise reduction operation (eg, spectral subtraction or winner filtering operation) on at least one channel of the multichannel signal based on the updated noise estimate.
The results from the proximity detection operation and the directed coherence detection operation (which define the bubble as shown in FIGS. 14B and / or FIGS. 15 and 16) to obtain an improved multi-channel voice activity detection (VAD) operation. May be combined. Combined VAD operation may be used for rapid rejection of non-voice frames and / or to establish a noise reduction scheme to operate on the primary microphone channel. Such a method may include performing calibration, a combination of direction and proximity information for the VAD, and performing a noise reduction operation based on the results of the VAD operation. For example, it may be desirable to use this combined VAD operation in methods M160, M170 or M180 instead of proximity detection task T700.
Acoustic noise in a typical environment may include loud noise, airport noise, distance noise, competing speaker's voices, and / or sounds from interference sources (eg, a TV set or radio). As a result, such noise is typically abnormal and may have an average spectrum close to the spectrum of the user's own voice. The noise power reference signal when calculated from a single microphone signal is generally only an approximate normal noise estimate. In addition, this calculation generally involves a noise power estimation delay, so that the corresponding adjustment of the subband gain can only be performed after a significant delay. It may be desirable to obtain a reliable and concurrent estimate of environmental noise.
Examples of noise estimates include a single channel long term estimate based on a single channel VAD, and a noise reference when generated by a multichannel BSS filter. Task T810 may be configured to calculate a single channel noise reference by using (dual-channel) information from the proximity detection operation to classify components and / or segments of the primary microphone channel. Since these noise estimates do not require long-term estimates, they may be available much faster than other approaches. This single channel noise reference can also capture abnormal noise, unlike a long term estimate based approach that typically cannot support the removal of abnormal noise. Such a method may provide a fast, accurate and abnormal noise reference. For example, this method may be configured to update the noise reference for any frames that are not in the forward cone as shown in FIG. 14B. The noise reference may be smoothed (eg, using a 1 degree smoother for each frequency component if possible). The use of proximity detection may cause a device using this method to reject nearby transients, such as the sound of the noise of the car passing through the forward lobe of the directional masking function.
Rather than waiting for a multichannel BSS scheme for convergence, it may be desirable to configure task T810 to take a noise reference directly from the primary channel. Such a noise reference may be constructed using a combined phase gain VAD or just using phase VAD. This approach may also help to avoid the problem of the BSS scheme, which attenuates voice while converging to a new spatial configuration between the speaker and the telephone, or when the headset is being used in a suboptimal spatial configuration.
The VAD indication as described above may be used to support the calculation of the noise reference signal. For example, when the VAD indication indicates that the frame is noise, the frame may be used to update the noise reference signal (eg, the spectral profile of the noise component of the primary microphone channel). Such updating may be performed in the frequency domain, for example, by temporarily smoothing the frequency component values (eg, updating the previous value of each component with the value of the corresponding component of the current noise estimate). In one example, the Wiener filter uses a noise reference signal to perform noise reduction operations on the primary microphone channel. In another example, the spectral subtraction operation uses a noise reference signal to perform a noise reduction operation on the primary microphone channel (eg, by subtracting the noise spectrum from the primary microphone channel). When the VAD indication indicates that the frame is not noise, the frame may be used to update the spectral profile of the signal component of the primary microphone channel, which profile may also be used by the Wiener filter to perform a noise reduction operation. have. The resulting operation may be considered to be a quasi-single-channel noise reduction algorithm using dual channel VAD operation.
Note that as described herein, proximity detection may also be applied to situations where channel correction is not required (eg, microphone channels are already balanced). FIG. 19A illustrates a VAD operation T900 based on coherency measurement and proximity detection (eg, a bubble as shown in FIG. 14B) as described herein and tasks T100 and as described herein. A flowchart of the method M300 according to the general configuration, including the cases of T360, is shown. 19B shows a flowchart of an implementation M310 of method M300 that includes a noise estimate calculation task T910 (eg, as described with reference to task T810), and FIG. For example, a flowchart of an implementation M320 of method M310 that includes a noise reduction task T920 (as described with reference to task T820).
20B shows a block diagram of an apparatus G100 according to a general configuration. Apparatus G100 includes means F100 for obtaining a plurality of phase differences (eg, as described herein with reference to task T100). Apparatus G100 also includes means F200 for calculating levels of the first and second channels of the multichannel signal (eg, as described herein with reference to task T200). Apparatus G100 also includes means F300 for updating the gain factor value (eg, as described herein with reference to task T300). The apparatus G100 also includes means for changing the amplitude of the second channel for the first channel based on the updated gain factor value (eg, as described herein with reference to task T400). F400).
21A shows a block diagram of an apparatus A100 according to a general configuration. Apparatus A100 is configured to obtain a plurality of phase differences from channels S10-1 and S10-2 of a multichannel signal (eg, as described herein with reference to task T100). 100. Apparatus A100 also includes a level calculator 200 configured to calculate levels of the first and second channels of the multichannel signal (eg, as described herein with reference to task T200). . Apparatus A100 also includes a gain factor calculator 300 configured to update the gain factor value (eg, as described herein with reference to task T300). Apparatus A100 is also multi-processed by changing the amplitude of the second channel for the first channel based on the updated gain factor value (eg, as described herein with reference to task T400). A gain control element 400 configured to generate a channel signal.
21B shows the apparatus A100; FFT modules TM10a and TM10b configured to generate each of signals S10-1 and S10-2 in the frequency domain; And a block select processing module SS100 configured to perform a space select processing operation (eg, as described herein) on the processed multichannel signal. FIG. 22 shows a block diagram of an apparatus A120 that includes apparatus A100 and FFT modules TM10a and TM10b. Apparatus A120 is also configured to perform a proximity detection operation (eg, voice activity detection operation) on the processed multichannel signal (eg, as described herein with reference to task T700). Proximity detection module 700 (eg, voice activity detector), noise reference calculator 810, configured to update the noise estimate (eg, as described herein with reference to task T810) (eg, For example, a noise reduction module 820 configured to perform a noise reduction operation on at least one channel of the processed multichannel signal, as described herein with reference to task T820, and time the noise reduced signal. An inverse FFT module IM10 configured to convert to a domain. In addition to or as an alternative to the proximity detection module 700, the apparatus A110 may detect voice activity based on directional processing (eg, forward lobes as shown in FIG. 14B) of the processed multichannel signal. It may also include a module for.
Some multichannel signal processing operations use information from more than one channel of the multichannel signal to generate each channel of the multichannel output. Examples of such operations include beamforming and blind source separation (BBS) operations. Integrating this technique with echo cancellation may be difficult because operation tends to change the residual echo in each output channel. As described herein, the method M100 uses information from the calculated phase differences to single channel time and / or frequency dependent for each of one or more channels (eg, primary channel) of the multichannel signal. It may be implemented to perform amplitude control (eg, noise reduction operation). This single channel operation may be implemented such that the residual echo remains substantially unchanged. As a result, the integration of the echo cancellation operation with the implementation of the method M100 including such a noise reduction operation may be easier than the integration of the noise reduction operation with the echo cancellation operation operating on two or more microphone channels.
It may be desirable to whiten residual background noise. For example, a VAD operation (e.g., It may be desirable to use directivity and / or proximity based VAD operation as described). Such noise whitening may produce a residual normal noise floor and / or may reach perception of noise that is placed in the background or backed out of the background. A smoothing scheme, such as a temporal smoothing scheme, to handle transitions between intervals where whitening is not applied (eg speech intervals) and intervals where heightening is applied (eg noise intervals). It may be desirable. Such smoothing may help support smooth transitions between gaps.
Note that the microphones (eg, MC10 and MC20) may be more generally implemented as transducers sensitive to radiation or emission other than sound. In one such example, the microphone pair is implemented as a pair of ultrasonic transducers (eg, transducers sensitive to acoustic frequencies greater than 15, 20, 25, 30, 40, or 50 kilohertz).
For directional signal processing applications (eg, identifying a forward lobe as shown in FIG. 14B), certain frequency components that a speech signal (or other desired signal) may be expected to be directionally coherent. Or, it may be desirable to target a specific frequency range. It may be expected that background noise such as directional noise and / or distributed noise (eg, from sources such as automobiles) will not be directionally coherent over the same range. Speech tends to have low power in the range of 4 to 8 kHz, so it may be desirable to determine the directional coherence with reference to frequencies not greater than 4 kHz. For example, it may be desirable to determine directional coherence over a range from about 700 hertz to about 2 kilohertz.
As mentioned above, it may be desirable to configure task T360 to calculate a coherency measurement based on phase differences of frequency components over a defined frequency range. Additionally or alternatively, define task T360 and / or other directional processing task (in particular, a forward lobe as shown in FIG. 14B) to calculate a coherency measure based on frequency components in multiples of pitch frequency. It may be desirable to configure for speech applications such as).
The energy spectrum of voiced speech (eg, vowels) tends to have local peaks at harmonics of pitch frequency. On the other hand, the energy spectrum of background noise tends to be relatively unstructured. As a result, the components of the input channels at harmonics of the pitch frequency may be expected to have a higher signal to noise ratio (SNR) than the other components. For the directional processing task for the speech processing application (eg, voice activity detection application) of the method M100, configuring the task to only consider phase differences corresponding to multiples of the estimated pitch frequency (eg, Configuring a forward lobe identification task) may be desirable.
Typical pitch frequencies range from about 70 to 100 Hz for male speakers to about 150 to 200 Hz for female speakers. The current pitch frequency may be estimated by calculating the pitch period as the distance between adjacent pitch peaks (eg, in the primary microphone channel). The sample of the input channel is based on a measure of its energy (e.g., based on the ratio between the sample energy and the frame average energy) and / or how well the sample's neighborhood correlates with a similar neighborhood of known pitch peaks. May be identified as the pitch peak. Pitch estimation procedures are described, for example, in Section 4.6.3 (pp. 4-44 to 4-49) of Enhanced Variable Rate Codec (EVRC) document C.S0014-C available online at www-dot-3gpp-dot-org. ). The current estimate of the pitch frequency (eg, in the form of a pitch period or an "pitch lag" estimate) is typically used in applications that include speech encoding and / or decoding (eg, code excitation linear prediction (CELP). ) And voice communications using codecs including pitch estimation such as prototype waveform interpolation (PWI).
By considering only the phase differences corresponding to multiples of the pitch frequency, the number of phase differences to be considered may be significantly reduced. It may also be expected that the frequency coefficients from which these selected phase differences are calculated have high SNRs for other frequency coefficients within the considered frequency range. In a more general case, other signal characteristics may also be considered. For example, it may be desirable to configure the directional processing task such that at least 25, 50, or 75 percent of the calculated phase differences correspond to multiples of the estimated pitch frequency. The same principle may also be applied to other desired harmonic signals.
As mentioned above, it may be desirable to manufacture a portable audio sensing device having an array R100 of two or more microphones configured to receive acoustic signals. Examples of portable audio sensing devices that may be implemented to include such an array and that may be used for audio recording and / or voice communication applications include, but are not limited to, telephone handsets (eg, cellular telephone handsets); Wired or wireless headsets (eg, Bluetooth headsets); Handheld audio and / or video recorders; A personal media player configured to record audio and / or video content; A portable assistant (PDA) or other handheld computing device; And notebook computers, laptop computers, netbook computers, or other portable computing devices.
Each microphone of array R100 may have a response that is omnidirectional, bidirectional, or unidirectional (eg, cardioid). Various types of microphones that may be used in array R100 include piezoelectric microphones, dynamic microphones, and electret microphones. In devices for portable voice communications such as handsets or headsets, the center-to-center spacing between adjacent microphones of array R100 is typically also greater in devices such as handsets with larger spacing (eg, up to 10 or 15 cm). Possible, but in the range of about 1.5 cm to about 4.5 cm. For listening assistance, the center to center spacing between adjacent microphones of array R100 may be as small as about 4 or 5 mm. The microphones of array R100 may be arranged along a line, or alternatively, their centers are at the vertices of a two-dimensional (eg, triangular) or three-dimensional shape.
During operation of a multi-microphone audio sensing device (e.g., a device D100, D200, D300, D400, D500, or D600 as described herein), the array R100 can be used as a microphone for each acoustic environment. Generate a multichannel signal based on the corresponding one of these responses. One microphone may receive a particular sound more directly than the other, so that the corresponding channels are different from one another to collectively provide a more complete representation of the acoustic environment than can be captured using a single microphone.
It may be desirable for array R100 to perform one or more processing operations on the signals generated by the microphones to generate a multichannel signal S10. FIG. 23A illustrates an audio preprocessing stage (AP10) configured to perform one or more such operations that may include (without limitation) impedance matching, analog-to-digital conversion, gain control, and / or filtering in the analog and / or digital domains. Shows a block diagram of an implementation R200 of an array R100.
23B shows a block diagram of an implementation R210 of array R200. Array R210 includes an implementation AP20 of audio preprocessing stage AP10 that includes analog preprocessing stages P10a and P10b. In one example, the stages P10a and P10b are each configured to perform a high pass filtering operation (eg, at a cutoff frequency of 50, 100, or 200 Hz) on the corresponding microphone signal.
It may be desirable for array R100 to generate a multichannel signal as a digital signal, ie as a sequence of samples. Array R210 includes, for example, analog-to-digital converters (ADCs) C10a and C10b that are each arranged to sample a corresponding analog channel. Typical sampling rates for acoustic applications include 8 kHz, 12 kHz, 16 kHz, and other frequencies in the range of about 8 to about 16 kHz, although sampling rates as high as about 44 kHz may also be used. In this particular example, array R210 is also each digital preconfigured to perform one or more preprocessing operations (eg, echo cancellation, noise reduction, and / or spectral shaping) on the corresponding digitized channel. Processing stages P20a and P20b.
Note that the microphones of array R100 may be more generally implemented as transducers sensitive to radiation or emission in addition to sound. In one such example, the microphones of array R100 are implemented as ultrasonic transducers (eg, transducers sensitive to acoustic frequencies greater than 15, 20, 25, 30, 40, or 50 kilohertz). .
24A shows a block diagram of the device D10 according to the general configuration. Device D10 includes the case of any implementation of microphone array R100 disclosed herein, and any audio sensing devices disclosed herein may be implemented as one case of device D10. Device D10 also includes the case of an implementation of apparatus A10 that is configured to process a multichannel signal as generated by array R100 to calculate a value of a coherency measure. For example, apparatus A10 may be configured to process a multichannel audio signal in accordance with the case of any implementation of the method M100 disclosed herein. The apparatus A10 may be implemented in hardware and / or software (eg, firmware). For example, apparatus A10 may be used to perform spatial processing operations (e.g., determining the distance between an audio sensing device and a particular sound source, reducing noise, and from a particular direction, as described above for the processed multichannel signal. May be implemented on a processor of device D10 that is also configured to perform one or more operations to enhance the signal components that arrive and / or separate one or more sound components from other environmental sounds. Device A10 as described above may be implemented as the case of device A10.
24B shows a block diagram of communication device D20, which is an implementation of device D10. Device D20 includes a chip or chipset CS10 (eg, a mobile station modem (MSM) chipset) that includes apparatus A10. Chip / chipset CS10 may include one or more processors that may be configured to execute all or a portion of apparatus A10 (eg, as instructions). Chip / chipset CS10 may also include processing elements of array R100 (eg, elements of audio preprocessing stage AP10). The chip / chipset CS10 is based on a receiver configured to receive a radio frequency (RF) communication signal and to decode and reproduce an audio signal encoded in the RF signal, and based on the processed signal generated by the apparatus A10. And a transmitter configured to encode the audio signal and to transmit an RF communication signal that describes the encoded audio signal. For example, one or more processors of chip / chipset CS10 may be configured to perform the noise reduction operation as described above for one or more channels of the multichannel signal, such that the encoded audio signal is subjected to the noise reduced signal. Based.
Device D20 is configured to receive and transmit RF communication signals via antenna C30. Device D20 may also include a diplexer and one or more power amplifiers in the path to antenna C30. Chip / chipset CS10 may also be configured to receive user input via keypad C10 and display information via display C20. In this example, device D20 may also include one or more antennas for supporting short range communication with an external device, such as Global Positioning System (GPS) location services and / or a wireless (eg, Bluetooth ™) headset ( C40). In another example, this communication device itself is a Bluetooth headset and lacks a keypad C10, a display C20, and an antenna C30.
Implementations of apparatus A10 as described herein may be implemented in various audio sensing devices, including headsets and handsets. One example of a handset implementation includes a front-facing dual microphone implementation of array R100 with 6.5 centimeter spacing between microphones. Implementation of a dual microphone masking approach may include directly analyzing the phase relationships of the microphone pairs in spectrograms and masking time-frequency points from unwanted directions.
25A-25D show various views of a multi-microphone portable audio sensing implementation D100 of device D10. Device D100 is a wireless headset that includes a housing Z10 carrying a two-microphone implementation of array R100 and earphones Z20 extending from the housing. Such a device may be half or full duplex via communication with a telephone device, such as a cellular telephone handset (e.g., using a version of the Bluetooth ™ protocol as advertised by Bluetooth Special Interest Group, Inc., Bellevue, WA). It may also be configured to support telephony. In general, the housing of the headset may be rectangular or otherwise slender (eg, miniboom, more round or even circular, as shown in FIGS. 25A, 25B, and 25D). May also enclose a battery and processor and / or other processing circuitry (e.g., a printed circuit board and components mounted thereon), an electrical port (e.g., a mini-USB (Universal Serial Bus) or Other ports for battery charging) and user interface features such as one or more button switches and / or LEDs Typically, the length of the housing along the major axis is in the range of 1 to 3 inches.
Typically, each microphone of array R100 is mounted in the device behind one or more small holes in the housing that act as acoustic ports. 25B-25D show the locations of acoustic port Z40 for the primary microphone of the array of device D100 and acoustic port Z50 for the secondary microphone of the array of device D100.
The headset may also include a fixing device such as an ear hook Z30 that is typically removable from the headset. The external ear hook may be reversible, for example, to allow a user to configure the headset for use on an ear. Alternatively, the earphones of the headset may include removable earpieces to allow different users to use different size (eg, diameter) earpieces for good fitting to the external portion of the particular user's ear conduit. It may be designed as an internal fixation device (eg, an earplug) that may be.
26A-26D show various views of a multi-microphone portable audio sensing implementation D200 of device D10 that is another example of a wireless headset. Device D200 includes a round oval housing Z12 and earphone Z22, which may be configured as earplugs. 26A-26D also show the locations of acoustic port Z42 for the primary microphone of the array of device D200 and acoustic port Z52 for the secondary microphone. It is possible that the secondary microphone port Z52 is at least partially blocked (eg by a user interface button).
FIG. 27A shows a cross-sectional view (along the center axis) of a multi-microphone portable audio sensing implementation D300 of device D10 that is a communication handset. Device D300 includes an implementation of array R100 with primary microphone MC10 and secondary microphone MC20. In this example, device D300 also includes a primary loudspeaker SP10 and a secondary loudspeaker SP20. Such a device may be configured to wirelessly transmit and receive voice communication data via one or more encoding and decoding schemes (also called "codecs"). Examples of such codecs are the third generation partnership project 2 (3GPP2) document C.S0014-C, v1.0, entitled "Enhanced Variable Rate Codec, Speech Service Options 3, 68, and 70 for Wideband Spread Spectrum Digital Systems". Ehanced Variable Rate Codec as described in February 2007 (available online at www-dot-3gpp-dot-org); 3GPP2, entitled "Selectable Mode Vocoder (SMV) Service Option for Wideband Spread Spectrum Communication Systems," C.S0030-0, v3.0, January 2004 (available online at www-dot-3gpp-dot-org) The Selectable Mode Vocoder speech codec; The Adaptive Multi Rate (AMR) speech codec as described in document ETSI TS 126 092 V6.0.0 (European Telecommunications Standards Institute (ETSI), Sophia Antipolis Cedex, FR, December 2004); And the AMR Wideband speech codec as described in document ETSI TS 126 192 V6.0.0 (ETSI, Dec. 2004). In the example of FIG. 3A, the handset D300 is a cram cell type cellular telephone handset (also referred to as a “flip” handset). Other configurations of such multi-microphone communication handsets include bar and slider phone handsets. FIG. 27B shows a cross-sectional view of an implementation D310 of device D300 that includes a three-microphone implementation of array R100 that includes third microphone MC30.
28A shows a diagram of a multi-microphone portable audio sensing implementation D400 of device D10 that is a media player. These devices are available in standard compression formats (e.g., Video Expert Group (MPEG) -1 Audio Layer 3 (MP3), MPEG-4 Part 14 (MP4), Windows Media Audio / Video (WMA / WMV) (Microsoft , Redmond, WA), Advanced Audio Coding (AAC), International Telecommunications Association (ITU) -T H.264, etc.) may be configured for playback of compressed audio or audiovisual information, such as encoded files or streams. have. The device D400 includes a display screen SC10 and a loudspeaker SP10 disposed in front of the device, and the microphones MC10 and MC20 of the array R100 are arranged on the same side of the device (eg, in this example). On the opposite side of the top surface, or on the opposite side of the front face). FIG. 28B shows another implementation D410 of device D400 in which microphones MC10 and MC20 are disposed on opposite surfaces of the device, and FIG. 28C shows microphones MC10 and MC20 at adjacent surfaces of the device. Another implementation D420 of device D400 that is deployed is shown. The media player may also be designed such that the long axis is horizontal during its intended use.
FIG. 29 shows a diagram of a multi-microphone portable audio sensing implementation D500 of device D10 that is a handsfree car kit. Such a device may be configured to be installed or detachably secured to or on a dashboard, windshield, rearview mirror, visor, or other interior surface of the vehicle. Device D500 includes an implementation of loudspeaker 85 and array R100. In this particular example, device D500 includes an implementation R102 of array R100 as four microphones arranged in a linear array. Such a device may be configured to wirelessly transmit and receive voice communication data via one or more codecs, such as the examples listed above. Alternatively, or in addition, such a device may employ half or full duplex telephony via communication with a telephony device such as a cellular telephone handset (e.g., using a version of the Bluetooth ™ protocol as described above). It may be configured to support.
30 shows a diagram of a multi-microphone portable audio sensing implementation D600 of device D10 for a handheld application. Device D600 includes a touchscreen display T10, three front microphones MC10 to MC30, a back microphone MC40, two loudspeakers SP10 and SP20, a left user interface control (e.g., for selection U110, and right user interface control (eg, for navigation) U120. Each of the user interface controls may be implemented using one or more of pushbuttons, trackballs, click-wheels, touchpads, joysticks and / or other pointing devices, and the like. A typical size of device D800 that may be used in browse-talk mode or game play mode is about 15 centimeters by about 20 centimeters. It is explicitly disclosed that the applicability of the systems, methods, and apparatuses disclosed herein is not limited to the particular examples shown in FIGS. 25A-30. Other examples of portable audio sensing devices to which such systems, methods, and apparatuses may apply include hearing aids.
The methods and apparatuses disclosed herein may be generally applied to any transmit and receive and / or audio sensing application, in particular mobile or otherwise portable instances of such applications. For example, the scope of the configurations disclosed herein includes communication devices residing in a wireless telephony system configured to use a code division multiple access (CDMA) air interface. Nevertheless, a method and apparatus having the features as described herein provides for Voice ove IP (VoIP) (eg, CDMA, TDMA, FDMA, and / or TD-SCDMA) transmission channels over wired and / or wireless. It will be understood by those skilled in the art that they may reside in any of a variety of communication systems that use a wide variety of techniques known to those skilled in the art, such as systems that utilize the system.
The communication devices disclosed herein may be configured for use in networks that are packet switching (e.g., wired and / or wireless networks arranged to carry audio transmissions in accordance with protocols such as VoIP) and / or circuit switching. It is expressly contemplated and disclosed thereby. In addition, the communication devices disclosed herein are for use in narrowband coding systems (eg, systems that encode an audio frequency range of about 4 or 5 kilohertz) and / or full band wideband coding systems and split bands. It is expressly contemplated and disclosed herein that it may be configured for use in wide area coding systems (eg, systems that encode audio frequencies greater than 5 kilohertz), including wide area coding systems.
The presentation of the configurations described herein is provided to enable any person skilled in the art to make or use the methods and other structures disclosed herein. The flowcharts, block diagrams, and other structures shown and described herein are merely examples, and other variations of these structures are also within the scope of this disclosure. Various modifications to these configurations are possible, and the general principles provided herein may also be applied to other configurations. Accordingly, the present disclosure is not intended to be limited to the configurations shown above, but rather, the principles and novel techniques disclosed herein in any manner, included in the appended claims at the time of forming a part of the original disclosure. It is intended to give the widest range consistent with one feature.
Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, and symbols, which may be referenced throughout the above description, may include voltage, current, electromagnetic waves, magnetic or magnetic particles, optical fields or optical particles, or their It may be represented by a combination.
Important design requirements for the implementation of the configuration as disclosed herein include, in particular, the playback of compressed audio or audiovisual information (e.g., a file or stream encoded according to a compression format, such as one of the examples identified herein) and For computationally intensive applications or applications for wide area communication (eg, voice communication at sampling rates higher than 8 kilohertz, such as 12, 16, or 44 kHz), typically millions of instructions per second Or processing complexity) and / or computational complexity (as measured in MIPS).
The purpose of a multi-microphone processing system is to achieve 10 to 12 dB in total noise reduction, preserve speech levels and colors during the movement of the desired speaker, and recognize that noise has moved to the background instead of speech's active noise canceling reverberation. And / or enabling the option of post-processing for more aggressive noise reduction.
Various elements of the implementation of the ANC device disclosed herein may be implemented in any combination of hardware, software, and / or firmware that is deemed suitable for the intended application. For example, such elements may be manufactured, for example, as electronic and / or optical devices residing on the same chip or among two or more chips of a chipset. One example of such a device is a fixed or programmable array of logic elements such as transistors or logic gates, and any of these elements may be implemented as one or more such arrays. Any two or more, or even all of these arrays may be implemented within the same array or arrays. Such an array or arrays may be implemented in one or more chips (eg, in a chipset comprising two or more).
One or more elements of the various implementations of the ANC apparatus disclosed herein may also include microprocessors, embedded processors, IP cores, digital signal processors, FPGA (field programmable gate array), ASSP (application specific standard product), and It may be implemented in whole or in part as one or more sets of instructions arranged to execute one or more fixed or programmable arrays of logic elements, such as an application specific integrated circuit (ASIC). Any various elements of an implementation of an apparatus as disclosed herein may also be referred to as "processors," machines, including one or more computers (eg, one or more arrays programmed to execute one or more sets or sequences of instructions). Or any two or more or even all of these elements may be implemented within the same such computer or computers.
A processor or means for processing as disclosed herein may be manufactured, for example, as one or more electronic and / or optical devices residing among two or more chips on the same chip or in a chipset. One example of such a device is a fixed or programmable array of logic elements such as transistors or logic gates, and any of these elements may be implemented as one or more such arrays. Such an array or arrays may be implemented in one or more chips (eg, in a chipset comprising two or more chips). Examples of such arrays include fixed or programmable arrays of logic elements such as microprocessors, embedded processors, IP cores, DSPs, FPGAs, ASSPs and ASICs. The processor or means as disclosed herein may also be implemented as one or more computers (eg, machines comprising one or more arrays programmed to execute one or more sets or sequences of instructions) or other processors. Another set of instructions that a processor as described herein is not directly related to a coherency detection procedure, such as tasks relating to other operations of the device or system (e.g., audio sensing device) in which the processor is embedded. It is possible to be used to execute tasks or perform tasks. It is also possible that part of the method as disclosed herein is performed by a processor of the audio sensing device, and that another part of the method is performed under the control of one or more other processors.
Those skilled in the art will appreciate that various exemplary modules, logic blocks, circuits, and tests described in connection with the configurations disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. Such modules, logic blocks, circuits, and operations may be general purpose processors, digital signal processors (DSPs), ASICs or ASSPs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or disclosed herein. It may be implemented or performed in any combination thereof designed to generate a configuration as such. For example, such a configuration may be a hardware wired circuit, a circuit configuration made of an application specific integrated circuit, or a firmware program loaded in non-volatile storage or a software program loaded in or on a data storage medium as machine-readable code. At least partially implemented, such code is instructions executable by an array of logic elements, such as a general purpose processor or other digital signal processing unit. A general purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The software module includes random access memory (RAM), read-only memory (ROM), nonvolatile RAM (NVRAM) such as flash RAM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, It may reside in a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from and write information to the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
Note that the various methods disclosed herein may be performed by an array of logic elements such as a processor, and the various elements of the apparatus as described herein may be implemented as modules designed to execute such an array. As used herein, the term “module” or “sub-module” is any method, apparatus, device, unit, or apparatus that includes computer instructions (eg, logical representations) in the form of software, hardware or firmware. Computer-readable data storage media. It should be understood that multiple modules or systems can be combined into one module or system, and that one module or system can be separated into multiple modules or systems to perform the same functions. When implemented in software or other computer executable instructions, the elements of the process are essentially code segments for performing related tasks with, for example, routines, programs, objects, components, data structures, and the like. . The term "software" means source code, assembly language code, machine code, binary code, firmware, macrocode, microcode, any one or more sets or sequences of instructions executable by an array of logic elements, and such examples. Should be understood to include any combination. The program or code segments may be stored in a processor readable medium or transmitted by a computer data signal contained in a carrier via a transmission medium or communication link.
Implementations of the methods, methods, and techniques disclosed herein may also be readable by a machine that includes an array of logic elements (eg, a processor, microprocessor, microcontroller, or other finite state machine). It may also be included in the form as one or more sets of executable instructions (eg, in one or more computer readable media as listed herein). The term “computer readable medium” may include any medium capable of storing or transmitting information, including volatile, nonvolatile, removable and non-removable media. Examples of computer readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROM (EROM), floppy diskettes or other magnetic storage, CD-ROM / DVD or other optical storage, hard disks, optical fibers Media, radio frequency (RF) links, or any other media that can be used and accessed to store desired information. The computer data signal may include any signal capable of propagating through a transmission medium, such as electronic network channels, optical fibers, aerial, electromagnetic, RF links, and the like. Code segments may be downloaded via computer networks such as the Internet or intranets. In either case, the scope of the present disclosure should not be construed as limited by these embodiments.
Each of the tasks of the methods described herein may be implemented in hardware, in a software module executed by a processor, or in a combination of the two. In a typical application of the implementation of a method as disclosed herein, the array of logic elements (eg, logic gates) is configured to perform one, two or more, or even all of the various tasks of the method. One or more (and possibly all) of the tasks are also read by a machine (eg, a computer) that includes an array of logic elements (eg, a processor, microprocessor, microcontroller, or other finite state machine). Code (eg, one of instructions) included in a computer program product (eg, one or more data storage media such as disks, flash or other nonvolatile memory cards, semiconductor memory chips, etc.) that is possible and / or executable May be implemented as the above sets). Tasks of the implementation of the method as disclosed herein may also be performed by two or more such arrays or machines. In these or other implementations, the tasks may be performed within a device for wireless communication, such as a cellular telephone or other device having such communication capability. Such a device may be configured to communicate with circuit switching and / or packet switching networks (eg, using one or more protocols such as VoIP). For example, such a device may include RF circuitry configured to receive and / or transmit encoded frames.
It is evident that the various methods disclosed herein may be performed by a portable communication device, such as a handset, a headset, or a portable information terminal (PDA), and the apparatuses described herein may be included in such a device. A typical real time (eg, online) application is a telephone conversation conducted using such a mobile device.
In one or more illustrative embodiments, the operations described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these operations may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. The term “computer readable medium” includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. The storage medium may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer readable media may comprise semiconductor memory, ferroelectrics, magnetoresistances, obonics, polymers, or phases (which may include, without limitation, dynamic or static RAM, ROM, EEPROM, and / or flash RAM). Store desired program code in the form of instructions or data structures in a change memory, CD-ROM or other optical disk storage device, magnetic disk storage device or other magnetic storage device, or in structures of a type that can be accessed by a computer It may also include an array of storage elements, such as any other medium that can be used to. Also, any context is appropriately termed a computer readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a wireless technology such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared, radio and microwave, Wireless technologies such as cable, fiber optic cable, twisted pair, DSL, or infrared, radio and microwave are included in the definition of media. As used herein, discs and discs include compact discs (CDs), laser discs, optical discs, DVDs, floppy discs and Blu-ray Disc ™ (Blu-Ray Disc Association, Universal). City, CA), where disks generally reproduce data magnetically, while disks optically reproduce data with a laser. Combinations of the above should also be included within the scope of computer-readable media.
An acoustic signal processing apparatus as described herein may be integrated into an electronic device that accepts a speech input to control certain operations, or otherwise benefit from the separation of desired noises from background noises, such as communication devices. You can also get Multiple applications may benefit from enhancing or separating clean desired sound from background sounds resulting from multiple directions. Such applications may include human-machine interfaces in electronic or computing devices that include capabilities such as speech recognition and detection, speech enhancement, and separation, voice excitation control, and the like. It may be desirable to implement such an acoustic signal processing apparatus to be suitable for devices providing only limited processing capabilities.
The elements of the various implementations of the modules, elements, and devices described herein may be manufactured, for example, as electronic and / or optical devices residing on the same chip or among two or more chips in a chipset. One example of such a device is a fixed or programmable array of logic elements such as transistors or gates. One or more elements of the various implementations of the apparatus described herein may also be one or more fixed or programmatic of logic elements, such as microprocessors, embedded processors, IP cores, digital signal processors, FPGAs, ASSPs, and ASICs. It may be implemented in whole or in part as one or more sets of instructions arranged to execute on possible arrays.
For one or more elements of an implementation of an apparatus as described herein, another set of instructions or performing tasks that are not directly related to the operation of the apparatus, such as tasks relating to other operations of the device or system in which the apparatus is included. It is possible to be used to run them. Furthermore, for one or more elements of the implementation of such apparatuses, a processor used to execute a different number of portions of code corresponding to different elements, for example, a processor, different tasks corresponding to different elements. It is possible to have a set of instructions executed to perform a number of times, or an apparatus of electronic and / or optical devices that perform a different number of operations on different elements.
A method of processing a multichannel signal,
For each of a plurality of segments of the multichannel signal, and for each of a plurality of different frequency components of the multichannel signal, (A) the phase of the frequency component in the first channel of the multichannel signal during the segment and ( B) calculating a difference between the phases of the frequency components in the second channel of the multichannel signal during the segment to obtain a plurality of calculated phase differences for the segment;
Coherency indicating the degree of coherency among the arrival directions of at least the plurality of different frequency components during the segment, for each of the plurality of segments and based on a corresponding plurality of calculated phase differences. Calculating a corresponding value of the measurement;
For each of the plurality of segments, comparing the corresponding value of the coherency measure to a threshold;
For each of the plurality of segments, the amplitude of the frequency component in the second channel during the segment relative to the corresponding amplitude of the frequency component in the first channel during the segment, at least the plurality of different frequency components, respectively. Generating a corresponding segment of the processed multichannel signal by modifying for a signal, wherein the changing is based on a value of a gain factor corresponding to the segment. ; And
For at least one of the plurality of segments, and in response to a result of the comparison, updating a value of the gain factor, wherein updating comprises at least one of the plurality of calculated phase differences and the first channel. Updating the value of the gain factor based on the relationship between the calculated level and the calculated level of the second channel.
The calculated level of the first channel is a calculated level of one of the plurality of different frequency components in the first channel, and the calculated level of the second channel is the calculated level of the frequency component in the second channel. Calculated level,
The updating step is based on a relationship between the calculated level of the frequency component in the first channel and the calculated level of the frequency component in the second channel, for each of the plurality of different frequency components, A method of processing a multichannel signal.
Wherein said updating is based on a value of said gain factor corresponding to a previous segment.
And the method includes selecting the plurality of different frequency components based on an estimated pitch frequency of the multichannel signal.
The relationship between the calculated level of the first channel and the calculated level of the second channel is a ratio between the calculated level of the first channel and the calculated level of the second channel. How to process.
Generating a corresponding segment of the processed multichannel signal includes reducing an imbalance between the calculated levels of the first channel and the second channel.
And the calculated level of the first channel and the calculated level of the second channel are the same.
The method includes indicating the presence of voice activity based on a relationship between a level of a first channel of the processed multichannel signal and a level of a second channel of the processed multichannel signal. How to process.
Each of the first channel and the second channel is based on signals generated by corresponding microphones in an array,
The method comprising indicating that the multichannel signal is coherent directionally in an endfire direction of the array,
Indicating the voice activity is performed in response to the step of indicating that the multichannel signal is directionally coherent.
The method is based on a relationship between the level of the first channel and the level of the second channel in the segment of the processed multichannel signal, and comparing the corresponding value of the coherency measurement to a threshold value. In response to a result of the step, updating the noise estimate according to acoustic information from at least one of the first channel and the second channel of the multichannel signal.
11. A computer readable medium comprising tangible features that, when read by a processor, cause the processor to perform the method of any one of claims 1-10.
An apparatus for processing a multichannel signal,
(A) the difference between the phase of the frequency component in the first channel of the multichannel signal during the segment and (B) the phase of the frequency component in the second channel of the multichannel signal during the segment of the multichannel signal. A first calculator configured to obtain a plurality of calculated phase differences for the segment for each of the plurality of segments of the multichannel signal by calculating for each of a plurality of different frequency components;
For each of the plurality of segments, and based on a corresponding plurality of calculated phase differences, correspondence of a coherency measure indicative of the degree of coherence among at least directions of arrival of the plurality of different frequency components during the segment. A second calculator configured to calculate a value of
For each of the plurality of segments, a module configured to compare the corresponding value of the coherency measure to a threshold value;
For each of the plurality of segments, the amplitude of the frequency component in the second channel during the segment relative to the corresponding amplitude of the frequency component in the first channel during the segment, at least the plurality of different frequency components, respectively. A gain control element configured to generate a corresponding segment of the processed multichannel signal by modifying for the gain control element, the changing being based on a value of a gain factor corresponding to the segment; And
A third calculator configured to update a value of the gain factor for at least one of the plurality of segments and in response to a result of the comparison, wherein the updating comprises at least one of the plurality of calculated phase differences and the first And a third calculator based on the relationship between the calculated level of the channel and the calculated level of the second channel.
The updating is based on a relationship between the calculated level of the frequency component in the first channel and the calculated level of the frequency component in the second channel, for each of the plurality of different frequency components. A device for processing channel signals.
And said updating is based on a value of said gain factor corresponding to a previous segment.
And the first calculator is configured to select the plurality of different frequency components based on an estimated pitch frequency of the multichannel signal.
The relationship between the calculated level of the first channel and the calculated level of the second channel is a ratio between the calculated level of the first channel and the calculated level of the second channel. A device for processing.
The gain control element is configured to reduce an imbalance between the calculated levels of the first channel and the second channel.
The apparatus includes a voice activity detector configured to indicate the presence of voice activity based on a relationship between a level of a first channel of the processed multichannel signal and a level of a second channel of the processed multichannel signal. And an apparatus for processing a multichannel signal.
The gain factor calculator is configured to indicate whether the multichannel signal is coherent directionally in an endfire direction of the array,
And the voice activity detector is configured to indicate the presence of the voice activity in response to an indication by the gain factor calculator that the multichannel signal is directionally coherent.
The apparatus is further configured to compare the corresponding value of the coherency measurement to a threshold based on a relationship between the level of the first channel and the level of the second channel in the segment of the processed multichannel signal. In response to a result, a noise reference calculator configured to update a noise estimate according to acoustic information from at least one of the first channel and the second channel of the multichannel signal.
For each of a plurality of segments of the multichannel signal, and for each of a plurality of different frequency components of the multichannel signal, (A) the phase of the frequency component in the first channel of the multichannel signal during the segment and ( B) means for calculating a difference between the phases of the frequency components in the second channel of the multichannel signal during the segment to obtain a plurality of calculated phase differences for the segment;
For each of the plurality of segments, and based on a corresponding plurality of calculated phase differences, correspondence of a coherency measure indicative of the degree of coherence among at least directions of arrival of the plurality of different frequency components during the segment. Means for calculating a value to make;
For each of the plurality of segments, means for comparing the corresponding value of the coherency measure to a threshold value;
For each of the plurality of segments, the amplitude of the frequency component in the second channel during the segment relative to the corresponding amplitude of the frequency component in the first channel during the segment, at least the plurality of different frequency components, respectively. Means for generating a corresponding segment of the processed multichannel signal by modifying a variable, wherein the modifying is based on a value of a gain factor corresponding to the segment. ; And
Means for updating a value of the gain factor for at least one of the plurality of segments, and in response to a result of the comparison, wherein the updating comprises calculating at least one of the plurality of calculated phase differences and the first channel Means for updating a value of the gain factor based on a relationship between a calculated level and a calculated level of the second channel.
And the apparatus comprises means for selecting the plurality of different frequency components based on an estimated pitch frequency of the multichannel signal.
Means for generating a corresponding segment of the processed multichannel signal is configured to reduce an imbalance between the calculated levels of the first channel and the second channel.
The apparatus includes means for indicating the presence of voice activity based on a relationship between a level of a first channel of the processed multichannel signal and a level of a second channel of the processed multichannel signal. A device for processing.
The apparatus comprises means for indicating that the multichannel signal is coherently directional in an endfire direction of the array,
Means for indicating the presence of voice activity is configured to indicate the presence of the voice activity in response to an indication by the means indicating that the multichannel signal is directionally coherent. Device.
The apparatus is based on the relationship between the level of the first channel and the level of the second channel in the segment of the processed multichannel signal, and comparing the corresponding value of the coherency measurement to a threshold value. In response to a result, means for updating a noise estimate according to acoustic information from at least one of the first channel and the second channel of the multichannel signal.
KR1020127000692A 2009-06-09 2010-06-09 Systems, methods, apparatus, and computer-readable media for phase-based processing of multichannel signal KR101275442B1 (en)
US61/185,518 2009-06-09
US22703709P true 2009-07-20 2009-07-20
US61/227,037 2009-07-20
US24032009P true 2009-09-08 2009-09-08
US61/240,318 2009-09-08
US61/240,320 2009-09-08
US12/796,566 2010-06-08
US12/796,566 US8620672B2 (en) 2009-06-09 2010-06-08 Systems, methods, apparatus, and computer-readable media for phase-based processing of multichannel signal
KR20120027510A KR20120027510A (en) 2012-03-21
KR101275442B1 true KR101275442B1 (en) 2013-06-17
KR1020127000692A KR101275442B1 (en) 2009-06-09 2010-06-09 Systems, methods, apparatus, and computer-readable media for phase-based processing of multichannel signal
TW (1) TW201132138A (en)
WO2009012491A2 (en) * 2007-07-19 2009-01-22 Personics Holdings Inc. Device and method for remote acoustic porting and magnetic acoustic connection
CN107293311A (en) 2011-12-21 2017-10-24 华为技术有限公司 Very short pitch determination and coding
CN102404273B (en) * 2011-12-29 2015-04-15 电子科技大学 Method for transmitting OFDM signals based on new companding transform
CN103426441B (en) 2012-05-18 2016-03-02 华为技术有限公司 A method and apparatus for detecting the correctness of a pitch period
US10217467B2 (en) 2016-06-20 2019-02-26 Qualcomm Incorporated Encoding and decoding of interchannel phase differences between audio signals
EP0164097A2 (en) * 1984-06-06 1985-12-11 Officine Carrozzerie PATAVIUM A. Zanin S.p.A. Fluid-operated portable tool for end chamfering large diameter pipe sections particularly in pipeline applications
KR950035103A (en) * 1994-05-31 1995-12-30 김광호 A multi-channel audio masking processing unit
KR20080092404A (en) * 2006-01-05 2008-10-15 오디언스 인코포레이티드 System and method for utilizing inter-microphone level differences for speech enhancement
EP1198974B1 (en) * 1999-08-03 2003-06-04 Widex A/S Hearing aid with adaptive matching of microphones
KR100657912B1 (en) 2004-11-18 2006-12-14 삼성전자주식회사 Noise reduction method and apparatus
2010-06-08 US US12/796,566 patent/US8620672B2/en active Active
2010-06-09 CN CN201080025957.1A patent/CN102461203B/en active IP Right Grant
2010-06-09 KR KR1020127000692A patent/KR101275442B1/en active IP Right Grant
2010-06-09 TW TW099118854A patent/TW201132138A/en unknown
2010-06-09 EP EP10729976A patent/EP2441273A1/en active Pending
2010-06-09 WO PCT/US2010/037973 patent/WO2010144577A1/en active Application Filing
2010-06-09 JP JP2012515105A patent/JP5410603B2/en not_active Expired - Fee Related
JP5410603B2 (en) 2014-02-05
CN102461203A (en) 2012-05-16
TW201132138A (en) 2011-09-16
EP2441273A1 (en) 2012-04-18
KR20120027510A (en) 2012-03-21
JP2012529868A (en) 2012-11-22
WO2010144577A1 (en) 2010-12-16
CN102461203B (en) 2014-10-29
US20100323652A1 (en) 2010-12-23
US8620672B2 (en) 2013-12-31