Source: http://www.google.com/patents/US8170875?dq=7751826
Timestamp: 2016-02-07 23:09:46
Document Index: 341459534

Matched Legal Cases: ['application No. 2', 'Application No. 06721766', 'Application No. 2007', 'Application No. 2007', 'Application No. 200680000746', 'Application No. 2007', 'Application No. 10', 'Application No. 10']

Patent US8170875 - Speech end-pointer - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA rule-based end-pointer isolates spoken utterances contained within an audio stream from background noise and non-speech transients. The rule-based end-pointer includes a plurality of rules to determine the beginning and/or end of a spoken utterance based on various speech characteristics. The rules...http://www.google.com/patents/US8170875?utm_source=gb-gplus-sharePatent US8170875 - Speech end-pointerAdvanced Patent SearchPublication numberUS8170875 B2Publication typeGrantApplication numberUS 11/152,922Publication dateMay 1, 2012Priority dateJun 15, 2005Fee statusPaidAlso published asCA2575632A1, CA2575632C, CN101031958A, CN101031958B, EP1771840A1, EP1771840A4, US8165880, US8554564, US20060287859, US20070288238, US20120265530, WO2006133537A1Publication number11152922, 152922, US 8170875 B2, US 8170875B2, US-B2-8170875, US8170875 B2, US8170875B2InventorsPhil Hetherington, Alex EscottOriginal AssigneeQnx Software Systems LimitedExport CitationBiBTeX, EndNote, RefManPatent Citations (128), Non-Patent Citations (30), Referenced by (2), Classifications (9), Legal Events (8) External Links: USPTO, USPTO Assignment, EspacenetSpeech end-pointer
US 8170875 B2Abstract
A rule-based end-pointer isolates spoken utterances contained within an audio stream from background noise and non-speech transients. The rule-based end-pointer includes a plurality of rules to determine the beginning and/or end of a spoken utterance based on various speech characteristics. The rules may analyze an audio stream or a portion of an audio stream based upon an event, a combination of events, the duration of an event, or a duration relative to an event. The rules may be manually or dynamically customized depending upon factors that may include characteristics of the audio stream itself, an expected response contained within the audio stream, or environmental conditions.
ASR systems enable a user to speak into a microphone and have signals translated into a command that is recognized by a computer. Upon recognition of the command, the computer may implement an application. One factor in implementing an ASR system is correctly recognizing spoken utterances. This requires locating the beginning and/or the end of the utterances (“end-pointing”).
A rule-based end-pointer comprises one or more rules that determine a beginning, an end, or both a beginning and end of an audio speech segment in an audio stream. The rules may be based on various factors, such as the occurrence of an event or combination of events, or the duration of a presence/absence of a speech characteristic. Furthermore, the rules may comprise, analyzing a period of silence, a voiced audio event, a non-voiced audio event, or any combination of such events; the duration of an event; or a duration relative to an event. Depending upon the rule applied or the contents of the audio stream being analyzed, the amount of the audio stream the rule-based end-pointer sends to an ASR may vary.
A dynamic end-pointer may analyze one or more dynamic aspects related to the audio stream, and determine a beginning, an end, or both a beginning and end of an audio speech segment based on the analyzed dynamic aspect. The dynamic aspects that may be analyzed include, without limitation: (1) the audio stream itself, such as the speaker's pace of speech, the speaker's pitch, etc.; (2) an expected response in the audio stream, such as an expected response (e.g., “yes” or “no”) to a question posed to the speaker; or (3) the environmental conditions, such as the background noise level, echo, etc. Rules may utilize the one or more dynamic aspects in order to end-point the audio speech segment.
A rule-based end-pointer may examine one or more characteristics of the audio stream for a triggering characteristic. A triggering characteristic may include voiced or non-voiced sounds. Voiced speech segments (e.g. vowels), generated when the vocal cords vibrate, emit a nearly periodic time-domain signal. Non-voiced speech sounds, generated when the vocal cords do not vibrate (such as when speaking the letter “f” in English), lack periodicity and have a time-domain signal that resembles a noise-like structure. By identifying a triggering characteristic in an audio stream and employing a set of rules that operate on the natural characteristics of speech sounds, the end-pointer may improve the determination of the beginning and/or end of a speech utterance.
Alternatively, an end-pointer may analyze at least one dynamic aspect of an audio stream. Dynamic aspects of the audio stream that may be analyzed include, without limitation: (1) the audio stream itself, such as the speaker's pace of speech, the speaker's pitch, etc.; (2) an expected response in an audio stream, such as an expected response (e.g., “yes” or “no”) to a question posed to the speaker; or (3) the environmental conditions, such as the background noise level, echo, etc. The dynamic end-pointer may be rule-based. The dynamic nature of the end-pointer enables improved determination of the beginning and/or end of a speech segment.
FIG. 3 is a flowchart of a speech end-pointer system. The system may operate by dividing an input audio stream into discrete sections, such as frames, so that the input audio stream may be analyzed on a frame-by-frame basis. Each frame may comprise anywhere from about 10 ms to about 100 ms of the entire input audio stream. The system may buffer a predetermined amount of data, such as about 350 ms to about 500 ms of input audio data, before it begins processing the data. An energy detector, as shown at block 302, may be used to determine if energy, apart from noise, is present. The energy detector examines a portion of the audio stream, such as a frame, for the amount of energy present, and compares the amount to an estimate of the noise energy. The estimate of the noise energy may be constant or may be dynamically determined. The difference in decibels (dB), or ratio in power, may be the instantaneous signal to noise ratio (SNR). Prior to analysis, frames may be assumed to be non-speech so that, if the energy detector determines that energy exists in the frame, the frame is marked as non-speech, as shown at block 304. After energy is detected, voicing analysis of the current frame, designated as framen may occur, as shown at block 306. Voicing analysis may occur as described in U.S. Ser. No. 11/131,150, filed May 17, 2005, whose specification is incorporated herein by reference. The voicing analysis may check for any triggering characteristic that may be present in framen. The voicing analysis may check to see if an audio “S” or “X” is present in framen. Alternatively, the voicing analysis may check for the presence of a vowel. For purposes of explanation and not for limitation, the remainder of FIG. 3 is described as using a vowel as the triggering characteristic of the voicing analysis.
When the voicing analysis determines that a vowel is present in framen, framen is marked as speech, as shown at block 310. The system then may examine one or more previous frames. The system may examine the immediate preceding frame, framen−1, as shown at block 312. The system may determine whether the previous frame was previously marked as containing speech, as shown at block 314. If the previous frame was already marked as speech (i.e., answer of “Yes” to block 314), the system has already determined that speech is included in the frame, and moves to analyze a new audio frame, as shown at block 304. If the previous frame was not marked as speech (i.e., answer of “No” to block 314), the system may use one or more rules to determine whether the frame should be marked as speech.
As shown in FIG. 3, block 316, designated as decision block “Outside EndPoint” may use a routine that uses one or more rules to determine whether the frame should be marked as speech. One or more rules may be applied to any part of the audio stream, such as a frame or a group of frames. The rules may determine whether the current frame or frames under examination contain speech. The rules may indicate if speech is or is not present in a frame or group of frames. If speech is present, the frame may be designated as being inside the end-point.
If energy is present in the frame or a group of frames being analyzed, an “Energy” counter is incremented at block 404. “Energy” counter counts an amount of time. It is incremented by the frame length. If the frame size is about 32 ms, then block 404 increments the “Energy” counter by about 32 ms. At decision 406, a check is performed to see if the value of the “Energy” counter exceeds a time threshold. The threshold evaluated at decision block 406 corresponds to the continuous non-voiced energy rule which may be used to determine the presence and/or absence of speech. At decision block 406, the threshold for the maximum duration of continuous non-voiced energy may be evaluated. If decision 406 determines that the threshold setting is exceeded by the value of the “Energy” counter, then the frame or group of frames being analyzed are designated as being outside the end-point (e.g. no speech is present) at block 408. As a result, referring back to FIG. 3, the system jumps back to block 304 where a new frame, framen+1, is input into the system and marked as non-speech. Alternatively, multiple thresholds may be evaluated at block 406.
If no time threshold is exceeded by the value of the “Energy” counter at block 406, then a check is performed at decision block 410 to determine if the “noEnergy” counter exceeds an isolation threshold. Similar to the “Energy” counter 404, “noEnergy” counter 418 counts time and is incremented by the frame length when a frame or group of frames being analyzed does not possess energy above the noise level. The isolation threshold is a time threshold defining an amount of time between two plosive events. A plosive is a consonant that literally explodes from the speaker's mouth. Air is momentarily blocked to build up pressure to release the plosive. Plosives may include the sounds “P”, “T”, “B”, “D”, and “K”. This threshold may be in the range of about 10 ms to about 50 ms, and may be about 25 ms. If the isolation threshold is exceeded an isolated non-voiced energy event, a plosive surrounded by silence (e.g. the P in STOP) has been identified, and “isolatedEvents” counter 412 is incremented. The “isolatedEvents” counter 412 is incremented in integer values. After incrementing the “isolatedEvents” counter 412 “noEnergy” counter 418 is reset at block 414. This counter is reset because energy was found within the frame or group of frames being analyzed. If the “noEnergy” counter 418 does not exceed the isolation threshold, then “noEnergy” counter 418 is reset at block 414 without incrementing the “isolatedEvents” counter 412. Again, “noEnergy” counter 418 is reset because energy was found within the frame or group of frames being analyzed. After resetting “noEnergy” counter 418, the outside end-point analysis designates the frame or frames being analyzed as being inside the end-point (e.g. speech is present) by returning a “NO” value at block 416. As a result, referring back to FIG. 3, the system marks the analyzed frame as speech at 318 or 322.
Alternatively, if decision 402 determines there is no energy above the noise level then the frame or group of frames being analyzed contain silence or background noise. In this case, “noEnergy” counter 418 is incremented. At decision 420, a check is performed to see if the value of the “noEnergy” counter exceeds a time threshold. The threshold evaluated at decision block 420 corresponds to the continuous non-voiced energy rule threshold which may be used to determine the presence and/or absence of speech. At decision block 420, the threshold for a duration of continuous silence may be evaluated. If decision 420 determines that the threshold setting is exceeded by the value of the “noEnergy” counter, then the frame or group of frames being analyzed are designated as being outside the end-point (e.g. no speech is present) at block 408. As a result, referring back to FIG. 3, the system jumps back to block 304 where a new frame, framen+1, is input into the system and marked as non-speech. Alternatively, multiple thresholds may be evaluated at block 420.
If no time threshold is exceed by the value of the “noEnergy” counter 418, then a check is performed at decision block 422 to determine if the maximum number of allowed isolated events has occurred. An “isolatedEvents” counter provides the necessary information to answer this check. The maximum number of allowed isolated events is a configurable parameter. If a grammar is expected (e.g. a “Yes” or a “No” answer) the maximum number of allowed isolated events may be set accordingly so as to “tighten” the end-pointer's results. If the maximum number of allowed isolated events has been exceeded, then the frame or frames being analyzed are designated as being outside the end-point (e.g. no speech is present) at block 408. As a result, referring back to FIG. 3, the system jumps back to block 304 where a new frame, framen+1, is input into the system and marked as non-speech.
If the maximum number of allowed isolated events has not been reached, “Energy” counter 404 is reset at block 424. “Energy” counter 404 may be reset when a frame of no energy is identified. After resetting “Energy” counter 404, the outside end-point analysis designates the frame or frames being analyzed as being inside the end-point (e.g. speech is present) by returning a “NO” value at block 416. As a result, referring back to FIG. 3, the system marks the analyzed frame as speech at 318 or 322.
FIGS. 5-9 show some raw time series of a simulated audio stream, various characterization plots of these signals, and spectrographs of the corresponding raw signals. In FIG. 5, block 502, illustrates the raw time series of a simulated audio stream. The simulated audio stream comprises the spoken utterances “NO” 504, “YES” 506, “NO” 504, “YES” 506, “NO” 504, “YESSSSS” 508, “NO” 504, and a number of “clicking” sounds 510. These clicking sounds may represent the sound generated when a vehicle's turn signal is engaged. Block 512 illustrates various characterization plots for the raw time series audio stream. Block 512 displays the number of samples along the x-axis. Plot 514 is one representation of the end-pointer's analysis. When plot 514 is at a zero level, the end-pointer has not determined the presence of a spoken utterance. When plot 514 is at a non-zero level the end-pointer bounds the beginning and/or end of a spoken utterance. Plot 516 represents energy above the background energy level. Pilot 518 represents a spoken utterance in the time-domain. Block 520 illustrates a spectral representation of the corresponding audio stream identified in block 502.
Block 512 illustrates how the end-pointer may respond to an input audio stream. As shown in FIG. 5, end-pointer plot 514 correctly captures the “NO” 504 and the “YES” 506 signals. When the “YESSSSS” 508 is analyzed, the end-pointer plot 514 captures the trailing “S” for a while, but when it finds that the maximum time period after a vowel or the maximum duration of continuous non-voiced energy has been exceeded the end-pointer cuts off. The rule-based end-pointer sends the portion of the audio stream that is bound by end-pointer plot 514 to an ASR. As illustrated in block 512, and FIGS. 6-9, the portion of the audio stream sent to an ASR varies depending upon which rule is applied. The “clicks” 510 were detected as having energy. This is represented by the above background energy plot 516 at the right most portion of block 512. However, because no vowel was detected in the “clicks” 510, the end-pointer excludes these audio sounds.
FIG. 6 is a close up of one end-pointed “NO” 504. Spoken utterance plot 518 lags by a frame or two due to time smearing. Plot 518 continues throughout the period in which energy is detected, which is represented by above energy plot 516. After spoken utterance plot 518 rises, it levels off and follows above background energy plot 516. End-pointer plot 514 begins when the speech energy is detected. During the period represented by plot 518 none of the end-pointer rules are violated and the audio stream is recognized as a spoken utterance. The end-pointer cuts off at the right most side when either the maximum duration of continuous silence after a vowel rule or the maximum time after a vowel rule may have been violated. As illustrated, the portion of the audio stream that is sent to an ASR comprises approximately 3150 samples.
FIG. 7 is a close up of one end-pointed “YES” 506. Spoken utterance plot 518 again lags by a frame or two due to time smearing. End-pointer plot 514 begins when the energy is detected. End-pointer plot 514 continues until the energy falls off to noise; when the maximum duration of continuous non-voiced energy rule or the maximum time after a vowel rule may have been violated. As illustrated, the portion of the audio stream that is sent to an ASR comprises approximately 5550 samples. The difference between the amounts of the audio stream sent to an ASR in FIG. 6 and FIG. 7 results from the end-pointer applying different rules.
FIG. 8 is a close up of one end-pointed “YESSSSS” 508. The end-pointer accepts the post-vowel energy as a possible consonant, but only for a reasonable amount of time. After a reasonable time period, the maximum duration of continuous non-voiced energy rule or the maximum time after a vowel rule may have been violated and the end-pointer falls off limiting the data passed to an ASR. As illustrated, the portion of the audio stream that is sent to an ASR comprises approximately 5750 samples. Although the spoken utterance continues on for an additional approximately 6500 samples, because the end-pointer cuts off the after a reasonable amount of time the amount of the audio stream sent to an ASR differs from that sent in FIG. 6 and FIG. 7.
FIG. 9 is a close up of an end-pointed “NO” 504 followed by several “clicks” 510. As with FIGS. 6-8, spoken utterance plot 518 lags by a frame or two because of time smearing. End-pointer plot 514 begins when the energy is detected. The first click is included within end-point plot 514 because there is energy above the background noise energy level and this energy could be a consonant, i.e. a trailing “T”. However, there is about 300 ms of silence between the first click and the next click. This period of silence, according the threshold values used for this example, violates the end-pointer's maximum duration of continuous silence after a vowel rule. Therefore, the end-pointer excluded the energies after the first click.
The end-pointer may also be configured to determine the beginning and/or end of an audio speech segment by analyzing at least one dynamic aspect of an audio stream. FIG. 10 is a partial flowchart of an end-pointer system that analyzes at least one dynamic aspect of an audio stream. An initialization of global aspects may be performed at 1002. Global aspects may include characteristics of the audio stream itself. For purposes of explanation and not for limitation, these global aspects may include a speaker's pace of speech or a speaker's pitch. At 1004, an initialization of local aspects may be performed. For purposes of explanation and not for limitation, these local aspects may include an expected speaker response (e.g. a “YES” or a “NO” answer), environmental conditions (e.g. an open or closed environment, effecting the presence of echo or feedback in the system), or estimation of the background noise.
The operation of a dynamic end-pointer may be similar to the end-pointer described with reference to FIGS. 3 and 4, except that one or more thresholds of the one or more rules of the “Outside Endpoint” routine, block 316, may be dynamically configured. If there is a large amount of background noise, the threshold for the energy above noise decision, block 402, may be dynamically raised to account for this condition. Upon performing this re-configuration, the dynamic end-pointer may reject more transient and non-speech sounds thereby reducing the number of false positives. Dynamically configurable thresholds are not limited to the background noise level. Any threshold utilized by the dynamic end-pointer may be dynamically configured.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS55201 *May 29, 1866SanfoedImprovement in machinery for printing railroad-ticketsUS4435617 *Aug 13, 1981Mar 6, 1984Griggs David TSpeech-controlled phonetic typewriter or display device using two-tier approachUS4486900Mar 30, 1982Dec 4, 1984At&T Bell LaboratoriesReal time pitch detection by stream processingUS4531228Sep 29, 1982Jul 23, 1985Nissan Motor Company, LimitedSpeech recognition system for an automotive vehicleUS4532648 *Sep 29, 1982Jul 30, 1985Nissan Motor Company, LimitedSpeech recognition system for an automotive vehicleUS4630305Jul 1, 1985Dec 16, 1986Motorola, Inc.Automatic gain selector for a noise suppression systemUS4701955 *Oct 21, 1983Oct 20, 1987Nec CorporationVariable frame length vocoderUS4811404Oct 1, 1987Mar 7, 1989Motorola, Inc.Noise suppression systemUS4843562Jun 24, 1987Jun 27, 1989Broadcast Data Systems Limited PartnershipBroadcast information classification system and methodUS4856067Aug 6, 1987Aug 8, 1989Oki Electric Industry Co., Ltd.Speech recognition system wherein the consonantal characteristics of input utterances are extractedUS4945566Nov 18, 1988Jul 31, 1990U.S. Philips CorporationMethod of and apparatus for determining start-point and end-point of isolated utterances in a speech signalUS4989248Mar 3, 1989Jan 29, 1991Texas Instruments IncorporatedSpeaker-dependent connected speech word recognition methodUS5027410Nov 10, 1988Jun 25, 1991Wisconsin Alumni Research FoundationAdaptive, programmable signal processing and filtering for hearing aidsUS5056150Nov 8, 1989Oct 8, 1991Institute Of Acoustics, Academia SinicaMethod and apparatus for real time speech recognition with and without speaker dependencyUS5146539Nov 8, 1988Sep 8, 1992Texas Instruments IncorporatedMethod for utilizing formant frequencies in speech recognitionUS5151940 *Dec 7, 1990Sep 29, 1992Fujitsu LimitedMethod and apparatus for extracting isolated speech wordUS5152007Apr 23, 1991Sep 29, 1992Motorola, Inc.Method and apparatus for detecting speechUS5201028 *Sep 21, 1990Apr 6, 1993Theis Peter FSystem for distinguishing or counting spoken itemized expressionsUS5293452Jul 1, 1991Mar 8, 1994Texas Instruments IncorporatedVoice log-in using spoken name inputUS5305422 *Feb 28, 1992Apr 19, 1994Panasonic Technologies, Inc.Method for determining boundaries of isolated words within a speech signalUS5313555Feb 7, 1992May 17, 1994Sharp Kabushiki KaishaLombard voice recognition method and apparatus for recognizing voices in noisy circumstanceUS5400409Mar 11, 1994Mar 21, 1995Daimler-Benz AgNoise-reduction method for noise-affected voice channelsUS5408583Jul 14, 1992Apr 18, 1995Casio Computer Co., Ltd.Sound outputting devices using digital displacement data for a PWM sound signalUS5479517Dec 23, 1993Dec 26, 1995Daimler-Benz AgMethod of estimating delay in noise-affected voice channelsUS5495415Nov 18, 1993Feb 27, 1996Regents Of The University Of MichiganMethod and system for detecting a misfire of a reciprocating internal combustion engineUS5502688Nov 23, 1994Mar 26, 1996At&T Corp.Feedforward neural network system for the detection and characterization of sonar signals with characteristic spectrogram texturesUS5526466Apr 11, 1994Jun 11, 1996Matsushita Electric Industrial Co., Ltd.Speech recognition apparatusUS5568559Dec 13, 1994Oct 22, 1996Canon Kabushiki KaishaSound processing apparatusUS5572623Oct 21, 1993Nov 5, 1996Sextant AvioniqueMethod of speech detectionUS5584295Sep 1, 1995Dec 17, 1996Analogic CorporationSystem for measuring the period of a quasi-periodic signalUS5596680 *Dec 31, 1992Jan 21, 1997Apple Computer, Inc.Method and apparatus for detecting speech activity using cepstrum vectorsUS5617508Aug 12, 1993Apr 1, 1997Panasonic Technologies Inc.Speech detection device for the detection of speech end points based on variance of frequency band limited energyUS5677987Jul 18, 1994Oct 14, 1997Matsushita Electric Industrial Co., Ltd.Feedback detector and suppressorUS5680508May 12, 1993Oct 21, 1997Itt CorporationEnhancement of speech coding in background noise for low-rate speech coderUS5687288 *Sep 14, 1995Nov 11, 1997U.S. Philips CorporationSystem with speaking-rate-adaptive transition values for determining words from a speech signalUS5692104Sep 27, 1994Nov 25, 1997Apple Computer, Inc.Method and apparatus for detecting end points of speech activityUS5701344Aug 5, 1996Dec 23, 1997Canon Kabushiki KaishaAudio processing apparatusUS5732392 *Sep 24, 1996Mar 24, 1998Nippon Telegraph And Telephone CorporationMethod for speech detection in a high-noise environmentUS5794195May 12, 1997Aug 11, 1998Alcatel N.V.Start/end point detection for word recognitionUS5933801Nov 27, 1995Aug 3, 1999Fink; Flemming K.Method for transforming a speech signal using a pitch manipulatorUS5949888Sep 15, 1995Sep 7, 1999Hughes Electronics CorporatonComfort noise generator for echo cancelersUS5963901Dec 10, 1996Oct 5, 1999Nokia Mobile Phones Ltd.Method and device for voice activity detection and a communication deviceUS6011853Aug 30, 1996Jan 4, 2000Nokia Mobile Phones, Ltd.Equalization of speech signal in mobile phoneUS6029130 *Aug 20, 1997Feb 22, 2000Ricoh Company, Ltd.Integrated endpoint detection for improved speech recognition method and systemUS6098040Nov 7, 1997Aug 1, 2000Nortel Networks CorporationMethod and apparatus for providing an improved feature set in speech recognition by performing noise cancellation and background maskingUS6163608Jan 9, 1998Dec 19, 2000Ericsson Inc.Methods and apparatus for providing comfort noise in communications systemsUS6167375Mar 16, 1998Dec 26, 2000Kabushiki Kaisha ToshibaMethod for encoding and decoding a speech signal including background noiseUS6173074Sep 30, 1997Jan 9, 2001Lucent Technologies, Inc.Acoustic signature recognition and identificationUS6175602May 27, 1998Jan 16, 2001Telefonaktiebolaget Lm Ericsson (Publ)Signal noise reduction by spectral subtraction using linear convolution and casual filteringUS6192134Nov 20, 1997Feb 20, 2001Conexant Systems, Inc.System and method for a monolithic directional microphone arrayUS6199035May 6, 1998Mar 6, 2001Nokia Mobile Phones LimitedPitch-lag estimation in speech codingUS6216103 *Oct 20, 1997Apr 10, 2001Sony CorporationMethod for implementing a speech recognition system to determine speech endpoints during conditions with background noiseUS6240381 *Feb 17, 1998May 29, 2001Fonix CorporationApparatus and methods for detecting onset of a signalUS6304844 *Mar 30, 2000Oct 16, 2001Verbaltek, Inc.Spelling speech recognition apparatus and method for communicationsUS6317711 *Feb 14, 2000Nov 13, 2001Ricoh Company, Ltd.Speech segment detection and word recognitionUS6324509 *Feb 8, 1999Nov 27, 2001Qualcomm IncorporatedMethod and apparatus for accurate endpointing of speech in the presence of noiseUS6356868 *Oct 25, 1999Mar 12, 2002Comverse Network Systems, Inc.Voiceprint identification systemUS6405168Sep 30, 1999Jun 11, 2002Conexant Systems, Inc.Speaker dependent speech recognition training using simplified hidden markov modeling and robust end-point detectionUS6434246Oct 2, 1998Aug 13, 2002Gn Resound AsApparatus and methods for combining audio compression and feedback cancellation in a hearing aidUS6453285Aug 10, 1999Sep 17, 2002Polycom, Inc.Speech activity detector for use in noise reduction system, and methods thereforUS6453291 *Apr 16, 1999Sep 17, 2002Motorola, Inc.Apparatus and method for voice activity detection in a communication systemUS6487532Sep 24, 1998Nov 26, 2002Scansoft, Inc.Apparatus and method for distinguishing similar-sounding utterances speech recognitionUS6507814Sep 18, 1998Jan 14, 2003Conexant Systems, Inc.Pitch determination using speech classification and prior pitch estimationUS6535851Mar 24, 2000Mar 18, 2003Speechworks, International, Inc.Segmentation approach for speech recognition systemsUS6574592 *Mar 20, 2000Jun 3, 2003Kabushiki Kaisha ToshibaVoice detecting and voice control systemUS6574601 *Jan 13, 1999Jun 3, 2003Lucent Technologies Inc.Acoustic speech recognizer system and methodUS6587816Jul 14, 2000Jul 1, 2003International Business Machines CorporationFast frequency-domain pitch estimationUS6643619Oct 22, 1998Nov 4, 2003Klaus LinhardMethod for reducing interference in acoustic signals using an adaptive filtering method involving spectral subtractionUS6687669Jul 2, 1997Feb 3, 2004Schroegmeier PeterMethod of reducing voice signal interferenceUS6711540Sep 25, 1998Mar 23, 2004Legerity, Inc.Tone detector with noise detection and dynamic thresholding for robust performanceUS6721706 *Oct 30, 2000Apr 13, 2004Koninklijke Philips Electronics N.V.Environment-responsive user interface/entertainment device that simulates personal interactionUS6782363May 4, 2001Aug 24, 2004Lucent Technologies Inc.Method and apparatus for performing real-time endpoint detection in automatic speech recognitionUS6822507Jan 2, 2003Nov 23, 2004William N. BucheleAdaptive speech filterUS6850882Oct 23, 2000Feb 1, 2005Martin RothenbergSystem for measuring velar function during speechUS6859420Jun 13, 2002Feb 22, 2005Bbnt Solutions LlcSystems and methods for adaptive wind noise rejectionUS6873953 *May 22, 2000Mar 29, 2005Nuance CommunicationsProsody based endpoint detectionUS6910011Aug 16, 1999Jun 21, 2005Haman Becker Automotive Systems - Wavemakers, Inc.Noisy acoustic signal enhancementUS6996252 *Apr 5, 2004Feb 7, 2006Digimarc CorporationLow visibility watermark using time decay fluorescenceUS7117149Aug 30, 1999Oct 3, 2006Harman Becker Automotive Systems-Wavemakers, Inc.Sound source classificationUS7146319Mar 31, 2003Dec 5, 2006Novauris Technologies Ltd.Phonetically based speech recognition system and methodUS7535859Oct 8, 2004May 19, 2009Nxp B.V.Voice activity detection with adaptive noise floor trackingUS20010028713Apr 4, 2001Oct 11, 2001Michael WalkerTime-domain noise suppressionUS20020071573Feb 21, 2001Jun 13, 2002Finn Brian M.DVE system with customized equalizationUS20020176589Apr 12, 2002Nov 28, 2002Daimlerchrysler AgNoise reduction method with self-controlling interference frequencyUS20030040908Feb 12, 2002Feb 27, 2003Fortemedia, Inc.Noise suppression for speech signal in an automobileUS20030120487Dec 20, 2001Jun 26, 2003Hitachi, Ltd.Dynamic adjustment of noise separation in data handling, particularly voice activationUS20030216907May 14, 2002Nov 20, 2003Acoustic Technologies, Inc.Enhancing the aural perception of speechUS20040078200Oct 17, 2002Apr 22, 2004Clarity, LlcNoise reduction in subbanded speech signalsUS20040138882Oct 31, 2003Jul 15, 2004Seiko Epson CorporationAcoustic model creating method, speech recognition apparatus, and vehicle having the speech recognition apparatusUS20040165736Apr 10, 2003Aug 26, 2004Phil HetheringtonMethod and apparatus for suppressing wind noiseUS20040167777Oct 16, 2003Aug 26, 2004Hetherington Phillip A.System for suppressing wind noiseUS20050096900Oct 31, 2003May 5, 2005Bossemeyer Robert W.Locating and confirming glottal events within human speech signalsUS20050114128Dec 8, 2004May 26, 2005Harman Becker Automotive Systems-Wavemakers, Inc.System for suppressing rain noiseUS20050240401Apr 23, 2004Oct 27, 2005Acoustic Technologies, Inc.Noise suppression based on Bark band weiner filtering and modified doblinger noise estimateUS20060034447Aug 10, 2004Feb 16, 2006Clarity Technologies, Inc.Method and system for clear signal captureUS20060053003 *Jun 3, 2004Mar 9, 2006Tetsu SuzukiAcoustic interval detection method and deviceUS20060074646Sep 28, 2004Apr 6, 2006Clarity Technologies, Inc.Method of cascading noise reduction algorithms to avoid speech distortionUS20060080096Sep 29, 2005Apr 13, 2006Trevor ThomasSignal end-pointing method and systemUS20060100868Oct 17, 2005May 11, 2006Hetherington Phillip AMinimization of transient noises in a voice signalUS20060115095Dec 1, 2004Jun 1, 2006Harman Becker Automotive Systems - Wavemakers, Inc.Reverberation estimation and suppression systemUS20060116873Jan 13, 2006Jun 1, 2006Harman Becker Automotive Systems - Wavemakers, IncRepetitive transient noise removalUS20060136199Dec 23, 2005Jun 22, 2006Haman Becker Automotive Systems - Wavemakers, Inc.Advanced periodic signal enhancementUS20060178881Jan 27, 2006Aug 10, 2006Samsung Electronics Co., Ltd.Method and apparatus for detecting voice regionUS20060251268May 9, 2005Nov 9, 2006Harman Becker Automotive Systems-Wavemakers, Inc.System for suppressing passing tire hissUS20070033031Sep 29, 2006Feb 8, 2007Pierre ZakarauskasAcoustic signal classification systemUS20070219797Mar 16, 2006Sep 20, 2007Microsoft CorporationSubword unit posterior probability for measuring confidenceUS20070288238May 18, 2007Dec 13, 2007Hetherington Phillip ASpeech end-pointerCA2157496A1Mar 31, 1994Oct 13, 1994British TelecommConnected Speech RecognitionCA2158064A1Mar 31, 1994Oct 13, 1994British TelecommSpeech ProcessingCA2158847A1Mar 25, 1994Sep 29, 1994British TelecommA Method and Apparatus for Speaker RecognitionCN1042790ANov 16, 1988Jun 6, 1990中国科学院声学研究所Real-time phonetic recognition method and device with or without function of identfying a personEP0076687A1Oct 4, 1982Apr 13, 1983Signatron, Inc.Speech intelligibility enhancement system and methodEP0543329B1Nov 17, 1992Feb 6, 2002Kabushiki Kaisha ToshibaSpeech dialogue system for facilitating human-computer interactionEP0629996A2Jun 3, 1994Dec 21, 1994Ontario HydroAutomated intelligent monitoring systemEP0750291A1May 29, 1987Dec 27, 1996BRITISH TELECOMMUNICATIONS public limited companySpeech processorEP1450353A1Feb 18, 2004Aug 25, 2004Harman Becker Automotive Systems-Wavemakers, Inc.System for suppressing wind noiseEP1450354A1Feb 19, 2004Aug 25, 2004Harman Becker Automotive Systems-Wavemakers, Inc.System for suppressing wind noiseEP1669983A1Dec 8, 2005Jun 14, 2006Harman Becker Automotive Systems-Wavemakers, Inc.System for suppressing rain noiseJP06269084A2 Title not availableJP06319193A Title not availableJP2000250565A Title not availableJPH06269084A Title not availableJPH06319193A Title not availableKR19990077910A Title not availableKR20010091093A Title not availableWO2000041169A1Jan 7, 2000Jul 13, 2000Ravi ChandranMethod and apparatus for adaptively suppressing noiseWO2001056255A1Dec 15, 2000Aug 2, 2001Acoustic Tech IncMethod and apparatus for removing audio artifactsWO2001073761A1Mar 2, 2001Oct 4, 2001Ravi ChandranRelative noise ratio weighting techniques for adaptive noise cancellation* Cited by examinerNon-Patent CitationsReference1Avendano, C., Hermansky, H., "Study on the Dereverberation of Speech Based on Temporal Envelope Filtering," Proc. ICSLP '96, pp. 889-892, Oct. 1996.2Berk et al., "Data Analysis with Microsoft Excel", Duxbury Press, 1998, pp. 236-239 and 256-259.3Canadian Examination Report of related application No. 2,575, 632, Issued May 28, 2010.4European Search Report dated Aug. 31, 2007 from corresponding European Application No. 06721766.1, 13 pages.5Fiori, S., Uncini, A., and Piazza, F., "Blind Deconvolution by Modified Bussgang Algorithm", Dept. of Electronics and Automatics-University of Ancona (Italy), ISCAS 1999.6Fiori, S., Uncini, A., and Piazza, F., "Blind Deconvolution by Modified Bussgang Algorithm", Dept. of Electronics and Automatics—University of Ancona (Italy), ISCAS 1999.7International Preliminary Report on Patentability dated Jan. 3, 2008 from corresponding PCT Application No. PCT/CA2006/000512, 10 pages.8International Search Report and Written Opinion dated Jun. 6, 2006 from corresponding PCT Application No. PCT/CA2006/000512, 16 pages.9Learned, R.E. et al., A Wavelet Packet Approach to Transient Signal Classification, Applied and Computational Harmonic Analysis, Jul. 1995, pp. 265-278, vol. 2, No. 3, USA, XP 000972660. ISSN: 1063-5203. abstract.10Nakatani, T., Miyoshi, M., and Kinoshita, K., "Implementation and Effects of Single Channel Dereverberation Based on the Harmonic Structure of Speech," Proc. of IWAENC-2003, pp. 91-94, Sep. 2003.11Office Action dated Aug. 17, 2010 from corresponding Japanese Application No. 2007-524151, 3 pages.12Office Action dated Jan. 7, 2010 from corresponding Japanese Application No. 2007-524151, 7 pages.13Office Action dated Jun. 12, 2010 from corresponding Chinese Application No. 200680000746.6, 11 pages.14Office Action dated Jun. 6, 2011 for corresponding Japanese Patent Application No. 2007-524151, 9 pages.15Office Action dated Mar. 27, 2008 from corresponding Korean Application No. 10-2007-7002573, 11 pages.16Office Action dated Mar. 31, 2009 from corresponding Korean Application No. 10-2007-7002573, 2 pages.17Puder, H. et al., "Improved Noise Reduction for Hands-Free Car Phones Utilizing Information on a Vehicle and Engine Speeds", Sep. 4-8, 2000, pp. 1851-1854, vol. 3, XP009030255, 2000. Tampere, Finland, Tampere Univ. Technology, Finland Abstract.18Quatieri, T.F. et al., Noise Reduction Using a Soft-Dection/Decision Sine-Wave Vector Quantizer, International Conference on Acoustics, Speech & Signal Processing, Apr. 3, 1990, pp. 821-824, vol. Conf. 15, IEEE ICASSP, New York, US XP000146895, Abstract, Paragraph 3.1.19Quelavoine, R. et al., Transients Recognition in Underwater Acoustic with Multilayer Neural Networks, Engineering Benefits from Neural Networks, Proceedings of the International Conference EANN 1998, Gibraltar, Jun. 10-12, 1998 pp. 330-333, XP 000974500. 1998, Turku, Finland, Syst. Eng. Assoc., Finland. ISBN: 951-97868-0-5. abstract, p. 30 paragraph 1.20Savoji, M. H. "A Robust Algorithm for Accurate Endpointing of Speech Signals" Speech Communication, Elsevier Science Publishers, Amsterdam, NL, vol. 8, No. 1, Mar. 1, 1989 (pp. 45-60).21Seely, S., "An Introduction to Engineering Systems", Pergamon Press Inc., 1972, pp. 7-10.22Shust, Michael R. and Rogers, James C., "Electronic Removal of Outdoor Microphone Wind Noise", obtained from the Internet on Oct. 5, 2006 at: , 6 pages.23Shust, Michael R. and Rogers, James C., "Electronic Removal of Outdoor Microphone Wind Noise", obtained from the Internet on Oct. 5, 2006 at: <http://www.acoustics.org/press/136th/mshust.htm>, 6 pages.24Shust, Michael R. and Rogers, James C., Abstract of "Active Removal of Wind Noise From Outdoor Microphones Using Local Velocity Measurements", J. Acoust. Soc. Am., vol. 104, No. 3, Pt 2, 1998, 1 page.25Simon, G., Detection of Harmonic Burst Signals, International Journal Circuit Theory and Applications, Jul. 1985, vol. 13, No. 3, pp. 195-201, UK, XP 000974305. ISSN: 0098-9886. abstract.26 *Turner, John M. And Dickinson, Bradley W. , "A Variable Frame Length Linear Predicitive Coder", "Acoustics, Speech, and Signal Processing, IEEE International Conference on ICASSP '78." , vol. 3, pp. 454-457.27Vieira, J., "Automatic Estimation of Reverberation Time", Audio Engineering Society, Convention Paper 6107, 116th Convention, May 8-11, 2004, Berlin, Germany, pp. 1-7.28Wahab A. et al., "Intelligent Dashboard With Speech Enhancement", Information, Communications, and Signal Processing, 1997. ICICS, Proceedings of 1997 International Conference on Singapore, Sep. 9-12, 1997, New York, NY, USA, IEEE, pp. 993-997.29 *Ying et al. "Endpoint Detection of Isolated Utterances Based on a Modified Teager Energy Estimate". In Proc. IEEE ICASSP, vol. 2 pp. 732-735, 1993.30Zakarauskas, P., Detection and Localization of Nondeterministic Transients in Time series and Application to Ice-Cracking Sound, Digital Signal Processing, 1993, vol. 3, No. 1, pp. 36-45, Academic Press, Orlando, FL, USA, XP 000361270, ISSN: 1051-2004. entire document.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS8775191Nov 13, 2013Jul 8, 2014Google Inc.Efficient utterance-specific endpointer triggering for always-on hotwordingUS8843369Dec 27, 2013Sep 23, 2014Google Inc.Speech endpointing based on voice profileClassifications U.S. Classification704/253, 704/210, 704/233, 704/215International ClassificationG10L15/20, G10L15/04, G10L11/06Cooperative ClassificationG10L25/87European ClassificationG10L25/87Legal EventsDateCodeEventDescriptionJun 15, 2005ASAssignmentOwner name: HARMAN BECKER AUTOMOTIVE SYSTEMS - WAVEMAKERS, INCFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HETHERINGTON, PHIL;ESCOTT, ALEX;REEL/FRAME:016702/0510Effective date: 20050615Nov 14, 2006ASAssignmentOwner name: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.,CANADAFree format text: CHANGE OF NAME;ASSIGNOR:HARMAN BECKER AUTOMOTIVE SYSTEMS - WAVEMAKERS, INC.;REEL/FRAME:018515/0376Effective date: 20061101May 8, 2009ASAssignmentOwner name: JPMORGAN CHASE BANK, N.A.,NEW YORKFree format text: SECURITY AGREEMENT;ASSIGNORS:HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED;BECKER SERVICE-UND VERWALTUNG GMBH;CROWN AUDIO, INC.;AND OTHERS;REEL/FRAME:022659/0743Effective date: 20090331Jun 3, 2010ASAssignmentOwner name: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED,CONNFree format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045Effective date: 20100601Owner name: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.,CANADAFree format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045Effective date: 20100601Owner name: QNX SOFTWARE SYSTEMS GMBH & CO. KG,GERMANYFree format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045Effective date: 20100601Owner name: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, CONFree format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045Effective date: 20100601Owner name: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC., CANADAFree format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045Effective date: 20100601Owner name: QNX SOFTWARE SYSTEMS GMBH & CO. KG, GERMANYFree format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045Effective date: 20100601Jul 9, 2010ASAssignmentOwner name: QNX SOFTWARE SYSTEMS CO., CANADAFree format text: CONFIRMATORY ASSIGNMENT;ASSIGNOR:QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.;REEL/FRAME:024659/0370Effective date: 20100527Feb 27, 2012ASAssignmentOwner name: QNX SOFTWARE SYSTEMS LIMITED, CANADAFree format text: CHANGE OF NAME;ASSIGNOR:QNX SOFTWARE SYSTEMS CO.;REEL/FRAME:027768/0863Effective date: 20120217Apr 4, 2014ASAssignmentOwner name: 8758271 CANADA INC., ONTARIOFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:QNX SOFTWARE SYSTEMS LIMITED;REEL/FRAME:032607/0943Effective date: 20140403Owner name: 2236008 ONTARIO INC., ONTARIOFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:8758271 CANADA INC.;REEL/FRAME:032607/0674Effective date: 20140403Nov 2, 2015FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services