Source: http://www.google.com/patents/US8015000?dq=5,884,272
Timestamp: 2015-04-21 02:35:14
Document Index: 104536530

Matched Legal Cases: ['Application No. 60', 'art 200', 'art 300', 'art 300', 'art 400', 'art 500', 'art 500', 'art 1600', 'art 1800', 'art 2000', 'art 2200', 'art 2200', 'art 2200']

Patent US8015000 - Classification-based frame loss concealment for audio signals - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsAn audio decoding system performs frame loss concealment (FLC) when portions of a bit stream representing an audio signal are lost within the context of a digital communication system. The audio decoding system employs two different FLC methods: one designed to perform well for music, and the other designed...http://www.google.com/patents/US8015000?utm_source=gb-gplus-sharePatent US8015000 - Classification-based frame loss concealment for audio signalsAdvanced Patent SearchPublication numberUS8015000 B2Publication typeGrantApplication numberUS 11/734,800Publication dateSep 6, 2011Filing dateApr 13, 2007Priority dateAug 3, 2006Also published asUS20080033718Publication number11734800, 734800, US 8015000 B2, US 8015000B2, US-B2-8015000, US8015000 B2, US8015000B2InventorsRobert W. Zopf, Juin-Hwey Chen, Jes ThyssenOriginal AssigneeBroadcom CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (25), Non-Patent Citations (4), Referenced by (4), Classifications (27), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetClassification-based frame loss concealment for audio signals
US 8015000 B2Abstract
An audio decoding system performs frame loss concealment (FLC) when portions of a bit stream representing an audio signal are lost within the context of a digital communication system. The audio decoding system employs two different FLC methods: one designed to perform well for music, and the other designed to perform well for speech. When a frame is deemed lost, the audio decoding system analyzes a previously-decoded audio signal corresponding to previously-decoded frames of an audio bit-stream. Based on the results of the analysis, the lost frame is classified as either speech or music. Using this classification, other signal analysis, and knowledge of the employed FLC methods, the audio decoding system selects the appropriate FLC method which then performs FLC on the lost frame.
1. A method for performing frame loss concealment (FLC) in an audio decoder, comprising:
performing a first analysis on a previously-decoded portion of an audio signal, wherein performing the first analysis includes generating a feature set, wherein the feature set includes at least a short-term speech likelihood measure and a long-term speech likelihood measure;
classifying a lost frame as either speech or music based on the results of the first analysis;
performing a second analysis on the previously-decoded portion of the audio signal, wherein performing the second analysis comprises using at least the short-term speech likelihood measure and the long-term speech likelihood measure; and
selecting either a first FLC technique or a second FLC technique for replacing the lost frame based on the classification and the results of the second analysis.
2. The method of claim 1, wherein selecting a first FLC technique comprises selecting an FLC technique designed for speech.
3. The method of claim 2, wherein selecting an FLC technique designed for speech comprises selecting a periodic waveform extrapolation (PWE) based FLC technique.
4. The method of claim 1, wherein selecting a second FLC technique comprises selecting an FLC technique designed for music.
5. The method of claim 4, wherein selecting an FLC technique designed for music comprises selecting a frame repeat FLC technique.
6. The method of claim 1, further comprising modifying at least one of the first or second FLC techniques based on the results of the second analysis.
7. The method of claim 6, wherein modifying at least one of the first or second FLC techniques comprises:
using a pitch multiple in a periodic waveform extrapolation (PWE) based FLC technique.
8. The method of claim 6, wherein modifying at least one of the first or second FLC techniques comprises:
mixing a shaped noise signal with the output of a frame repeat FLC technique.
9. A system for performing frame loss concealment (FLC) in an audio decoder, comprising:
a signal classifier, executed by a processor, configured to perform a first analysis on a previously-decoded portion of an audio signal and to classify a lost frame as either speech or music based on the results of the first analysis, wherein the first analysis generates a feature set, wherein the feature set includes at least a short-term speech likelihood measure and a long-term speech likelihood measure; and
decision logic coupled to the signal classifier, the decision logic configured to perform a second analysis on the previously-decoded portion of the audio signal and to select either a first FLC technique or a second FLC technique for replacing the lost frame based on the classification and the results of the second analysis, wherein the second analysis uses at least the short-term speech likelihood measure and the long-term speech likelihood measure.
10. The system of claim 9, wherein the first FLC technique is an FLC technique designed for speech.
11. The system of claim 10, wherein the FLC technique designed for speech is a periodic waveform extrapolation (PWE) based FLC technique.
12. The system of claim 9, wherein the second FLC technique is an FLC technique designed for music.
13. The system of claim 12, wherein the FLC technique designed for music is a frame repeat FLC technique.
14. The system of claim 9, wherein the decision logic is further configured to modify at least one of the first or second FLC techniques based on the results of the second analysis.
15. The system of claim 14, wherein the decision logic is configured to modify at least one of the first or second FLC techniques by using a pitch multiple in a periodic waveform extrapolation (PWE) based FLC technique.
16. The system of claim 14, wherein the decision logic is configured to modify at least one of the first or second FLC techniques by mixing a shaped noise signal with the output of a frame repeat FLC technique.
This application claims priority to provisional U.S. Patent Application No. 60/835,106, filed Aug. 3, 2006, the entirety of which is incorporated by reference herein.
In audio coding (sometimes called �audio compression�), a coder encodes an input audio signal into a compressed digital bit stream for transmission or storage, and a decoder decodes the transmitted or stored bit stream into an output audio signal. The combination of the coder and the decoder is called a codec. The compressed bit stream is usually partitioned into frames. When the decoder decodes the bit stream, certain frames of the compressed bit stream may be deemed �lost� and thus not available for the normal decoding operation. This frame loss may be due to late or dropped packets in a packet transmission system or to severely corrupted frames in a wireless transmission system. Frame loss may even occur in audio storage applications for a variety of reasons.
One of the simplest and most common FLC techniques consists of repeating the bit stream of the last good frame preceding the lost frame, and decoding the repeated bit stream normally as if it were the received bit stream for the lost frame. This scheme is commonly called the �Frame Repeat� method. If the audio codec performs instantaneous quantization (such as Pulse Code Modulation (PCM)) without any overlap-add operation, the application of such a frame repeat method will generally cause waveform discontinuities at the frame boundaries. These waveform discontinuities will give rise to undesired audible artifacts that may be perceived as �clicks� by the listener.
On the other hand, modern audio codecs typically perform frequency-domain transforms, such as Fast Fourier Transform (FFT) or Modified Discrete Cosine Transform (MDCT), and such transforms are typically performed on a windowed version of the input signal, wherein adjacent windows are to some extent overlapping. The corresponding audio decoders typically synthesize the output audio signals by using an overlap-add technique that is well-known in the art. When used with such modern audio codecs, the frame repeat FLC method generally will not cause waveform discontinuities at the frame boundaries, because the overlap-add operation gradually transitions between one piece of waveform and the next overlapping piece of waveform, thus smoothing out waveform discontinuities at the frame boundaries.
Even though the frame repeat method will not cause waveform discontinuities if it is used with audio codecs that employ overlap-add synthesis at the decoder, it can still result in audible distortion for certain types of audio signals, especially those signals that are nearly periodic, such as the vowels portions of speech signals (voiced speech). This is understandable since the waveform repeated at the frame rate is generally not aligned or �in phase� with the original input waveform in the lost frame. When the frame repeat method overlaps such two �out-of-phase� waveforms and adds them together, the resulting output signal usually includes an audible disturbance that will make the output signal sound a little �busy� and not as �clean� as the original signal. Therefore, the frame repeat method generally performs poorly for nearly periodic signals such as voiced speech.
What is surprising is that when used with audio codecs employing overlap-add synthesis at the decoder (which include most of the modern audio codec standards), the frame repeat FLC method has been found to work surprisingly well for a large variety of audio signals that are �busy-sounding� and far from periodic. This is because for such busy-sounding audio signals, there is not a well-defined �phase�, and the disturbance resulting from out-of-phase overlap-add is not nearly as pronounced as in the case of nearly periodic signals. In other words, any residual disturbance in the output audio signal is likely hidden by the busy sounds in the audio signal. For such audio signals, it is actually quite difficult to perceive the distortion caused by the frame repeat FLC method.
This class of PWE-based FLC methods is usually tuned for speech signals, and thus these methods usually work quite well for speech. However, when applied to general audio signals such as music, these methods do not perform as well and tend to generate more audible distortion. One of the most common problems is that for busy-sounding music signals, the use of periodic waveform extrapolation often generates a �buzzing� sound. This is due to the fact that the periodically-extrapolated waveform is more periodic than the original waveform corresponding to the lost frames.
To summarize, when used with audio codecs employing overlap-add synthesis in the decoder, the frame repeat FLC method works well for most music signals but performs poorly for speech signals. On the other hand, PWE-based FLC methods work well for speech signals but often produce an audible �buzzing� for busy, non-periodic music signals. However, many audio signals, such as those associated with movie soundtracks, television, and radio programs, frequently change between pure speech, pure music, and a combination of speech and music. Consequently, using either a frame repeat or a PWE-based FLC method will result in performance problems for at least some portion(s) of the audio signal.
What is needed therefore is an FLC technique that works well for both speech and music. Ideally, the desired FLC method should be �universal� in that it works well for any kind of audio signal, but at the very least, the desired FLC method should work well for both speech and music, since speech and music are the dominant types of audio signals in soundtracks for movie, television, and radio. The present invention addresses this problem and can achieve good performance for both speech and music signals.
It is noted that the classification-based frame loss concealment system of the present invention is an improvement over the classification-based frame loss concealment system described in co-owned, commonly pending U.S. patent application Ser. No. 11/285,311 to Chen, filed Nov. 23, 2005, and entitled �Classification-Based Frame Loss Concealment for Audio Signals,� the entirety of which is incorporated by reference herein.
In the most general form of the present invention, an audio decoding system employs at least two different frame loss concealment (FLC) methods, wherein one method is designed to perform well for music and the other is designed to perform well for speech. When a frame is deemed lost, the audio decoding system analyzes an audio signal corresponding to previously-decoded frames of an audio bit-stream. Based on the results of the analysis, the lost frame is classified as either speech or music. Using this classification, other signal analysis, and knowledge of the employed FLC methods, the audio decoding system selects the appropriate FLC method which then performs FLC on the lost frame.
In accordance with one implementation of the present invention, the speech-based FLC method is a modified version of that described in U.S. patent application Ser. No. 11/234,291 to Juin-Hwey Chen, filed Sep. 26, 2005, and entitled �Packet Loss Concealment for Block-Independent Speech Codecs� (the entirety of which is incorporated by reference herein) and the music-based FLC method is an advanced frame repeat scheme.
The present invention is appropriate for audio systems that employ overlap-add synthesis at the decoder as well as those that do not. A system in accordance with an embodiment of the present invention makes use of any overlap-add synthesis employed at the decoder to improve analysis and concealment. If unavailable, the system generates a ringing signal to maintain smooth transitions from received frames to lost frames.
A. Improved Classification-Based FLC System and Method in Accordance with an Embodiment of the Present Invention
In general, audio decoding system 100 operates to decode each of a series of frames of an input audio bit-stream into corresponding frames of an output audio signal. System 100 decodes the input audio bit-stream one frame at a time. As used herein, the term �current frame� refers to a frame of the input audio bit-stream that system 100 is currently decoding, whereas �previous frame� refers to a frame of the input audio bit-stream that system 100 has already decoded. As also used herein, the term �decoding� may include both normal decoding of a received frame of the input audio bit-stream into corresponding output audio signal samples as well as generating output audio signal samples for a lost frame of the input audio bit-stream using an FLC technique. The function of each of the components of system 100 will now be described in more detail.
If a current frame of the input audio bit-stream is deemed received, audio decoder 110 decodes the current frame using any of a variety of known audio decoding techniques to generate output audio signal samples. Output signal selection switch 180 is controlled by a lost frame indicator, which indicates whether the current frame of the input audio bit-stream is deemed received or is lost. If the current frame is deemed received, switch 180 is placed in the upper position shown in FIG. 1 (connected to the node labeled �Frame Received�) and the decoded audio signal at the output of audio decoder 110 is used as the output audio signal for the current frame. Additionally, if the current frame is deemed received, the decoded audio signal for the current frame is also stored in decoded signal buffer 120 in preparation for possible FLC operations for future frames.
In contrast, if the current frame of the input audio bit-stream is deemed lost, then output signal selection switch 180 is placed in the lower position shown in FIG. 1 (connected to the node labeled �Frame Lost�). In this case, signal classifier 130 and FLC decision/control logic 140 operate together to select one of two possible FLC methods to perform the necessary FLC operations.
As shown in FIG. 1, there are two possible FLC methods that audio decoding system 100 can use. These two possible FLC methods are implemented in first and second processing blocks 161 and 162, respectively, in FIG. 1. In one embodiment of the invention, processing block 161 (labeled �First FLC Method�) is designed or tuned to perform FLC for an audio signal that has been classified as speech, while processing block 162 (labeled �Second FLC Method�) is designed or tuned to perform FLC for an audio signal that has been classified as music.
FLC decision/control logic 140 selects the FLC method for the current frame based on a classification output from signal classifier 130 and other decision logic. FLC decision/control logic selects the FLC method by generating a signal (labeled �FLC Method Decision� in FIG. 1) that controls the operation of first and second FLC method selection switches 150 and 170 to apply either the FLC method of processing block 161 or the FLC method of processing block 162. In the particular example shown in FIG. 1, switches 150 and 170 are in the uppermost position so that the FLC method of processing block 161 is selected. Of course, this is just an example. For a different frame that is lost, FLC decision/control logic 140 may select the FLC method of processing block 162.
If signal classifier 130 classifies the input signal as speech, FLC decision/control logic 140 performs further logic and analysis to determine which FLC technique to use. In one example implementation, signal classifier passes FLC decision/control logic 140 a feature set used in performing speech classification. FLC decision/control logic 140 then uses this information along with the knowledge of the FLC algorithms to determine which FLC method would perform best for the current frame.
Once a particular FLC method is selected, this FLC method uses the previously-decoded audio signal, or some portion thereof, stored in decoded signal buffer 120 and performs the associated FLC operations. The resulting output signal is then routed through switches 170 and 180 and becomes the output audio signal for the audio decoding system 100. Note that although it is not depicted in FIG. 1 for the sake of simplicity, it is understood and generally advisable that the FLC audio signal picked up by switch 170 is also passed back to decoded signal buffer 120 so that the audio signal produced by the selected FLC method for the current lost frame is also stored as the newest portion of the �previously-decoded audio signal.� This is done to prepare decoded signal buffer 120 for the next frame in case the next frame is also lost. In other words, it is generally advantageous for decoded signal buffer 120 to store the audio signal corresponding to the last frame immediately processed before a lost frame, whether or not the audio signal was produced by audio decoder 110 or one of FLC processing blocks 161 or 162.
As shown in FIG. 2, the beginning of flowchart 200 is indicated at step 202 labeled �start�. Processing immediately proceeds to step 204, in which a decision is made as to whether the next frame of the input audio bit-stream to be received by audio decoder 110 is received or lost. If the frame is deemed received, then audio decoder 110 performs normal decoding operations on the received frame to generate corresponding decoded audio signal samples, as shown at step 206. Processing then proceeds to step 208 in which the decoded audio signal corresponding to the received frame is stored in decoded signal buffer 120.
At step 210, a determination is made whether or not this is the first good frame after erasure or loss. If it is, then a portion of the frame and an extrapolated signal provided by one of FLC processing blocks 161 or 162 are overlap-added, as shown in step 212. In an embodiment, a �ramp up� operation is also performed for the first good frame. The overlap-add and ramp up operations will be described in more detail below in reference to the operation of processing blocks 161 and 162.
The decoded audio signal is then provided as the output audio signal of audio decoding system 100, as shown at step 214. With reference to FIG. 1, this is achieved through the operation of output signal selection switch 180 (under the control of the lost frame indicator) to couple the output of audio decoder 110 to the ultimate output of system 100. Processing then proceeds to step 216, where it is determined whether or not there are more frames in the input audio bit-stream to be processed by audio decoding system 100. If there are more frames, then processing returns to decision step 204; otherwise, processing ends as shown at step 236 labeled �end�.
With reference to FIG. 1, the selection of the FLC method by FLC decision/control logic 140 is performed via the generation of the signal labeled �FLC Method Decision�, which controls FLC method selection switches 150 and 170 to select one of the processing blocks 161 or 162.
Modifications can also be performed on the FLC method designed for music. For example, if signal classifier 130 classifies the input signal as speech, but FLC decision/control logic 140 selects the FLC method designed for music, the FLC method designed for music may be modified to be more appropriate for speech. For example, the signal can be analyzed for the degree of mix between periodic and noise-like components in a manner similar to that described in U.S. patent application Ser. No. 11/234,291 to Chen (explaining the calculation of a �voicing measure�), the entirety of which has been incorporated by reference herein. The output of the FLC method designed for music can then be mixed with a speech-like derived (LPC analysis) noise signal.
After either the FLC method designed for speech has been applied at step 226 or the FLC method designed for music has been applied at step 230, the audio signal generated by application of the selected FLC method is then provided as the output audio signal of audio decoding system 100, as shown at step 232. In the implementation shown in FIG. 1, this is achieved through the operation of output signal selection switch 180 (under the control of the lost frame indicator) to couple the output at switch 170 to the ultimate output of system 100. The audio signal generated by application of the selected FLC method is also stored in decoded signal buffer 120 as shown in step 234. Processing then proceeds to step 216, where it is determined whether or not there are more frames in the input audio bit-stream to be processed by audio decoding system 100. If there are more frames, then processing returns to decision step 204; otherwise, processing ends at step 236 labeled �end�.
As shown in FIG. 3, the beginning of flowchart 300 is indicated by step 302 labeled �start�. Processing immediately proceeds to step 304, in which a dynamic threshold for SLM is determined based on LTSLM. In one implementation, this step is carried out by setting the dynamic threshold to −4 if LTSLM is greater than 2.18, and otherwise setting the dynamic threshold to (1.8/LTSLM)3 if LTSLM is less than or equal to 2.18. This has the effect of eliminating the dynamic threshold for signals that exhibit a strong long-term tendency for speech, while setting the dynamic threshold to a value that is inversely proportional to LTSLM for signals that do not. As will be made evident below, the higher the dynamic threshold is set, the less likely it is that the method of flowchart 300 will select the FLC method designed for speech.
After the FLC method designed for speech has been selected at step 310, additional tests are performed to see if the pitch period should be doubled prior to application of the FLC method. First, a series of tests are applied to determine if the speech classification is a borderline one as shown at step 312. This series of tests may include determining if SLM is less than a certain threshold and/or determining if LTSLM is less than a certain threshold. For example, in one implementation, these additional tests include determining if SLM is less than 1.4 and if LTSLM is less than 2.4. If either of these conditions is evaluated as true, then a borderline classification is indicated and processing proceeds via decision step 314 to decision step 316. Otherwise, the pitch period is not doubled and processing ends at step 328 labeled �end.�
As shown in FIG. 4, the beginning of flowchart 400 is indicated by step 402 labeled �start�. Processing immediately proceeds to step 404, in which a dynamic scaling factor is determined based on LTSLM. In one implementation, the dynamic scaling factor is set to a value that is inversely proportional to LTSLM. For example, in one implementation, the dynamic scaling factor is set to 1.8/LTSLM. As will be made evident below, the higher the scaling factor, the less likely that the FLC method designed for speech will be selected.
At step 404, a series of tests are performed to detect speech in music and thereby determine if the FLC method designed for speech should be applied. These tests may include determining if SLM exceeds a certain threshold, if the sum total of one or more SLM values associated with prior frames exceeds certain thresholds, or a combination of both. If the results of these tests indicate speech in music, then processing proceeds via decision step 408 to step 410, wherein the FLC method designed for speech is selected. Processing then ends as shown at step 422 denoted �end�.
After the FLC method designed for speech has been selected at step 416, the pitch period is set to the largest multiple of the pitch period that will fit within frame size. This is done because there is a weak indication of speech in the recent past but a long-term indication of music. Consequently, the FLC method designed for speech is used but with a larger pitch multiple, thereby making it act more like an FLC method designed for music (e.g., a frame repeat FLC method). After this, processing ends at step 422 labeled �end�.
For both FLC methods described in this section, a �ringing� signal, r, is obtained to maintain continuity between the previously-decoded frame and the lost frame. For the case where there is no audio overlap-add synthesis at the decoder (AOLA=0), this ringing signal is calculated as the zero-input response of a synthesis filter associated with the audio decoder 110. As discussed in U.S. patent application Ser. No. 11/234,291 to Chen, filed Sep. 26, 2005, and entitled �Packet Loss Concealment for Block-Independent Speech Codecs� (the entirety of which is incorporated by reference herein), an effective approach is to use the ringing of the cascaded long-term and short-term synthesis filters of the decoder.
In accordance with an embodiment of the present invention, the FLC method designed for music is an improved frame repeat method. As discussed in U.S. patent application Ser. No. 11/285,311 to Chen, filed Nov. 23, 2005, and entitled �Classification-Based Frame Loss Concealment for Audio Signals�, a frame repeat method combined with the overlapping windows of typical audio coders produces surprisingly sufficient quality for most music.
FIG. 5 is a flowchart 500 illustrating an improved frame repeat method in accordance with an embodiment of the present invention. As shown in FIG. 5, the beginning of flowchart 500 is indicated by a step 502 labeled �start�. Processing immediately proceeds to step 504, in which it is determined whether the current frame is the first bad (i.e., erased) frame since a good (i.e., non-erased) frame was received. If so, step 506 is performed. In step 506, the last good frame played out, denoted Lgf, is overlap-added with the ringing signal, r, to form the �correlated� repeat component frcor:
if ( AOLA > 0 ) fr cor ( n ) = Lfg ( n ) � wc i n ( n ) + r ( n ) � wc out ( n ) n = 0. . AOLA - 1 fr cor ( n ) = Lgf ( n ) n = AOLA .. FS - 1 else fr cor ( n ) = Lfg ( n ) � wc i n ( n ) + r ( n ) � wc out ( n ) n = 0. . ROLA - 1 fr cor ( n ) = Lgf ( n ) n = ROLA .. FS - 1 where wcin is a correlated fade-in window, wcout is a correlated fade-out window, AOLA is the length in samples of the overlap-add window, ROLA is the length in samples of the ringing signal for overlap-add, and FS is the number of samples in a frame (i.e., the frame size).
w ci(n)+wc out(n)=1.
At step 510, an appropriate mixture of the repeated signal frcor and the filtered noise signal nlpc is determined. Many different methods can be used to perform this step. In one implementation, a �voicing measure� or figure of merit (fom) such as that described in U.S. patent application Ser. No. 11/234,291 to Chen is used to compute a scale factor, β, that ranges from 0 to 1. The scale is overwritten to 0 if the current classification from signal classifier 130 is MUSIC.
sq ( N + n ) = fr cor ( n ) � ( 1 - β ) + n = 0. . AOLA - 1 ( A out ( n ) � wu out ( n ) + n lpc ( n ) � wu in ( n ) ) � β sq ( N + n ) = fr cor ( n ) � ( 1 - β ) + n lpc ( n ) � β n = AOLA .. FS - 1 where sq is the output signal buffer, N is the position of the first sample of the current frame in the output signal buffer, frcor is the correlated repeat component, β is the scale factor described in the preceding paragraph, nlpc is the filtered noise signal, Aout is the audio fade-out signal, wuout is the uncorrelated fade-out window, wuin is the uncorrelated fade-in window, AOLA is the overlap add window length, and FS is the frame size. Where there is no overlap-add synthesis at the decoder, AOLA=0, and the foregoing simply becomes:
At step 514, denoted �update speech-FLC�, any frame-to-frame memory is updated in order to maintain continuity (signal buffer, decimation filters, LPC filters, pitch buffers, etc.).
If the frame erasure lasts for an extended period of time, the output of the FLC scheme is preferably ramped down to zero in a gradual manner in order to avoid buzzy sounds or other artifacts. At step 516, a measure of the time in frame erasure is compared to a predetermined threshold, and if it exceeds the threshold, step 518 is performed which attenuates the signal in the output signal buffer denoted sq(N . . . FS−1). A linear ramp starting at 43 ms and ending at 63 ms is preferably used. Finally, at step 520, the samples in sq(N . . . FS−1) are released to a playback buffer. After this, processing ends as indicated by step 522 labeled �end�.
sq ( N + n ) = ( fr cor ( n ) � wc out ( n ) + sq ( N + n ) � wc in ( n ) ) � n = 0. . OLAG - 1 ( 1 - β ) + ( n lpc ( n + FS ) � wu out ( n ) + sq ( N + n ) � wu in ( n ) ) � β where sq is the output signal buffer, N is the position of the first sample of the current frame in the output signal buffer, frcor is the correlated repeat component, β is the scale factor, nlpc is the filtered noise signal, wcout is the correlated fade-out window, wcin is the correlated fade-in window, wuout is the uncorrelated fade-out window, wuin is the uncorrelated fade-in window, OLAG is the overlap-add window length, and FS is the frame size. It should be noted that sq(N+n) likely has a portion or all of Wcin already applied if the frame is from an audio decoder. Typically, the audio encoder applies √{square root over (wcin(n))} and the decoder does the same. It should be understood that whatever portion of the window has been applied is not reapplied.
As described above in reference to step 212 of FIG. 2, a �ramp up� operation is performed on the first good frame after erasure for both FLC methods. In particular, in order to avoid an abrupt energy change from FLC frames to the first good frame, the output signal in the first good frame is ramped up from a scale factor associated with a last sample in the previously-described gain attenuation step, to 1, over a period of
In an embodiment of the present invention, the FLC method applied by processing block 161 is a modified version of that described in U.S. patent application Ser. No. 11/234,291 to Chen, which is incorporated by reference herein. A flowchart of the modified approach is collectively depicted in FIGS. 6 and 7 of the present application. Because the flowchart is large, it has been divided into two portions, one depicted in FIG. 6 and one depicted in FIG. 7, with a node �A� as the connecting point between the two portions.
The method begins at step 602, which is located in the upper left corner of FIG. 6 and is labeled �start�. Processing then immediately proceeds to decision step 604, in which it is determined whether the current frame is erased. If the current frame is not erased, then processing proceeds to decision step 606, in which it is determined whether the current frame is the first good frame after an erasure. If the current frame is not the first good frame after an erasure, then the decoded speech samples in the current frame are copied to a corresponding location in the output buffer as shown at step 608.
sq ( N + n ) = ( 1 - β ) � ( sq ( N + n ) � wc in ( n ) + sq ( N + FS + n ) � wc out ( n ) ) + β � ( sq ( N + n ) � wu in ( n ) + n lpc ( FS + n ) � wu out ( n ) ) where sq is the output signal buffer, N is the position of the first sample of the current frame in the output signal buffer, β is a scale factor that will be described in more detail herein, wcout is the correlated fade-out window, wcin is the correlated fade-in window, wuout is the uncorrelated fade-out window, wuin is the uncorrelated fade-in window, OLAG is the overlap-add window length for the first good frame, and FS is the frame size.
After step 610, control flows to step 612 in which a �ramp up� operation is performed on the current frame. In particular, in order to avoid an abrupt energy change from FLC frames to the first good frame, the output signal in the first good frame is ramped up from a scale factor associated with a last sample in a gain attenuation step (described herein in reference to step 648 of FIG. 6) to 1, over a period of
After step 608 or 612 is completed, processing proceeds to step 614, which updates the coefficients of a short-term predictor by performing a so-called �LPC analysis�, a technique that is well-known by persons skilled in art. One method of performing this step is described in more detail in U.S. patent application Ser. No. 11/234,291. After step 614 is completed, control flows to node 650, labeled �A�. This node is identical to node 702 in FIG. 7.
At decision step 624, it is determined whether a voicing measure (the calculation of which is described below in reference to step 718 of FIG. 7) has a value greater than a first threshold value T1. If the answer is �No�, the waveform in the last frame is considered not periodic enough to warrant doing any periodic waveform extrapolation. As a result, steps 626, 628 and 630 are bypassed and control flows directly to decision step 632. On the other hand, if the answer is �Yes�, the waveform in the last frame is considered to have at least some degree of periodicity. Consequently, control flows to decision step 626.
Δ0=min(127, ┌pp*0.2┐)
sq ( N + n ) = sq ( N + n - ppmr ) � wc i n ( n ) + ring ( n ) � wc out ( n ) n = 0. . ROLA - 1 sq ( N + n ) = sq ( N + n - ppmr ) n = ROLA .. FS + OLAG where sq is the output signal buffer, N is the position of the first sample of the current frame in the output signal buffer, ppmr is the refined pitch, wcin is the correlated fade-in window, wcout is the correlated fade-out window, ring is the ringing signal, ROLA is the length in samples of the ringing signal for overlap-add, OLAG is the overlap-add length for the first good frame, and FS is the frame size. Note that Aout likely has a portion or all of wcout already applied. Typically, the audio encoder applies √{square root over (wcout(n))} and the decoder does the same. It should be understood that whatever portion of the window has been applied is not reapplied.
After decision step 624 or step 630 is complete, processing then proceeds to decision step 632, in which it is determined whether the voicing measure (the calculation of which is described below in reference to step 718 of FIG. 7) is less than a second threshold T2. If the answer is �No�, the waveform in the last frame is considered highly periodic and there is no need to mix in any random, noisy component in the output audio signal; hence, control flows directly to decision step 640 as shown in FIG. 6.
If, on the other hand, the answer to decision 632 is �Yes�, then control flows to step 634. At step 634, a sequence of pseudo-random white noise is generated. Following step 634, the sequence of pseudo-random white noise is passed through a short-term synthesis filter to generate a filtered noise signal, as shown at step 636. The manner in which steps 634 and 636 are performed is described in detail in U.S. patent application Ser. No. 11/234,291 to Chen, except that in the present embodiment, scaling is applied to the noise signal after it has been passed through the short-term synthesis filter rather than before, and the scaling factor is based on the average magnitude of the speech signal associated with the last frame rather than on the average magnitude of the LPC prediction residual signal of the last frame.
sq ( N + n ) = ( 1 - β ) � ( sq ( N + n ) � wc i n ( n ) + A out ( n ) � wc out ( n ) ) + β � ( n lpc ( n ) wu i n ( n ) + A out ( n ) � wu out ( n ) ) n = 0. . AOLA - 1 sq ( N + n ) = ( 1 - β ) � ( sq ( N + n ) ) + β � n lpc ( n ) n = AOLA .. FS - 1 where sq is the output signal buffer, N is the position of the first sample of the current frame in the output signal buffer, β is the scale factor, nlpc is the noise signal, Aout is the audio fade-out signal, wcout is the correlated fade-out window, wcin is the correlated fade-in window, wuout is the uncorrelated fade-out window, wuin is the uncorrelated fade-in window, AOLA is the overlap-add window length, and FS is the frame size. Note that if β=0, then only the extrapolated signal and the audio fade-out signal are combined and if β=1, then only the LPC generated noise and the audio fade-out signal are combined.
sq ( N + n ) = ( 1 - β ) � ( sq ( N + n ) ) + β � n lpc ( n ) n = 0. . FS - 1. In this instance, even though there is no audio fade-out signal for overlapping, a smooth signal transition will still occur at the frame boundary because the ringing signal was overlap-added with the extrapolated signal contained in the output signal buffer during step 630.
After step 642 or step 644 completes, processing proceeds to step 646, which determines whether the current erasure is too long�that is, whether the current frame is too �deep� into erasure. If the length of the current erasure has not exceeded a predetermined threshold, then control flows to node 650 (labeled �A�) in FIG. 6, which is the same as node 702 in FIG. 7. However, if the length of the current erasure has exceeded this threshold, then step 648 is performed. Step 648 attenuates the signal in the output signal buffer denoted sq(N . . . FS−1) in a manner similar to that described in U.S. patent application Ser. No. 11/234,291 to Chen. This is done to avoid buzzy artifacts. A linear ramp starting at 43 ms and ending at 63 ms is preferably used.
After step 708, processing proceeds to decision step 710, in which it is determined whether the current frame is erased. If the answer is �Yes�, then steps 712, 714, 716 and 718 are skipped, and control flows directly to step 720. If the answer is �No�, then the current frame is a good frame, and steps 712, 714, 716 and 718 are performed.
After step 726, control flows to step 728, which is labeled �end�. Node 728 denotes the end of the frame processing loop. Then, the control flow goes back to node 602 labeled �start� to start the frame processing for the next frame.
StMaxEst=StMaxEst�StMaxBeta+lg�(1−StMaxBeta)
LtMaxEst=LtMaxEst�LtMaxBeta+lg�(1−LtMaxBeta)
if (StMaxEst > LtMaxEst) LtMaxEst = LtMaxEst � LtMaxAlpha + StMaxEst � (1 − LtMaxAlpha) where LtMaxAlpha is set between 0 and 1 (e.g., tuned to 0.5 in one embodiment). Thus, as described above, if StMaxEst is greater than LtMaxEst, LtMaxEst is adjusted with the sum of a long term running average component (LtMaxEst�LtMaxAlpha) and a component based on StMaxEst (StMaxEst�(1−LtMaxAlpha)). If the frame energy is less than the short term maximum estimate StMaxEst, the more likely the long term maximum estimate LtMaxEst is lagging, so LtMaxBeta may be decreased in order to increase a change in long term maximum estimate LtMaxEst when there is an update:
if ( lg ≤ StMaxEst ) LtMaxBeta = LtMaxBeta � LtMaxBetaDecay where LtMaxBetaDecay = 0.9998 � FS 344 � 16 SF and FS is the frame size, and SF is the sampling frequency in kHz.
if ( StMaxEst > LtMinEst ) StMaxEst = StMaxEst - ( StMaxEst - LtMinEst ) � StMaxStepSize else StMaxEst - LtMinEst where StMaxStepSize = 0.0005 � FS 344 � 16 SF , In this way, the short-term estimate adaptation rate increases with the input dynamic range.
StMinEst=StMinEst�StMinBeta+lg�(1−StMinBeta)
LtMinEst=LtMinEst�LtMinBeta+lg�(1−LtMinBeta)
if (StMinEst < LtMinEst) LtMinEst = LtMinEst � LtMinAlpha + StMinEst � (1 − LtMinAlpha) where LtMinAlpha is set between 0 and 1 (e.g., tuned to 0.5 in one embodiment). Thus, as described above, if StMinEst is less than LtMinEst, LtMinEst is adjusted with the sum of a long term running average component (LtMinEst�LtMinAlpha) and a component based on StMinEst (StMinEst�(1−LtMinAlpha)).
LtMinBeta = LtMinBeta � LtMinBetaDecay where LtMinBetaDecay = 0.9998 � FS 344 � 16 SF As described above, the short term minimum estimate StMinEst is then updated by increasing it slightly by a factor that depends on the dynamic range of input signal 802. As shown in FIG. 10, minimum energy tracker module 1004 receives maximum energy tracking signal 1008 from maximum energy tracker module 1002. Maximum energy tracking signal 1008 includes long term maximum energy estimate, LtMaxEst, generated by maximum energy tracker module 1002, which is used as an indication of the input dynamic range:
StMinEst
LtMaxEst
StMinStepSize
ThActive=max(min(ThMax, ThMin),11.0)
if (lg > ThActive) ActiveSignal = TRUE else ActiveSignal = FALSE If ActiveSignal is TRUE, then input signal 802 is currently active. If ActiveSignal is FALSE, then input signal 802 is not active. Active signal detector module 1006 outputs ActiveSignal on active signal indicator signal 1012. Energy tracker module 810 outputs maximum energy tracking signal 1008, minimum energy tracking signal 1010, and active signal indicator signal 1008 in a serial, parallel, or other fashion on energy tracking signal 804.
pp Δ =  pp i - pp i - 1  pp i where:
ppg = 10 � log 10 ( E R ) , where:
E = ∑ n = N - K + 1 N x 2 ( n ) , where:
R = E - c 2 ( pp i ) ∑ n = N - K + 1 N x 2 ( n - pp i ) , where:
c(�)=the signal correlation, which may be calculated by:
As shown in FIG. 12, feature extraction module 820 outputs an extracted feature signal 806, which includes the results of the analysis of the one or more analyzed signal features, such as change in pitch period, ppΔ (from module 1202), pitch prediction gain, ppg (from module 1204), first normalized autocorrelation coefficient, ρ1 (from module 1206), and logarithmic signal gain, lg (from module 1208).
N � pp Δ=(1−min(3�pp Δ,1))�2−1
N_ppg = max ( min ( ppg , 10 ) , 0 ) 5 - 1 In other embodiments, other equations for normalizing pitch prediction gain may alternatively be used.
N_ρ1=max(ρ1,0)�2−1
if ( ( LtMaxEst - LtMinEst ) > 6 ) & ( lg > ThActive ) N_lg = max ( min ( lg - ( LtMaxEst - 10 ) 5 - 1 , 1 ) , - 1 ) else N_lg = 0 In other embodiments, other equations for normalizing logarithmic signal gain may alternatively be used.
SLM=N � pp Δ +N � ppg+N_ρ1 +N � lg. In an embodiment, where each normalized feature is in a range (−1 to +1), SLM is in the range {−4 to +4}. Values close to the minimum or maximum values of the range indicate a likelihood that speech is present in input signal 802, while values close to zero indicate the likelihood of the presence of music or other non-speech signals.
if (lg > ThActive) LTSLM = LTSLM * LtslAlpha + |SLM| * (1 − LtslAlpha) where LtslAlpha is a variable that may be set between 0 and 1 (e.g., tuned to 0.99 in one embodiment). As indicated above, in an embodiment, the long term average is updated by module 850 only when an active signal is indicated by ThActive on energy tracking signal 804. This provides classification robustness during background noise.
if (Class(i − 1) == SPEECH) if (LTSLM > 1.75) Class(i) = SPEECH else Class(i) = NONSPEECH else if (LTSLM > 1.85) Class(i) = SPEECH else Class(i) = NONSPEECH where Class(i−1) is the classification of the prior (i−1) classified frame of input signal 802. Threshold values other than 1.75 and 1.85 may alternatively be used by module 860, in other embodiments.
s ( n ) = s out ( n ) � w out ( n ) + s in ( n ) � w in ( n ) n = 0. . N - 1 where sout is the signal to be faded out, sin is the signal to be faded in, wout is a fade-out window, win is the fade-in window, and N is the overlap-add window length.
Overlapping Decomposed Signals with Decomposed Signals
In this embodiment, the signals for overlapping are decomposed into a correlated component, scout and scin, and an uncorrelated component, suout and suin The overlapped signal s(n) is then given by the following equation (Equation C.1):
Flowchart 1600 begins with step 1602. In step 1602, a correlated component of the first segment is added to a correlated component of the second segment to generate a combined correlated component. For example, as shown in FIG. 17, the correlated component of the first segment, SCout, is multiplied with a correlated fade-out window, wcout, by a first multiplier 1702, to generate a first product. The correlated component of the second segment, scin, is multiplied with a correlated fade-in window, wcin, by a second multiplier 1704, to generate a second product. The first product is added to the second product by a first adder 1710 to generate the combined correlated component, scout(n)�wcout(n)+scin(n)�wcin(n).
In step 1604, an uncorrelated component of the first segment is added to an uncorrelated component of the second segment to generate a combined uncorrelated component. For example, as shown in FIG. 17, the uncorrelated component of the first segment, suout, is multiplied with an uncorrelated fade-out window, wuout, by third multiplier 1706, to generate a first product. The uncorrelated component of the second segment, suin, is multiplied with an uncorrelated fade-in window, wuin, by fourth multiplier 1708, to generate a second product. The first product is added to the second product by a second adder 1712 to generate the combined uncorrelated component suout(n)�wuout(n)+suin(n)�wuin(n).
Overlapping a Mixed Signal with a Decomposed Signal
s ( n ) = [ s out ( n ) � wc out ( n ) ] � β + sc in ( n ) � wc in ( n ) + [ ( s out ( n ) � wu out ( n ) ] � ( 1 - β ) + su in ( n ) � wu in ( n ) n = 0. . N - 1 where β is the desired fraction of correlated signal in the final overlapped signal s(n), or an estimate of the cross-correlation between sout and scin+suin. The above formulation is given for a mixed sout signal and decomposed sin signal. A similar formulation for the opposite case, where sout is decomposed and Sin mixed, is provided by the following equation (Equation C.2.b):
Flowchart 1800 begins with step 1802. In step 1802, the first segment is multiplied by an estimate β of the correlation between the first segment and the second segment to generate a first product. For example, as shown in FIG. 19, the first segment, sout, is multiplied with a correlated fade-out window, wcout, by a first multiplier 1902, to generate a third product, sout(n)�wcout(n). The third product is multiplied with β by a second multiplier 1904 to generate the first product.
In step 1804, the first product is added to a correlated component of the second segment to generate a combined correlated component. For example, as shown in FIG. 19, the correlated component of the second segment, scin(n), is multiplied with a correlated fade-in window, wcin(n), by a third multiplier 1906, to generate a fourth product, scin(n)�wcin(n). The first product is added to the fourth product by a first adder 1914 to generate the combined correlated component.
In step 1806, the first segment is multiplied by (1−β) to generate a second product. For example, the first segment, sout, is multiplied with an uncorrelated fade-out window, wuout(n), by a fourth multiplier 1908, to generate a fifth product, sout(n)�wuout(n). The fifth product is multiplied with (1−β) by a fifth multiplier 1910 to generate the second product.
In step 1808, the second product is added to an uncorrelated component of the second segment to generate a combined uncorrelated component. For example, the uncorrelated component of the second segment, suin(n), is multiplied with an uncorrelated fade-in window, wuin(n), by a sixth multiplier 1912, to generate a sixth product, suin(n)�wuin(n). The second product is added to the sixth product by a second adder 1916 to generate the combined uncorrelated component.
Overlapping a Mixed Signal with a Mixed Signal
s ( n ) = [ s out ( n ) � wc out ( n ) + s in ( n ) � wc in ( n ) ] � β + [ s out ( n ) � wu out ( n ) + s in ( n ) � wu in ( n ) ] � ( 1 - β ) n = 0. . N - 1 where β is an estimate of the cross-correlation between sout and sin. Again, notice that if the signals are completely correlated (β=1) or completely uncorrelated (β=0), the solution is optimal.
Flowchart 2000 begins with step 2002. In step 2002, the first segment is added to the second segment to generate a first combined component. For example, as shown in FIG. 21, the first segment, sout(n), is multiplied with a correlated fade-out window, wcout(n), by a first multiplier 2102, to generate a third product, sout(n)�wcout(n). The second segment, sin(n), is multiplied with a correlated fade-in window, wcin(n), by a second multiplier 2104, to generate a fourth product, sin(n)�wcin(n). The third product is added to the fourth product by a first adder 2110 to generate the first combined component.
In step 2006, the first segment is added to the second segment to generate a second combined component. For example, as shown in FIG. 21, the first segment, sout(n), is multiplied with an uncorrelated fade-out window, wuout(n), by a fourth multiplier 2106, to generate a fifth product. The second segment, sin(n), is multiplied with an uncorrelated fade-in window, wuin(n), by a fifth multiplier 2108, to generate a sixth product, sin(n)�wuin(n). The fifth product is added to the sixth product by a second adder 2112 to generate the second combined component.
Flowchart 2200 begins with step 2202. In step 2202, a coarse pitch lag associated with the audio signal is set as a best pitch lag. The initial pitch estimate, also referred to as a �coarse pitch,� is denoted P0. The coarse pitch may be a pitch value from a prior received signal frame used as a best pitch lag estimate, or the coarse pitch may be obtained by other ways.
c ( k ) = ∑ n = 1 M x ( n ) x ( n - k ) ∑ n = 1 M x 2 ( n ) ∑ n = 1 M x 2 ( n - k ) where M is the pitch analysis window length. The parameters P0 and c(P0) are assumed to be available before the pitch refinement is performed in subsequent steps. The normalized correlation may be calculated by one of modules 2310, 2320, 2330 or other module not shown in FIG. 23 (e.g., a normalized correlation calculator module).
In step 2404, decimate the signal x(n). Let D(�) represent a decimator with decimation factor D. Then
If c d(k)>c(P i) then c(P i)=C d(k) and P i=P i-1+k In step 2214, for one or more additional iterations, a new refinement pitch range is calculated and steps 2208, 2210, and 2212 are repeated. Step 2214 may perform as many additional iterations as necessary, until no further decimation is practical, until an acceptable pitch value is determined, etc. As shown in FIG. 23, decimated bisectional search module 2330 outputs pitch estimate Pi.
An example of the iterative process of flowchart 2200 is illustrated in FIGS. 25A-25D. FIGS. 25A-25D show plots of normalized correlation values (cd(k)) versus values of k. For the initial conditions of the search, P0=Δ0=16, and cd(P0) is calculated.
In the first iteration shown in FIG. 25A, Δi=Di=8, and cd(P0�8) is evaluated on the decimated signal. The time resolution of the decimated correlation is noted by the darkened sample points. The candidate that maximizes cd(k) is P0−8 and is selected as P1.
In the second iteration, shown in FIG. 25B, Δi=Di=4, and the search is centered around P1. This time, neither candidate at cd(P1�4) is greater than cd(P1), and so P2=P1.
In the third iteration, shown in FIG. 25C, Δi=Di=2, and the search is centered around P2(P1). The candidate that maximizes cd(k) is P2+2, and is selected as P3.
In the fourth iteration, shown in FIG. 25D, Δi=Di=1 (hence no decimation) and the search is centered around P3. The candidate at P0−7(P3−1) maximizes cd(k), and is selected as the final pitch value.
Note that the process of flowchart 2200 shown in FIG. 22 may be adapted to determining/refining parameters other than just a pitch period parameter. For example, in a process for refining a parameter (e.g., a generic parameter �Q�) of a signal, an adapted step 2202 may include setting a coarse value for the parameter associated with the signal to a best parameter value. An adapted step 2204 may include setting a value of a function f(Q) associated with the coarse parameter value as a best function value. An adapted step 2206 may include calculating a refinement parameter range. An adapted step 2208 may include calculating a value of the function f(Q) at a first midpoint of the refinement parameter range preceding the best parameter value and at a second midpoint of the refinement parameter range following the best parameter value. An adapted step 2210 may include comparing the calculated function value at each of the first and second midpoints to the best function value. An adapted step 2212 may include, responsive to a determination that the calculated function value at either of the first and second midpoints is better than the best function value, setting the better function value associated with each of the first and second midpoints to the best function value and setting the midpoint associated with the better function value to the best parameter value.
As used herein, the terms �computer program medium� and �computer usable medium� are used to generally refer to media such as removable storage units 2628 and 2630, a hard disk installed in hard disk drive 2622, and signals received by communications interface 2640. These computer program products are means for providing software to computer system 2600.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5550543 *Oct 14, 1994Aug 27, 1996Lucent Technologies Inc.Frame erasure or packet loss compensation methodUS5611019 *May 19, 1994Mar 11, 1997Matsushita Electric Industrial Co., Ltd.Method and an apparatus for speech detection for determining whether an input signal is speech or nonspeechUS5712953Jun 28, 1995Jan 27, 1998Electronic Data Systems CorporationSystem and method for classification of audio or audio/video signals based on musical contentUS6134518 *Mar 4, 1998Oct 17, 2000International Business Machines CorporationDigital audio signal coding using a CELP coder and a transform coderUS6157670Aug 10, 1999Dec 5, 2000Telogy Networks, Inc.Background energy estimationUS6490556May 28, 1999Dec 3, 2002Intel CorporationAudio classifier for half duplex communicationUS6570991Dec 18, 1996May 27, 2003Interval Research CorporationMulti-feature speech/music discrimination systemUS6647366 *Dec 28, 2001Nov 11, 2003Microsoft CorporationRate control strategies for speech and music codingUS6694293Feb 13, 2001Feb 17, 2004Mindspeed Technologies, Inc.Speech coding system with a music classifierUS6952668 *Apr 19, 2000Oct 4, 2005At&T Corp.Method and apparatus for performing packet loss or frame erasure concealmentUS7069208 *Jan 24, 2001Jun 27, 2006Nokia, Corp.System and method for concealment of data loss in digital audio transmissionUS7243063Jul 17, 2002Jul 10, 2007Mitsubishi Electric Research Laboratories, Inc.Classifier-based non-linear projection for continuous speech segmentationUS7328149Nov 29, 2004Feb 5, 2008Microsoft CorporationAudio segmentation and classificationUS7565286 *Jul 16, 2004Jul 21, 2009Her Majesty The Queen In Right Of Canada, As Represented By The Minister Of Industry, Through The Communications Research Centre CanadaMethod for recovery of lost speech dataUS7596489 *Sep 5, 2001Sep 29, 2009France TelecomTransmission error concealment in an audio signalUS20010014857Aug 14, 1998Aug 16, 2001Zifei Peter WangA voice activity detector for packet voice networkUS20030009325 *Jan 22, 1999Jan 9, 2003Raif KirchherrMethod for signal controlled switching between different audio coding schemesUS20030074197 *Jun 28, 2002Apr 17, 2003Juin-Hwey ChenMethod and system for frame erasure concealment for predictive speech coding based on extrapolation of speech waveformUS20040083110 *Oct 23, 2002Apr 29, 2004Nokia CorporationPacket loss recovery based on music signal classification and mixingUS20050044471 *Nov 15, 2002Feb 24, 2005Chia Pei YenError concealment apparatus and methodUS20050154584 *May 30, 2003Jul 14, 2005Milan JelinekMethod and device for efficient frame erasure concealment in linear predictive based speech codecsUS20050166124 *Feb 24, 2005Jul 28, 2005Yoshiteru TsuchinagaVoice packet loss concealment device, voice packet loss concealment method, receiving terminal, and voice communication systemUS20050192798Feb 22, 2005Sep 1, 2005Nokia CorporationClassification of audio signalsUS20050228649Jul 8, 2003Oct 13, 2005Hadi HarbMethod and apparatus for classifying sound signalsUS20060265216Sep 26, 2005Nov 23, 2006Broadcom CorporationPacket loss concealment for block-independent speech codecs* Cited by examinerNon-Patent CitationsReference1"ITU-T Recommendation G.711-Appendix I: A High Quality Low-Complexity Algorithm for Packet Loss Concealment with G.711", prepared by ITU-T Study Group 16, (Sep. 1999), 26 pages.2"ITU-T Recommendation G.711�Appendix I: A High Quality Low-Complexity Algorithm for Packet Loss Concealment with G.711", prepared by ITU-T Study Group 16, (Sep. 1999), 26 pages.3Goodman, et al., "Waveform Substitution Techniques for Recovering Missing Speech Segments in Packet Voice Communications", IEEE Transaction on Acoustics, Speech and Signal Processing, (Dec. 1986), pp. 1440-1448.4Office Action for U.S. Appl. No. 11/734,806 mailed on Oct. 7, 2010, 23 pages.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8489406 *Aug 12, 2011Jul 16, 2013Huawei Technologies Co., Ltd.Stereo encoding method and apparatusUS8798991 *Nov 13, 2012Aug 5, 2014Fujitsu LimitedNon-speech section detecting method and non-speech section detecting deviceUS20090070117 *Sep 5, 2008Mar 12, 2009Fujitsu LimitedInterpolation methodUS20110301962 *Aug 12, 2011Dec 8, 2011Wu WenhaiStereo encoding method and apparatus* Cited by examinerClassifications U.S. Classification704/208, 704/214, 704/207, 704/220, 704/200, 704/210, 704/213, 704/228, 704/215, 704/209, 704/216, 704/219, 704/227, 704/221, 704/500, 704/226, 704/201International ClassificationG10L19/06, G10L19/12, G10L21/02, G10L11/06, G10L21/00, G10L11/00, G10L19/00Cooperative ClassificationG10L25/78, G10L19/005European ClassificationG10L19/005Legal EventsDateCodeEventDescriptionDec 6, 2011CCCertificate of correctionApr 13, 2007ASAssignmentOwner name: BROADCOM CORPORATION, CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZOPF, ROBERT W.;CHEN, JUIN-HWEY;THYSSEN, JES;REEL/FRAME:019156/0035Effective date: 20070412RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services