Abstract:
The method and system disclosed herein reduces total bandwidth requirement for communication in a voice over Internet protocol application. Sample [ 101 ] and convert [ 102 ] the analog input audio signal into digital signals and derive sampled frames [ 103 ]. Compute spacings of order statistics [ 104 ]. Measure the entropy for each of the sampled frames [ 105 ]. Set a threshold for entropy [ 106 ]. Mark the audio frames as active speech frames or inactive speech frames [ 107 ]. Mark an audio frame as an&#39; inactive speech frame when the entropy is greater than the threshold, and mark the audio frame as an active speech frame when the entropy is lesser than the threshold [ 107 ]. Transmit only the active speech frames [ 108].

Description:
BACKGROUND 
       [0001]    This invention in general relates to a method for reducing the total bandwidth requirement for voice-enabled applications over the Internet and specifically relates to a method of separating speech signal from non-speech signal. 
         [0002]    Given the rapid growth in Internet traffic, there is a shortage in the bandwidth available for the transfer of data for voice over IP applications. Speech signals consist of non-speech segments and speech segments. Non-speech segments do not contribute to comprehension and may contain noise or disturbances which are undesirable, and may cause deterioration. However, all segments, speech or otherwise, demand bandwidth for transmission. Moreover, in the context of speech recognition, segmentation of the input speech stream into “speech” and “non-speech” is the precursor to applying recognition algorithms. 
         [0003]    Bandwidth optimization is achieved by speech compression using low bit rate codecs integrated with Voice Activity Detection (VAD). Further optimization is usually achieved by the following two methods. In the first method, VAD scheme, usually based on energy and zero-crossing methods, is embedded in codecs. Examples are G.729, Global System for Mobile communication (GSM), Adaptive Multi Rate (AMR), G.722 and 3 rd  Generation Partnership Project (3GPP). In the second method, VAD scheme may not be embedded in the codec block. Selecting talk spurts and avoiding codec processing of non-speech segments at the transmitter has the additional advantage of reducing the computational load on the codec itself. This is particularly significant as the number of streams grows. In such a setup, VAD coding is independent of the speech code. Portability across codecs is an added advantage since one can use any codec after applying a stand-alone VAD that removes the non-speech part of the stream. 
         [0004]    There is an unmet market need for a method and a system that effectively removes the non-speech component in a voice over internet protocol (VoIP) based communication system. 
       SUMMARY 
       [0005]    The method and system disclosed herein seeks to separate the speech segments from the non-speech segments in an audio signal and transmit only the speech segments over the Internet. An entropy measure derived from spacings of order statistics of speech frames is used to differentiate non-speech and/or silent (inactive) zones from speech (active) zones. Non-speech segments are not transmitted; they are replaced, in general, by “comfort noise” during playout at the receiver&#39;s end, thereby increasing the proportion of available bandwidth for other users of the Internet. The present invention accomplishes a greater saving in bandwidth by detection of the speech or active signal by efficaciously discriminating it from non-speech. A threshold is devised and applied for detection of the speech and non-speech segments in real-time. 
         [0006]    The method and system disclosed herein enables speech activity detection through Adaptation of Threshold computed from Entropy derived from Spacings of Order Statistics—which we hereinafter refer to as ATESOS. 
         [0007]    The method and the system disclosed herein are scalable across different frame sizes. 
         [0008]    The method and the system disclosed herein determine the boundaries between contiguous active and inactive zones with sharper accuracy, thereby improving the effectiveness of speech spurt detection and speech recognition. 
         [0009]    The method and the system disclosed herein can be implemented in low Signal-to-noise ratio (SNR) environments since frame classification, or packet classification in the context of packet-switched network, for example, VoIP, is independent of the signal energy, and depends only on the signal entropy. 
         [0010]    Thus, the method and the system disclosed herein are applicable to packet switched networks for improving bandwidth utility. 
         [0011]    The method and the system disclosed herein initialize the threshold by observing merely two initial packets. Hence, the decision to differentiate speech from non-speech segments is made almost instantaneously. This rapid decision-making process minimizes delay to the extent that it is perceived as effectively on-line and real-time in its implementation. 
         [0012]    The method and the system disclosed herein is applicable for VoIP, speech recognition, speech-to-text, biometrics, etc. Speech boundary segmentation in the context of speech-to-text, speech recognition and spealer recognition are some examples in point. Further, the system and method is adaptable to varying quantization levels of 8-bit, 16-bit, for instance. This scheme is therefore portable in its present form, equally efficaciously, across different quantization levels. 
         [0013]    The method and the system disclosed herein are less sensitive to the characteristics of the microphone employed to capture the original speech stream. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    The foregoing summary, as well as the following detailed description of the embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there is shown in the drawings exemplary constructions of the invention; however, the invention is not limited to the specific methods and instrumentalities disclosed. 
           [0015]      FIG. 1  illustrates a method for reducing total bandwidth requirement for voice-enabled applications over the internet by transmitting only the frames containing active speech segments. 
           [0016]      FIG. 2  illustrates the flowchart for the ATESOS algorithm used in the detection and the separation of the active speech frames from the inactive speech frames in an audio signal. 
           [0017]      FIG. 3  illustrates the Pseudo code of the ATESOS algorithm used in the detection and the separation of the active speech frames from the inactive speech frames in an audio signal. 
           [0018]      FIG. 4  illustrates the system diagram that implements the separation of the active speech frames from the inactive speech frames. 
           [0019]      FIG. 5A  illustrates the speech signal for utterances of “/Hello/, /One/, /Two/, /Three/” with deliberate pauses in between the words. 
           [0020]      FIGS. 5B and 5C  illustrate the output waveforms from the application of the ATESOS algorithm, for a zero noise condition and 20 ms frame size. 
           [0021]      FIG. 6  illustrates the output waveform for speech activity detection (SpAD) with 5 dB babble noise. 
           [0022]      FIG. 6A  illustrates the speech signal of utterances of “/Hello/, /One/, /Two/, /Three/” with deliberate pauses in between the words. 
           [0023]      FIG. 6B  illustrates the entropy obtained from the spacings of the order statistics. 
           [0024]      FIG. 6C  illustrates the decision taken by the ATESOS algorithm. 
           [0025]      FIG. 7A  illustrates the speech signal for utterances of “/Hello/, /One/, /Two/, /Three/” with deliberate pauses in between the words. 
           [0026]      FIGS. 7B and 7C  illustrate the output waveform from the implementation of the speech activity detection (SPAD) algorithm. 
           [0027]      FIG. 8  illustrates how x min  and x max  are determined for calculating the threshold. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0028]      FIG. 1  illustrates a method for reducing total bandwidth requirement for voice-enabled applications over the internet by transmitting only the frames that consist of active speech segments. The analog input audio signal is sampled  101  and converted into a digital signal  102 . The sampled digital audio signal is then converted into audio frames of a fixed size  103 . 
         [0029]    The spacings of order statistics is computed for the above audio frames  104 . Any intelligible speech segment, such as human speech or music, contains redundant information while noise or non-intelligible speech is characterized by lesser redundancy; i.e., it possesses “high” information content. Entropy reflects a measure of information content. Thus it follows that all intelligible speech segments have lower entropy or randomness in them and the non-intelligible speech segments have higher entropy. A statistical analysis of intelligible speech vis-à-vis non-intelligible speech reveals that over the mixed sample, non-intelligible speech segments have probabilities closer to the mean of the sample, whereas the probabilities associated with intelligible speech lie away from the mean and have a larger variance. 
         [0030]    Entropy for each of the frames is calculated  105 . Entropy is measured at each of the input instances, i.e. at the occurrence of the audio signal at a given time. A threshold is set for a first set of frames based on the entropy measured  106 . The first set of frames may comprise one or more frames. The threshold for a second set of frames is equal to the threshold for the first set of frames plus an increment. The increment may be positive or negative. The threshold for each frame in the second set may vary depending on the entropy of the frame and the threshold of the past frame plus the increment. The second set of frames may comprise one or more frames. The maximum and the minimum values of entropy are calculated for different input instances. If the entropy of the frame under consideration is greater than the threshold, then the frame is marked inactive; otherwise the frame is marked active  107 . The active speech frames are transmitted  108 . 
         [0031]    An adaptive threshold is achieved by sensitizing the threshold to varying entropy values of input frames as they stream in. The value to be added to, or subtracted from the threshold, called the increment, is determined by two variables: x max  and x min . x max  is the maximum entropy attained until the current frame; x min  depends on whether the frame is active or inactive. The increment is calculated as a percentage of the sum of x max  and x min . In particular, a non limiting example of the invention uses 10% of the sum of x max  and x min  as the increment. If the current frame is active then x min  will be equal to minimum entropy observed over all the frames until the current frame in the given talk spurt. A talk spurt consists of consecutive frames marked as active. Usually, speech frames occur in bursts, and similarly silence frames occur in bursts. If the frames are marked as inactive, then x min  will be equal to x max . Therefore x min  will be high if the frame is inactive and x min  will be low if current frame is active. 
         [0032]      FIG. 8  illustrates the calculation of x min  and x max . Initially x min  and x max  are calculated. 
         [0033]    As stated above, x max  is the maximum entropy attained until the current frame; and the graph for x max  monotonically increases. Hence there is a need to sensitize the threshold to the varying nature of the input to the sensor  401 . A change in x max  results in a new threshold value. This new threshold value is a step closer to x max . The x min  depends on whether the current frame is active or inactive. For an active frame, it is checked if the entropy of the current frame is lesser than x min . If the entropy of the current frame is lesser than x min , then x min  is updated and a new threshold is calculated  211 . Once the least value is hit upon in that active speech segment, the x min  and, hence, the threshold do not change. 
         [0034]    In an inactive speech segment, the x min  and x max  are equal. A change only in x max  results in adaptation of the threshold to the dynamics of the input. 
         [0035]    Due to the variation of the x min , the increment will be a small step in the direction of the movement of the entropy curve. The threshold is calculated only if there is a change in either the x max  or x min . The frames consisting of active speech frames are separated from the inactive speech frames and are transmitted over the Internet  108 . Thus the transmitted frames consist of only the active speech frames, thereby reducing bandwidth requirement for voice-enabled applications over the Internet.  FIG. 2  illustrates the flowchart for the ATESOS algorithm used in the detection and the separation of the active speech frames from the inactive speech frames in an audio signal. The ATESOS algorithm marks the speech frame as active or inactive with reference to the threshold. 
         [0036]    For each of the recorded speech frames  201 , the entropy is calculated using spacings of order statistics for the first two frames. The first two frames are represented by j=1, 2. For the values of j=1 to 2,  202  the entropy H for the frames is calculated using the formula 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       H 
                       ^ 
                     
                     
                       m 
                       , 
                       
                         N 
                          
                         
                           ( 
                           Y 
                           ) 
                         
                       
                     
                   
                   = 
                   
                     
                       
                         1 
                         N 
                       
                        
                       
                         
                           ∑ 
                           
                             i 
                             = 
                             1 
                           
                           
                             N 
                             - 
                             m 
                           
                         
                          
                         
                             
                         
                          
                         
                           ln 
                            
                           
                             ( 
                             
                               
                                 N 
                                  
                                 
                                   ( 
                                   
                                     
                                       Y 
                                       
                                         i 
                                         + 
                                         m 
                                       
                                     
                                     - 
                                     
                                       Y 
                                       i 
                                     
                                   
                                   ) 
                                 
                               
                               m 
                             
                             ) 
                           
                         
                       
                     
                     + 
                     
                       ln 
                        
                       
                         ( 
                         m 
                         ) 
                       
                     
                   
                 
               
               
                 
                   Equation 
                    
                   
                       
                   
                    
                   
                     ( 
                     1 
                     ) 
                   
                 
               
             
           
         
       
     
         [0037]    where
       Y i+m −Y i  for 1≦i&lt;i+m≦N is m-spacings of the nth order statistic.   N is the number of samples in a frame   Y is the set of ordered samples of a frame       
 
         [0041]    For j=3  203  the maximum value, x max , in the first two frames is calculated, wherein x max  is the maximum entropy attained until the current frame. 
         [0000]        x   max =max{ Ĥ ( j )}∀ j= 1 to 2 
         [0000]    The threshold is a moving average and it is initialized to the mean of the first two entropy values 
         [0000]      threshold=mean{ Ĥ (1),  Ĥ (2)} 
         [0000]    x min  is initialized to x max , wherein if the current frame is active, then x min  will be equal to minimum entropy observed over all the frames until the current frame. If the frame is marked inactive then the x min  is equal to x max . 
         [0000]      x min =x max    
         [0000]    The entropy is calculated  204  for each frame starting from the third frame and as-and-when a recorded speech frame is available. A check is performed to determine if the entropy calculated is greater than the x max    205 . If the entropy calculated  204  is greater than the x max    205 , the x max  and the threshold are calculated as follows  206 , 
         [0000]      ( x   max   &lt;Ĥ ( j )) 
         [0000]        x   max   =Ĥ ( j ) 
         [0000]      incr=( x   max   +x   min )/10 
         [0000]      threshold=(threshold+incr)/1.23 
         [0000]    If the entropy obtained for the current frame is less than the threshold  207 , the frame is marked as active by assigning bspeechframe to 1  208 . 
         [0000]      ( Ĥ ( j )&lt;threshold) 
         [0000]      bSpeechFrame=1 
         [0000]        n Compression= n Compression+1 
         [0000]    A check is performed to determine if a new x min  is achieved, and if the x min  is greater than entropy  210 , a new threshold is calculated as follows  211 : 
         [0000]      ( x   min   &gt;Ĥ ( j )) 
         [0000]        x   min   =Ĥ ( j ) 
         [0000]      Incr=( x   max   +x   min )/10 
         [0000]      threshold=(threshold+Incr)/1.23 
         [0000]    If the entropy calculated for the frame is greater than the threshold, then the frame is marked as inactive by assigning bspeechframe to 0 and initializing x min  to x max    209 . A new threshold value is calculated as 81.3% of the sum of the threshold and increment. 
         [0000]      bSpeechFrame=0 
         [0000]      x min =x max    
         [0000]    If bSpeechFrame is zero, the transmission of speech frames is withheld, i.e., the conversation is in ‘silence’. Similarly the consecutive frames marked as bSpeechFrame=1 results from a talk spurt. 
         [0042]      FIG. 3  illustrates the Pseudo code of the ATESOS algorithm used in the detection and the separation of the active speech frames from the inactive speech frames in an audio signal. 
         [0043]      FIG. 4  illustrates the system diagram that implements the separation of the active speech frames from the inactive speech frames. The analog audio input is taken from the sensor  401 . Optionally, the analog audio input can be a speech file that is directly fed to the analog to digital converter  402 . The analog audio input is passed through an analog to digital converter  402  for analog to digital conversion. The digital audio signal is then passed into a fixed-sized buffer  403  to convert the digital audio signal into frames of a particular size. The digitized and buffered audio signal converted to frames is then passed through the central processing unit  404 . The microprocessor located in the central processing unit  404  applies the ATESOS algorithm and differentiates the active speech frames from the inactive speech frames. The network interface module  405  accepts only the active speech frames and transmits them over the internet in the form of packets. The central processing unit  404  computes spacings of order statistics in a statistical sample for said sampled frames, measures the entropy of each of said sampled frames, sets threshold for entropy and marks the audio frames. The step of marking comprises marking the audio frame as an inactive speech frame when the entropy is greater than the threshold, and marking the audio frame as an active speech frame when the entropy is lesser than the threshold. 
         [0044]    At the receiver, the inactive speech frames are not received. During playout of the buffers, the silence created by inactive frames at the transmitter is substituted by comfort noise making the listener perceive that the inactive frames were transmitted. 
         [0045]      FIG. 5A  illustrates the speech signal for utterances of “/Hello/, /One/, /Two/, /Three/” with deliberate pauses in between the words. 
         [0046]      FIGS. 5B and 5C  illustrate the output waveforms from the application of the ATESOS algorithm, for a zero noise condition and 20 ms frame size. 
         [0047]      FIG. 5B  illustrates the entropy obtained from the spacings of the order statistics using the equation (1), described earlier under the description of  FIG. 2 . The dotted line  501  in  FIG. 5B  illustrates the threshold values for the respective entropy values. 
         [0048]      FIG. 5C  illustrates the decision taken by the ATESOS algorithm. The speech frame is marked as active if decision is 1, and inactive if the decision is 0. The decision is 1 when the entropy value is less than the threshold. 
         [0049]      FIG. 6A  illustrates the speech signal of utterances of “/Hello/, /One/, /Two/, /Three/” with deliberate pauses in between the words. The signal is corrupted with additive babble noise and the overall SNR is 5 dB. 
         [0050]      FIG. 6B  illustrates the entropy obtained from the spacings of the Order Statistics using equation (1). The frame size considered is 20 ms. The dotted line in  FIG. 6B  illustrates the threshold values for the respective entropy values. 
         [0051]      FIG. 6C  illustrates the decision taken by the ATESOS algorithm. The speech frame is marked as active if the decision is 1 and inactive if the decision is 0. The decision is 1 when the entropy value is less than the threshold and the decision is 0 if the entropy value is greater than the threshold. 
         [0052]      FIG. 7A  illustrates the speech signal for utterances of “/Hello/, /One/, /Two/, /Three/” with deliberate pauses in between the words.  FIG. 7B  and  FIG. 7C  illustrate the output waveform from the implementation of the speech activity detection (SpAD) algorithm. 
         [0053]      FIG. 7B  illustrates the entropy obtained from the spacings of the order statistics using equation (1). The frame size considered is 60 ms. The dotted line in  FIG. 7B  illustrates the adaptive threshold values for the respective entropy values. The adaptive threshold is computed using the ATESOS algorithm described in  FIG. 2  and  FIG. 3 . 
         [0054]      FIG. 7C  illustrates the decision taken by the ATESOS algorithm. The speech frame is marked as active if the decision is 1 and inactive if the decision is 0. The decision is 1 when the entropy value is less than the threshold and the decision is 0 if the entropy value is greater than the threshold. 
         [0055]      FIG. 8  illustrates the calculation of x min  and x max .  801  points to the location on the entropy curve from where the x min  starts decreasing in that active speech region.  802  points to the least value that x min  reaches in that active speech region.  803  points to the location in the entropy curve where x max  reaches the highest value. 
         [0056]    The method and system disclosed herein accomplishes a greater saving in bandwidth by detection of the speech/active signal by efficaciously discriminating from non-speech. 
         [0057]    Using the ATESOS algorithm, the speech burst is located accurately. Information on the location of the speech burst may be provided to an echo cancellation module (not shown in figure). The identification of the location of the speech burst aids in the process of subtracting the return signal in VoIP systems. 
         [0058]    The location of the speech burst can be provided to a speech recognition module (not shown in figure) for accurately mapping and identifying the words. The ATESOS algorithm may be used to preprocess audio data for a speech recognition module. 
         [0059]    The location of the speech burst can be provided to coding modules for reducing the level of computation required for coding the speech data. 
         [0060]    The foregoing examples have been provided merely for the purpose of explanation and are in no way to be construed as limiting of the present method and system disclosed herein. While the invention has been described with reference to various embodiments, it is understood that the words that have been used herein are words of description and illustration, rather than words of limitations. Further, although the invention has been described herein with reference to particular means, materials and embodiments, the invention is not intended to be limited to the particulars disclosed herein; rather, the invention extends to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims. Those skilled in the art, having the benefit of the teachings of this specification, may effect numerous modifications thereto and changes may be made without departing from the scope and spirit of the invention in its aspects.