Abstract:
A voice encoding method includes the steps of encoding a first frame that contains a plurality of voice data into encoded parameters, locally decoding the encoded parameters of the first frame into a second frame, performing a plurality of interpolation recovery processes that generate respective frames approximating to the first frame by using a frame or frames other than the first frame, comparing the second frame with the frames approximating to the first frame generated by the plurality of interpolation recovery processes, calculating a signal to noise ratio of each of the frames approximating to the first frame by treating the second frame as the signal, determining an index number that indicates an interpolation recovery process which provides a highest signal to noise ratio, and multiplexing and transmitting the index number with the encoded parameters.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    The present invention generally relates to a voice encoding method for voice transmission through an IP (Internet protocol) network, and particularly relates to the voice encoding method that alleviates deterioration in voice quality at a receiving end when a packet is lost in the transmission.  
           [0003]    2. Description of the Related Art  
           [0004]    VOIP (Voice Over IP) has been known as a technology to transmit voice over an IP network. FIG. 1 shows a basic structure of a VOIP transmission system. The VOIP transmission system is principally comprised of such user terminals as telephone sets  101  and  107 , access/conventional networks  102  and  106 , VOPI gateways (VOIPGW)  103  and  105  and the Internet  104 . VOIPGW  103  and  105  are located in between the access/conventional networks  102  and  106  and the Internet  104 , respectively. FIG. 2 shows a basic structure of a voice processing unit of the VOIPGW. The VOIPGW voice processing unit is principally comprised of an access/conventional network interface  201 , a voice encoding unit  202 , a packet assembling unit  203 , a voice decoding unit  204  and a packet disassembling unit  205 . In VOIP, a voice signal that is input to the VOIPGW  103  and  105  from the access/conventional networks  102  and  106 , respectively, is transmitted after encoding by the voice encoding unit  202  at a low bit rate. The encoded voice signal is multiplexed with data packets, thereby economizing the cost of voice communication.  
           [0005]    However, the basic structure as shown in FIG. 1 suffers problems as follows. One of the problems is that a delay time becomes lengthy as packets are transmitted via a plurality of routers in the IP network. The second problem is that there is a fluctuation (i.e., jitter) in the time of packet arrivals as the packets are transmitted via various buffers. The third problem is that a packet may be lost due to data overflow at these buffers or due to errors occurring during data transmission, which deteriorates quality of voice reproduced at a receiving end.  
           [0006]    Conventional techniques for compensating for lost packets on the transmitting side are as follows, for example. The first technique is to return information about the packet loss from the receiving end to the transmitting side so that a frame corresponding to the lost packet is retransmitted. The second technique employs an interleave process, which alleviates an effect of packet loss by randomizing errors. The third technique employs an FEC (Forward Error Correction) encoding.  
           [0007]    Examples of conventional techniques that can be employed on the receiving side are as follows. The first is a method of inserting a waveform with respect to a lost frame. The second method interpolates a waveform from waveforms of the frames preceding and following the lost frame, or interpolates a waveform from a waveform of the preceding frame. The third method is to interpolate voice codec parameters from those of preceding and following frames so as to reproduce voice from the interpolated parameters. These techniques are described in “A Survey of Packet Loss Recovery Techniques for Streaming Audio,” IEEE Network Magazine, the September/October issue, pp.40-48, 1998, and “Internet Telephony: Services Technical Challenges, and Products,” IEEE Communication Magazine, the April issue, pp 96-103, 2000.  
           [0008]    The first and the second techniques employed on the transmitting side are principally used in delivery services where time delays are permissible. FIG. 3 shows an example of a media specific interpolation process that corresponds to the third technique employed on the transmission side described above.  
           [0009]    In FIG. 3, frames of an original voice stream are referred to by reference numerals  301  through  304 . In this example, four frames are shown. Here, the frame  303  is coded into an coded parameter  313 - 3  that is ordinarily used, and is also encoded into another coded parameter  314 - 3  corresponding to a voice encoder having a bit rate lower than the ordinarily used bit rate. The coded parameter  313 - 3  that is ordinarily used and the coded parameter  314 - 3  corresponding to the lower bit rate voice encoder are inserted into a frame  313  and a frame  314 , respectively, which have respective FEC codes added thereto, and are then transmitted as packets. If the packet  313  is lost during the transmission, the encoded parameter  314 - 3  of the lower bit rate voice encoder is used in place of the ordinarily used encoded parameter  313 - 3 , thereby reproducing a waveform corresponding to the voice frame  303  that should have been transmitted by the packet  313 . The processing delay in this method is one frame interval. In order to obtain voice quality of a desired level, the lower bit rate encoder is required to be capable of encoding at about 2 to 4 kbps. Accordingly, redundant data (i.e., overhead) of about 40 to 80 bits is necessary to add the encoded parameter  314 - 3  of the lower bit rate voice encoder in the case of a frame length of 20 msec.  
           [0010]    Conversely, in the conventional techniques where the lost packet is interpolated on the receiving end, the interpolation process can be performed without the overhead. FIG. 4 shows a basic structure for performing a conventional interpolation method on the receiving end. FIG. 4 shows the voice decoding unit  204  that principally includes a packet disassembling unit  401 , a voice decoding unit  402 , and an interpolation process unit  403 . An encoded parameter output from the packet disassembling unit  401  is provided to the voice decoding unit  402 , which reproduces and outputs a voice waveform. If there is a packet loss in the received packets, a packet loss index indicative of the lost packet is supplied to the interpolation process unit  403 . The interpolation process unit  403  performs an interpolation process, an example of which will be described in the following.  
           [0011]    A first example is to multiply a reproduced waveform by a window function where the reproduced waveform is that of a frame preceding the lost packet, and uses the obtained waveform as the waveform of the frame that has suffered the packet loss. Alternatively, a second example is to interpolate coded parameters from frames preceding and following the frame that has suffered packet loss, thereby reproducing the voice of the frame of packet loss based on the interpolated parameters. In this case, LPC (Linear Prediction Coding) parameters, for example, are obtained by linear interpolation from parameters obtained from the frames preceding and following the frame of packet loss. As for other parameters, the same parameter values as those of the preceding frame are used.  
           [0012]    It has been known that the method based on parameter interpolation has an advantage of better reproduction quality over other techniques employed on the receiver end for interpolating and recovering the lost packet. However, this method has following problems.  
           [0013]    A first problem is that, despite presence of a plurality of available interpolation and recovery processes, the conventional method is configured to use only one of such processes. Accordingly, the process employed for interpolation and recovery of a lost packet may not be the best method from the viewpoint of an S/N (signal to noise) ratio or the viewpoint of subjective quality.  
           [0014]    A second problem is that if the lost packet contains a consonant section, the interpolation recovery process may still loose clarity of voice.  
           [0015]    HoHooHo  
         SUMMARY OF THE INVENTION  
         [0016]    It is a general object of the present invention to provide a voice encoding scheme that substantially obviates one or more of the problems caused by the limitations and disadvantages of the related art.  
           [0017]    It is another and more specific object of the present invention to provide a voice encoding method employing a packet recovery process, which is capable of providing a high S/N ratio and high subjective quality, and is capable of providing clear voice during consonant intervals.  
           [0018]    To achieve the first part of the object, a plurality of interpolation recovery processes are provided on the transmitting side. On the transmitting side, each and every frame is assumed to be lost, and all the interpolation recovery processes are performed with respect to each frame. Waveforms that are interpolated and recovered are compared with a waveform that is locally decoded and reproduced from the relevant packet. An interpolation recovery process that provides the closest waveform to the locally decoded and reproduced waveform is determined. An index number of this process is transmitted with the packet to the receiver end. At the receiving end, the plurality of interpolation recovery processes are provided in the same manner as in the transmitting end. When packet loss is detected, an interpolation recovery process indicated by the index number that is transmitted together with the frame is used to select a proper interpolation process, which is then performed. In this manner, the present invention obtains an interpolated and recovered waveform closest to the waveform that would have been recovered if the packet had not been lost.  
           [0019]    For the second part of the object described above, a detection process is performed frame by frame on the transmitting side to detect whether a frame contains a consonant interval. If a consonant is included in the frame, the frame is transmitted with higher priority. The higher priority may be attained by transmitting the frame having a consonant a number of times. Alternatively, if a setting can be made to indicate frame priority, the frame having a consonant is given a setting indicative of higher priority.  
           [0020]    Other objects and further features of the present invention will be apparent from the following detailed description when read in conjunction with the accompanying drawings. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0021]    [0021]FIG. 1 shows a basic structure of a VOIP transmission system;  
         [0022]    [0022]FIG. 2 shows a basic structure of a VOIPGW voice processing unit;  
         [0023]    [0023]FIG. 3 shows an example of a conventional media specific interpolation process on the transmitting side;  
         [0024]    [0024]FIG. 4 shows a basic structure for performing a conventional interpolation method on the receiving end;  
         [0025]    [0025]FIG. 5A is a block diagram of the transmitting end (encoding side) according to a first embodiment;  
         [0026]    [0026]FIG. 5B is a block diagram of the receiving end (decoding side) according to the first embodiment;  
         [0027]    [0027]FIG. 6 is an illustrative drawing showing a process of the first embodiment of the present invention;  
         [0028]    [0028]FIG. 7 shows an example of packet structure;  
         [0029]    [0029]FIG. 8A is a block diagram of an encoder according to a second embodiment;  
         [0030]    [0030]FIG. 8B is a block diagram of a decoder according to the second embodiment;  
         [0031]    [0031]FIG. 9 shows a basic structure of a CELP encoder;  
         [0032]    [0032]FIG. 10 shows transmission timing of parameters;  
         [0033]    [0033]FIG. 11 is a block diagram of a voice encoding unit and a packet assembly unit according to a third embodiment of the present invention;  
         [0034]    [0034]FIG. 12 is an illustrative drawing showing processes of the third embodiment of the present invention;  
         [0035]    [0035]FIG. 13 is a block diagram of the transmission side according to a fourth embodiment of the present invention;  
         [0036]    [0036]FIGS. 14A through 14C show examples of distributions of a zero crossing number Z, a log level L, and a first-order autocorrelation value R, respectively; and  
         [0037]    [0037]FIG. 15 is a block diagram of the receiving end. 
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0038]    In the following, embodiments of the present invention will be described with reference to the accompanying drawings.  
         [0039]    The present invention is applied to the VOIPGWs  103  and  105  as shown in FIG. 1. FIGS. 5A and 5B show a structure of a first embodiment of the present invention, which solves the first problem mentioned above. FIG. 5A exhibits a sample structure of the voice encoding unit  202  provided on the transmitting side shown in FIG. 2. FIG. 5B exhibits a sample structure of the voice decoding unit  204  on the receiving end shown in FIG. 2. The voice encoding unit  202  includes principally a voice encoding unit  501 , a plurality of interpolation processing units such as interpolation processing units  502  through  504 , an S/N calculation comparison unit  505  and a multiplexing unit  506 . The voice encoding unit  501  includes a local decoding unit that locally decodes parameters encoded in the encoding unit. The local decoding unit may share components with an encoding part of the encoding unit. The voice decoding unit  204  includes a disassembly unit  511 , a voice decoding unit  512 , an interpolation processing unit  513 . On the transmitting side, the interpolation processing units  502  through  504  always assume that a frame is lost, and attempt their respective interpolation recovery processes. Then, waveforms interpolated and recovered by the interpolation recovery units  502  through  504  are compared with a waveform locally decoded from the relevant packet by the voice encoding unit  501 . This comparison is made with respect to S/N ratios by the S/N calculation comparison unit  505 . An index number, which indicates an interpolation and recovery process of the interpolation processing unit that has provided the highest S/N, is supplied to the multiplexing unit  506 , by which the index number is multiplexed with the encoded parameters, followed by transmission thereof. On the receiving end, when there is no packet loss, a voice decoding process is performed by the voice decoding unit  512  using the encoded parameters output from the disassembly unit  511 . When a packet loss is detected at the disassembly unit  511 , an interpolation recovery process is carried out by using the index number of the interpolation recovery processing method that is received from the transmission side.  
         [0040]    [0040]FIG. 6 is an illustrative drawing showing a process of the first embodiment of the present invention. In FIG. 6, (A) shows input voice signal frames  601 ,  602  and  603 . (B) shows process intervals  611  through  616 . (C) shows output packets  621 ,  622  and  623 , as well as an example structure of the packet  622 . (D) shows received packets  631 ,  632  and  633  on the receiving end when there is no packet loss and decoded voice outputs  641 ,  642  and  643 , respectively. When there is a packet loss, the received packets  631 ,  632  and  633  and their respective decoded voice outputs  641 ,  644  and  643  are as shown in (E).  
         [0041]    On the transmitting side, the voice input frames  601 ,  602  and  603  are encoded during the process intervals  611 ,  612  and  613 , respectively. Further, during the process intervals  614 ,  615  and  616 , interpolation recovery processes take place at the interpolation process units  502 ,  503  and  504 , respectively, as described above, assuming that every one of the packets is lost. For example, during the process interval  616 , these interpolation recovery processes are performed for the frame  602  by using the encoded parameters of the frames  601  and  603 . An index number indicative of the interpolation recovery process that provides the highest S/N is identified, and is packetized together with the encoded parameter. The packet may be composed of, for example, a header  625 , a control bit portion  626 , the index number  627  of the selected optimum interpolation process, and the encoded parameter  628 . FIG. 7 shows another example of the structure of a packet. Here, the packet includes an IP header  701 , a UDP header  702 , an RTP header  703 , and voice encoded data  704 . The index number obtained as above may be loaded at an unused area such as bits  6  and  7  of a TOS (Type Of Service) field  705  in the IP header  701 . By loading the index number outside the encoded data area  704  of the packet, the index number can be transmitted without deteriorating voice quality. Similarly, if there is an unused area available in the RTP header  703 , the index number may be loaded into this area. Further, in the encoded data area  704 , there is an area whose error sensitivity is low. Therefore, the obtained index number may be loaded to the area that has the lowest error sensitivity, minimizing an impact on the voice quality when sending the index number in the encoded data area  704 .  
         [0042]    In an implementation where the index number is loaded into the least error sensitive area of the encoded data area  704 , the index number may be transmitted once in several frames, thereby further minimizing voice quality deterioration. In this case, the process mentioned above is performed once in several frames. Alternatively, the process may be performed and the index number may be transmitted only when the encoded parameters greatly differ between adjacent frames.  
         [0043]    On the receiving end, the voice outputs  641 ,  642  and  643  are generated by decoding the received packets  631 ,  632  and  633  by using the encoded parameters for each of the frames as shown in FIG. 6, (D). On the other hand, if the packet  632  was lost, for example, as shown in (E), the voice frame  644  is reproduced by an interpolation recovery process using the-frames  631  and  633  and the index number received together with these frames.  
         [0044]    Here, a second embodiment of the present invention is described. FIG. 8A shows an embodiment wherein the CELP method is employed in the voice encoding. The voice encoding unit  202  includes a CELP encoder  801 , frame buffers  802 ,  803  and  804 , interpolation processing units  805 ,  806 ,  807  and  808 , local decoding units  809 ,  810 ,  811  and  812 , an S/N calculation comparison unit  813 , and a multiplexing unit  814 . FIG. 9 is a block diagram of the CELP encoder  801 , comprising principally an LPC analysis unit  901 , an LPC quantization unit  902 , a synthesis filter unit  903 , a subtraction unit  904 , an audibility weight filter unit  905 , a distortion minimizing unit  906 , an adaptive codebook  907 , a fixed codebook  908 , gain adjustment units  909  and  910 , and an adder  911 .  
         [0045]    The CELP method is a voice compression method wherein a most appropriate codebook is selected by AbS (Analysis by Synthesis). In the CELP encoder  801 , LPC parameters are computed by an LPC analysis unit  901  for every frame that is 20 msec long, for example. Further, an index and a gain in an adaptive codebook and an index and a gain in a fixed codebook that provide the best voice quality are computed and output for every subframe that is 5 msec long, for example. FIG. 10 shows relationships between frames and subframes. In FIG. 8A, the parameters that are computed by the CELP encoder  801  as described above are stored in the frame buffer  802  for two previous frames. Similarly, the internal state of the local decoder and an output of the synthesis filter  903  for a frame immediately preceding the current frame are stored in the frame buffers  803  and  804 , respectively. Further, interpolation recovery processes are performed by the interpolation processing units  805  through  808  for every frame, assuming that the frame immediately preceding the current frame is lost.  
         [0046]    In the interpolation processing unit  805  shown in FIG. 8A, a linear interpolation process is performed for the LPC parameters by using the values of the frame before the last and the values of the frame of the present. As for the index and gain of the adaptive codebook and the index and gain of the fixed codebook, values of the fourth subframe of the frame before the last are used without any change for all the four subframes.  
         [0047]    In the interpolation processing unit  806  in FIG. 8A, a linear interpolation process is performed on the LPC parameters in the same manner as in the interpolation processing unit  805 . As regards the index and gain of the adaptive codebook and the index and gain of the fixed codebook, values of the third subframe of the second last frame is used for a first subframe, and values of the fourth subframe of the second last frame is used for a second subframe, with values of the first subframe of the present frame being used for a third subframe, and values of the second subframe of the present frame being used for a fourth subframe.  
         [0048]    In the interpolation processing unit  807  shown in FIG. 8A, interpolation of the LPC parameters is performed by using the values of the second preceding frame and the values of the present frame based on the quadratic function interpolation. Other parameters are obtained in the same manner as performed by the interpolation processing unit  805 .  
         [0049]    In the interpolation processing unit  808 , the LPC parameter interpolation is performed by using the values of the second preceding frame and the values of the present frame by the quadratic function interpolation. Other parameters are obtained in the same manner as performed by the interpolation processing unit  806 . The local decoding units  809 ,  810 ,  811  and  812  carry out local decoding by using the four parameters obtained from the interpolation process as described above. Further, an output of the local decoding using encoded parameters of the frame immediately preceding the present frame is compared with the outputs of the local decoding units  809 ,  810 ,  811  and  812  by the S/N calculation comparison unit  813 , thereby obtaining S/N values. An interpolation method that provides the largest S/N value is selected, an index number of which is multiplexed with the CELP encoded parameters by the multiplexing unit  814 . The multiplexed signal is provided to the packet assembly unit  203 .  
         [0050]    For example, indices  00 ,  01 ,  10  and  11  are assigned to the processes of the interpolation processing units  805 ,  806 ,  807  and  808 , respectively. If the interpolation processing unit  807  provides the highest S/N value of the four, for example, the index number  10  is multiplexed.  
         [0051]    The processes described above may be implemented as a firmware process of a DSP (Digital Signal Processor).  
         [0052]    [0052]FIG. 8B shows a structure of a decoder. The voice decoding unit  204  includes a packet disassembly unit  821 , a frame buffer  822 , an interpolation processing unit  823 , a selector  824  and a CELP decoder  825 . The received encoded parameter is disassembled by the packet disassembly unit  821 , and, then, is stored in the frame buffer  822 , which has a storage capacity for two frames. If frame loss is reported by a received packet loss index, the interpolation processing unit  823  performs an interpolation recovery process of the most appropriate interpolation process indicated by the index number.  
         [0053]    [0053]FIG. 11 shows a third embodiment of the present invention, in which examples of the voice encoding unit  202  and the packet assembly unit  203  are shown. The voice encoding unit  202  includes a voice encoding means  1001  and a vowel/consonant detection unit  1002 . Input voice is encoded by the voice encoding unit  1001  while the presence or absence of consonants is checked by the vowel/consonant detection unit  1002  for each frame. If an interval that contains a consonant is detected, the detection result is provided to the packet assembly unit  203  together with the encoded parameters. If the frame contains a consonant interval, the packet assembly unit  203  transmits the same frame a number of times with the same sequence number attached thereto until the time comes for the next frame to be processed. This is done while monitoring occupancy of the packet transmission buffer.  
         [0054]    [0054]FIG. 12 is an illustrative drawing showing processes of the third embodiment of the present invention. In FIG. 12, (A) indicates input voice signal frames  1101 ,  1102  and  1103 . (B) indicates process intervals  1111  through  1116 . (C) indicates output packets  1121  through  1125 . (D) shows packets  1121  through  1125  that are received on the receiver side in the case that a packet containing a consonant is lost, and also shows their respective decoded voice outputs  1131 ,  1132  and  1133 .  
         [0055]    On the transmission side, the input voice frames as shown in (A) of FIG. 12 are encoded by the voice encoding unit  1001  during the process intervals  1111 ,  1112 , and  1113 , as shown in (B). During the process intervals  1114 ,  1115 , and  1116 , further, the consonant detection unit  1002  checks whether a consonant interval is included in these frames. For example, if the frame  1102  is found to contain a consonant interval, the packet assembly unit  203  transmits the same frame a number of times with a same sequence number attached thereto as exemplified by the frames  1122 ,  1123  and  1124 . This is done while monitoring occupancy of the packet transmission buffer until the next frame  1103  is processed.  
         [0056]    The receiving side expects to receive the next packet  1122  within a certain time period from the receiving of the packet  1121 . If the next packet  1122  is not received at an anticipated timing, packet loss is suspected, so that the receiving side waits for a subsequent packet during the time period in which the same frame having the same sequence number is transmitted a number of times. If the packet  1123  with the same sequence number attached thereto is received during this time period, the frame  1132  is decoded from this received packet.  
         [0057]    A fourth embodiment of the present invention will be described hereafter. FIG. 13 is a block diagram of the fourth embodiment of the present invention. FIG. 13 shows a structure of the transmission side which principally includes the voice encoding unit  204  and the packet assembly unit  203 . The voice encoder unit  204  further includes a CELP encoding unit  1201 , a zero crossing number detection unit  1202 , a log level detection unit  1203 , a first-order autocorrelation detection unit  1204  and a consonant interval detection unit  1205 . FIGS. 14A through 14C show examples of distributions of a zero crossing number Z, a log level L, and a first-order autocorrelation value R, respectively. In the present embodiment, consonant intervals are detected by the consonant interval detection unit  1205  for each subframe of a target frame. The consonant interval detection is performed by calculating the zero crossing number-Z, the log level L, and the first-order autocorrelation value R for each of the subframes. The obtained values are then compared with predetermined threshold values Thz, Thl, and Thr of the zero crossing number, the log level, and the first-order autocorrelation value, respectively. If three conditions Z&gt;Thz, L&lt;Thl, and R&gt;Thr are satisfied, then, the subframe is determined to be that of a consonant interval. Further, if a frame includes at least one consonant interval, then, the frame is determined to be a consonant frame. A method to determine each of the vowel, consonant and silent intervals is described in, for example, “A Pattern Recognition Approach to Voiced-Unvoiced-Silence Classification with Application of Speech Recognition”, IEEE Transaction on ASSP, ASSP-24, No.3, July 1976, pp. 201-212. The present embodiment employs a method based on the properties shown in FIGS. 2, 3 and  4  of this paper.  
         [0058]    [0058]FIG. 15 is a block diagram of the receiving end. The receiving end includes a frame buffer  1211 , a packet disassembly unit  1212  and a CELP decoding unit  1213 . As a precaution against packet loss, the frame buffer  1211  waits for an arrival of a packet during a time period in which the same packet is transmitted a number of times with the same sequence number attached thereto. When the packet having the same sequence number as a lost packet attached thereto is received, frame decoding is performed based on the received packet. The entire process in FIG. 15 may be implemented by using a firmware process of a DSP (Digital Signal Processor).  
         [0059]    Further, the present invention is not limited to these embodiments, but various variations and modifications may be made without departing from the scope of the present invention.  
         [0060]    The present application is based on Japanese priority application No. 2000-361874 filed on Nov. 28, 2000, with the Japanese Patent Office, the entire contents of which are hereby incorporated by reference.