Patent Description:
In the transmission of audio and acoustic signals (which are collectively referred to hereinafter as "audio signal") through an IP network or a mobile communication network, the audio signal is encoded into audio packets at regular time intervals and transmitted through a communication network. At the receiving end, the audio packets are received through the communication network and decoded into a decoded audio signal by server, a MCU (Multipoint Control Unit), a terminal or the like.

The audio signal is generally collected in digital format. Specifically, it is measured and accumulated as a sequence of numerals whose number is the same as a sampling frequency per second. Each element of the sequence is called a "sample". In audio encoding, each time a predetermined number of samples of an audio signal is accumulated in a built-in buffer, the audio signal in the buffer is encoded. The above-described specified number of samples is called a "frame length", and a set of the same number of samples as the frame length is called "frame". For example, at the sampling frequency of <NUM>, when the frame length is <NUM>, the frame length is <NUM> samples. Note that the length of the buffer may be more than one frame.

When transmitting audio packets through a communication network, a phenomenon (so-called "packet loss") can occur where some of the audio packets are lost, or an error can occur in part of information written in the audio packets due to congestion in the communication network or the like. In such a case, the audio packets cannot be correctly decoded at the receiving end, and therefore a desired decoded audio signal cannot be obtained. Further, the decoded audio signal corresponding to the audio packet where packet loss has occurred is detected as noise, which significantly degrades the subjective quality to a person who listens to the audio.

In order to overcome the above-described inconveniences, packet loss concealment technology is used as a way to interpolate a part of the audio/acoustic signal that is lost by packet loss. There are two types of packet loss concealment technology: "packet loss concealment technology without using side information" where packet loss concealment is performed only at the receiving end and "packet loss concealment technology using side information" where parameters that help packet loss concealment are obtained at the transmitting end and transmitted to the receiving end, where packet loss concealment is performed using the received parameters at the receiving end.

The "packet loss concealment technology without using side information" generates an audio signal corresponding to a part where packet loss has occurred by copying a decoded audio signal contained in a packet that has been correctly received in the past on a pitch-by-pitch basis and then multiplying it by a predetermined attenuation coefficient as described in Non Patent Literature <NUM>, for example. Because the "packet loss concealment technology without using side information" is based on the assumption that the properties of the part of the audio where packet loss has occurred are similar to those of the audio immediately before the occurrence of loss, the concealment effect cannot be sufficiently obtained when the part of the audio where packet loss has occurred has different properties from the audio immediately before the occurrence of loss or when there is a sudden change in power.

On the other hand, the "packet loss concealment technology using side information" includes a technique that encodes parameters required for packet loss concealment at the transmitting end and transmits them for use in packet loss concealment at the receiving end as described in Patent Literature <NUM>. In Patent Literature <NUM>, the audio is encoded by two encoding methods: main encoding and redundant encoding. The redundant encoding encodes the frame immediately before the frame to be encoded by the main encoding at a lower bit rate than the main encoding (see <FIG>). For example, the Nth packet contains an audio code obtained by encoding the Nth frame by major encoding and a side information code obtained by encoding the (N-<NUM>)th frame by redundant encoding.

The receiving end waits for the arrival of two or more temporally successive packets and then decodes the temporally earlier packet and obtains a decoded audio signal. For example, to obtain a signal corresponding to the Nth frame, the receiving end waits for the arrival of the (N+<NUM>)th packet and then performs decoding. In the case where the Nth packet and the (N+<NUM>)th packet are correctly received, the audio signal of the Nth frame is obtained by decoding the audio code contained in the Nth packet (see <FIG>). On the other hand, in the case where packet loss has occurred (when the (N+<NUM>)th packet is obtained in the condition where the Nth packet is lost), the audio signal of the Nth frame can be obtained by decoding the side information code contained in the (N+<NUM>)th packet (see <FIG>).

According to the method of Patent Literature <NUM>, after a packet to be decoded arrives, it is necessary to wait to perform decoding until one or more packet arrives, and algorithmic delay increases by one packet or more. Accordingly, in the method of Patent Literature <NUM>, although the audio quality can be improved by packet loss concealment, the algorithmic delay increases to cause the degradation of the voice communication quality.

Further, in the case of applying the above-described packet loss concealment technology to CELP (Code Excited Linear Prediction) encoding, another problem caused by the characteristics of the operation of CELP arises. Because CELP is an audio model based on linear prediction and is able to encode an audio signal with high accuracy and with a high compression ratio, it is used in many international standards.

In CELP, an audio signal is synthesized by filtering an excitation signal e(n) using an all-pole synthesis filter. Specifically, an audio signal s(n) is synthesized according to the following equation: <MAT> where a(i) is a linear prediction coefficient (LP coefficient), and a value such as P=<NUM>, for example, is used as a degree.

The excitation signal is accumulated in a buffer called an adaptive codebook. When synthesizing the audio for a new frame, an excitation signal is newly generated by adding an adaptive codebook vector read from the adaptive codebook and a fixed codebook vector representing a change in excitation signal over time based on position information called a pitch lag. The newly generated excitation signal is accumulated in the adaptive codebook and is also filtered by the all-pole synthesis filter, and thereby a decoded signal is synthesized.

In CELP, an LP coefficient is calculated for all frames. In the calculation of the LP coefficient, a look-ahead signal of about <NUM> is required. Specifically, in addition to a frame to be encoded, a look-ahead signal is accumulated in the buffer, and then the LP coefficient calculation and the subsequent processing are performed (see <FIG>). Each frame is divided into about four sub-frames, and processing such as the above-described pitch lag calculation, adaptive codebook vector calculation, fixed codebook vector calculation and adaptive codebook update are performed in each sub-frame. In the processing of each sub-frame, the LP coefficient is also interpolated so that the coefficient varies from sub-frame to sub-frame. Further, for quantization and interpolation, the LP coefficient is encoded after being converted into an ISP (Immittance Spectral Pair) parameter and an ISF (Immittance Spectral Frequency) parameter, which are equivalent representation(s) of the LP coefficient(s). A procedure for the interconversion of the LP coefficient(s) and the ISP parameter and the ISF parameter is described in Non Patent Literature <NUM>.

In CELP encoding, encoding and decoding are performed based on the assumption that both of the encoding end and the decoding end have adaptive codebooks, and those adaptive codebooks are always synchronized with each other. Although the adaptive codebook at the encoding end and the adaptive codebook at the decoding end are synchronized under conditions where packets are correctly received and decoding correctly, once packet loss has occurred, the synchronization of the adaptive codebooks cannot be achieved.

For example, if a value that is used as a pitch lag is different between the encoding end and the decoding end, a time lag occurs between the adaptive codebook vectors. Because the adaptive codebook is updated with those adaptive codebook vectors, even if the next frame is correctly received, the adaptive codebook vector calculated at the encoding end and the adaptive codebook vector calculated at the decoding end do not coincide, and the synchronization of the adaptive codebooks is not recovered. Due to such inconsistency of the adaptive codebooks, the degradation of the audio quality occurs for several frames after the frame where packet loss has happened.

In the packet loss concealment in CELP encoding, a more advanced technique is described in Patent Literature <NUM>. According to Patent Literature <NUM>, an index of a transition mode codebook is transmitted instead of a pitch lag or an adaptive codebook gain in a specific frame that is largely affected by packet loss. The technique of Patent Literature <NUM> focuses attentions on a transition frame (transition from a silent audio segment to a sound audio segment, or transition between two vowels) as the frame that is largely affected by packet loss. By generating an excitation signal using the transition mode codebook in this transition frame, it is possible to generate an excitation signal that is not dependent on the past adaptive codebook and thereby recover from the inconsistency of the adaptive codebooks due to the past packet loss.

However, because the method of Patent Literature <NUM> does not use the transition frame codebook in a frame where a long vowel continues, for example, it is not possible to recover from the inconsistency of the adaptive codebooks in such a frame. Further, in the case where the packet containing the transition frame codebook is lost, packet loss affects the frames after the loss. This is the same when the next packet after the packet containing the transition frame codebook is lost.

Although it is feasible to apply a codebook to all frames that is not dependent on the past frames, such as the transition frame codebook, because the encoding efficiency is significantly degraded, it is not possible to achieve a low bit rate and high audio quality under these circumstances. <CIT> describes that, when the vanished frame of a voice signal is caused in a voice decoding device, the compensation processing of the vanished frame is performed on the basis of pitch lug information for the compensation processing of varnished frame by computing a pitch lug for the compensation processing of frame vanishing in the voice coding device and by transmitting information expressing the pitch lug to the voice decoding device together with the coded voice signal. <CIT> describes saving valuable bitrate with maintaining the noise generation quality within inactive phases, by continuously updating the parametric background noise estimate during an active phase so that the noise generation may immediately be started with upon the entrance of an inactive phase following the active phase.

With use of the method of Patent Literature <NUM>, after the arrival of a packet to be decoded, decoding is not started before the arrival of the next packet. Therefore, although the audio quality is improved by packet loss concealment, the algorithmic delay increases, which causes the degradation of the voice communication quality.

In the event of packet loss in CELP encoding, the degradation of the audio quality occurs due to the inconsistency of the adaptive codebooks between the encoding unit and the decoding unit. Although the method as described in Patent Literature <NUM> can allow for recovery from the inconsistency of the adaptive codebooks, the method is not sufficient to allow recovery when a frame different from the frame immediately before the transition frame is lost.

The present invention has been accomplished to solve the above problems and an object of the present invention is thus to provide an audio encoding device, an audio encoding method, an audio encoding program, an audio decoding device, an audio decoding method, and an audio decoding program that recover audio quality without increasing algorithmic delay in the event of packet loss in audio encoding.

To achieve the above object, an audio coding device for encoding an audio signal according to one aspect of the present invention has the features of claim <NUM>.

The side information is related to a pitch lag in a look-ahead signal and may be related to a pitch gain in a look-ahead signal, or related to a pitch lag and a pitch gain in a look-ahead signal. Further, the side information may contain information related to availability of the side information.

The side information encoding unit calculates side information for a look-ahead signal part and may encode the side information, and also generate a concealment signal, and the audio encoding device may further include an error signal encoding unit configured to encode an error signal between an input audio signal and a concealment signal output from the side information encoding unit, and a main encoding unit configured to encode an input audio signal.

It is possible to recover audio quality without increasing algorithmic delay in the event of packet loss in audio encoding. Particularly, in CELP encoding, it is possible to reduce degradation of an adaptive codebook that occurs when packet loss happens and thereby improve audio quality in the event of packet loss.

Embodiments of the present invention are described hereinafter with reference to the attached drawings. Note that, where possible, the same elements are denoted by the same reference numerals and redundant description thereof is omitted.

An embodiment of the present invention relates to an encoder and a decoder that implement "packet loss concealment technology using side information" that encodes and transmits side information calculated on the encoder side for use in packet loss concealment on the decoder side.

In the embodiments of the present invention, the side information that is used for packet loss concealment is contained in a previous packet. <FIG> shows a temporal relationship between an audio code and a side information code contained in a packet. As illustrated in <FIG>, the side information in the embodiments of the present invention is parameters (pitch lag, adaptive codebook gain, etc.) that are calculated for a look-ahead signal in CELP encoding.

Because the side information is contained in a previous packet, it is possible to perform decoding without waiting for a packet that arrives after a packet to be decoded. Further, when packet loss is detected, because the side information for a frame to be concealed is obtained from the previous packet, it is possible to implement highly accurate packet loss concealment without waiting for the next packet.

In addition, by transmitting parameters for CELP encoding in a look-ahead signal as the side information, it is possible to reduce the inconsistency of adaptive codebooks even in the event of packet loss.

The embodiments of the present invention can be composed of an audio signal transmitting device (audio encoding device) and an audio signal receiving device (audio decoding device). A functional configuration example of an audio signal transmitting device is shown in <FIG>, and an example procedure of the same is shown in <FIG>. Further, a functional configuration example of an audio signal receiving device is shown in <FIG>, and an example procedure of the same is shown in <FIG>.

As shown in <FIG>, the audio signal transmitting device includes an audio encoding unit <NUM> and a side information encoding unit <NUM>. As shown in <FIG>, the audio signal receiving device includes an audio code buffer <NUM>, an audio parameter decoding unit <NUM>, an audio parameter missing processing unit <NUM>, an audio synthesis unit <NUM>, a side information decoding unit <NUM>, and a side information accumulation unit <NUM>.

The audio signal transmitting device encodes an audio signal for each frame and can transmit the audio signal by the example procedure shown in <FIG>.

The audio encoding unit <NUM> can calculate audio parameters for a frame to be encoded and output an audio code (Step S131 in <FIG>).

The side information encoding unit <NUM> can calculate audio parameters for a look-ahead signal and output a side information code (Step S132 in <FIG>).

It is determined whether the audio signal ends, and the above steps can be repeated until the audio signal ends (Step S133 in <FIG>).

The audio signal receiving device decodes a received audio packet and outputs an audio signal by the example procedure shown in <FIG>.

The audio code buffer <NUM> waits for the arrival of an audio packet and accumulates an audio code. When the audio packet has correctly arrived, the processing is switched to the audio parameter decoding unit <NUM>. On the other hand, when the audio packet has not correctly arrived, the processing is switched to the audio parameter missing processing unit <NUM> (Step S141 in <FIG>).

The audio parameter decoding unit <NUM> decodes the audio code and outputs audio parameters (Step S142 in <FIG>).

The side information decoding unit <NUM> decodes the side information code and outputs side information. The outputted side information is sent to the side information accumulation unit <NUM> (Step S143 in <FIG>).

The audio synthesis unit <NUM> synthesizes an audio signal from the audio parameters output from the audio parameter decoding unit <NUM> and outputs the synthesized audio signal (Step S144 in <FIG>).

The audio parameter missing processing unit <NUM> accumulates the audio parameters output from the audio parameter decoding unit <NUM> in preparation for packet loss (Step S145 in <FIG>).

The audio code buffer <NUM> determines whether the transmission of audio packets has ended, and when the transmission of audio packets has ended, stops the processing. While the transmission of audio packets continues, the above Steps S141 to S146 are repeated (Step S147 in <FIG>).

The audio parameter missing processing unit <NUM> reads the side information from the side information accumulation unit <NUM> and carries out prediction for the parameter(s) not contained in the side information and thereby outputs the audio parameters (Step S146 in <FIG>).

The audio synthesis unit <NUM> synthesizes an audio signal from the audio parameters output from the audio parameter missing processing unit <NUM> and outputs the synthesized audio signal (Step S144 in <FIG>).

The audio parameter missing processing unit <NUM> accumulates the audio parameters output from the audio parameter missing processing unit <NUM> in preparation for packet loss (Step S145 in <FIG>).

In this example of a case where a pitch lag is transmitted as the side information, the pitch lag can be used for generation of a packet loss concealment signal at the decoding end.

The functional configuration example of the audio signal transmitting device is shown in <FIG>, and the functional configuration example of the audio signal receiving device is shown in <FIG>. An example of the procedure of the audio signal transmitting device is shown in <FIG>, and an example of the procedure of the audio signal receiving device is shown in <FIG>.

In the audio signal transmitting device, an input audio signal is sent to the audio encoding unit <NUM>.

The audio encoding unit <NUM> encodes a frame to be encoded by CELP encoding (Step <NUM> in <FIG>). For the details of CELP encoding, the method described in Non Patent Literature <NUM> is used, for example. The details of the procedure of CELP encoding are omitted. Note that, in the CELP encoding, local decoding is performed at the encoding end. The local decoding is to decode an audio code also at the encoding end and obtain parameters (ISP parameter and corresponding ISF parameter, pitch lag, long-term prediction parameter, adaptive codebook, adaptive codebook gain, fixed codebook gain, fixed codebook vector, etc.) required for audio synthesis. The parameters obtained by the local decoding include: at least one or both of the ISP parameter and the ISF parameter, the pitch lag, and the adaptive codebook, which are sent to the side information encoding unit <NUM>. In the case where the audio encoding as described in Non Patent Literature <NUM> is used in the audio encoding unit <NUM>, an index representing the characteristics of a frame to be encoded may also be sent to the side information encoding unit <NUM>. In embodiments, encoding different from CELP encoding may be used in the audio encoding unit <NUM>. In embodiments using different encoding, at least one or both of the ISP parameter and the ISF parameter, the pitch lag, and the adaptive codebook can be separately calculated from an input signal, or a decoded signal obtained by the local decoding, and sent to the side information encoding unit <NUM>.

The side information encoding unit <NUM> calculates a side information code using the parameters calculated by the audio encoding unit <NUM> and the look-ahead signal (Step <NUM> in <FIG>). As shown in the example of <FIG>, the side information encoding unit <NUM> includes an LP coefficient calculation unit <NUM>, a target signal calculation unit <NUM>, a pitch lag calculation unit <NUM>, an adaptive codebook calculation unit <NUM>, an excitation vector synthesis unit <NUM>, an adaptive codebook buffer <NUM>, a synthesis filter <NUM>, and a pitch lag encoding unit <NUM>. An example procedure in the side information encoding unit is shown in <FIG>.

The LP coefficient calculation unit <NUM> calculates an LP coefficient using the ISF parameter calculated by the audio encoding unit <NUM> and the ISF parameter calculated in the past several frames (Step <NUM> in <FIG>). The procedure of the LP coefficient calculation unit <NUM> is shown in <FIG>.

First, the buffer is updated using the ISF parameter obtained from the audio encoding unit <NUM> (Step <NUM> in <FIG>). Next, the ISF parameter ω̇i in the look-ahead signal is calculated. The ISF parameter ω̇i is calculated by the following equation (Step <NUM> in <FIG>). <MAT> <MAT> where <MAT> is the ISF parameter, stored in the buffer, which is for the frame preceding by j-number of frames. Further, <MAT> is the ISF parameter during the speech period that is calculated in advance by learning or the like. β is a constant, and it may be a value such as <NUM>, for example, though not limited thereto. Further, α is also constant, and it may be a value such as <NUM>, for example, though not limited thereto. <MAT>, α and β may be varied by the index representing the characteristics of the frame to be encoded as in the ISF concealment described in Non Patent Literature <NUM>, for example.

In addition, the values of i are arranged so that ω̇i satisfies <NUM><ω̇<NUM> < ω̇<NUM> <. ω̇<NUM>, and the values of ω̇i can be adjusted so that the adjacent ω̇i is not too close. As a procedure to adjust the value of ω̇i, Non Patent Literature <NUM> (Equation <NUM>) may be used, for example (Step <NUM> in <FIG>).

After that, the ISF parameter ω̇i is converted into an ISP parameter and interpolation can be performed for each sub-frame. As a method of calculating the ISP parameter from the ISF parameter, the method described in the section <NUM>. <NUM> in Non Patent Literature <NUM> may be used, and as a method of interpolation, the procedure described in the section <NUM>. <NUM> in Non Patent Literature <NUM> may be used (Step <NUM> in <FIG>).

Then, the ISP parameter for each sub-frame is converted into an LP coefficient <MAT>. The number of sub-frames contained in the look-ahead signal is M1a. For the conversion from the ISP parameter to the LP coefficient, the procedure described in the section <NUM>. <NUM> in Non Patent Literature <NUM> may be used (Step <NUM> in <FIG>).

The target signal calculation unit <NUM> calculates a target signal x(n) and an impulse response h(n) by using the LP coefficient <MAT> (Step <NUM> in <FIG>). As described in the section <NUM>. <NUM> in Non Patent Literature <NUM>, the target signal is obtained by applying an perceptual weighting filter to a linear prediction residual signal (<FIG>).

First, a residual signal r(n) of the look-ahead signal <MAT> is calculated using the LP coefficient according to the following equation (Step <NUM> in <FIG>).

Note that L' indicates the number of samples of a sub-frame, and L indicates the number of samples of a frame to be encoded spre(n)(<NUM>≤n<L). Then, <MAT> is satisfied.

In addition, the target signal x(n)(<NUM>≤n<L') is calculated by the following equations (Step <NUM> in <FIG>). <MAT> <MAT> <MAT> <MAT> where an perceptual weighting filter y=<NUM>. The value of the perceptual weighting filter may be a different value according to the design policy of audio encoding.

Then, the impulse response h(n)(<NUM>≤n<L') is calculated by the following equations (Step <NUM> in <FIG>). <MAT> <MAT>.

The pitch lag calculation unit <NUM> calculates a pitch lag for each sub-frame by calculating k that maximizes the following equation (Step <NUM> in <FIG>). Note that, in order to reduce the amount of calculations, the above-described target signal calculation (Step <NUM> in <FIG>) and the impulse response calculation (Step <NUM> in <FIG>) may be omitted, and the residual signal may be used as the target signal. <MAT> <MAT> <MAT> <MAT>.

Note that yk(n) is obtained by convoluting the impulse response with the linear prediction residual. Int(i) indicates an interpolation filter. The details of the interpolation filter are as described in the section <NUM>. <NUM> in Non Patent Literature <NUM>. As a matter of course, v'(n)=u(n+Nadapt-Tp+i) may be employed without using the interpolation filter.

Although the pitch lag can be calculated as an integer by the above-described calculation method, the accuracy of the pitch lag may be increased to after the decimal point accuracy by interpolating the above Tk. For the details of the procedure to calculate the pitch lag after the decimal point by interpolation, the processing method described in the section <NUM>. <NUM> in Non Patent Literature <NUM> may be used.

The adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector v'(n) and a long-term prediction parameter from the pitch lag Tp and the adaptive codebook u(n) stored in the adaptive codebook buffer <NUM> according to the following equation (Step <NUM> in <FIG>).

For the details of the procedure to calculate the long-term parameter, the method described in the section <NUM> in Non Patent Literature <NUM> may be used.

The excitation vector synthesis unit <NUM> multiplies the adaptive codebook vector v'(n) by a predetermined adaptive codebook gain <MAT> and outputs an excitation signal vector according to the following equation (Step <NUM> in <FIG>).

Although the value of the adaptive codebook gain <MAT> may be <NUM> or the like, for example, a value obtained in advance by learning may be used, or it may be varied by the index representing the characteristics of the frame to be encoded.

Then, the state of the adaptive codebook u(n) stored in the adaptive codebook buffer <NUM> is updated by the excitation signal vector according to the following equations (Step <NUM> in <FIG>). <MAT> <MAT>.

The synthesis filter <NUM> synthesizes a decoded signal according to the following equation by linear prediction inverse filtering using the excitation signal vector as an excitation source (Step <NUM> in <FIG>).

The above-described Steps <NUM> to <NUM> in <FIG> are repeated for each sub-frame until the end of the look-ahead signal (Step <NUM> in <FIG>).

The pitch lag encoding unit <NUM> encodes the pitch lag <MAT> that is calculated in the look-ahead signal (Step <NUM> in <FIG>). The number of sub-frames contained in the look-ahead signal is M1a.

Encoding may be performed by a method such as one of the following methods, for example, although any method may be used for encoding.

For scalar quantization and vector quantization, a codebook determined empirically or a codebook calculated in advance by learning may be used. Further, a method that performs encoding after adding an offset value to the above pitch lag may also be included in the scope of the embodiment of the present invention as a matter of course.

As shown in <FIG>, an example of the audio signal receiving device includes the audio code buffer <NUM>, the audio parameter decoding unit <NUM>, the audio parameter missing processing unit <NUM>, the audio synthesis unit <NUM>, the side information decoding unit <NUM>, and the side information accumulation unit <NUM>. The procedure of the audio signal receiving device is as shown in the example of <FIG>.

The audio code buffer <NUM> determines whether a packet is correctly received or not. When the audio code buffer <NUM> determines that a packet is correctly received, the processing is switched to the audio parameter decoding unit <NUM> and the side information decoding unit <NUM>. On the other hand, when the audio code buffer <NUM> determines that a packet is not correctly received, the processing is switched to the audio parameter missing processing unit <NUM> (Step <NUM> in <FIG>).

The audio parameter decoding unit <NUM> decodes the received audio code and calculates audio parameters required to synthesize the audio for the frame to be encoded (ISP parameter and corresponding ISF parameter, pitch lag, long-term prediction parameter, adaptive codebook, adaptive codebook gain, fixed codebook gain, fixed codebook vector etc.) (Step <NUM> in <FIG>).

The side information decoding unit <NUM> decodes the side information code, calculates a pitch lag <MAT> and stores it in the side information accumulation unit <NUM>. The side information decoding unit <NUM> decodes the side information code by using the decoding method corresponding to the encoding method used at the encoding end (Step <NUM> in <FIG>).

The audio synthesis unit <NUM> synthesizes the audio signal corresponding to the frame to be encoded based on the parameters output from the audio parameter decoding unit <NUM> (Step <NUM> in <FIG>). The functional configuration example of the audio synthesis unit <NUM> is shown in <FIG>, and an example procedure of the audio synthesis unit <NUM> is shown in <FIG>. Note that, although the audio parameter missing processing unit <NUM> is illustrated to show the flow of the signal, the audio parameter missing processing unit <NUM> is not included in the functional configuration of the audio synthesis unit <NUM>.

An LP coefficient calculation unit <NUM> converts an ISF parameter into an ISP parameter and then performs interpolation processing, and thereby obtains an ISP coefficient for each sub-frame. The LP coefficient calculation unit <NUM> then converts the ISP coefficient into a linear prediction coefficient (LP coefficient) and thereby obtains an LP coefficient for each sub-frame (Step <NUM> in <FIG>). For the interpolation of the ISP coefficient and the ISP-LP coefficient, the method described in, for example, section <NUM>. <NUM> in Non Patent Literature <NUM> may be used. The procedure of parameter conversion is not the essential part of the embodiment of the present invention and thus not described in detail.

An adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector by using the pitch lag, a long-term prediction parameter and an adaptive codebook <NUM> (Step <NUM> in <FIG>). An adaptive codebook vector v'(n) is calculated from the pitch lag <MAT> and the adaptive codebook u(n) according to the following equation.

The adaptive codebook vector is calculated by interpolating the adaptive codebook u(n) using FIR filter Int(i). The length of the adaptive codebook is Nadapt. The filter Int(i) that is used for the interpolation is the same as the interpolation filter of <MAT>.

This is the FIR filter with a predetermined length <NUM>+<NUM>. L' is the number of samples of the sub-frame. It is not necessary to use a filter for the interpolation, whereas at the encoder end a filter is used for the interpolation.

The adaptive codebook calculation unit <NUM> carries out filtering on the adaptive codebook vector according to the value of the long-term prediction parameter (Step <NUM> in <FIG>). When the long-term prediction parameter has a value indicating the activation of filtering, filtering is performed on the adaptive codebook vector by the following equation.

On the other hand, when the long-term prediction parameter has a value indicating no filtering is needed, filtering is not performed, and v(n)=v'(n) is established.

An excitation vector synthesis unit <NUM> multiplies the adaptive codebook vector by an adaptive codebook gain gp (Step <NUM> in <FIG>). Further, the excitation vector synthesis unit <NUM> multiplies a fixed codebook vector c(n) by a fixed codebook gain gc (Step <NUM> in <FIG>). Furthermore, the excitation vector synthesis unit <NUM> adds the adaptive codebook vector and the fixed codebook vector together and outputs an excitation signal vector (Step <NUM> in <FIG>).

A post filter <NUM> performs post processing such as pitch enhancement, noise enhancement and low-frequency enhancement, for example, on the excitation signal vector. The details of techniques such as pitch enhancement, noise enhancement and low-frequency enhancement are described in the section <NUM> in Non Patent Literature <NUM>. The processing in the post filter is not significantly related to the essential part of the embodiment of the present invention and thus not described in detail (Step <NUM> in <FIG>).

The adaptive codebook <NUM> updates the state by an excitation signal vector according to the following equations (Step <NUM> in <FIG>). <MAT> <MAT>.

A synthesis filter <NUM> synthesizes a decoded signal according to the following equation by linear prediction inverse filtering using the excitation signal vector as an excitation source (Step <NUM> in <FIG>).

An perceptual weighting inverse filter <NUM> applies an perceptual weighting inverse filter to the decoded signal according to the following equation (Step <NUM> in <FIG>).

The value of β is typically <NUM> or the like, though not limited to this value.

The audio parameter missing processing unit <NUM> stores the audio parameters (ISF parameter, pitch lag, adaptive codebook gain, fixed codebook gain) used in the audio synthesis unit <NUM> into the buffer (Step <NUM> in <FIG>).

The audio parameter missing processing unit <NUM> reads a pitch lag <MAT> from the side information accumulation unit <NUM> and predicts audio parameters. The functional configuration example of the audio parameter missing processing unit <NUM> is shown in the example of <FIG>, and an example procedure of audio parameter prediction is shown in <FIG>.

An ISF prediction unit <NUM> calculates an ISF parameter using the ISF parameter for the previous frame and the ISF parameter calculated for the past several frames (Step <NUM> in <FIG>). The procedure of the ISF prediction unit <NUM> is shown in <FIG>.

First, the buffer is updated using the ISF parameter of the immediately previous frame (Step <NUM> in <FIG>). Next, the ISF parameter ω̇i is calculated according to the following equation (Step <NUM> in <FIG>). <MAT> <MAT> where <MAT> is the ISF parameter, stored in the buffer, which is for the frame preceding by j-number of frames. Further, <MAT>, α and β are the same values as those used at the encoding end.

In addition, the values of i are arranged so that ω̇i satisfies <NUM><ω̇<NUM><ω̇<NUM><. ω̇<NUM>, and values of ω̇i are adjusted so that the adjacent ω̇i is not too close. As an example procedure to adjust the value of ω̇i, Non Patent Literature <NUM> (Equation <NUM>) may be used (Step <NUM> in <FIG>).

A pitch lag prediction unit <NUM> decodes the side information code from the side information accumulation unit <NUM> and thereby obtains a pitch lag <MAT>. Further, by using a pitch lag <MAT> used for the past decoding, the pitch lag prediction unit <NUM> outputs a pitch lag <MAT>. The number of sub-frames contained in one frame is M, and the number of pitch lags contained in the side information is M1a. For the prediction of the pitch lag <MAT>, the procedure described in, for example, section <NUM>. <NUM> in Non Patent Literature <NUM> may be used (Step <NUM> in <FIG>).

An adaptive codebook gain prediction unit <NUM> outputs an adaptive codebook gain <MAT> by using a predetermined adaptive codebook gain <MAT> and an adaptive codebook gain <MAT> used in the past decoding. The number of sub-frames contained in one frame is M, and the number of pitch lags contained in the side information is M1a. For the prediction of the adaptive codebook gain <MAT>, the procedure described in, for example, section <NUM>. <NUM> in Non Patent Literature <NUM> may be used (Step <NUM> in <FIG>).

A fixed codebook gain prediction unit <NUM> outputs a fixed codebook gain <MAT> by using a fixed codebook gain <MAT> used in the past decoding. The number of sub-frames contained in one frame is M. For the prediction of the fixed codebook gain <MAT>, the procedure described in the section <NUM>. <NUM> in Non Patent Literature <NUM> may be used, for example (Step <NUM> in <FIG>).

A noise signal generation unit <NUM> outputs a noise vector, such as a white noise, with a length of L (Step <NUM> in <FIG>). The length of one frame is L.

The audio synthesis unit <NUM> synthesizes a decoded signal based on the audio parameters output from the audio parameter missing processing unit <NUM> (Step <NUM> in <FIG>). The operation of the audio synthesis unit <NUM> is the same as the operation of the audio synthesis unit <When audio packet is correctly received> and not redundantly described in detail (Step <NUM> in <FIG>).

Although the case of encoding and transmitting the side information for all sub-frames contained in the look-ahead signal is described in the above example, the configuration that transmits only the side information for a specific sub-frame may be employed.

As an alternative example of the previously discussed example <NUM>, an example that adds a pitch gain to the side information is described hereinafter. A difference between the alternative example <NUM>-<NUM> and the example <NUM> is only the operation of the excitation vector synthesis unit <NUM>, and therefore description of the other parts is omitted.

The procedure of the excitation vector synthesis unit <NUM> is shown in the example of <FIG>.

An adaptive codebook gain <MAT> is calculated from the adaptive codebook vector v'(n) and the target signal x(n) according to the following equation (Step <NUM> in <FIG>). <MAT> where y(n) is a signal y(n)=v(n)*h(n) that is obtained by convoluting the impulse response with the adaptive codebook vector.

The calculated adaptive codebook gain is encoded and contained in the side information code (Step <NUM> in <FIG>). For the encoding, scalar quantization using a codebook obtained in advance by learning may be used, although any other technique may be used for the encoding.

By multiplying the adaptive codebook vector by an adaptive codebook gain ĝp obtained by decoding the code calculated in the encoding of the adaptive codebook gain, an excitation vector is calculated according to the following equation (Step <NUM> in <FIG>).

The excitation vector synthesis unit <NUM> multiplies the adaptive codebook vector v'(n) by an adaptive codebook gain ĝp obtained by decoding the side information code and outputs an excitation signal vector according to the following equation (Step <NUM> in <FIG>).

As an alternative example of the example <NUM>, an example that adds a flag for determination of use of the side information to the side information is described hereinafter.

The functional configuration example of the side information encoding unit is shown in <FIG>, and the procedure of the side information encoding unit is shown in the example of <FIG>. A difference from the example <NUM> is only a side information output determination unit <NUM> (Step <NUM> in <FIG>), and therefore description of the other parts is omitted.

The side information output determination unit <NUM> calculates segmental SNR of the decoded signal and the look-ahead signal according to the following equation, and only when segmental SNR exceeds a threshold, sets the value of the flag to ON and adds it to the side information.

On the other hand, when segmental SNR does not exceed a threshold, the side information output determination unit <NUM> sets the value of the flag to OFF and adds it to the side information (Step <NUM> in <FIG>). Note that, the amount of bits of the side information may be reduced by adding the side information such as a pitch lag and a pitch gain to the flag and transmitting the added side information only when the value of the flag is ON, and transmitting only the value of the flag when the value of the flag is OFF.

The side information decoding unit decodes the flag contained in the side information code. When the value of the flag is ON, the audio parameter missing processing unit calculates a decoded signal by the same procedure as in the example <NUM>. On the other hand, when the value of the flag is OFF, it calculates a decoded signal by the packet loss concealment technique without using side information (Step <NUM> in <FIG>).

In this example, the decoded audio of the look-ahead signal part is also used when a packet is correctly received. For purposes of this discussion, the number of sub-frames contained in one frame is M sub-frames, and the length of the look-ahead signal is M' sub-frame(s).

As shown in the example of <FIG>, the audio signal transmitting device includes a main encoding unit <NUM>, a side information encoding unit <NUM>, a concealment signal accumulation unit <NUM>, and an error signal encoding unit <NUM>. The procedure of the audio signal transmitting device is shown in <FIG>.

The error signal encoding unit <NUM> reads a concealment signal for one sub-frame from the concealment signal accumulation unit <NUM>, subtracts it from the audio signal and thereby calculates an error signal (Step <NUM> in <FIG>).

The error signal encoding unit <NUM> encodes the error signal. As a specific example procedure, AVQ described in the section <NUM>. <NUM> in Non Patent Literature <NUM>, can be used. In the encoding of the error signal, local decoding is performed, and a decoded error signal is output (Step <NUM> in <FIG>).

By adding the decoded error signal to the concealment signal, a decoded signal for one sub-frame is output (Step <NUM> in <FIG>).

The above Steps <NUM> to <NUM> are repeated for M' sub-frames until the end of the concealment signal.

An example functional configuration of the main encoding unit <NUM> is shown in <FIG>. The main encoding unit <NUM> includes an ISF encoding unit <NUM>, a target signal calculation unit <NUM>, a pitch lag calculation unit <NUM>, an adaptive codebook calculation unit <NUM>, a fixed codebook calculation unit <NUM>, a gain calculation unit <NUM>, an excitation vector calculation unit <NUM>, a synthesis filter <NUM>, and an adaptive codebook buffer <NUM>.

The ISF encoding unit <NUM> obtains an LP coefficient by applying the Levinson-Durbin method to the frame to be encoded and the look-ahead signal. The ISF encoding unit <NUM> then converts the LP coefficient into an ISF parameter and encodes the ISF parameter. The ISF encoding unit <NUM> then decodes the code and obtains a decoded ISF parameter. Finally, the ISF encoding unit <NUM> interpolates the decoded ISF parameter and obtains a decoded LP coefficient for each sub-frame. The procedures of the Levinson-Durbin method and the conversion from the LP coefficient to the ISF parameter are the same as in the example <NUM>. Further, for the encoding of the ISF parameter, the procedure described in, for example, section <NUM>. <NUM> in Non Patent Literature <NUM> can be used. An index obtained by encoding the ISF parameter, the decoded ISF parameter, and the decoded LP coefficient (which is obtained by converting the decoded ISF parameter into the LP coefficient) can be obtained by the ISF encoding unit <NUM> (Step <NUM> in <FIG>).

The detailed procedure of the target signal calculation unit <NUM> is the same as in Step <NUM> in <FIG> in the example <NUM> (Step <NUM> in <FIG>).

The pitch lag calculation unit <NUM> refers to the adaptive codebook buffer and calculates a pitch lag and a long-term prediction parameter by using the target signal. The detailed procedure of the calculation of the pitch lag and the long-term prediction parameter is the same as in the example <NUM> (Step <NUM> in <FIG>).

The adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector by using the pitch lag and the long-term prediction parameter calculated by the pitch lag calculation unit <NUM>. The detailed procedure of the adaptive codebook calculation unit <NUM> is the same as in the example <NUM> (Step <NUM> in <FIG>).

The fixed codebook calculation unit <NUM> calculates a fixed codebook vector and an index obtained by encoding the fixed codebook vector by using the target signal and the adaptive codebook vector. The detailed procedure is the same as the procedure of AVQ used in the error signal encoding unit <NUM> (Step <NUM> in <FIG>).

The gain calculation unit <NUM> calculates an adaptive codebook gain, a fixed codebook gain and an index obtained by encoding these two gains using the target signal, the adaptive codebook vector and the fixed codebook vector. A detailed procedure which can be used is described in, for example, section <NUM>. <NUM> in Non Patent Literature <NUM> (Step <NUM> in <FIG>).

The excitation vector calculation unit <NUM> calculates an excitation vector by adding the adaptive codebook vector and the fixed codebook vector to which the gain is applied. The detailed procedure is the same as in example <NUM>. Further, the excitation vector calculation unit <NUM> updates the state of the adaptive codebook buffer <NUM> by using the excitation vector. The detailed procedure is the same as in the example <NUM> (Step <NUM> in <FIG>).

The synthesis filter <NUM> synthesizes a decoded signal by using the decoded LP coefficient and the excitation vector (Step <NUM> in <FIG>).

The above Steps <NUM> to <NUM> are repeated for M-M' sub-frames until the end of the frame to be encoded.

The side information encoding unit <NUM> calculates the side information for the look-ahead signal M' sub-frame. A specific procedure is the same as in the example <NUM> (Step <NUM> in <FIG>).

In addition to the procedure of the example <NUM>, the decoded signal output by the synthesis filter <NUM> of the side information encoding unit <NUM> is accumulated in the concealment signal accumulation unit <NUM> in the example <NUM> (Step <NUM> in <FIG>).

As shown in <FIG>, an example of the audio signal receiving device includes an audio code buffer <NUM>, an audio parameter decoding unit <NUM>, an audio parameter missing processing unit <NUM>, an audio synthesis unit <NUM>, a side information decoding unit <NUM>, a side information accumulation unit <NUM>, an error signal decoding unit <NUM>, and a concealment signal accumulation unit <NUM>. An example procedure of the audio signal receiving device is shown in <FIG>. An example functional configuration of the audio synthesis unit <NUM> is shown in <FIG>.

The audio code buffer <NUM> determines whether a packet is correctly received or not. When the audio code buffer <NUM> determines that a packet is correctly received, the processing is switched to the audio parameter decoding unit <NUM>, the side information decoding unit <NUM> and the error signal decoding unit <NUM>. On the other hand, when the audio code buffer <NUM> determines that a packet is not correctly received, the processing is switched to the audio parameter missing processing unit <NUM> (Step <NUM> in <FIG>).

The error signal decoding unit <NUM> decodes an error signal code and obtains a decoded error signal. As a specific example procedure, a decoding method corresponding to the method used at the encoding end, such as AVQ described in the section <NUM>. <NUM> in Non Patent Literature 4can be used (Step <NUM> in <FIG>).

A look-ahead excitation vector synthesis unit <NUM> reads a concealment signal for one sub-frame from the concealment signal accumulation unit <NUM> and adds the concealment signal to the decoded error signal, and thereby outputs a decoded signal for one sub-frame (Step <NUM> in <FIG>).

The audio parameter decoding unit <NUM> includes an ISF decoding unit <NUM>, a pitch lag decoding unit <NUM>, a gain decoding unit <NUM>, and a fixed codebook decoding unit <NUM>. The functional configuration example of the audio parameter decoding unit <NUM> is shown in <FIG>.

The ISF decoding unit <NUM> decodes the ISF code and converts it into an LP coefficient and thereby obtains a decoded LP coefficient. For example, the procedure described in the section <NUM>. <NUM> in Non Patent Literature <NUM> is used (Step <NUM> in <FIG>).

The pitch lag decoding unit <NUM> decodes a pitch lag code and obtains a pitch lag and a long-term prediction parameter (Step <NUM> in <FIG>).

The gain decoding unit <NUM> decodes a gain code and obtains an adaptive codebook gain and a fixed codebook gain. An example detailed procedure is described in the section <NUM>. <NUM> in Non Patent Literature <NUM> (Step <NUM> in <FIG>).

An adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector by using the pitch lag and the long-term prediction parameter. The detailed procedure of the adaptive codebook calculation unit <NUM> is as described in the example <NUM> (Step <NUM> in <FIG>).

The fixed codebook decoding unit <NUM> decodes a fixed codebook code and calculates a fixed codebook vector. The detailed procedure is as described in the section <NUM>. <NUM> in Non Patent Literature <NUM> (Step <NUM> in <FIG>).

An excitation vector synthesis unit <NUM> calculates an excitation vector by adding the adaptive codebook vector and the fixed codebook vector to which the gain is applied. Further, an excitation vector calculation unit updates the adaptive codebook buffer by using the excitation vector (Step <NUM> in <FIG>). The detailed procedure is the same as in the example <NUM>.

A synthesis filter <NUM> synthesizes a decoded signal by using the decoded LP coefficient and the excitation vector (Step <NUM> in <FIG>). The detailed procedure is the same as in the example <NUM>.

The functional configuration of the side information decoding unit <NUM> is the same as in the example <NUM>. The side information decoding unit <NUM> decodes the side information code and calculates a pitch lag (Step <NUM> in <FIG>).

The functional configuration of the audio parameter missing processing unit <NUM> is the same as in the example <NUM>.

The ISF prediction unit <NUM> predicts an ISF parameter using the ISF parameter for the previous frame and converts the predicted ISF parameter into an LP coefficient. The procedure is the same as in Steps <NUM>, <NUM> and <NUM> of the example <NUM> shown in <FIG> (Step <NUM> in <FIG>).

The adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector by using the pitch lag output from the side information decoding unit <NUM> and an adaptive codebook <NUM> (Step <NUM> in <FIG>). The procedure is the same as in Steps <NUM> and <NUM> in <FIG>.

The adaptive codebook gain prediction unit <NUM> outputs an adaptive codebook gain. A specific procedure is the same as in Step <NUM> in <FIG> (Step <NUM> in <FIG>).

The fixed codebook gain prediction unit <NUM> outputs a fixed codebook gain. A specific procedure is the same as in Step <NUM> in <FIG> (Step <NUM> in <FIG>).

The noise signal generation unit <NUM> outputs a noise, such as a white noise as a fixed codebook vector. The procedure is the same as in Step <NUM> in <FIG> (Step <NUM> in <FIG>).

The excitation vector synthesis unit <NUM> applies gain to each of the adaptive codebook vector and the fixed codebook vector and adds them together and thereby calculates an excitation vector. Further, the excitation vector synthesis unit <NUM> updates the adaptive codebook buffer using the excitation vector (Step <NUM> in <FIG>).

The synthesis filter <NUM> calculates a decoded signal using the above-described LP coefficient and the excitation vector. The synthesis filter <NUM> then updates the concealment signal accumulation unit <NUM> using the calculated decoded signal (Step <NUM> in <FIG>).

The above steps are repeated for M' sub-frames, and the decoded signal is output as the audio signal.

A concealment signal for one sub-frame is read from the concealment signal accumulation unit and is used as the decoded signal (Step <NUM> in <FIG>).

The above is repeated for M' sub-frames.

The ISF prediction unit <NUM> predicts an ISF parameter (Step <NUM> in <FIG>). As the procedure, Step <NUM> in <FIG> can be used.

The pitch lag prediction unit <NUM> outputs a predicted pitch lag by using the pitch lag used in the past decoding (Step <NUM> in <FIG>). The procedure used for the prediction is the same as in Step <NUM> in <FIG>.

The operations of the adaptive codebook gain prediction unit <NUM>, the fixed codebook gain prediction unit <NUM>, the noise signal generation unit <NUM> and the audio synthesis unit <NUM> are the same as in the example <NUM> (Step <NUM> in <FIG>).

The above steps are repeated for M sub-frames, and the decoded signal for M-M' sub-frames is output as the audio signal, and the concealment signal accumulation unit <NUM> is updated by the decoded signal for the remaining M' sub-frames.

A case of using glottal pulse synchronization in the calculation of an adaptive codebook vector is described hereinafter.

The functional configuration of the audio signal transmitting device is the same as in example <NUM>. The functional configuration and the procedure are different only in the side information encoding unit, and therefore only the operation of the side information encoding unit is described below.

The side information encoding unit includes an LP coefficient calculation unit <NUM>, a pitch lag prediction unit <NUM>, a pitch lag selection unit <NUM>, a pitch lag encoding unit <NUM>, and an adaptive codebook buffer <NUM>. The functional configuration of an example of the side information encoding unit is shown in <FIG>, and an example procedure of the side information encoding unit is shown in the example of <FIG>.

The LP coefficient calculation unit <NUM> is the same as the LP coefficient calculation unit in example <NUM> and thus will not be redundantly described (Step <NUM> in <FIG>).

The pitch lag prediction unit <NUM> calculates a pitch lag predicted value T̂p, using the pitch lag obtained from the audio encoding unit (Step <NUM> in <FIG>). The specific processing of the prediction is the same as the prediction of the pitch lag <MAT> in the pitch lag prediction unit <NUM> in the example <NUM> (which is the same as in Step <NUM> in <FIG>).

Then, the pitch lag selection unit <NUM> determines a pitch lag to be transmitted as the side information (Step <NUM> in <FIG>). The detailed procedure of the pitch lag selection unit <NUM> is shown in the example of <FIG>.

First, a pitch lag codebook is generated from the pitch lag predicted value T̂p and the value of the past pitch lag <MAT> according to the following equations (Step <NUM> in <FIG>).

The value of the pitch lag for one sub-frame before is <MAT>. Further, the number of indexes of the codebook is I. δj is a predetermined step width, and ρ is a predetermined constant.

Then, by using the adaptive codebook and the pitch lag predicted value T̂p, an initial excitation vector u<NUM>(n) is generated according to the following equation (Step <NUM> in <FIG>).

The procedure of calculating the initial excitation vector is the same as the equations (<NUM>) and (<NUM>) in Non Patent Literature <NUM>.

Then, glottal pulse synchronization is applied to the initial excitation vector by using all candidate pitch lags <MAT> in the pitch lag codebook to thereby generate a candidate adaptive codebook vector uj(n)(<NUM>≤j<I) (Step <NUM> in <FIG>). For the glottal pulse synchronization, the same procedure can be used as in the case described in section <NUM>. <NUM> in Non Patent Literature <NUM> where a pulse position is not available. Note, however, that u(n) in Non Patent Literature <NUM> corresponds to u<NUM>(n) in the embodiment of the present invention, and extrapolated pitch corresponds to <MAT> in the embodiment of the present invention, and the last reliable pitch(Tc) corresponds to <MAT> in the embodiment of the present invention.

For the candidate adaptive codebook vector uj(n)(<NUM>≤j<I), a rate scale is calculated (Step <NUM> in <FIG>). In the case of using segmental SNR as the rate scale, a signal is synthesized by inverse filtering using the LP coefficient, and segmental SNR is calculated with the input signal according to the following equation. <MAT> <MAT>.

Instead of performing inverse filtering, segmental SNR may be calculated in the region of the adaptive codebook vector by using a residual signal according to the following equation. <MAT> <MAT>.

In this case, a residual signal r(n) of the look-ahead signal s(n)(<NUM>≤n<L') is calculated by using the LP coefficient (Step <NUM> in <FIG>).

An index corresponding to the largest rate scale calculated in Step <NUM> is selected, and a pitch lag corresponding to the index is calculated (Step <NUM> in <FIG>).

The functional configuration of the audio signal receiving device is the same as in the example <NUM>. Differences from the example <NUM> are the functional configuration and the procedure of the audio parameter missing processing unit <NUM>, the side information decoding unit <NUM> and the side information accumulation unit <NUM>, and only those are described hereinbelow.

The side information decoding unit <NUM> decodes the side information code and calculates a pitch lag <MAT> and stores it into the side information accumulation unit <NUM>. The example procedure of the side information decoding unit <NUM> is shown in <FIG>.

In the calculation of the pitch lag, the pitch lag prediction unit <NUM> first calculates a pitch lag predicted value T̂p, by using the pitch lag obtained from the audio decoding unit (Step <NUM> in <FIG>). The specific processing of the prediction is the same as in Step <NUM> of <FIG> in the example <NUM>.

Then, a pitch lag codebook is generated from the pitch lag predicted value T̂p, and the value of the past pitch lag <MAT>, according to the following equations (Step <NUM> in <FIG>).

The procedure is the same as in Step <NUM> in <FIG>. The value of the pitch lag for one sub-frame before is <MAT>. Further, the number of indexes of the codebook is I. δj is a predetermined step width, and ρ is a predetermined constant.

Then, by referring to the pitch lag codebook, a pitch lag <MAT> corresponding to the index idx transmitted as part of the side information is calculated and stored in the side information accumulation unit <NUM> (Step <NUM> in <FIG>).

Although the functional configuration of the audio synthesis unit is also the same as in the example <NUM> (which is the same as in <FIG>), only the adaptive codebook calculation unit <NUM> that operates differently from that in the example <NUM> is described hereinbelow.

The audio parameter missing processing unit <NUM> reads the pitch lag from the side information accumulation unit <NUM> and calculates a pitch lag predicted value according to the following equation, and uses the calculated pitch lag predicted value instead of the output of the pitch lag prediction unit <NUM>. <MAT> where κ is a predetermined constant.

Then, by using the adaptive codebook and the pitch lag predicted value <MAT>, an initial excitation vector u<NUM>(n) is generated according to the following equation (Step <NUM> in <FIG>).

Then, glottal pulse synchronization is applied to the initial excitation vector by using the pitch lag <MAT> to thereby generate an adaptive codebook vector u(n). For the glottal pulse synchronization, the same procedure as in Step <NUM> of <FIG> is used.

Hereinafter, an audio encoding program <NUM> that causes a computer to execute the above-described processing by the audio signal transmitting device is described. As shown in <FIG>, the audio encoding program <NUM> is stored in a program storage area <NUM> formed in a recording medium <NUM> that is inserted into a computer and accessed, or included in a computer.

The audio encoding program <NUM> includes an audio encoding module <NUM> and a side information encoding module <NUM>. The functions implemented by executing the audio encoding module <NUM> and the side information encoding module <NUM> are the same as the functions of the audio encoding unit <NUM> and the side information encoding unit <NUM> in the audio signal transmitting device described above, respectively.

Note that a part or the whole of the audio encoding program <NUM> may be transmitted through a transmission medium such as a communication line, received and stored (including being installed) by another device. Further, each module of the audio encoding program <NUM> may be installed not in one computer but in any of a plurality of computers. In this case, the above-described processing of the audio encoding program <NUM> is performed by a computer system composed of the plurality of computers.

Hereinafter, an audio decoding program <NUM> that causes a computer to execute the above-described processing by the audio signal receiving device is described. As shown in <FIG>, the audio decoding program <NUM> is stored in a program is stored in a program storage area <NUM> formed in a recording medium <NUM> that is inserted into a computer and accessed, or included in a computer.

The audio decoding program <NUM> includes an audio code buffer module <NUM>, an audio parameter decoding module <NUM>, a side information decoding module <NUM>, a side information accumulation module <NUM>, an audio parameter missing processing module <NUM>, and an audio synthesis module <NUM>. The functions implemented by executing the audio code buffer module <NUM>, the audio parameter decoding module <NUM>, the side information decoding module <NUM>, the side information accumulation module <NUM>, an audio parameter missing processing module <NUM> and the audio synthesis module <NUM> are the same as the function of the audio code buffer <NUM>, the audio parameter decoding unit <NUM>, the side information decoding unit <NUM>, the side information accumulation unit <NUM>, the audio parameter missing processing unit <NUM> and the audio synthesis unit <NUM> described above, respectively.

Note that a part or the whole of the audio decoding program <NUM> may be transmitted through a transmission medium such as a communication line, received and stored (including being installed) by another device. Further, each module of the audio decoding program <NUM> may be installed not in one computer but in any of a plurality of computers. In this case, the above-described processing of the audio decoding program <NUM> is performed by a computer system composed of the plurality of computers.

An example that uses side information for pitch lag prediction at the decoding end is described hereinafter.

The functional configuration of the audio signal transmitting device is the same as in the example <NUM>. The functional configuration and the procedure are different only in the side information encoding unit <NUM>, and therefore the operation of the side information encoding unit <NUM> only is described hereinbelow.

The functional configuration of an example of the side information encoding unit <NUM> is shown in <FIG>, and an example procedure of the side information encoding unit <NUM> is shown in <FIG>. The side information encoding unit <NUM> includes an LP coefficient calculation unit <NUM>, a residual signal calculation unit <NUM>, a pitch lag calculation unit <NUM>, an adaptive codebook calculation unit <NUM>, an adaptive codebook buffer <NUM>, and a pitch lag encoding unit <NUM>.

The LP coefficient calculation unit <NUM> is the same as the LP coefficient calculation unit <NUM> in example <NUM> shown in <FIG> and thus is not redundantly described.

The residual signal calculation unit <NUM> calculates a residual signal by the same processing as in Step <NUM> in example <NUM> shown in <FIG>.

The pitch lag calculation unit <NUM> calculates a pitch lag for each sub-frame by calculating k that maximizes the following equation (Step <NUM> in <FIG>). Note that u(n) indicates the adaptive codebook, and L' indicates the number of samples contained in one sub-frame. <MAT> <MAT>.

The adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector v'(n) from the pitch lag Tp and the adaptive codebook u(n). The length of the adaptive codebook is Nadapt (Step <NUM> in <FIG>).

The adaptive codebook buffer <NUM> updates the state by the adaptive codebook vector v'(n) (Step <NUM> in <FIG>). <MAT> <MAT>.

The pitch lag encoding unit <NUM> is the same as that in example <NUM> and thus not redundantly described (Step <NUM> in <FIG>).

The audio signal receiving device includes the audio code buffer <NUM>, the audio parameter decoding unit <NUM>, the audio parameter missing processing unit <NUM>, the audio synthesis unit <NUM>, the side information decoding unit <NUM>, and the side information accumulation unit <NUM>, just like in example <NUM>. The procedure of the audio signal receiving device is as shown in <FIG>.

The operation of the audio code buffer <NUM> is the same as in example <NUM>.

The operation of the audio parameter decoding unit <NUM> is the same as in the example <NUM>.

The side information decoding unit <NUM> decodes the side information code, calculates a pitch lag <MAT> and stores it into the side information accumulation unit <NUM>. The side information decoding unit <NUM> decodes the side information code by using the decoding method corresponding to the encoding method used at the encoding end.

The audio synthesis unit <NUM> is the same as that of example <NUM>.

The ISF prediction unit <NUM> of the audio parameter missing processing unit <NUM> (see <FIG>) calculates an ISF parameter the same way as in the example <NUM>.

An example procedure of the pitch lag prediction unit <NUM> is shown in <FIG>. The pitch lag prediction unit <NUM> reads the side information code from the side information accumulation unit <NUM> and obtains a pitch lag <MAT> in the same manner as in example <NUM> (Step <NUM> in <FIG>). Further, the pitch lag prediction unit <NUM> outputs the pitch lag <MAT> by using the pitch lag <MAT> used in the past decoding (Step <NUM> in <FIG>). The number of sub-frames contained in one frame is M, and the number of pitch lags contained in the side information is M1a. In the prediction of the pitch lag <MAT>, the procedure as described in Non Patent Literature <NUM> can be used (Step <NUM> in <FIG>).

In the prediction of the pitch lag <MAT>, the pitch lag prediction unit <NUM> may predict the pitch lag <MAT> by using the pitch lag <MAT> used in the past decoding and the pitch lag <MAT>. Further, <MAT> may be established. The procedure of the pitch lag prediction unit in this case is as shown in <FIG>.

Further, the pitch lag prediction unit <NUM> may establish <MAT> only when the reliability of the pitch lag predicted value is low. The procedure of the pitch lag prediction unit in this case is shown in <FIG>. Instruction information as to whether the predicated value is used, or the pitch lag <MAT> obtained by the side information is used may be input to the adaptive codebook calculation unit <NUM>.

The adaptive codebook gain prediction unit <NUM> and the fixed codebook gain prediction unit <NUM> are the same as those of the example <NUM>.

The noise signal generation unit <NUM> is the same as that of the example <NUM>.

The audio synthesis unit <NUM> synthesizes, from the parameters output from the audio parameter missing processing unit <NUM>, an audio signal corresponding to the frame to be encoded.

The LP coefficient calculation unit <NUM> of the audio synthesis unit <NUM> (see <FIG>) obtains an LP coefficient in the same manner as in example <NUM> (Step S11301 in <FIG>).

The adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector in the same manner as in example <NUM>. The adaptive codebook calculation unit <NUM> may perform filtering on the adaptive codebook vector or may not perform filtering. Specifically, the adaptive codebook vector is calculated using the following equation. The filtering coefficient is fi.

In the case of decoding a value that does not indicate filtering, v(n)=v'(n) is established (adaptive codebook calculation step A).

The adaptive codebook calculation unit <NUM> may calculate an adaptive codebook vector in the following procedure (adaptive codebook calculation step B).

An initial adaptive codebook vector is calculated using the pitch lag and the adaptive codebook <NUM>. <MAT> v(n)=v'(n) may be established according to a design policy.

Then, glottal pulse synchronization is applied to the initial adaptive codebook vector. For the glottal pulse synchronization, the same procedure as in the case where a pulse position is not available in the section <NUM>. <NUM> in Non Patent Literature <NUM> is used. Note that, however, u(n) in Non Patent Literature <NUM> corresponds to v(n) in the embodiment of the present invention, and extrapolated pitch corresponds to <MAT> in the embodiment of the present invention, and the last reliable pitch(Tc) corresponds to <MAT> in the embodiment of the present invention.

Further, in the case where the pitch lag prediction unit <NUM> outputs the above-described instruction information for the predicated value, when the instruction information indicates that the pitch lag transmitted as the side information should not be used as the predicated value (NO in Step <NUM> in <FIG>), the adaptive codebook calculation unit <NUM> may use the above-described adaptive codebook calculation step A, and if it is indicated that the pitch value should be used (YES in Step <NUM> in <FIG>), the adaptive codebook calculation unit <NUM> may use the above-described adaptive codebook calculation step B. The procedure of the adaptive codebook calculation unit <NUM> in this case is shown in the example of <FIG>.

The excitation vector synthesis unit <NUM> outputs an excitation vector in the same manner as in example <NUM> (Step <NUM> in <FIG>).

The post filter <NUM> performs post processing on the synthesis signal in the same manner as in the example <NUM>.

The adaptive codebook <NUM> updates the state by using the excitation signal vector in the same manner as in the example <NUM> (Step <NUM> in <FIG>).

The synthesis filter <NUM> synthesizes a decoded signal in the same manner as in the example <NUM> (Step <NUM> in <FIG>).

The perceptual weighting inverse filter <NUM> applies a perceptual weighting inverse filter in the same manner as in the example <NUM>.

The audio parameter missing processing unit <NUM> stores the audio parameters (ISF parameter, pitch lag, adaptive codebook gain, fixed codebook gain) used in the audio synthesis unit <NUM> into the buffer in the same manner as in the example <NUM> (Step <NUM> in <FIG>).

In this embodiment, a configuration is described in which a pitch lag is transmitted as side information only in a specific frame class, and otherwise a pitch lag is not transmitted.

The audio encoding unit <NUM> in this example calculates an index representing the characteristics of a frame to be encoded and transmits the index to the side information encoding unit <NUM>. The other operations are the same as in example <NUM>.

In the side information encoding unit <NUM>, a difference from the examples <NUM> to <NUM> is only with regard to the pitch lag encoding unit <NUM>, and therefore the operation of the pitch lag encoding unit <NUM> is described hereinbelow. The configuration of the side information encoding unit <NUM> in the example <NUM> is shown in <FIG>.

The procedure of the pitch lag encoding unit <NUM> is shown in the example of <FIG>. The pitch lag encoding unit <NUM> reads the index representing the characteristics of the frame to be encoded (Step <NUM> in <FIG>) and, when the index representing the characteristics of the frame to be encoded is equal to a predetermined value, the pitch lag encoding unit <NUM> determines the number of bits to be assigned to the side information as B bits (B><NUM>). On the other hand, when the index representing the characteristics of the frame to be encoded is different from a predetermined value, the pitch lag encoding unit <NUM> determines the number of bits to be assigned to the side information as <NUM> bit (Step <NUM> in <FIG>).

When the number of bits to be assigned to the side information is <NUM> bit (No in Step <NUM> in <FIG>), a value indicating non-transmission of the side information, is used as the side information code, and is set to the side information index (Step <NUM> in <FIG>).

On the other hand, when the number of bits to be assigned to the side information is B bits (Yes in Step <NUM> in <FIG>), a value indicating transmission of the side information is set to the side information index (Step <NUM> in <FIG>), and further, a code of B-<NUM> bits obtained by encoding the pitch lag by the method described in example <NUM> is added, for use as the side information code (Step <NUM> in <FIG>).

The operation of the audio parameter decoding unit <NUM> is the same as in example <NUM>.

The procedure of the side information decoding unit <NUM> is shown in the example of <FIG>. The side information decoding unit <NUM> decodes the side information index contained in the side information code first (Step <NUM> in <FIG>). When the side information index indicates non-transmission of the side information, the side information decoding unit <NUM> does not perform any further decoding operations. Also, the side information decoding unit <NUM> stores the value of the side information index in the side information accumulation unit <NUM> (Step <NUM> in <FIG>).

On the other hand, when the side information index indicates transmission of the side information, the side information decoding unit <NUM> further performs decoding of B-<NUM> bits and calculates a pitch lag <MAT> and stores the calculated pitch lag in the side information accumulation unit <NUM> (Step <NUM> in <FIG>). Further, the side information decoding unit <NUM> stores the value of the side information index into the side information accumulation unit <NUM>. Note that the decoding of the side information of B-<NUM> bits is the same operation as the side information decoding unit <NUM> in example <NUM>.

The ISF prediction unit <NUM> of the audio parameter missing processing unit <NUM> (see <FIG>) calculates an ISF parameter the same way as in example <NUM>.

The procedure of the pitch lag prediction unit <NUM> is shown in the example of <FIG>. The pitch lag prediction unit <NUM> reads the side information index from the side information accumulation unit <NUM> (Step <NUM> in <FIG>) and checks whether it is the value indicating transmission of the side information (Step <NUM> in <FIG>).

In the same manner as in example <NUM>, the side information code is read from the side information accumulation unit <NUM> to obtain a pitch lag <MAT> (Step <NUM> in <FIG>). Further, the pitch lag <MAT> is output by using the pitch lag <MAT> used in the past decoding and <MAT> obtained as the side information (Step <NUM> in <FIG>). The number of sub-frames contained in one frame is M, and the number of pitch lags contained in the side information is Mla. In the prediction of the pitch lag <MAT>, the procedure as described in Non Patent Literature <NUM> can be used (Step <NUM> in <FIG>). Further, <MAT> may be established.

Further, the pitch lag prediction unit <NUM> may establish <MAT> only when the reliability of the pitch lag predicted value is low, and otherwise set the predicted value to <MAT> (Step <NUM> in <FIG>). Further, pitch lag instruction information indicating whether the predicated value is used, or the pitch lag <MAT> obtained by the side information is used, may be input into the adaptive codebook calculation unit <NUM>.

In the prediction of the pitch lag <MAT>, the pitch lag prediction unit <NUM> predicts the pitch lag <MAT> by using the pitch lag <MAT> used in the past decoding (Step <NUM> in <FIG>).

Further, the pitch lag prediction unit <NUM> may establish <MAT> only when the reliability of the pitch lag predicted value is low (Step <NUM> in <FIG>), and the pitch lag prediction unit <NUM> can otherwise set the predicted value to <MAT>. Further, pitch lag instruction information indicating whether the predicated value is used, or the pitch lag <MAT> used in past decoding is used, is input to the adaptive codebook calculation unit <NUM> (Step <NUM> in <FIG>).

The adaptive codebook gain prediction unit <NUM> and the fixed codebook gain prediction unit <NUM> are the same as those of example <NUM>.

The audio synthesis unit <NUM> synthesizes, from the parameters output from the audio parameter missing processing unit <NUM>, an audio signal which corresponds to the frame to be encoded.

The procedure of the adaptive codebook calculation unit <NUM> is shown in the example of <FIG>. The adaptive codebook calculation unit <NUM> calculates an adaptive codebook vector in the same manner as in example <NUM>. First, by referring to the pitch lag instruction information (Step <NUM> in <FIG>), when the reliability of the predicted value is low (YES in Step <NUM> in <FIG>), the adaptive codebook vector is calculated using the following equation (Step <NUM> in <FIG>). The filtering coefficient is fi.

Note that v(n)=v'(n) may be established according to the design policy.

By referring to the pitch lag instruction information, when the reliability of the predicted value is high (NO in Step <NUM> in <FIG>), the adaptive codebook calculation unit <NUM> calculates the adaptive codebook vector by the following procedure.

First, the initial adaptive codebook vector is calculated using the pitch lag and the adaptive codebook <NUM> (Step <NUM> in <FIG>). <MAT> v(n)=v'(n) may be established according to the design policy.

Then, glottal pulse synchronization is applied to the initial adaptive codebook vector. For the glottal pulse synchronization, the same procedure can be used as in the case where a pulse position is not available in section <NUM>. <NUM> in Non Patent Literature <NUM> (Step <NUM> in <FIG>). Note that, however, u(n) in Non Patent Literature <NUM> corresponds to v(n) in the embodiment of the present invention, extrapolated pitch corresponds to <MAT> in the embodiment of the present invention, and the last reliable pitch(Tc) corresponds to <MAT> in the embodiment of the present invention.

The excitation vector synthesis unit <NUM> outputs an excitation signal vector in the same manner as in the example <NUM> (Step <NUM> in <FIG>).

The post filter <NUM> performs post processing on the synthesis signal in the same manner as in example <NUM>.

The adaptive codebook <NUM> updates the state using the excitation signal vector in the same manner as in the example <NUM> (Step <NUM> in <FIG>).

The synthesis filter <NUM> synthesizes a decoded signal in the same manner as in example <NUM> (Step <NUM> in <FIG>).

The perceptual weighting inverse filter <NUM> applies an perceptual weighting inverse filter in the same manner as in example <NUM>.

The audio parameter missing processing unit <NUM> stores the audio parameters (ISF parameter, pitch lag, adaptive codebook gain, fixed codebook gain) used in the audio synthesis unit <NUM> into the buffer in the same manner as in example <NUM> (Step <NUM> in <FIG>).

Claim 1:
An audio coding device for coding an audio signal, the audio coding device comprising:
an audio encoder for coding the audio signal; and
a side information encoder for calculating a parameter for a look-ahead signal in CELP coding as side information to be used in packet loss concealment in the CELP coding,
wherein the audio encoder calculates an index representing characteristics of a frame to be coded and transmits the index to the side information encoder,
when the index indicates that the frame to be coded is in a specific frame class, a difference between a pitch lag of the frame to be coded and a pitch lag of a frame immediately after the frame to be coded is included, as the side information, in a packet of the frame to be coded,
an audio code being an encoded audio signal of the frame to be coded and the side information can be transmitted before the audio encoder encodes an audio signal of the frame immediately after the frame to be coded, and
when the index indicates that the frame to be coded is not in a specific frame class, the pitch lag of the frame immediately after the frame to be coded is not included in the packet of the frame to be coded.