# Mutual Information, the Linear Prediction Model, and CELP Voice Codes
###### Abstract  
We write the mutual information between an input speech utterance and its reconstruction by a code-excited linear prediction (CELP) codec in terms of the mutual information between the input speech and the contributions due to the short-term predictor, the adaptive codebook, and the fixed codebook. We then show that a recently introduced quantity, the log ratio of entropy powers, can be used to estimate these mutual informations in terms of bits/sample. A key result is that for many common distributions and for Gaussian autoregressive processes, the entropy powers in the ratio can be replaced by the corresponding minimum mean squared errors. We provide examples of estimating CELP codec performance using the new results and compare these to the performance of the adaptive multirate (AMR) codec and other CELP codecs. Similar to rate distortion theory, this method only needs the input source model and the appropriate distortion measure.  
autoregressive models; entropy power; linear prediction model; CELP voice codecs; mutual information +
Footnote †: journal: Information  
10.00.  
quality would be very useful in optimizing codec performance and for investigating new adaptive codec structures.  
In this paper, we develop and extend an approach suggested by [PERSON] (2008) and decompose the mutual information between the input speech and the speech reconstructed by a CELP codec into the sum of unconditional and conditional mutual informations between the input speech and the linear prediction component, the adaptive codebook, and the fixed codebook, and show that this decomposition can be used to predict CELP codec performance based upon analyzing the input speech utterance, without actually implementing the CELP codec and processing the speech. We present examples comparing the estimated CELP codec performance and the actual performance achieved by CELP codecs. The approach is highly novel and the agreement between the actual and estimated performance is surprising.  
The paper is outlined as follows. Section 2 provides an overview of the principles of Code-Excited Linear Prediction (CELP) needed for the remainder of the paper, while Section 3 develops the decomposition of the mutual information between the input speech and the speech reconstructed by the CELP codec. The concept of entropy power as defined by [PERSON] (1954) is presented in Section 4, and the ordering of mutual information as a signal is passed through a cascaded signal processing system is stated in Section 5. The recently proposed quantity, the log ratio of entropy powers, is given in Section 6, where it is shown that the mean squared estimation errors can be substituted for the entropy powers in the ratio for an important set of probability densities [PERSON] (2008); [PERSON] (2010); [PERSON] (2010). The mutual information between the input speech and the short-term prediction sequence is discussed in Section 7 and the mutual information provided by the adaptive and fixed codebooks about the input speech is developed in Section 8. The promised analysis of CELP codecs using these prior mutual information results based on only the input speech model and the distortion measure is presented in Section 9. Section 10 contains conclusions drawn from the results in the paper.