# Mutual Information, the Linear Prediction Model, and CELP Voice Codes
## 8 Mutual Information in the Fixed and Adaptive Codebooks
### Adaptive Codebook: Long-Term Prediction  
The long-term predictor that makes up the adaptive codebook in CELP may have one or three taps and is of the form  
\[P(z)=\beta_{-1}z^{-(P-1)}+\beta_{0}z^{-P}+\beta_{1}z^{-(P+1)}, \tag{19}\]  
where \(\beta_{i},i=-1,0,1\), are the predictor coefficients that are updated on a frame-by-frame basis, and \(P\) is the lag of the periodic correlation that captures the pitch excitation of the vocal tract. A three-tap predictor as shown in Equation (19) requires a stability check to guarantee that this component does not cause divergence [23]. The single tap form given by  
\[P(z)=\beta z^{-P} \tag{20}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Speech Frame No.** & \(SPER\) **in dB** & \(I(X;X_{10})-I(X;X_{0})\) \\ \hline
45 & 16 & 2.647 bits/letter \\
23 & 11.85 & 1.968 bits/letter \\
3314 & 7.74 & 1.29 bits/letter \\
87 & 5 & 0.83 bits/letter \\ \hline \hline \end{tabular}
\end{table}
Table 2: Change in mutual information from Equation (17) for 10 th order predictors and corresponding SPERs for several speech frames [5].  
Figure 6: Frame 3237 time domain waveform (bottom) and spectral envelope, signal-to-prediction error (SPER) = 9.15 dB.  
only needs the stability check that \(|\beta|<1\), which should hold for any normalized autocorrelation. The three-tap form can often provide improved performance over the single tap predictor, but at increased complexity.  
We denote the long-term predicted value as \(X_{P}^{P}\) to distinguish it from the short-term predictor of order \(N\), \(X_{N}\), which contains all terms up to and including \(N\). Thus, we can write the MMSPE between \(X\) and \(X_{P}^{P}\) as \(MMSPE(X,X_{P}^{P})\).  
The prediction gain in dB is often used as a performance indicator for a pitch predictor. The long-term prediction gain has the form of Equation (18) but with \(MMSPE(X,X_{10})\) replaced by \(MMSPE(X,X_{P}^{P})\), where \(X_{P}^{P}\) is the long-term predicted value for a pitch lag of \(P\) samples, where usually \(P=20\) up to \(P=140\) samples at 8000 samples/s. Rather than calculate the prediction gain for selected speech frames, we consult the literature to get a range of values that might be expected.  
[PERSON] and [PERSON] (2014) indicate a pitch prediction gain of 1 to 5 dB, but of around 2 dB on average. Other sources indicate that the prediction gains can be 3-4 dB ([PERSON], 2014) or up to 5-6 dB ([PERSON], 2014). Table 3 shows the mutual information in bits/letter that can be associated with prediction gains of 1, 2, 3, 4, and 6 dB.  
It is interesting to consult the standardized speech codecs to see how many bits/sample are allocated to coding the pitch gain (coefficient or coefficients) and the pitch delay (lag). For the narrowband form of the AMR codec, we see that for 7.95 bits/s rate, the pitch gain and pitch delay are allocated 16 and 28 bits, respectively, per 20 ms frame, or 44 bits/160 samples = 0.275 bits/sample (Beck  
We now have estimates of the bits/sample devoted to the short-term predictor, adaptive codebook, and fixed codebook for a CELP codec operating at different bit rates. In the following, we show how to exploit these estimates to predict the rate of CELP codecs for different speech sources.