# Enhanced Routing Algorithm Based on Reinforcement Machine Learning--A Case of VoIP Service
## 2 Theoretical Review
### Speech Quality Assessment in VoIP Services  
One of the major concerns in VoIP service is the cost associated with the transmission medium. Due to this fact, in a VoIP communication compression techniques are used, and they do not cause significant losses in the received signal quality [56]. Speech codecs are responsible for this compression. There are different speech codecs, one of the most adopted in current communication networks is the Adaptive Multi-Rate Wideband (AMR-WB) codec [57]. The AMR-WB is a speech codec used for mobile device communications. It is widely used by network operators to provide high quality conversations. AMR-WB is based on the linear prediction generated by the ACELP algebraic code that uses a vector quantization technique [58]. The AMR-WB uses nine operation modes, from 6.6 kbps to 23.85 kbps, and each of one has a different response to packet losses. The _WB_ E-Model algorithm [16] is a parametric method that predict a conversation quality using different impairment factors related to acoustic environment, network, and speech codec. The \(R_{\text{WB}}\) is the global quality rating that is obtained using all the impairment factors. This value is expressed on a quality scale from 0 to 129, the higher the value the better the quality. The \(R_{WB}\) score is determined by:  
\[R_{WB}=R_{0,WB}-I_{s,WB}-I_{\text{d,}WB}-I_{\text{e-eff,}WB}+A, \tag{4}\]  
where \(R_{0,WB}\) represents the basic signal-to-noise ratio (SNR), and for _WB_ networks the standardized value is 129; \(I_{s,WB}\) represents the combination of all impairments which occur simultaneously with the voice signal, for WB signals the adopted values of this factor is 0; \(I_{\text{d,}WB}\) represents the impairments caused by delay and; \(I_{\text{e-eff,}WB}\) is the quality degradation due to equipment, specifically the speech code; A represents an advantage factor, but in _WB_ E-model is not considered and it is equal to 0. In this paper, we mainly focus on \(I_{\text{e-eff,}WB}\), and the \(I_{\text{d,}WB}\) is also evaluated. The \(I_{\text{e-eff}}\) is determined by:  
\[I_{\text{e,eff,}WB}=I_{\text{e,}WB}+(95-I_{\text{e,}WB})\cdot\frac{Ppl}{Ppl+Bpl_ {WB}}, \tag{5}\]  
where \(I_{\text{e,}WB}\) is the equipment impairment factor at zero packet-loss, only related to codec impairment; \(Ppl\) is the probability of packet losses, and the \(Bpl_{WB}\) is the packet-loss robustness factor for a specific codec in WB networks. In Annex IV of ITU-T recommendation G.113 [59], _Ie_ and \(Bpl\) values for AMR-WB codec are defined. Table 2 presents the number of bits and bit-rate of each AMR-WB operation modes, and their existing standardized \(I_{e}\) and \(Bpl\) values. Note that some \(Bpl\) values are not defined (ND) in some cases. In this work, the operation modes used in the simulation tests were 2 and 8, because they have the Bpl parameter already defined, as can be observed in Table 2. Thus, the \(I_{\text{e,eff,WB}}\) value can be computed. The \(I_{\text{d}}\) is computed using the following relation:  
\[I_{\text{d,}WB}=I_{\text{dte,}WB}+I_{\text{dte,}WB}+I_{\text{dd,}WB}, \tag{6}\]  
where \(I_{\text{dte,}WB}\) gives an estimate for the impairments due to talker echo, \(I_{\text{dte,}WB}\) represents impairments due to listener echo, and \(I_{\text{dd,}WB}\) represents the impairment caused by an absolute delay \(T_{a}\) in the network. In this study, the \(I_{\text{dte,}WB}\) and \(I_{\text{dte,}WB}\) are not considered because they are related to echo and acoustic problems at the end-sides of the communication that is out of the scope of this research. The \(I_{\text{dd,}WB}\) is defined by:  
\[I_{\text{dd,}WB}= \begin{cases}1,&T_{a}<100\text{ ms}\\ 25[(1+\chi^{6})^{(\frac{1}{6})}-3(1+(\frac{\chi}{3})^{6})^{\frac{1}{6}}+2],&T_{ a}>100\text{ ms}\end{cases}