# Change-Point Detection in Autoregressive Processes via the Cross-Entropy Method
## 2 Model Selection Criteria
### The Principle of Minimum Description Length  
The MDL criterion can tailor penalties for parameters having different natures, while the other information criteria treat every parameter as imposing the same penalty; therefore, which parameter to be penalized and the amount to be encoded need to be formalized according to the general principles of [PERSON] [41]:  
* For an integer parameter \(N\), when its value is not bounded, approximately \(\log_{2}(N)\) bits are encoded. If \(N\) is limited by the upper bound value \(N_{u}\), then approximately \(\log_{2}(N_{u})\) bits are needed.
* To encode a real parameter, if it is estimated from \(T\) observations, \(\frac{1}{2}\log_{2}(T)\) bits are required.  
Then, we use the framework set out by [7] to derive the description length based on model (1); the MDL consists of the following four components:  
1. As mentioned above, the first part of two-stage MDL can be considered as a negative log-likelihood function of the whole process, by assuming that the segments are independent, and using the approximation in [PERSON] et al. [42] for the exact loglikelihood for the AR(\(p\)) model, the likelihood of segment \(i\) can be written as \(-\frac{T}{2}\log(\hat{\
u}^{2})-\frac{1}{2}\log(\Gamma_{s})\) (see [42] for further details), where \(\hat{\
u}^{2}\) is the Yule-Walker estimate, \(\log(\Gamma_{s})\) is the covariance determinant of first \(s\) observations.
2. For the number \(N\), the penalty term is \(\log_{2}(N)\). Since it can be considered as an integer parameter without being bounded, the code length is \(\log_{2}(N)\).
3. The length for each of \(N+1\) segments cannot exceed \(T\). This means that, according to the integer parameter principle, the lengths of the segments can be encoded with \(\log_{2}(T)\) bits.
4. \(p_{i}\) represents the order of the AR process for each segment, which should be an integer. So the parameter can be encoded with \(\log_{2}(p_{i})\) bits; see [7].
5. For each segment \(i\), \(i=1,\ldots,N+1\), we estimate three real-valued parameters. These are mean shift \(\delta_{i}\); autocorrelation coefficient \(\rho_{ij}\), where \(j=1,\ldots,p_{i}\); and variance \(\
u_{i}^{2}\). Under the real parameter principle, each of these parameters can be computed from \(\tau_{i}-\tau_{i-1}-1\) observations. Hence, each of them need \(\frac{1}{2}\log_{2}(\tau_{i}-\tau_{i-1}-1)\) bits. Thus, we can get the objective functions based on the MDL for the multiple change-points model: \[\text{MDL}(N,\tau,\delta,\rho,p,\
u^{2}) = -\sum_{i=1}^{N+1}\log\mathcal{L}(X_{(\tau_{i-1}+1):\tau_{i}}\mid \hat{\theta_{i}})+\mathcal{P}(N,\tau,\theta_{i})\] (8) \[= \underbrace{-\sum_{i=1}^{N+1}\left(\frac{T}{2}\log(\hat{\
u_{i} }^{2})+\frac{1}{2}\log(\Gamma_{is})\right)}_{\text{stochastic complexity of the model}}\] \[+ \underbrace{\log N+(N+1)\log T+\sum_{i=1}^{N+1}\log p_{i}+\sum_{ i=1}^{N+1}\frac{p_{i}+2}{2}\log(\tau_{i}-\tau_{i-1}-1)}_{\text{penalty terms}}.\]  
If there are no change-points in the model (\(N=0\)), the MDL for a stationary AR(\(p\)) process will be \[\text{MDL}(\delta,\rho,p,\
u^{2}) = -\log\mathcal{L}(X_{1},X_{2},\ldots,X_{T}\mid\delta)+\frac{p+2}{2} \log T \tag{9}\] \[= -\left(\frac{T}{2}\log(\hat{\
u}^{2})+\frac{1}{2}\log(\Gamma_{s}) \right)+\frac{p+2}{2}\log T.\]