# Change-Point Detection in Autoregressive Processes via the Cross-Entropy Method
## 2 Model Selection Criteria  
### Change-Point Detection Problem  
Firstly, let us formulate the change-point detection problem. Let \(X=(X_{1},X_{2},\ldots,X_{T})\) denote a data sequence of length \(T\). Assume there are \(N\) unknown change-points, so \(X\) is segmented into \(N+1\) segments by the set of locations \(\tau=(\tau_{1},\tau_{2},\ldots,\tau_{N})\), \(0=\tau_{0}<\tau_{1}<\tau_{2}<\cdots<\tau_{N}<\tau_{N+1}=T\), where the \(i\)-th segment consists of observations \((X_{\tau_{i-1}+1},\ldots,X_{\tau_{i}})\); \(i\) is numbered from \(1\) to \(N+1\). Next, we assume that the data sequence \(X\) is generated by piecewise stationary AR(\(p_{i}\)) processes, the model assumes that the observations within each segment are generated by a stationary AR(\(p_{i}\)) process with autocorrelation coefficients \(\rho_{i}=(\rho_{i1},\ldots,\rho_{i\rho_{i}})\), mean \(\delta_{i}\), and variance \(\
u_{i}^{2}\). The segments are assumed to be independent of each other. The parameters \(\delta_{i}\), \(\
u_{i}^{2}\), and \(\rho_{i}\) are not known in advance, whereas \(p_{i}\) is known. \[U_{t,j} = \sum_{j=1}^{p_{i}}\rho_{ij}U_{t-j,i}+\epsilon_{t,i},\qquad t=1, \ldots,T,\quad i=1,\ldots,N+1,\quad j=1,\ldots,p_{i}\] \[X_{t,j} = \delta_{i}+U_{t,i} \tag{1}\]  
where the noise \(\epsilon_{t,i}\sim\) i.i.d. Normal \((0,\
u_{i}^{2})\), \(p=(p_{1},p_{2},\ldots,p_{N+1})\), \(\rho=(\rho_{1},\rho_{2},\ldots,\rho_{N+1})\), and the model allows for shifts in the mean, \(\delta=(\delta_{1},\delta_{2},\ldots,\delta_{N+1})\); also define \(\
u^{2}=(\
u_{1}^{2},\ldots,\
u_{N+1}^{2})\), \(\theta=(\delta,\rho,p,\
u^{2})\). It is necessary to specify the unknown parameter \(N\), the number of change-points. Theoretically, \(N\) could be any integer between \(0\) and \(T-1\), while in practice, we can set a maximum value for \(N\), perhaps via the rough graphical inspection. Additionally, the change-point problem is characterized by \(N\); if \(N\) is greater than \(1\), we call it multiple change-point problem; otherwise,the process is stationary. A model selection procedure is often used to determine the number of change-points; one can minimize an objective function of the form [38]:  
\[\sum_{i=1}^{N+1}\left[\mathcal{C}(X_{(\tau_{i-1}+1);\tau_{i}})\right]+\mathcal{ P}(\tau,N,\theta) \tag{2}\]  
where the \(\mathcal{C}\) is a cost function for a segment and \(\mathcal{P}\) is a penalty term. In this paper, we consider the negative log-likelihood as the cost function by rewriting (2) as a penalized likelihood information criterion:  
\[-\sum_{i=1}^{N+1}\left[\mathcal{L}(X_{(\tau_{i-1}+1);\tau_{i}})\right]+ \mathcal{P}(\tau,N,\theta). \tag{3}\]  
Akaike's information criterion (AIC), Bayesian information criterion (BIC), and modified BIC are well-known information criteria. The significant difference between these information criteria is the amount of penalty added to each parameter of the model. The penalty terms for the AIC and the BIC are given below:  
\[\text{AIC} : \mathcal{P}=k, \tag{4}\] \[\text{BIC} : \mathcal{P}=\frac{1}{2}\log(T)k \tag{5}\]  
The total number of parameters in the model is denoted by \(k\). Thus, each parameter is penalized by the same amount. Because the AIC and BIC are not theoretically designed for the change-point problems, [PERSON] and [PERSON] [39] proposed the modified BIC (mBIC) for determining the number of change-points in array-based comparative genomic hybridization data. The mBIC follows the classic BIC, while it differs in the penalty term. The general form that penalizes for model dimensions is given:  
\[\text{mBIC} : \mathcal{P}\approx\frac{3}{2}N\log(T)+\frac{1}{2}\sum_{i=1}^{N+1} \log(\tau_{i}-\tau_{i-1}). \tag{6}\]  
In addition, [PERSON] [40] proposed a penalty function with an adjustable shrinkage parameter \(\beta\) for the change-point problem:  
\[\text{ShrinkageIC} : \mathcal{P}\approx\beta N\log(T). \tag{7}\]  
In this paper, we introduce the two-stage minimum description length (MDL) for our change-point problem based on model (1); the superior flexibility is one strength of MDL.