# Modeling and Profiling of Aggregated Industrial Network Traffic
## 3 Modelling Aggregated Traffic Break-Down
### Objective 2: State Modelling and Profiling
#### 3.2.1 Model Selection  
Model selection has been one of the main concerns in deploying learning algorithms in real scenarios. Dynamic systems are hard to be deterministically modeled as there are many factors involved in the problem, from system parameters to the effect of the surrounding environment on system behavior. Furthermore, the process of identifying the complex relation and correlation between all parameters, as well as the element of noise, can be costly and time-consuming, if not impossible. In data-driven system identification, a learning algorithm is trained to uncover system model over recorded historical data, and then deployed for various configuration and provisioning management, diagnostic and prognostic purposes [19; 20; 21; 22; 23]. The question here is how to select a learning algorithm in the absence of an identified true model? To answer this question, we first need to find models with various orders that give the best approximation, or fit, for the targeted data. There exist many methods in the literature for comparing models accuracy for various datasets and data types with different characteristics [24; 25]. The majority of the methods are based on likelihood model selection where the model parameters, such as the number of states and samples, are not considered. Thus, increasing the number of states leads to a higher likelihood adds to system complexity without providing additional information [24]. Therefore, methods that consider the number of model parameters are desirable. Commonly applied methods for order estimation of sequential data with parameter consideration are [PERSON]'s information criterion (AIC) [26] and Bayesian information criterion (BIC) [27] and efficient determination criterion (EDC) [28]. Akaike information criterion is one mathematically supported evaluation criterion of models. It is an estimator of expected relative (K-L) information based on the maximized log-likelihood function:  
\[AIC=-2\log(\hat{L})+2k \tag{4}\]  
where \(k\) is the number of estimated parameters in the approximated model, and \(\hat{L}\) is the maximum likelihood of the model with the true order \(k\). For small sample size where \(\frac{n}{k}\lesssim 40\), \(AIC\) becomes:  
\[AIC_{c}=-2\log(\hat{L})+2k+\frac{2k(k+1)}{(n-k-1)} \tag{5}\]  
where \(n\) is the cardinality of the set, that is the number of elements in the sample set. Baysian information criterion is closely related to \(AIC\) model selection methods, but introduces the sample size in the penalty term that provides  
\[BIC=-2\log(\hat{L})+k\log n \tag{6}\]Efficient determination criterion encompasses _AIC_ and _BIC_ and introduces a strictly increasing function that results in a strongly consistent order estimation. \[EDC=-2\log(\hat{L})+k\log\log n \tag{7}\]  
As all assumptions are excluded from the model selection process, in this work, the order of the model is set where the difference between the three methods are minimized. The identified work-cycles from the first objectives are the data used for state identification. The throughput in each work-cycle is influenced by the same conditions with minor differences, i.e., varying transmission intensity posed by unknown parameters. The results of applying the three methods for model selection are presented in Figure 3(a). There are two possible choices of orders: 2 and 4, with the second-order shared between all the three methods. However, the results of the fourth-order are less consistent between the methods; the model order by EDC maps those by AIC and BIC for the order of 2, but it shows signs of overfitting with higher orders, i.e., 4. Therefore, the model selection process suggests two different states in the communication dynamics within each work-cycle. A second-order HMM model identified the various states and transition probabilities between each state. Figure 3(b) shows the mapping of the identified states on the work-cycle data. The colour-coded identified states indicate consideration of both temporal and spatial features.