# Root Cause Analysis of Reduced Accessibility in 4G Networks

Diogo Ferreira, Carlos Senna, Paulo Salvador, Luís Cortesão, Cristina Pires, Rui Pedro, Susana Sargento

► To cite this version:

Diogo Ferreira, Carlos Senna, Paulo Salvador, Luís Cortesão, Cristina Pires, et al. Root Cause Analysis of Reduced Accessibility in 4G Networks. 2nd International Conference on Machine Learning for Networking (MLN), Dec 2019, Paris, France. pp.117-133, 10.1007/978-3-030-45778-5_9. hal-03266468

HAL Id: hal-03266468

https://inria.hal.science/hal-03266468

Submitted on 21 Jun 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Root Cause Analysis of Reduced Accessibility in 4G Networks

Diogo Ferreira¹[0000-1111-2222-3333], Carlos Senna²[0000-0002-6791-4291], Paulo Salvador¹[0000-1111-2222-3333], Luís Cortesão³, Cristina Pires³, Rui Pedro³, and Susana Sargento¹,²[0000-0001-8761-8281]

`{diogodanielsoaresferreira,salvador,susana}@ua.pt,`
`cr.senna@av.it.pt,`

`{luis-a-cortesao,cristina-j-pires,rui-d-pedro}@alticelabs.com`

¹ DETI, University of Aveiro, Aveiro, Portugal

² Instituto de Telecomunicações, Aveiro, Portugal

³ Altice Labs, Aveiro, Portugal

**Abstract.** The increased programmability of communication networks makes them more autonomous, and with the ability to actuate fast in response to users and networks’ events. However, it is usually a difficult task to understand the root cause of the network problems, so that autonomous actuation can be provided in advance.

This paper analyzes the probable root causes of reduced accessibility in 4G networks, taking into account the information of important Key Performance Indicators (KPIs), and considering their evolution in previous time-frames. This approach resorts to interpretable machine learning models to measure the importance of each KPI in the decrease of the network accessibility in a posterior time-frame.

The results show that the main root causes of reduced accessibility in the network are related with the number of failure handovers, the number of phone calls and text messages in the network, the overall download volume and the availability of the cells. However, the main causes of reduced accessibility in each cell are more related to the number of users in each cell and its download volume produced. The results also show the number of PCA components required for a good prediction, as well as the best machine learning approach for this specific use case.

**Keywords:** Cellular networks · root cause analysis · machine learning.

## 1 Introduction

In communication networks, root cause analysis of network problems or failures is essential, so that a fast reaction to these failures, or even an anticipation and prevention of these failures can take place. However, usually it is difficult to assess the cause of reduced network accessibility, since it may happen due to a large number of issues, and impacting in a large number of metrics simultaneously. Knowing the causes that lead to such events can help to prematurely detect
them. Besides, when such events do happen, it is easier to know where and how
to autonomously actuate in the network to mitigate the failure.

With the increased requirements proposed for the 5G networks, e.g. 1 000 000 devices per km², 20 Gbit/s of download peak data rate [4], the new generation of cellular networks promises to handle more traffic than ever before. The incorporation of network slicing, as well as Software-Defined Networking (SDN) and Network Function Virtualization (NFVs) in the 5G architecture, overly increases the management complexity of those networks. With so many metrics to monitor, it is becoming harder to detect the cause of an event due to the complex combinations of various Key Performance Indicators (KPIs). Traditional approaches to detect the root cause of failures, with a knowledge base and a set of rules, is becoming obsolete due to the flexibility of the network. With the advances in machine learning, it is easier to indirectly analyze dependent variables with reduced complexity, but with increased uncertainty.

This work identifies the KPIs that may cause reduced accessibility in 4G networks, using machine learning techniques. Knowing those KPIs helps to create a proactive management of the network, detecting an eventual future drop in network accessibility and having the possibility to avoid it by acting on the network, adjusting the resources that have the most impact on those KPIs. In this work, two different approaches for root cause analysis using machine learning techniques are explained and discussed. The first approach measures the feature importance using internal calculations in the model to determine the importance of each KPI in a reduced accessibility event. However, due to the high number of combinations of the KPIs, it is not feasible to test all possibilities. It is then important to perform feature selection. The second approach proposes a dimensionality reduction algorithm to reduce the number of features and apply the machine learning algorithms. In the evaluation results we present the most important KPIs that are able to predict if the number of E-UTRAN Radio Access Bearer (E-RAB) setup failures is above a specific threshold. Then, we present the most important KPIs that are able to predict if the number of E-RAB establishment failures has high variations, and therefore, are highly correlated to the reduction of the network accessibility.

The remaining of the paper is organized as follows. Section II addresses the relevant related work. Section III discusses how KPIs impact the network accessibility. Section IV presents the proposed approaches for root cause analysis, while Section V discusses the results of the approach to root cause detection. Finally, Section VI presents the conclusion and the future works.

## 2 Related Work

Understanding the root cause of an observed symptom in a complex system has been a major problem for decades. The main question is how to find appropriate real-time mechanisms to determine root causes [10]. Most root cause analysis approaches for network operators is currently based on Bayesian Networks [7]. They have the capability of representing network metrics and events in nodes,
and their relations represent the dependencies, along with a conditional probability. To obtain the most probable cause of an error, a probabilistic inference can be done. In [1, 2], the authors argue that a Bayesian network is not suitable for large-scale systems with a large number of components, because the complexity of inference increases exponentially with the number of nodes and dependencies between them. To solve that, they combine the Bayesian network with case-based reasoning techniques to prune the nodes needed to analyze in the network. The results show that the technique used reduces drastically the inference time, as well as the need for human intervention.

In [11], a generic framework for root cause analysis in large IP networks was proposed. To determine the root cause of events, two reasoning engines are included: Bayesian inference and rule-based reasoning. The authors discuss that rule-based logic is often preferred over Bayesian inference, because it is easier to configure, it has an easier interpretation of results and it is effective in most applications. However, Bayesian networks are preferred when the root cause is unobservable (no direct evidence can be collected). In our work, the root cause of reduced accessibility is mostly unobservable.

The proposed solution in [3] computes what are parameters that are most relevant across all different types of failure modes, and use them to build a Bayesian Network to model the cause-effect relationship between the degradation parameters (cause) and failure modes (effect) that occur on the field. Two real life field issues are used as examples to demonstrate the accuracy of the network once it is modelled and built. This paper shows that accurately modelling the hardware system as a Bayesian Network substantially accelerates the process of root cause analysis.

A self-healing method based on network data analysis is proposed to diagnose problems in future 5G radio access networks [8]. The proposed system analyzes the temporal evolution of a plurality of metrics and searches for potential inter-dependence under the presence of faults. The work in [6] is the only one, that we are aware, that uses the concept of “variable importance” (“feature importance” in our work) to measure how much a feature contributes to predicting an objective variable on a machine learning model. The influence of each variable is then represented in an influence matrix, that represents the influence that each variable has for each event. However, only the Random Forest algorithm is used to create a model. The variable importance is then calculated using the permutation feature importance approach.

# 3 Problem Statement

The objective of this work is to understand what are the most important KPIs to anticipate reduced accessibility in a 4G network. A 4G network is chosen, since it is a running network with real data. The data available from 5G networks, because they are still not widely implemented, is scarce and can be unbiased by the users (only users with newer phones with 5G available can use it.)
In this work, a 31-day data set with 4G network KPIs was used. The interval period between each measurement of a KPI is 1 hour, which means that there are 24 values for each KPI for each cell every day. A metric must be chosen to represent reduced accessibility. After that, with time-series analysis, it is possible to calculate the importance of each KPI to the reduced accessibility metric.

The metric used to indicate low accessibility in the network is the number of E-UTRAN Radio Access Bearer (E-RAB) setup failures per hour in the network. The E-RAB setup in a 4G network is a major KPI for accessibility. The E-RAB is a bearer that the User Agents (UEs) need to establish communications in the network. Figure 1 shows the E-RAB setup phase. After the UE has established a connection with the E-UTRAN Node B (eNB), it is needed to setup a context with the Mobility Management Entity (MME), to enable the UE to communicate and send data to the network.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Sequence diagram indicating the E-RAB setup phase.</figcaption></figure>
When the MME sends a context setup request, it is called an E-RAB setup attempt. After some configuration messages between the UE and the eNB, the context setup response from the eNB to the MME is called an E-RAB setup success. There are more E-RAB setup attempts than E-RAB setup successes. When the network is congested, the difference between the E-RAB setup attempts and the E-RAB setup successes is higher, because some messages after the E-RAB setup attempt are lost due to network problems, such as congestion.

A new accessibility metric is used to measure the accessibility of the network: the number of E-RAB setup failures. If the number of E-RAB setup failures has a high value, the network is congested. The E-RAB setup failure formula is presented in Equation 1, where esf is the number of E-RAB setup failures, esa is the number of E-RAB setup attempts and ess is the number of E-RAB setup successes.

$$esf(t) = esa(t) - ess(t) \qquad (1)$$

The E-RAB setup failure is depicted in Figure 2. The biggest congestion happened on the 17th day. The objective of this paper is now to understand which KPIs most contribute to the forecast of this metric.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Number of E-RAB setup failures.</figcaption></figure>

# 4 Methodology for Root Cause Detection

In this section we describe two different approaches for root cause analysis using machine learning techniques. One approach, that measures feature importance using internal calculations in the model, is used to determine the importance of each KPI in a reduced accessibility event. Our second approach proposes a
dimensionality reduction algorithm to decrease the number of features (feature selection).

## 4.1 Approaches to measure the feature importance

A simple test to check if any KPI can accurately forecast the number of E-RAB setup failures is to calculate the Pearson correlation coefficient<fnref n="4" /> between the number of E-RAB setup failures and all other KPIs. Since the goal is to understand which KPIs can forecast low accessibility before the number of E-RAB setup failures increases, the KPI values will be shifted (lagged) behind the number of E-RAB setup failures by one hour (the values are sampled hourly; one hour is the minimum time interval).

There are two disadvantages of analyzing the KPIs' importance with the Pearson correlation coefficient. For each KPI, it is only measured its linear contribution to the number of E-RAB setup failures. Besides, combinations of KPIs that can be important are not being taken into account, because it is assumed that the KPIs are independent of each other.

There are other approaches to measure the importance of input features to an output value that takes into account these considerations. These approaches take advantage of machine learning techniques, and they can be divided into two categories: some approaches take into account the error of the model in a test set to calculate the importance of the input features; other approaches measure the feature importance with an internal calculation (algorithm-specific) of the coefficients associated with each input feature.

**Approaches considering the model error:** One of the approaches that fall into the first category is the drop column feature importance. In this approach, the importance of a feature is measured by comparing the test error of a model when all features are available as input, with the test error of a model when one feature is dropped for training. The higher the error for the model with one feature dropped, the more importance is given to that feature.

A big disadvantage of this approach is that, for each feature, it is needed to train a new model with that feature dropped, which causes the approach to be inefficient for many features, or for models that take a significant training time. In the permutation feature importance approach [5], a similar approach is used without the need to re-train the model for each feature. Instead of dropping the feature, it is applied random shuffling to the test values of that feature among the various examples, to preserve the distribution of that variable. If the model error is almost unchanged, the feature is not much important for the forecast. However, if the model error increases much, it is a sign that the feature is important for the forecast.

Both previous approaches have some advantages. It is possible to apply them to black-box models, given that the feature importance is measured by the model

<footnote n="4">⁴ https://www.statisticssolutions.com/pearsons-correlation-coefficient/</footnote>
error. They also take into account all interactions between features, which is an advantage when compared with the correlation tests.

In the permutation feature importance approach, a disadvantage is that the results are dependent on the shuffling of the features. If the tests are repeated, the results may vary.

**Approaches considering feature importance using internal calculations:** The approaches that measure the feature importance by inspecting the internals of the models are algorithm-dependent. For some algorithms, like neural networks or Support Vector Machines (SVMs), it is impossible to calculate the importance of each feature, due to the non-linear transformations applied. However, for other algorithms such as Logistic Regression<fnref n="5" />, Extra Trees, Random Forest<fnref n="6" />, Gradient Boosting<fnref n="7" /> or AdaBoost<fnref n="8" />, it is possible to estimate the importance of each feature. For linear regression, the importance of each feature can be measured by the absolute value of the coefficient associated with each input (if all features are within the same scale). For the other four tree-based ensemble algorithms (Extra Trees, Random Forest, Gradient Boosting and AdaBoost), the feature importance can be calculated with resort to the mean decrease impurity (or Gini impurity<fnref n="9" />). The importance of a node *j* in a decision tree is computed as described in Equation 2, where *w*<sub>*j*</sub> is the weighted number of samples in node *j*, *C*<sub>*j*</sub> is the impurity of this node, and *left*(*j*) and *right*(*j*) are the respective children nodes.

$$ ni_j = w_j C_j - w_{left(j)} C_{left(j)} - w_{right(j)} C_{right(j)} \quad (2) $$

The feature importance of feature *i* across all nodes is computed as described in Equation 3¹⁰.

$$ F_{i} = \frac{\sum_{j:\text{node j splits on feature i}} ni_{j}}{\sum_{j \in \text{all nodes}} ni_{j}} \quad (3) $$

For this approach, it is important that the features are all normalized within the same scale, and it is recommendable that they are from the same type (continuous/categorical) for better importance estimation.

The advantage of the approaches that measure the feature importance by inspecting the internals of the models is that they do not depend on the test set, only on the model. If the model is accurate and it is not underfitting or

<footnote n="5">⁵ https://www.statisticssolutions.com/what-is-logistic-regression/</footnote>

<footnote n="6">⁶ https://towardsdatascience.com/an-intuitive-explanation-of-random-forest-and-extra-trees-classifiers-8507ac21d54b</footnote>

<footnote n="7">⁷ https://towardsdatascience.com/understanding-gradient-boosting-machines-9be756fe76ab</footnote>

<footnote n="8">⁸ https://towardsdatascience.com/understanding-adaboost-2f94f22d5bfe</footnote>

<footnote n="9">⁹ https://victorzhou.com/blog/gini-impurity/</footnote>

<footnote n="10">¹⁰ https://stats stackexchange.com/questions/311488/summing-feature-importance-in-scikit-learn-for-a-set-of-features/</footnote>
overfitting, the feature importance can be calculated more reliably than the previous methods. Otherwise, the feature importance will be highly biased. The biggest challenge is to create accurate models that do not overfit the train set. This second category of approaches (measure feature importance with an internal calculation algorithm-specific) will be used to measure the importance of the KPIs.

## 4.2 Feature Selection

If all KPIs are included as features, the performance of the model will be degraded, because some features are uncorrelated with the output and do not contribute to the output classification. It is then important to perform feature selection. The ideal scenario is to test all combinations of features in the input, and determine the best features by the test error. However, it is not feasible to test all combinations, due to their high number.

The approach chosen was to use a dimensionality reduction algorithm to reduce the number of features. Principal Component Analysis [9] will be used as the algorithm to perform dimensionality reduction. The five algorithms chosen to train the model and to calculate the feature importance were the following: Logistic Regression, Extra Trees, Random Forest, Gradient Boosting and Adaboost. The feature importance of each model will calculate the importance of each PCA component. Each component importance is then multiplied by the PCA coefficients, to get the KPI importance for each component. Finally, all the importances of the same KPI are added, as described in Equation 4, where $fi_j$ is the feature importance of the j PCA component and $pcacoefficient(j, i)$ is the PCA coefficient i for the component j, where i is the number of a KPI and j is a PCA component.

$$kpi_{i} = \sum_{j=0}^{n\_pca\_components} fi_{j} * pca\_coefficient(j, i) \quad (4)$$

To get the best possible model, the number of PCA components cannot be too small, because relevant features can be lost; however, the number cannot be too big either, with the risk of creating overfitted models. The number of PCA components tested will vary from 1 to the number of KPIs, for the five algorithms. The model with a lower test error will be used to calculate the KPI importance.

## 4.3 Defining the input and the output of the algorithms

The input values will be based on the KPIs. There will be two types of input values: normalized values and normalized variations. For each KPI, the normalized value of the previous hour will be used as input. The number of lags could be increased besides one hour, but in this test it is considered that the low accessibility indicators appear at most one hour before the congestion. For each KPI, the normalized variation will be calculated according to the Equation 5.
The normalized variations are added as features because the network accessibil-
ity can depend, not only on the previous values, but also on sudden variations
of other KPIs.

$$
normalized\_variation(t) = \frac{value(t - 1) - value(t - 2)}{value(t - 2)} \quad (5)
$$

Two types of classification problems will be used to calculate the importance of the KPIs. It is important to understand which KPIs are more important when forecasting the possibility of a low accessibility event in a network. It is also important to understand which KPIs are more important to forecast sudden increases and decreases in the network accessibility. For each case, a different output value will be calculated. For the first case, it will be set a threshold in the 90th percentile of the data, with all the values above the threshold being classified as one, and all other values as zero, turning the problem into a binary classification problem. For the second case, the output will be classified as one if the absolute difference between two consecutive values is higher than the 90th percentile of the data, and zero otherwise. Doing that, it is possible to analyze which KPIs are most important for classifying low accessibility events and also for forecasting bigger increases and decreases in data, which can be important for resource allocation.

The two classification problems will be applied in two different ways, ac-
cording to the data split. First, the tests will be done with aggregated data. The
network KPIs will be added and will be taken into account in the whole network.
In this way, it is possible to forecast low accessibility of the network. Another
way of performing the tests is to split the data per cell. In this approach, the
data will not be aggregated, and 75% of all cells in the network will be used to
train, with the other 25% cells to test. The threshold for the tasks will be defined
using the specific values of each cell (each cell will have a different threshold,
based on its values). With this approach, it is built a model that is capable of de-
tecting low accessibility per cell. This case is expected to perform worse than the
aggregated network, because forecasting low accessibility per cell is harder than
forecasting low accessibility in the network. However, for a network operator it is
very important to forecast low accessibility per cell for various reasons. For a cell
in a crowded region, it can be made management adjustments to avoid the low
accessibility of the cell, such as the installation of another temporary cell, or the
allocation of resources for that cell. Besides, it can be made a time-series analysis
about the future accessibility of the cells in a region, for expansion purposes.

4.4 Performance metric

For both classification problems, a performance metric must be used. Since in
the problems previously described the number of positive samples is lower than
the number of negative samples, the accuracy performance metric would give
similar cost to the false negative and false positive errors. The performance
metric used will be the F1-score (Equation 6). The F1-score is the harmonic
mean of precision (Equation 7) and recall (Equation 8), and it encompasses the False Negative and False Positive errors, weighted according to the number of samples of each class.

$$F_1 = 2 * \frac{precision * recall}{precision + recall} \qquad (6)$$

$$precision = \frac{true\_positives}{true\_positives/false\_positives} \qquad (7)$$

$$recall = \frac{true\_positives}{true\_positives/false\_negatives} \qquad (8)$$

# 5 Root Cause Detection Results

Traffic monitoring is essential to provide a good quality do the services in communication networks. Network conditions such as bandwidth, packet loss, delay and jitter are important for traffic engineering to track the quality needs of applications. Therefore, it is important not only to measure the network conditions, but also to analyze them to understand the causes of reduced accessibility in cellular networks. In the following, we show how machine learning techniques can help to understand which network KPIs can indicate a low accessibility event that will happen in the future.

## 5.1 Aggregated network tests

In this subsection, the results for the aggregated network tests will be presented. As explained before, two scenarios are proposed. First, it will be presented (i) the most important KPIs for predicting if the number of E-RAB setup failures is above a threshold. Second, it will be presented (ii) the most important KPIs for predicting if the number of E-RAB establishment failures has high variations.

In Figure 3 it is shown that the best model for scenario (i) is the Extra Trees algorithm, achieving an F1-score of 86.6%, with 23 PCA components. With a higher number of PCA components, the performance of most of the algorithms starts to deteriorate.

Table 1 shows the ten KPIs that are considered more important for predicting if the number of E-RAB setup failures is above a threshold (i). The handover failure is considered to be the most important KPI, both inter- and intra-frequency. The third KPI is the Circuit Switched FallBack (CSFB) preparation success. The CSFB is a technology to create circuit-switched calls over a 4G network that does not support LTE voice call standard (VoLTE), which has to fall back on the 3G network. This indicates that the number of phone calls and SMS messages has a high impact on the network accessibility. The Packet Data Convergence Protocol (PDCP) download transmission time, the variation of the inter-frequency handovers and the cell available time are also important KPIs to predict variations in the number of E-RAB setup failures. Finally, the ERAB
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** F1-score with different algorithms varying the number of PCA components for the task (i) in the aggregated network.</figcaption></figure>

normal release, handovers intrafrequency success and attempt, and upload traffic volume have also a significant importance.

<table><caption>**Table 1.** Ten KPIs that were considered more important for the scenario (i) in the aggregated network.</caption><thead><tr><td>KPI</td><td>Importance</td></tr></thead><tbody><tr><td>Handover interfreq failure</td><td>0.194</td></tr><tr><td>Handover intrafreq failure</td><td>0.188</td></tr><tr><td>CSFB Prep Success</td><td>0.181</td></tr><tr><td>PDCP Download TX Time</td><td>0.169</td></tr><tr><td>Variation Handover interfreq failure</td><td>0.150</td></tr><tr><td>Cell availability</td><td>0.131</td></tr><tr><td>ERAB Normal Release</td><td>0.124</td></tr><tr><td>Handover intrafrequency success</td><td>0.121</td></tr><tr><td>Handover intrafrequency attempt</td><td>0.121</td></tr><tr><td>PDCP Upload Volume (Mb)</td><td>0.115</td></tr></tbody></table>

The best model for scenario (ii) is achieved with the AdaBoost algorithm, achieving an F1-score of 40.0%, with 24 PCA components. The F1-score is lower than in scenario (i) because the task of predicting variations is harder than the one of predicting if the value is above a threshold. The number of PCA components is almost the same as in scenario (i); however in Figure 4 it can be seen that the algorithms' F1-score is very unstable, and that it is hard to build a model to predict accurately the variations of the number of E-RAB establishment failures in the aggregated network.
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** F1-score with different algorithms varying the number of PCA components for the task (ii) in the aggregated network.</figcaption></figure>

Table 2 shows the ten KPIs that are considered more important for predicting if the number of E-RAB establishment failures has high variations (ii). The handover failures are still important, but they are not the most important KPI. In this case, the CSFB preparation success is the most important KPI to predict the variations in the number of E-RAB setup failures. Similar to the task (i), PDCP download transmission time, the variation of the inter-frequency handovers, the number of active subscribers, download and upload ones, and the cell available time are also important KPIs.

<table><caption>**Table 2.** Ten KPIs that were considered more important for the scenario (ii) in the aggregated network.</caption><thead><tr><td>KPI</td><td>Importance</td></tr></thead><tbody><tr><td>CSFB Prep Success</td><td>0.188</td></tr><tr><td>Handover intrafreq failure</td><td>0.170</td></tr><tr><td>PDCP Download TX Time</td><td>0.157</td></tr><tr><td>Handover interfreq failure</td><td>0.148</td></tr><tr><td>Variation Handover interfreq failure</td><td>0.141</td></tr><tr><td>Download Active Subscribers (Max)</td><td>0.138</td></tr><tr><td>Active subscribers (Max)</td><td>0.125</td></tr><tr><td>RRC Setup Failure</td><td>0.121</td></tr><tr><td>Cell availability</td><td>0.120</td></tr><tr><td>Upload active subscribers (Max)</td><td>0.119</td></tr></tbody></table>
## 5.2 Individual cells tests

In this subsection, the results for the individual cells will be presented. In this case, 75% of the cells in the network will be used for training and the other 25% for testing. Just like in the previous subsection, first it will be presented (i) the most important KPIs for predicting if the number of E-RAB establishment failures is above a threshold, and then it will be presented (ii) the most important KPIs for predicting if the number of E-RAB setup failures has high variations.

The best model for scenario (i) achieved an F1-score of 30.79%, using the Gradient Boosting algorithm, with 66 PCA components. For the cell prediction, more information is needed to obtain the best model when compared with the previous subsection. Figure 5 shows the F1-score varying with the number of components and the different algorithms. In the PCA component 62 there is an increase in the F1-score of all algorithms.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** F1-score with different algorithms varying the number of PCA components for the task (i) in the individual cells tests.</figcaption></figure>

Table 3 shows the ten KPIs that are considered more important for the task (i). Two KPIs have much more importance than all others: Radio Resource Control (RRC) setup success ratio ($\frac{\text{RRC setup success}}{\text{RRC setup attempt}}$) and E-RAB setup success ratio ($\frac{\text{ERAB setup success}}{\text{ERAB setup attempt}}$), since they are directly correlated with the network accessibility.

The best model for scenario (ii) has an F1-score of 20.39%, using the AdaBoost algorithm with 68 PCA components. The F1-score for different PCA components is similar to the scenario (i), where the F1-score improved its performance significantly after 60 PCA components (Figure 6).
<table><caption>**Table 3.** Ten KPIs that were considered more important for the scenario (i) in the individual cells tests.</caption><thead><tr><td>KPI</td><td>Importance</td></tr></thead><tbody><tr><td>RRC Setup Success Rate</td><td>0.560</td></tr><tr><td>ERAB Setup Success Rate</td><td>0.547</td></tr><tr><td>Connected subscribers (Max)</td><td>0.069</td></tr><tr><td>Connected subscribers (Avg)</td><td>0.066</td></tr><tr><td>Connected active subscribers (Avg)</td><td>0.043</td></tr><tr><td>Variation Connected Subscribers (Max)</td><td>0.034</td></tr><tr><td>Variation Connected active Subscribers (Max)</td><td>0.032</td></tr><tr><td>ERAB normal release</td><td>0.031</td></tr><tr><td>Variation Radio Bearers (Avg)</td><td>0.031</td></tr><tr><td>Variation Connected subscribers (Avg)</td><td>0.031</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>**Fig. 6.** F1-score with different algorithms varying the number of PCA components for the task (ii) in the individual cells tests.</figcaption></figure>
Table 4 shows the ten KPIs that are considered more important for the scenario (ii). Like in the previous scenario, RRC setup success ratio and E-RAB setup success rate are the most important KPIs for the output. In this task, other KPIs have also similar importance, such as the variation of the cell availability, the average connected subscribers and the average number of radio bearers. A radio bearer is a connection between the eNB and the UE at layer 2, and defines the communication configurations for upper layers.

<table><caption>**Table 4.** Ten KPIs that were considered more important for the scenario (ii) in the individual cells tests.</caption><thead><tr><td>KPI</td><td>Importance</td></tr></thead><tbody><tr><td>ERAB Setup Success Rate</td><td>0.147</td></tr><tr><td>RRC Setup Success Rate</td><td>0.147</td></tr><tr><td>Variation Cell availablility</td><td>0.138</td></tr><tr><td>Connected Subscribers (Avg)</td><td>0.102</td></tr><tr><td>Radio Bearers (Avg)</td><td>0.095</td></tr><tr><td>Variation RRC Setup Success Rate</td><td>0.093</td></tr><tr><td>Variation ERAB Setup Success Rate</td><td>0.092</td></tr><tr><td>ERAB Normal Release</td><td>0.088</td></tr><tr><td>Connected subscribers (Max)</td><td>0.087</td></tr><tr><td>Variation Radio Bearers (Avg)</td><td>0.081</td></tr></tbody></table>

## 5.3 Discussion

**Aggregated network tests:** From the results of both aggregated network tests, it can be concluded that most KPIs that cause the number of E-RAB setup failures in a network to be above a threshold are the same that cause it to have high variations. Those KPIs are the number of failed handovers (intra- and inter-frequency), the CSFB preparation success (number of phone calls and SMSs in the network), the PDCP download volume and the cell availability. The maximum number of active subscribers (downloading subscribers and overall subscribers) also causes the number of E-RAB setup failures to vary.

Interpreting the KPIs, the results achieved are according to the intuition about lower network accessibility. When the number of failure handovers is high, the cells are crowded with user sessions and cannot accept any more sessions, which leads to lower network accessibility in the next hour. The high number of CSFB preparation success shows that there is a clear relationship between the high number of phone calls and SMS messages in the network with its lower accessibility. The KPIs of PDCP download volume and the maximum number of active subscribers also show that the number of active subscribers and their download volume influence the network accessibility (more than the number of connected subscribers). Finally, the cell availability indicates that, if many cells are unavailable in the current hour, it is likely that the network accessibility will be lower in the next hour.
**Individual cells tests:** The results of the forecasts of low network accessibility in individual cells had higher error than the results with the aggregated network. In the tests done, the best models needed more data than the aggregated network models to achieve the best result, because more features were needed to be able to generalize the predictions for different cells.

The results show that, just like in the aggregated network tests, the most important KPIs for forecasting the number of E-RAB setup failures in a cell to be above a threshold are the same that cause it to have high variations. However, the most important KPIs that cause lower accessibility in a network are different from the KPIs that cause lower accessibility in a cell. The two most important KPIs are the RRC and the E-RAB setup success rate. These KPIs cannot be understood as the cause for lower network accessibility, but as a consequence. Because they are intrinsically related to the E-RAB setup failure, being themselves accessibility metrics, they can be understood as an indicator of the high autocorrelation between consecutive hours. These results show that the network accessibility per cell is highly dependent on the network accessibility of that cell in the previous hour. If a cell has lower network accessibility in an hour, it is likely that the network accessibility in the next hour for that cell is still low, and vice-versa. It is essential for a network operator to monitor the right KPIs for the different tasks: forecast lower accessibility in a network or forecast lower accessibility per cell.

As opposed to the results for the aggregated network, the important KPIs for low network accessibility are not related with the CSFB preparation success, or with any of the handover metrics. The results by cell show that, besides the RRC and the E-RAB setup success ratio, the most important KPIs are counters related to the number of users in a cell and its utilization: maximum number of connected subscribers, average number of connected subscribers, average number of active subscribers download data, variation of the maximum number connected subscribers, variation of the maximum number of active download subscribers, average number of radio bearers or variation of the average number of radio bearers. It is expected that, as these KPIs have higher values, the accessibility of a cell decreases.

# 6 Conclusion

Understanding the causes of events in a network, such as low accessibility, helps the network operators to forecast and avoid them to happen, by adjusting network resources that influence their causes. In this work, the goal was to determine the causes of reduced network accessibility in 4G networks, using only historic data.

Two different analysis were made. Besides analyzing the causes of reduced accessibility in the whole network, it was also analyzed the causes of reduced accessibility for each cell. The results showed that the causes of reduced accessibility for each analysis are very different. While for the overall network, the KPIs that most influence the accessibility are the number of failure handovers,
the number of phone calls and SMSs in the network, the overall download volume and the availability of the cells, the KPIs that most influence the accessibility of each cell are related with the number of users in a cell and its download volume. For a network operator, it is important to know if it is important to monitor low accessibility in a cell, in a network, or in both, to make the right measurements in the network.

As future work, the next step will be to detect the patterns of those KPIs that indicate future low accessibility, to be able to predict it and adapt the network to prevent it to happen. For example, if the network operator knows that the network accessibility will be lower in the next hour when the number of handover failures intra-frequency and the maximum connected users both exceed a threshold, he can take proactive measures to adapt the network and avoid the low accessibility.

**Acknowledgments** This work is supported by the European Regional Development Fund (FEDER), through the Regional Operational Programme of Lisbon (POR LISBOA 2020) and the Competitiveness and Internationalization Operational Programme (COMPETE 2020) of the Portugal 2020 framework [Project 5G with Nr. 024539 (POCI-01-0247-FEDER-024539)], and by FCT/MEC through national funds and when applicable co-funded by FEDER – PT2020 partnership agreement under the project UID/EEA/50008/2019.

## References

1. Bennacer, L., Ciavaglia, L., Ghamri-Doudane, S., Chibani, A., Amirat, Y., Mellouk, A.: Scalable and fast root cause analysis using inter cluster inference. In: 2013 IEEE International Conference on Communications (ICC). pp. 3563–3568 (June 2013). https://doi.org/10.1109/ICC.2013.6655104

2. Bennacer, L., Amirat, Y., Chibani, A., Mellouk, A., Ciavaglia, L.: Self-diagnosis technique for virtual private networks combining bayesian networks and case-based reasoning. IEEE Transactions on Automation Science and Engineering 12(1), 354–366 (Jan 2015). https://doi.org/10.1109/tase.2014.2321011, https://doi.org/10.1109/tase.2014.2321011

3. Chigurupati, A., Lassar, N.: Root cause analysis using artificial intelligence. In: 2017 Annual Reliability and Maintainability Symposium (RAMS). pp. 1–5 (Jan 2017). https://doi.org/10.1109/RAM.2017.7889651

4. David Kennedy: D2.6 Final report on programme progress and KPIs (2017), https://5g-ppp.eu/wp-content/uploads/2017/10/Euro-5G-D2.6_Final-report-on-programme-progress-and-KPIs.pdf

5. Fisher, A., Rudin, C., Dominici, F.: All models are wrong but many are useful: Variable importance for black-box, proprietary, or misspecified prediction models, using model class reliance (2018)

6. Gonzalez, J.M.N., Jimenez, J.A., Lopez, J.C.D., G., H.A.P.: Root cause analysis of network failures using machine learning and summarization techniques. IEEE Communications Magazine 55(9), 126–131 (2017). https://doi.org/10.1109/mcom.2017.1700066, https://doi.org/10.1109/mcom.2017.1700066
7. Mfula, H., Nurminen, J.K.: Adaptive root cause analysis for self-healing in 5g networks. In: 2017 International Conference on High Performance Computing Simulation (HPCS). pp. 136–143 (July 2017). https://doi.org/10.1109/HPCS.2017.31

8. Muñoz, P., de la Bandera, I., Khatib, E.J., Gómez-Andrades, A., Serrano, I., Barco, R.: Root cause analysis based on temporal analysis of metrics toward self-organizing 5g networks. IEEE Transactions on Vehicular Technology 66(3), 2811–2824 (March 2017). https://doi.org/10.1109/TVT.2016.2586143

9. Pearson, K.: Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2(11), 559–572 (1901). https://doi.org/10.1080/14786440109462720

10. Solé, M., Muntés-Mulero, V., Rana, A.I., Estrada, G.: Survey on models and techniques for root-cause analysis. CoRR abs/1701.08546 (2017), http://arxiv.org/abs/1701.08546

11. Yan, H., Breslau, L., Ge, Z., Massey, D., Pei, D., Yates, J.: G-RCA: A generic root cause analysis platform for service quality management in large IP networks. IEEE/ACM Transactions on Networking 20(6), 1734–1747 (Dec 2012). https://doi.org/10.1109/tnet.2012.2188837, https://doi.org/10.1109/tnet.2012.2188837