# Detection and Localization of Failures in Hybrid Fiber-Coaxial Network Using Big Data Platform
## 5 Failure Detection and Localization Based on Big Data
### Failure Detection  
The detection part of the FDLBD mechanism should be constantly observing to the number of online users and triggering the failure localization in case of sudden drops. The number of online users is not constant and fluctuates with time. Figure 4 shows the number of online users in one network part for one day. Figure 4: Number of online users. Figure 3: Failure localization example. Obviously, the number of online users goes up and down depending on the period of the day. In Figure 4, it can also be noticed that the number of online users has dropped sharply, which represents an example of an outage of corresponding network part. To detect such cases, we propose a failure detection part of FDLBD as follows. First, the average number of active users \(N_{out\_avg}\) in the last \(K\) iterations is calculated:  
\[N_{out\_avg}=\frac{1}{K}\sum_{i=1}^{K}N_{out}[T_{CURR}-i], \tag{1}\]  
where \(T_{CURR}\) is the current measurement period, and \(N_{out}[j]\) is the number of active users in measurement period \(j\). \(K\) can be set to the desired value depending on the particular network behavior. For the real HFC network, where FDLBD is applied and a one minute collection period occurs, we have determined that \(K\) = 20 gives the best results. However, we suggest testing prior to the deployment of the FDLBD mechanism to determine the optimal value for \(K\) in the corresponding HFC network. We define failure if the following condition is met:  
\[Failure=\frac{N_{out}[T_{curr}]}{N_{out\_avg}}\leq 1-\textit{Threshold\_d},\ \ \ 0<\textit{Threshold\_d}<1, \tag{2}\]  
where _Threshold_d_ is, as the name suggests, the threshold for failure detection. The practice in the real HFC network has shown that _Threshold_d_ = 0.15 gives optimal results. Note that Equations (1) and (2), as well as suggested parameter values (\(K\) and _Threshold_d_), have been devised through experiments on the real HFC network data and by comparing algorithm results with real failures in the network. For the HFC network we tested, the proposed approach detects over 95% of real network failures. Our approach can be used in other HFC networks. However, depending on the social environment where HFC network is deployed, clients' behavior (nation culture, habits, working hours, etc.) can be different, and thus cdxCmtsCmRegisteredtime series might differ. For this reason, it is necessary to tune parameters to obtain the best results. Therefore, to get the best possible results using our proposed algorithm, tests should be conducted prior to FDLBD deployment to determine the optimal _Threshold_d_ value for the corresponding HFC network. Our parameters are tuned for one cable operator in Europe. Note that the similar threshold approach can also be used to detect situations when a problem is resolved. This is very useful in situations when a problem is resolved by itself. For example, electricity loss for a few minutes can cause such behavior. In the case of integration with fault management service, these two events can correlate and automatically close previously opened corresponding issue. The condition that detects resolution is:  
\[Resolution=\frac{N_{out}[T_{curr}]}{N_{out\_avg}}\geq 1+\textit{Threshold\_r},\ \ \ 0<\textit{Threshold\_r}<1, \tag{3}\]  
where _Threshold_r_ is the threshold for problem resolution detection. The only limitation it has is that the problem may happen before condition (3) is applied. In other words, problem resolution detection is activated only after the failure is detected. Note that FDLBD deploys the previously described problem resolution detection as well. Table 1 shows one example of detection and resolution of one failure that lasted a few minutes. We have only provided relevant measurements (\(N_{out}[T_{curr}]\), \(N_{out\_avg}\)) in Table 1 because the data sets collected from the real HFC network are confidential, and we are not able to disclose them. However, the values provided in Table 1 can help to demonstrate our algorithm.