# Detection and Localization of Failures in Hybrid Fiber-Coaxial Network Using Big Data Platform
## 2 Related Work
Similarly, machine learning can be used for failure localization in optical networks as well [20; 21; 22]. The proposed machine learning-based solutions require some data (such as power spectrum density [18], bit error rate samples [19], routed lightpaths [20], mean time between failures [21], etc.) from the optical network to perform failure detection or localization. Failure detection and localization are important for computer networks and data centers as well. Many protocols and mechanisms are defined for fast failure detection and fast rerouting, especially for the transport (core) parts of the network, such as MPLS (MultiProtocol Label Switching) fast re-route mechanisms [23]. A survey of failure diagnostic techniques (localization, detection) in computer networks is given in [24]. Failure diagnostic can be passive or active. Passive techniques rely on monitoring agents deployed on network devices, where monitoring agents can signal various alarms to network management system. Active techniques rely on sending probes across various paths in network to detect and localize the failures in the network. Optimal selection of probes is a major aspect of active techniques [25]. Besides alarms generated by the devices, logs can be used for failure detection as well [16; 26]. Logs can not only be processed to detect failures, but to predict failures as well. The main issue is that logs are not structured data, and the log format depends on the vendors but also on the versions of software installed on the network devices [26]. This increases the complexity of log processing and requires updates of log processing tools whenever a device from another vendor is installed in the network or when devices are updated with new firmware versions. Data centers represent computer networks with huge numbers of devices. For the efficient data center performance, it is important to swiftly detect and localize failures. The problem is even harder given the existence of at least two routes between any node pair, and the equal-cost, multi-path routing that is commonly used in data centers [27]. Active probing was proposed in [27] for fast failure detection and localization. There are a few papers that deal with the failure detection and localization in access networks as well. In [28], failure detection solution that uses RADIUS (Remote Authentication Dial-in User Service) protocol was proposed for an xDSL (x Digital Subscriber Line) access network. A failure localization solution for FTTH (Fiber-to-the-Home) networks was proposed in [29]. In case of large networks with enormous number of devices, big data is unavoidable for network monitoring purposes. However, as we stated in Section 1, big data collected from communication networks can be used for many purposes. In [30], a very detailed survey is given on big data usage in wired and wireless networks. Some of the discussed aspects of big data usage include traffic prediction, QoS (Quality of Service) improvement, cybersecurity, network performance optimization, etc. [30]. There are not many papers about big data use for failure detection and localization. Mainly, these papers cover the mobile networks [31; 32]. XDR (External Data Representation) data is used for failure detection in mobile networks [31], while in [32], bandwidth trends were analyzed to predict the equipment failures. HFC networks are addressed in the literature mostly regarding their technical aspects. The focus is on the improvement and optimization of devices, throughput, etc. However, there are no papers that deal with hard failure detection and localization in complete HFC network. Furthermore, big data usage in HFC networks is also not covered adequately in the literature. There are papers that cover big data usage in other communication networks, but HFC networks have their specific properties that differ them from other networking technologies. Additionally, neither of the papers that discuss big data usage in other communication networks cover fast failure detection and localization jointly.