# Detection and Localization of Failures in Hybrid Fiber-Coaxial Network Using Big Data Platform
## 1 Introduction  
HFC (Hybrid Fiber-Coaxial) networks evolved from traditional cable TV networks to offer their users a broader spectrum of services, i.e., triple-play service (voice, data, video). HFC networks employ DOCSIS (Data over Cable Service Interface Specification) standards. The DOCSIS 3.1 supports 10 Gbps downstream and 1 Gbps upstream [1]. DOCSIS 4.0 will support even higher capacities and full-duplex transmission [2]. Thus, HFC networks can offer high-quality broadband access to their users.  
Given the trend of remote work, which has been further increased due to the COVID-19 (Coronavirus Disease 2019) pandemic [3, 4], it is of great importance that users are provided high network access availability. Therefore, it is essential for telecom service providers to detect and localize malfunctions and failures in their networks as soon as possible. Thus, an efficient monitoring system needs to be deployed in the network.  
The telecom service provider networks typically comprise huge numbers of network devices and links to provide service to large number of users--usually in the order of millions. Network devices and devices at user premises (CPE--Customer Premises Equipment) can provide a lot of useful collected information. However, the amount of data that can be collected is enormous. Thus, big data technologies must be able to collect, process, and store such an amount of data. Today, many telecom service operators deploy big data platforms to exploit the collected data from their networks [5, 6, 7, 8].  
In this paper, we propose a novel approach to efficiently detect and localize malfunctions and failures in HFC networks using a big data platform. In the remainder of the paper, we refer to our proposed approach as FDLBD--Failure Detection and Localization using Big Data platform. Since big data platforms collect data on a regular basis, the collected data can be used to estimate the health of all devices in the network. However,some devices in the HFC network, such as amplifiers, cannot directly provide their status, i.e., data cannot be collected from such devices. Our proposed solution can indirectly estimate the health of such devices from the data collected from the devices that can provide their status. By using the network topology information, FDLBD can also localize the problematic devices. In this way, complete automatized monitoring of all devices is achieved. Furthermore, the monitoring costs are reduced, since our approach uses the big data platform that already collects data, and there is no need for additional equipment and capacities for the monitoring purposes. Note that the proposed FDLBD is currently successfully deployed in the real HFC network comprising over one million users. The main contributions of the paper are the following:  
* Efficient automatized failure detection and localization in HFC networks.
* Failure detection of devices that cannot be monitored and probed directly.
* Utilization of big data technology for failure detection and localization.  
The remainder of the paper is organized as follows. Related work is presented in Section 2. Section 3 briefly covers typical HFC network architecture. In Section 4, we provide a high-level architecture description of the big data platform that FDLBD uses. Section 5 gives detailed description of FDLBD. Finally, Section 6 concludes the paper.