# Predicting Hard Disk Failure by Means of Automatized Labeling and Machine Learning Approach
## 1 Introduction  
The new frontiers of industrial production and scientific research are reflected in a growing demand for computing and, consequently, in an evolution of storage technologies. The increased demand for data storage and processing pushes the data-centers to provide increasingly efficient services [1]. This efficiency requirement can be compromised by the devices that underpin storage: hard drives. Magnetic hard disks are currently among the most widely used and devices for storing data. Magnetic hard disks are currently among the most widely used and among the most frequently failing components of storage systems and disk failures, resulting in about 78% of cloud server system failures [2]. In addition to the impact of a service outage for the user, these events also result in significant costs for the service provider. As unscheduled interventions, they may require rapid (and hence costly) human intervention or a resource intensive consistency check and re-validation of recent processing steps by a combination of the service provider and user. In the worst case, disk failures can lead to permanent data loss and hence economical damage to the user and service provider due to a breach of service level agreements [2, 3, 4]. While fault tolerance techniques can reduce the risk of loss, they also often impact negatively on system performance or price per usable volume. In order to facilitate the automated interpretation of the disk operational status, the disk vendors introduced Self-Monitoring, Analysis and Reporting technology (SMART),a standardized monitoring system with the goal to warn about an increasing likelihood of disk failures. The SMART subsystem is a firmware component running on the disk processor and continuously collects operational metrics in order to generate a warning or error messages before disk failures that affect the integrity of application data. The characteristics of SMART technology include a number of attributes for diagnostics, the SMART attribute implementation technology varies from model to model, even for different disk models of the same manufacturer [5]. The goal of the SMART system is to warn a user at least 24 h before device failure [6]. Each manufacturing company implements SMART sensors according to their proprietary technology choices and sets alarm thresholds with the aim of an acceptable rate of false-alarms and its potential impact on drive warranty obligations. Generally, the false-alarm rate is around 0.1% per year [7,8]. Several methods based on modeling techniques have been proposed to recognize hard drives that need to be replaced using SMART attributes [8, 9, 10, 11], but none of them meet all the needs of adapting to heterogeneous sets of hard drives, low false positive and false negative rates, automatic preliminary dataset labeling. To overcome the limitations considered before, we propose an inference method to retrieve information about failed disks and a supervised machine learning approach to build a predictor of near-failure disks. The method presented in this paper does not need a human intervention for dataset labeling and is capable of operating with heterogeneous sets of magnetic hard disks. The proposal has been validated in a case study at the European Organization for Nuclear Research (CERN), where the High Energy Physics (HEP) challenges have led storage services to become a crucial part of the cloud services [12]. CERN uses a variety of models [13], and an effective prediction of a likely drive malfunction would allow increasing the user's perceived service quality and, at the same time, decrease the resources to operate the service. The remainder of this paper is organized as follows: in Section 2, we set the scene with a brief review of the state-of-the-art technology in hard disk failure prediction; in Section 3, we provide a short summary of the machine learning algorithm we have chosen, followed, in Section 4, by a detailed description of its application to our classification problem.