  Classification algorithms have been widely adopted to detect anomalies for
various systems, e.g., IoT, cloud and face recognition, under the common
assumption that the data source is clean, i.e., features and labels are
correctly set. However, data collected from the wild can be unreliable due to
careless annotations or malicious data transformation for incorrect anomaly
detection. In this paper, we extend a two-layer on-line data selection
framework: Robust Anomaly Detector (RAD) with a newly designed ensemble
prediction where both layers contribute to the final anomaly detection
decision. To adapt to the on-line nature of anomaly detection, we consider
additional features of conflicting opinions of classifiers, repetitive
cleaning, and oracle knowledge. We on-line learn from incoming data streams and
continuously cleanse the data, so as to adapt to the increasing learning
capacity from the larger accumulated data set. Moreover, we explore the concept
of oracle learning that provides additional information of true labels for
difficult data points. We specifically focus on three use cases, (i) detecting
10 classes of IoT attacks, (ii) predicting 4 classes of task failures of big
data jobs, and (iii) recognising 100 celebrities faces. Our evaluation results
show that RAD can robustly improve the accuracy of anomaly detection, to reach
up to 98.95% for IoT device attacks (i.e., +7%), up to 85.03% for cloud task
failures (i.e., +14%) under 40% label noise, and for its extension, it can
reach up to 77.51% for face recognition (i.e., +39%) under 30% label noise. The
proposed RAD and its extensions are general and can be applied to different
anomaly detection algorithms.
