Networks are constantly generating an enormous amount of rich diverse information. Such information creates exciting opportunities for network analytics. However, a major challenge to enable effective network analytics is the presence of missing data, measurement errors, and anomalies. Despite significant work in network analytics, fundamental issues remain: (i) the existing works do not explicitly account for anomalies or measurement noise, and incur serious performance degradation under significant noise or anomalies, and (ii) they assume network matrices have low-rank structure, which may not hold in reality. To address these issues, in this paper we develop LENS decomposition, a novel technique to accurately decompose a network matrix into a low-rank matrix, a sparse anomaly matrix, an error matrix, and a small noise matrix. LENS has the following nice properties: (i) it is general: it can effectively support matrices with or without anomalies, and having low-rank or not, (ii) its parameters are self tuned so that it can adapt to different types of data, (iii) it is accurate by incorporating domain knowledge, such as temporal locality, spatial locality, and initial estimate (e.g., obtained from models), (iv) it is versatile and can support many applications including missing value interpolation, prediction, and anomaly detection. We apply LENS to a wide range of network matrices from 3G, WiFi, mesh, sensor networks, and the Internet. Our results show that LENS significantly out-performs state-of-the-art compressive sensing schemes.
Chen et al. REF developed a novel decomposition technique to accurately decompose a network matrix into a low-rank matrix, a sparse anomaly matrix, an error matrix, and a small noise matrix, which is used for removing noisy data.
Robust network compressive sensing
MobiCom '14
Computer Science