Abstract-Online learning has become increasingly popular on handling massive data. The sequential nature of online learning, however, requires a centralized learner to store data and update parameters. In this paper, we consider online learning with distributed data sources. The autonomous learners update local parameters based on local data sources and periodically exchange information with a small subset of neighbors in a communication network. We derive the regret bound for strongly convex functions that generalizes the work by Ram et al. [1] for convex functions. More importantly, we show that our algorithm has intrinsic privacypreserving properties, and we prove the sufficient and necessary conditions for privacy preservation in the network. These conditions imply that for networks with greater-than-one connectivity, a malicious learner cannot reconstruct the subgradients (and sensitive raw data) of other learners, which makes our algorithm appealing in privacy-sensitive applications.
In REF , the authors proposed distributed online learning with intrinsic privacy-preserving properties, where local parameters at each participating user are updated based on local data sources, and parameters are periodically exchanged among a small subset of neighbors, and they showed that malicious users cannot reconstruct the subgradients of other users.
Distributed Autonomous Online Learning: Regrets and Intrinsic Privacy-Preserving Properties
IEEE Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering
Mathematics Computer Science