instruction:
In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
question:
Latent variable models are a fundamental modeling tool in machine learning applications, but they present significant computational and analytical challenges. The popular EM algorithm and its variants, is a much used algorithmic tool; yet our rigorous understanding of its performance is highly incomplete. Recently, work in Balakrishnan et al. (2014) has demonstrated that for an important class of problems, EM exhibits linear local convergence. In the high-dimensional setting, however, the M -step may not be well defined. We address precisely this setting through a unified treatment using regularization. While regularization for high-dimensional problems is by now well understood, the iterative EM algorithm requires a careful balancing of making progress towards the solution while identifying the right structure (e.g., sparsity or low-rank). In particular, regularizing the M -step using the state-of-the-art high-dimensional prescriptions (e.g., à la Wainwright (2014)) is not guaranteed to provide this balance. Our algorithm and analysis are linked in a way that reveals the balance between optimization and statistical errors. We specialize our general framework to sparse gaussian mixture models, high-dimensional mixed regression, and regression with missing variables, obtaining statistical guarantees for each of these examples.
answer:
Regularized EM Algorithms: A Unified Framework and Statistical Guarantees


question:
Self-paced learning (SPL) has been attracting increasing attention in machine learning and computer vision. Albeit empirically substantiated to be effective, the investigation on its theoretical insight is still a blank. It is even unknown that what objective a general SPL regime converges to. To this issue, this study attempts to initially provide some new insights under this “heuristic” learning scheme. Specifically, we prove that the solving strategy on SPL exactly accords with a majorization minimization algorithm, a well known technique in optimization and machine learning, implemented on a latent objective. A more interesting finding is that, the loss function contained in this latent objective has a similar configuration with non-convex regularized penalty, an attractive topic in statistics and machine learning. In particular, we show that the previous hard and linear self-paced regularizers are equivalent to the capped norm and minimax concave plus penalties, respectively, both being widely investigated in statistics. Such connections between SPL and previous known researches enhance new insightful comprehension on SPL, like convergence and parameter setting rationality. The correctness of the proposed theory is substantiated by experimental results on synthetic and UCI data sets.
answer:
What Objective Does Self-paced Learning Indeed Optimize?


question:
A basic problem in the design of privacy-preserving algorithms is the private maximization problem: the goal is to pick an item from a universe that (approximately) maximizes a data-dependent function, all under the constraint of differential privacy. This problem has been used as a sub-routine in many privacy-preserving algorithms for statistics and machine-learning. Previous algorithms for this problem are either range-dependent—i.e., their utility diminishes with the size of the universe—or only apply to very restricted function classes. This work provides the first general-purpose, range-independent algorithm for private maximization that guarantees approximate differential privacy. Its applicability is demonstrated on two fundamental tasks in data mining and machine learning.
answer:

The Large Margin Mechanism for Differentially Private Maximization