In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

[Q]: We consider the problem of accurately recovering a matrix B of size M ×M , which represents a probability distribution overM outcomes, given access to an observed matrix of “counts” generated by taking independent samples from the distribution B. How can structural properties of the underlying matrix B be leveraged to yield computationally efficient and information theoretically optimal reconstruction algorithms? When can accurate reconstruction be accomplished in the sparse data regime? This basic problem lies at the core of a number of questions that are currently being considered by different communities, including community detection in sparse random graphs, learning structured models such as topic models or hidden Markov models, and the efforts from the natural language processing community to compute “word embeddings”. Many aspects of this problem—both in terms of learning and property testing/estimation and on both the algorithmic and information theoretic sides—remain open. Our results apply to the setting where B has a particular rank 2 structure. For this setting, we propose an efficient (and practically viable) algorithm that accurately recovers the underlying M × M matrix using Θ(M) samples. This result easily translates to Θ(M) sample algorithms for learning topic models with two topics over dictionaries of size M , and learning hidden Markov Models with two hidden states and observation distributions supported on M elements. These linear sample complexities are optimal, up to constant factors, in an extremely strong sense: even testing basic properties of the underlying matrix (such as whether it has rank 1 or 2) requires Ω(M) samples. Furthermore, we provide an even stronger lower bound where distinguishing whether a sequence of observations were drawn from the uniform distribution over M observations versus being generated by an HMM with two hidden states requires Ω(M) observations. This precludes sublinear-sample hypothesis tests for basic properties, such as identity or uniformity, as well as sublinear sample estimators for quantities such as the entropy rate of HMMs. This impossibility of sublinear-sample property testing in these settings is intriguing and underscores the significant differences between these structured settings and the standard setting of drawing i.i.d samples from an unstructured distribution of support size M . ∗MIT. Email: qqh@mit.edu. †University of Washington. Email: sham@cs.washington.edu ‡Stanford University. Email: kweihao@gmail.com §Stanford University. Email: valiant@stanford.edu. Gregory and Weihao’s contributions were supported by NSF CAREER Award CCF-1351108, and a research grant from the Okawa Foundation. ar X iv :1 60 2. 06 58 6v 1 [ cs .L G ] 2 1 Fe b 20 16
[A]: Recovering Structured Probability Matrices


[Q]: We study the problem of Robust Least Squares Regression (RLSR) where several response variables can be adversarially corrupted. More specifically, for a data matrix X ∈ Rp×n and an underlying model w∗, the response vector is generated as y = XTw∗+b where b ∈ R is the corruption vector supported over at most C ·n coordinates. Existing exact recovery results for RLSR focus solely on L1-penalty based convex formulations and impose relatively strict model assumptions such as requiring the corruptions b to be selected independently of X. In this work, we study a simple hard-thresholding algorithm called Torrent which, under mild conditions on X, can recover w∗ exactly even if b corrupts the response variables in an adversarial manner, i.e. both the support and entries of b are selected adversarially after observing X and w∗. Our results hold under deterministic assumptions which are satisfied if X is sampled from any sub-Gaussian distribution. Finally unlike existing results that apply only to a fixed w∗, generated independently of X, our results are universal and hold for any w∗ ∈ R. Next, we propose gradient descent-based extensions of Torrent that can scale efficiently to large scale problems, such as high dimensional sparse recovery and prove similar recovery guarantees for these extensions. Empirically we find Torrent, and more so its extensions, offering significantly faster recovery than the state-of-the-art L1 solvers. For instance, even on moderate-sized datasets (with p = 50K) with around 40% corrupted responses, a variant of our proposed method called Torrent-HYB is more than 20× faster than the best L1 solver. “If among these errors are some which appear too large to be admissible, then those equations which produced these errors will be rejected, as coming from too faulty experiments, and the unknowns will be determined by means of the other equations, which will then give much smaller errors.” A. M. Legendre, On the Method of Least Squares. 1805.
[A]: Robust Regression via Hard Thresholding


[Q]: We propose a method for learning from streaming visual data using a compact, constant size representation of all the data that was seen until a given moment. Specifically, we construct a “coreset” representation of streaming data using a parallelized algorithm, which is an approximation of a set with relation to the squared distances between this set and all other points in its ambient space. We learn an adaptive object appearance model from the coreset tree in constant time and logarithmic space and use it for object tracking by detection. Our method obtains excellent results for object tracking on three standard datasets over more than 100 videos. The ability to summarize data efficiently makes our method ideally suited for tracking in long videos in presence of space and time constraints. We demonstrate this ability by outperforming a variety of algorithms on the TLD dataset with 2685 frames on average. This coreset based learning approach can be applied for both real-time learning of small, varied data and fast learning of big data.
[A]:
Coreset-Based Adaptive Tracking