TASK DEFINITION: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
PROBLEM: This paper presents to the best of our knowledge the first end-to-end object tracking approach which directly maps from raw sensor input to object tracks in sensor space without requiring any feature engineering or system identification in the form of plant or sensor models. Specifically, our system accepts a stream of raw sensor data at one end and, in real-time, produces an estimate of the entire environment state at the output including even occluded objects. We achieve this by framing the problem as a deep learning task and exploit sequence models in the form of recurrent neural networks to learn a mapping from sensor measurements to object tracks. In particular, we propose a learning method based on a form of input dropout which allows learning in an unsupervised manner, only based on raw, occluded sensor data without access to ground-truth annotations. We demonstrate our approach using a synthetic dataset designed to mimic the task of tracking objects in 2D laser data – as commonly encountered in robotics applications – and show that it learns to track many dynamic objects despite occlusions and the presence of sensor noise.

SOLUTION: Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks

PROBLEM: Common statistical practice has shown that the full power of Bayesian methods is not realized until hierarchical priors are used, as these allow for greater “robustness” and the ability to “share statistical strength.” Yet it is an ongoing challenge to provide a learning-theoretically sound formalism of such notions that: offers practical guidance concerning when and how best to utilize hierarchical models; provides insights into what makes for a good hierarchical prior; and, when the form of the prior has been chosen, can guide the choice of hyperparameter settings. We present a set of analytical tools for understanding hierarchical priors in both the online and batch learning settings. We provide regret bounds under log-loss, which show how certain hierarchical models compare, in retrospect, to the best single model in the model class. We also show how to convert a Bayesian log-loss regret bound into a Bayesian risk bound for any bounded loss, a result which may be of independent interest. Risk and regret bounds for Student’s t and hierarchical Gaussian priors allow us to formalize the concepts of “robustness” and “sharing statistical strength.” Priors for feature selection are investigated as well. Our results suggest that the learning-theoretic benefits of using hierarchical priors can often come at little cost on practical problems.

SOLUTION: Risk and Regret of Hierarchical Bayesian Learners

PROBLEM: Low dimensional representations of words allow<lb>accurate NLP models to be trained on limited<lb>annotated data. While most representations ig-<lb>nore words’ local context, a natural way to in-<lb>duce context-dependent representations is to per-<lb>form inference in a probabilistic latent-variable<lb>sequence model. Given the recent success of<lb>continuous vector space word representations,<lb>we provide such an inference procedure for con-<lb>tinuous states, where words’ representations are<lb>given by the posterior mean of a linear dynam-<lb>ical system. Here, efficient inference can be<lb>performed using Kalman filtering. Our learn-<lb>ing algorithm is extremely scalable, operating<lb>on simple cooccurrence counts for both param-<lb>eter initialization using the method of moments<lb>and subsequent iterations of EM. In our exper-<lb>iments, we employ our inferred word embed-<lb>dings as features in standard tagging tasks, ob-<lb>taining significant accuracy improvements. Fi-<lb>nally, the Kalman filter updates can be seen as a<lb>linear recurrent neural network. We demonstrate<lb>that using the parameters of our model to ini-<lb>tialize a non-linear recurrent neural network lan-<lb>guage model reduces its training time by a day<lb>and yields lower perplexity.

SOLUTION:
A Linear Dynamical System Model for Text