instruction:
In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
question:
Common statistical practice has shown that the full power of Bayesian methods is not realized until hierarchical priors are used, as these allow for greater “robustness” and the ability to “share statistical strength.” Yet it is an ongoing challenge to provide a learning-theoretically sound formalism of such notions that: offers practical guidance concerning when and how best to utilize hierarchical models; provides insights into what makes for a good hierarchical prior; and, when the form of the prior has been chosen, can guide the choice of hyperparameter settings. We present a set of analytical tools for understanding hierarchical priors in both the online and batch learning settings. We provide regret bounds under log-loss, which show how certain hierarchical models compare, in retrospect, to the best single model in the model class. We also show how to convert a Bayesian log-loss regret bound into a Bayesian risk bound for any bounded loss, a result which may be of independent interest. Risk and regret bounds for Student’s t and hierarchical Gaussian priors allow us to formalize the concepts of “robustness” and “sharing statistical strength.” Priors for feature selection are investigated as well. Our results suggest that the learning-theoretic benefits of using hierarchical priors can often come at little cost on practical problems.
answer:
Risk and Regret of Hierarchical Bayesian Learners


question:
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.
answer:
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling


question:
Named Entity Recognition (NER) is a key NLP task, which is all the more chal-<lb>lenging on Web and user-generated content with their diverse and continuously<lb>changing language. This paper aims to quantify how this diversity impacts<lb>state-of-the-art NER methods, by measuring named entity (NE) and context<lb>variability, feature sparsity, and their effects on precision and recall. In particu-<lb>lar, our findings indicate that NER approaches struggle to generalise in diverse<lb>genres with limited training data. Unseen NEs, in particular, play an impor-<lb>tant role, which have a higher incidence in diverse genres such as social media<lb>than in more regular genres such as newswire. Coupled with a higher incidence<lb>of unseen features more generally and the lack of large training corpora, this<lb>leads to significantly lower F1 scores for diverse genres as compared to more<lb>regular ones. We also find that leading systems rely heavily on surface forms<lb>found in training data, having problems generalising beyond these, and offer<lb>explanations for this observation.
answer:

Generalisation in Named Entity Recognition: A Quantitative Analysis