instruction:
In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
question:
In this paper we propose to view the acceptance rate of the Metropolis-Hastings algorithm as a universal objective for learning to sample from target distribution -- given either as a set of samples or in the form of unnormalized density. This point of view unifies the goals of such approaches as Markov Chain Monte Carlo (MCMC), Generative Adversarial Networks (GANs), variational inference. To reveal the connection we derive the lower bound on the acceptance rate and treat it as the objective for learning explicit and implicit samplers. The form of the lower bound allows for doubly stochastic gradient optimization in case the target distribution factorizes (i.e. over data points). We empirically validate our approach on Bayesian inference for neural networks and generative models for images.
answer:
Learning to sample via lower bounding the acceptance rate of the Metropolis-Hastings algorithm

question:
Likelihood-based generative models are a promising resource to detect out-of-distribution (OOD) inputs which could compromise the robustness or reliability of a machine learning system. However, likelihoods derived from such models have been shown to be problematic for detecting certain types of inputs that significantly differ from training data. In this paper, we pose that this problem is due to the excessive influence that input complexity has in generative models' likelihoods. We report a set of experiments supporting this hypothesis, and use an estimate of input complexity to derive an efficient and parameter-free OOD score, which can be seen as a likelihood-ratio, akin to Bayesian model comparison. We find such score to perform comparably to, or even better than, existing OOD detection approaches under a wide range of data sets, models, model sizes, and complexity estimates.
answer:
We pose that generative models' likelihoods are excessively influenced by the input's complexity, and propose a way to compensate it when detecting out-of-distribution inputs

question:
Understanding the groundbreaking performance of Deep Neural Networks is one
of the greatest challenges to the scientific community today. In this work, we
introduce an information theoretic viewpoint on the behavior of deep networks
optimization processes and their generalization abilities. By studying the Information
Plane, the plane of the mutual information between the input variable and
the desired label, for each hidden layer. Specifically, we show that the training of
the network is characterized by a rapid increase in the mutual information (MI)
between the layers and the target label, followed by a longer decrease in the MI
between the layers and the input variable. Further, we explicitly show that these
two fundamental information-theoretic quantities correspond to the generalization
error of the network, as a result of introducing a new generalization bound that is
exponential in the representation compression. The analysis focuses on typical
patterns of large-scale problems. For this purpose, we introduce a novel analytic
bound on the mutual information between consecutive layers in the network.
An important consequence of our analysis is a super-linear boost in training time
with the number of non-degenerate hidden layers, demonstrating the computational
benefit of the hidden layers.
answer:
Introduce an information theoretic viewpoint on the behavior of deep networks optimization processes and their generalization abilities