TASK DEFINITION: In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
PROBLEM: Bayesian neural networks (BNNs) hold great promise as a flexible and principled solution to deal with uncertainty when learning from finite data. Among approaches to realize probabilistic inference in deep neural networks, variational Bayes (VB) is theoretically grounded, generally applicable, and computationally efficient. With wide recognition of potential advantages, why is it that variational Bayes has seen very limited practical use for BNNs in real applications? We argue that variational inference in neural networks is fragile: successful implementations require careful initialization and tuning of prior variances, as well as controlling the variance of Monte Carlo gradient estimates. We provide two innovations that aim to turn VB into a robust inference tool for Bayesian neural networks: first, we introduce a novel deterministic method to approximate moments in neural networks, eliminating gradient variance; second, we introduce a hierarchical prior for parameters and a novel Empirical Bayes procedure for automatically selecting prior variances. Combining these two innovations, the resulting method is highly efficient and robust. On the application of heteroscedastic regression we demonstrate good predictive performance over alternative approaches.

SOLUTION: A method for eliminating gradient variance and automatically tuning priors for effective training of bayesian neural networks

PROBLEM: In this work we propose a simple and efficient framework for learning sentence representations from unlabelled data. Drawing inspiration from the distributional hypothesis and recent work on learning sentence representations, we reformulate the problem of predicting the context in which a sentence appears as a classification problem. Given a sentence and the context in which it appears, a classifier distinguishes context sentences from other contrastive sentences based on their vector representations. This allows us to efficiently learn different types of encoding functions, and we show that the model learns high-quality sentence representations. We demonstrate that our sentence representations outperform state-of-the-art unsupervised and supervised representation learning methods on several downstream NLP tasks that involve understanding sentence semantics while achieving an order of magnitude speedup in training time.

SOLUTION: A framework for learning high-quality sentence representations efficiently.

PROBLEM: Memory Network based models have shown a remarkable progress on the task of relational reasoning.
 Recently, a simpler yet powerful neural network module called Relation Network (RN) has been introduced. 
 Despite its architectural simplicity, the time complexity of relation network grows quadratically with data, hence limiting its application to tasks with a large-scaled memory.
 We introduce Related Memory Network, an end-to-end neural network architecture exploiting both memory network and relation network structures. 
 We follow memory network's four components while each component operates similar to the relation network without taking a pair of objects. 
 As a result, our model is as simple as RN but the computational complexity is reduced to linear time.
 It achieves the state-of-the-art results in jointly trained bAbI-10k story-based question answering and  bAbI dialog dataset.

SOLUTION:
A simple reasoning architecture based on the memory network (MemNN) and relation network (RN), reducing the time complexity compared to the RN and achieving state-of-the-are result on bAbI story based QA and bAbI dialog.