ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Utility Analysis of Network Architectures for 3D Point Cloud Processing

Paper content:
In this paper, we diagnose deep neural networks for 3D point cloud processing to explore the utility of different network architectures.
We propose a number of hypotheses on the effects of specific network architectures on the representation capacity of DNNs.
In order to prove the hypotheses, we design five metrics to diagnose various types of DNNs from the following perspectives, information discarding, information concentration, rotation robustness, adversarial robustness, and neighborhood inconsistency.
We conduct comparative studies based on such metrics to verify the hypotheses, which may shed new lights on the architectural design of neural networks.
Experiments demonstrated the effectiveness of our method.
The code will be released when this paper is accepted.

assistant: text: We diagnose deep neural networks for 3D point cloud processing to explore the utility of different network architectures.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: One Generation Knowledge Distillation by Utilizing Peer Samples

Paper content:
Knowledge Distillation (KD) is a widely used technique in recent deep learning research to obtain small and simple models whose performance is on a par with their large and complex counterparts.
Standard Knowledge Distillation tends to be time-consuming because of the training time spent to obtain a teacher model that would then provide guidance for the student model.
It might be possible to cut short the time by training a teacher model on the fly, but it is not trivial to have such a high-capacity teacher that gives quality guidance to student models this way.
To improve this, we present a novel framework of Knowledge Distillation exploiting dark knowledge from the whole training set.
In this framework, we propose a simple and effective implementation named Distillation by Utilizing Peer Samples (DUPS) in one generation.
We verify our algorithm on numerous experiments.
Compared with standard training on modern architectures, DUPS achieves an average improvement of 1%-2% on various tasks with nearly zero extra cost.
Considering some typical Knowledge Distillation methods which are much more time-consuming, we also get comparable or even better performance using DUPS.

assistant: text: Proposes a method for improving the effectiveness of knowledge distillation by softening the labels used and employing a dataset instead of a single sample.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: What Can Neural Networks Reason About?

Paper content:
Neural networks have succeeded in many reasoning tasks.
Empirically, these tasks require specialized network structures, e.g., Graph Neural Networks (GNNs) perform well on many such tasks, while less structured networks fail.
Theoretically, there is limited understanding of why and when a network structure generalizes better than other equally expressive ones.
We develop a framework to characterize which reasoning tasks a network can learn well, by studying how well its structure aligns with the algorithmic structure of the relevant reasoning procedure.
We formally define algorithmic alignment and derive a sample complexity bound that decreases with better alignment.
This framework explains the empirical success of popular reasoning models and suggests their limitations.
We unify seemingly different reasoning tasks, such as intuitive physics, visual question answering, and shortest paths, via the lens of a powerful algorithmic paradigm, dynamic programming (DP).
We show that GNNs can learn DP and thus solve these tasks.
On several reasoning tasks, our theory aligns with empirical results.

assistant: text: We develop a theoretical framework to characterize which reasoning tasks a neural network can learn well.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adaptive Quantization of Neural Networks

Paper content:
Despite the state-of-the-art accuracy of Deep Neural Networks (DNN) in various classification problems, their deployment onto resource constrained edge computing devices remains challenging due to their large size and complexity.
Several recent studies have reported remarkable results in reducing this complexity through quantization of DNN models.
However, these studies usually do not consider the changes in the loss function when performing quantization, nor do they take the different importances of DNN model parameters to the accuracy into account.
We address these issues in this paper by proposing a new method, called adaptive quantization, which simplifies a trained DNN model by finding a unique, optimal precision for each network parameter such that the increase in loss is minimized.
The optimization problem at the core of this method iteratively uses the loss function gradient to determine an error margin for each parameter and assigns it a precision accordingly.
Since this problem uses linear functions, it is computationally cheap and, as we will show, has a closed-form approximate solution.
Experiments on MNIST, CIFAR, and SVHN datasets showed that the proposed method can achieve near or better than state-of-the-art reduction in model size with similar error rates.
Furthermore, it can achieve compressions close to floating-point model compression methods without loss of accuracy.

assistant: text: Proposes a method for quantizing neural networks that allow weights to be quantized with different precision depending on their importance, taking into account the loss.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Augmentation via Data Subsampling

Paper content:
Data augmentation is commonly used to encode invariances in learning methods.
However, this process is often performed in an inefficient manner, as artificial examples are created by applying a number of transformations to all points in the training set.
The resulting explosion of the dataset size can be an issue in terms of storage and training costs, as well as in selecting and tuning the optimal set of transformations to apply.
In this work, we demonstrate that it is possible to significantly reduce the number of data points included in data augmentation while realizing the same accuracy and invariance benefits of augmenting the entire dataset.
We propose a novel set of subsampling policies, based on model influence and loss, that can achieve a 90% reduction in augmentation set size while maintaining the accuracy gains of standard data augmentation.

assistant: text: The authors propose to use influence or loss-based methods to select a subset of points to use in augmenting data sets for training models where the loss is additive over data points.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Foveated Downsampling Techniques

Paper content:
Foveation is an important part of human vision, and a number of deep networks have also used foveation.
However, there have been few systematic comparisons between foveating and non-foveating deep networks, and between different variable-resolution downsampling methods.
Here we define several such methods, and compare their performance on ImageNet recognition with a Densenet-121 network.
The best variable-resolution method slightly outperforms uniform downsampling.
Thus in our experiments, foveation does not substantially help or hinder object recognition in deep networks.

assistant: text: We compare object recognition performance on images that are downsampled uniformly and with three different foveation schemes.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: $\textrm{D}^2$GAN: A Few-Shot Learning Approach with Diverse and Discriminative Feature Synthesis

Paper content:
The rich and accessible labeled data fuel the revolutionary success of deep learning.
Nonetheless, massive supervision remains a luxury for many real applications, boosting great interest in label-scarce techniques such as few-shot learning (FSL).
An intuitively feasible approach to FSL is to conduct data augmentation via synthesizing additional training samples.
The key to this approach is how to guarantee both discriminability and diversity of the synthesized samples.
In this paper, we propose a novel FSL model, called $\textrm{D}^2$GAN, which synthesizes Diverse and Discriminative features based on Generative Adversarial Networks (GAN).
$\textrm{D}^2$GAN secures discriminability of the synthesized features by constraining them to have high correlation with real features of the same classes while low correlation with those of different classes.
Based on the observation that noise vectors that are closer in the latent code space are more likely to be collapsed into the same mode when mapped to feature space, $\textrm{D}^2$GAN incorporates a novel anti-collapse regularization term, which encourages feature diversity by penalizing the ratio of the logarithmic similarity of two synthesized features and the logarithmic similarity of the latent codes generating them.
Experiments on three common benchmark datasets verify the effectiveness of $\textrm{D}^2$GAN by comparing with the state-of-the-art.

assistant: text: A meta-learning method that learns a generative model that can augment the support set of a few-shot learner which optimizes a combination of losses.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Scalable Rule Learning in Probabilistic Knowledge Bases

Paper content:
Knowledge Bases (KBs) are becoming increasingly large, sparse and probabilistic.
These KBs are typically used to perform query inferences and rule mining.
But their efficacy is only as high as their completeness.
Efficiently utilizing incomplete KBs remains a major challenge as the current KB completion techniques either do not take into account the inherent uncertainty associated with each KB tuple or do not scale to large KBs.
Probabilistic rule learning not only considers the probability of every KB tuple but also tackles the problem of KB completion in an explainable way.
For any given probabilistic KB, it learns probabilistic first-order rules from its relations to identify interesting patterns.
But, the current probabilistic rule learning techniques perform grounding to do probabilistic inference for evaluation of candidate rules.
It does not scale well to large KBs as the time complexity of inference using grounding is exponential over the size of the KB.
In this paper, we present SafeLearner -- a scalable solution to probabilistic KB completion that performs probabilistic rule learning using lifted probabilistic inference -- as faster approach instead of grounding.
We compared SafeLearner to the state-of-the-art probabilistic rule learner ProbFOIL+ and to its deterministic contemporary AMIE+ on standard probabilistic KBs of NELL (Never-Ending Language Learner) and Yago.
Our results demonstrate that SafeLearner scales as good as AMIE+ when learning simple rules and is also significantly faster than ProbFOIL+.

assistant: text: Probabilistic Rule Learning system using Lifted Inference
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Explanation  by Progressive  Exaggeration

Paper content:
As machine learning methods see greater adoption and implementation in high stakes applications such
as medical image diagnosis, the need for model interpretability and explanation has become more
critical.
Classical approaches that assess feature importance (eg saliency maps) do not explain how and why a particular region of an image is relevant to the prediction.
We propose
a method that explains the outcome of a classification black-box by gradually exaggerating
the semantic effect of a given class.
Given a query input to a classifier, our method produces a
progressive set of plausible variations of that query, which gradually change the posterior probability
from its original class to its negation.
These counter-factually generated samples preserve features
unrelated to the classification decision, such that a user can employ our method as a ``tuning knob'' to traverse a data manifold while crossing the decision boundary.
Our method is model agnostic and only requires the output value and gradient of the predictor with respect to its input.

assistant: text: A model that when given a query input to a black-box, aims to explain the outcome by providing plausible and progressive variations to the query that can result in a change to the output.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Inference Amortization in Graphical Models using Structured Continuous Conditional Normalizing Flows

Paper content:
We introduce a more efficient neural architecture for amortized inference, which combines continuous and conditional normalizing flows using a principled choice of structure.
Our gradient flow derives its sparsity pattern from the minimally faithful inverse of its underlying graphical model.
We find that this factorization reduces the necessary numbers both of parameters in the neural network and of adaptive integration steps in the ODE solver.
Consequently, the throughput at training time and inference time is increased, without decreasing performance in comparison to unconstrained flows.
By expressing the structural inversion and the flow construction as compilation passes of a probabilistic programming language, we demonstrate their applicability to the stochastic inversion of realistic models such as convolutional neural networks (CNN).

assistant: text: We introduce a more efficient neural architecture for amortized inference, which combines continuous and conditional normalizing flows using a principled choice of sparsity structure.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generalizing Natural Language Analysis through Span-relation Representations

Paper content:
A large number of natural language processing tasks exist to analyze syntax, semantics, and information content of human language.
These seemingly very different tasks are usually solved by specially designed architectures.
In this paper, we provide the simple insight that a great variety of tasks can be represented in a single unified format consisting of labeling spans and relations between spans, thus a single task-independent model can be used across different tasks.
We perform extensive experiments to test this insight on 10 disparate tasks as broad as dependency parsing (syntax), semantic role labeling (semantics), relation extraction (information content), aspect based sentiment analysis (sentiment), and many others, achieving comparable performance as state-of-the-art specialized models.
We further demonstrate benefits in multi-task learning.
We convert these datasets into a unified format to build a benchmark, which provides a holistic testbed for evaluating future models for generalized natural language analysis.

assistant: text: We use a single model to solve a great variety of natural language analysis tasks by formulating them in a unified span-relation format.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Leveraging Simple Model Predictions for Enhancing its Performance

Paper content:
There has been recent interest in improving performance of simple models for multiple reasons such as interpretability, robust learning from small data, deployment in memory constrained settings as well as environmental considerations.
In this paper, we propose a novel method SRatio that can utilize information from high performing complex models (viz. deep neural networks, boosted trees, random forests) to reweight a training dataset for a potentially low performing simple model such as a decision tree or a shallow network enhancing its performance.
Our method also leverages the per sample hardness estimate of the simple model which is not the case with the prior works which primarily consider the complex model's confidences/predictions and is thus conceptually novel.
Moreover, we generalize and formalize the concept of attaching probes to intermediate layers of a neural network, which was one of the main ideas in previous work \citep{profweight}, to other commonly used classifiers and incorporate this into our method.
The benefit of these contributions is witnessed in the experiments where on 6 UCI datasets and CIFAR-10 we outperform competitors in a majority (16 out of 27) of the cases and tie for best performance in the remaining cases.
In fact, in a couple of cases, we even approach the complex model's performance.
We also conduct further experiments to validate assertions and intuitively understand why our method works.
Theoretically, we motivate our approach by showing that the weighted loss minimized by simple models using our weighting upper bounds the loss of the complex model.

assistant: text: Method to improve simple models performance given a (accurate) complex model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Restricting the Flow: Information Bottlenecks for Attribution

Paper content:
Attribution methods provide insights into the decision-making of machine learning models like artificial neural networks.
For a given input sample, they assign a relevance score to each individual input variable, such as the pixels of an image.
In this work we adapt the information bottleneck concept for attribution.
By adding noise to intermediate feature maps we restrict the flow of information and can quantify (in bits) how much information image regions provide.
We compare our method against ten baselines using three different metrics on VGG-16 and ResNet-50, and find that our methods outperform all baselines in five out of six settings.
The method’s information-theoretic foundation provides an absolute frame of reference for attribution values (bits) and a guarantee that regions scored close to zero are not necessary for the network's decision.

assistant: text: The paper proposes a novel perturbation-based method for computing attribution/saliency maps for deep neural network based image classifiers, by injecting crafted noise into an early layer of the network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Fully Quantized Transformer for Improved Translation

Paper content:
State-of-the-art neural machine translation methods employ massive amounts of parameters.
Drastically reducing computational costs of such methods without affecting performance has been up to this point unsolved.
In this work, we propose a quantization strategy tailored to the Transformer architecture.
We evaluate our method on the WMT14 EN-FR and WMT14 EN-DE translation tasks and achieve state-of-the-art quantization results for the Transformer, obtaining no loss in BLEU scores compared to the non-quantized baseline.
We further compress the Transformer by showing that, once the model is trained, a good portion of the nodes in the encoder can be removed without causing any loss in BLEU.

assistant: text: We fully quantize the Transformer to 8-bit and improve translation quality compared to the full precision model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Inference and Exploration for Reinforcement Learning

Paper content:
Despite an ever growing literature on reinforcement learning algorithms and applications, much less is known about their statistical inference.
In this paper, we investigate the large-sample behaviors of the Q-value estimates with closed-form characterizations of the asymptotic variances.
This allows us to efficiently construct confidence regions for Q-value and optimal value functions, and to develop policies to minimize their estimation errors.
This also leads to a policy exploration strategy that relies on estimating the relative discrepancies among the Q estimates.
Numerical experiments show superior performances of our exploration strategy than other benchmark approaches.

assistant: text: We investigate the large-sample behaviors of the Q-value estimates and proposed an efficient exploration strategy that relies on estimating the relative discrepancies among the Q estimates.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarial AutoAugment

Paper content:
Data augmentation (DA) has been widely utilized to improve generalization in training deep neural networks.
Recently, human-designed data augmentation has been gradually replaced by automatically learned augmentation policy.
Through finding the best policy in well-designed search space of data augmentation, AutoAugment (Cubuk et al., 2019) can significantly improve validation accuracy on image classification tasks.
However, this approach is not computationally practical for large-scale problems.
In this paper, we develop an adversarial method to arrive at a computationally-affordable solution called Adversarial AutoAugment, which can simultaneously optimize target related object and augmentation policy search loss.
The augmentation policy network attempts to increase the training loss of a target network through generating adversarial augmentation policies, while the target network can learn more robust features from harder examples to improve the generalization.
In contrast to prior work, we reuse the computation in target network training for policy evaluation, and dispense with the retraining of the target network.
Compared to AutoAugment, this leads to about 12x reduction in computing cost and 11x shortening in time overhead on ImageNet.
We show experimental results of our approach on CIFAR-10/CIFAR-100, ImageNet, and demonstrate significant performance improvements over state-of-the-art.
On CIFAR-10, we achieve a top-1 test error of 1.36%, which is the currently best performing single model.
On ImageNet, we achieve a leading performance of top-1 accuracy 79.40% on ResNet-50 and 80.00% on ResNet-50-D without extra data.

assistant: text: We introduce the idea of adversarial learning into automatic data augmentation to improve the generalization  of a targe network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: $\textrm{D}^2$GAN: A Few-Shot Learning Approach with Diverse and Discriminative Feature Synthesis

Paper content:
The rich and accessible labeled data fuel the revolutionary success of deep learning.
Nonetheless, massive supervision remains a luxury for many real applications, boosting great interest in label-scarce techniques such as few-shot learning (FSL).
An intuitively feasible approach to FSL is to conduct data augmentation via synthesizing additional training samples.
The key to this approach is how to guarantee both discriminability and diversity of the synthesized samples.
In this paper, we propose a novel FSL model, called $\textrm{D}^2$GAN, which synthesizes Diverse and Discriminative features based on Generative Adversarial Networks (GAN).
$\textrm{D}^2$GAN secures discriminability of the synthesized features by constraining them to have high correlation with real features of the same classes while low correlation with those of different classes.
Based on the observation that noise vectors that are closer in the latent code space are more likely to be collapsed into the same mode when mapped to feature space, $\textrm{D}^2$GAN incorporates a novel anti-collapse regularization term, which encourages feature diversity by penalizing the ratio of the logarithmic similarity of two synthesized features and the logarithmic similarity of the latent codes generating them.
Experiments on three common benchmark datasets verify the effectiveness of $\textrm{D}^2$GAN by comparing with the state-of-the-art.
The rich and accessible labeled data fuel the revolutionary success of deep learning.
Nonetheless, massive supervision remains a luxury for many real applications, boosting great interest in label-scarce techniques such as few-shot learning (FSL).
An intuitively feasible approach to FSL is to conduct data augmentation via synthesizing additional training samples.
The key to this approach is how to guarantee both discriminability and diversity of the synthesized samples.
In this paper, we propose a novel FSL model, called D 2 GAN, which synthesizes Diverse and Discriminative features based on Generative Adversarial Networks (GAN).
D 2 GAN secures discriminability of the synthesized features by constraining them to have high correlation with real features of the same classes while low correlation with those of different classes.
Based on the observation that noise vectors that are closer in the latent code space are more likely to be collapsed into the same mode when mapped to feature space, D 2 GAN incorporates a novel anti-collapse regularization term, which encourages feature diversity by penalizing the ratio of the logarithmic similarity of two synthesized features and the logarithmic similarity of the latent codes generating them.
Experiments on three common benchmark datasets verify the effectiveness of D 2 GAN by comparing with the state-of-the-art.

assistant: text: A new GAN based few-shot learning algorithm by synthesizing  diverse and discriminative Features
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: What do you learn from context? Probing for sentence structure in contextualized word representations

Paper content:
Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks.
Building on recent token-level probing work, we introduce a novel edge probing task design and construct a broad suite of sub-sentence tasks derived from the traditional structured NLP pipeline.
We probe word-level contextual representations from four recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena.
We find that existing models trained on language modeling and translation produce strong representations for syntactic phenomena, but only offer comparably small improvements on semantic tasks over a non-contextual baseline.

assistant: text: We probe for sentence structure in ELMo and related contextual embedding models. We find existing models efficiently encode syntax and show evidence of long-range dependencies, but only offer small improvements on semantic tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SDGM: Sparse Bayesian Classifier Based on a Discriminative Gaussian Mixture Model

Paper content:
In probabilistic classification, a discriminative model based on Gaussian mixture exhibits flexible fitting capability.
Nevertheless, it is difficult to determine the number of components.
We propose a sparse classifier based on a discriminative Gaussian mixture model (GMM), which is named sparse discriminative Gaussian mixture (SDGM).
In the SDGM, a GMM-based discriminative model is trained by sparse Bayesian learning.
This learning algorithm improves the generalization capability by obtaining a sparse solution and automatically determines the number of components by removing redundant components.
The SDGM can be embedded into neural networks (NNs) such as convolutional NNs and can be trained in an end-to-end manner.
Experimental results indicated that the proposed method prevented overfitting by obtaining sparsity.
Furthermore, we demonstrated that the proposed method outperformed a fully connected layer with the softmax function in certain cases when it was used as the last layer of a deep NN.

assistant: text: This paper proposes a classifier, called SDGM, based on discriminative Gaussian mixture and its sparse parameter estimation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Weight-space symmetry in neural network loss landscapes revisited

Paper content:
Neural network training depends on the structure of the underlying loss landscape, i.e. local minima, saddle points, flat plateaus, and loss barriers.
In relation to the structure of the landscape, we study the permutation symmetry of neurons in each layer of a deep neural network, which gives rise not only to multiple equivalent global minima of the loss function but also to critical points in between partner minima.
In a network of $d-1$ hidden layers with $n_k$ neurons in layers $k = 1, \ldots, d$, we construct continuous paths between equivalent global minima that lead through a `permutation point' where the input and output weight vectors of two neurons in the same hidden layer $k$ collide and interchange.
We show that such permutation points are critical points which lie inside high-dimensional subspaces of equal loss, contributing to the global flatness of the landscape.
We also find that a permutation point for the exchange of neurons $i$ and $j$ transits into a flat high-dimensional plateau that enables all $n_k!$
permutations of neurons in a given layer $k$ at the same loss value. Moreover
, we introduce higher-order permutation points by exploiting the hierarchical structure in the loss landscapes of neural networks, and find that the number of $K$-th order permutation points is much larger than the (already huge) number of equivalent global minima -- at least by a polynomial factor of order $K$. In two
tasks, we demonstrate numerically with our path finding method that continuous paths between partner minima exist: first, in a toy network with a single hidden layer on a function approximation task and, second, in a multilayer network on the MNIST task. Our geometric
approach yields a lower bound on the number of critical points generated by weight-space symmetries and provides a simple intuitive link between previous theoretical results and numerical observations.

assistant: text: Weight-space symmetry in neural network landscapes gives rise to numerous number of saddles and flat high-dimensional subspaces.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Meta-Learning with Latent Embedding Optimization

Paper content:
Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems.
However, they have practical difficulties when operating on high-dimensional parameter spaces in extreme low-data regimes.
We show that it is possible to bypass these limitations by learning a data-dependent latent generative representation of model parameters, and performing gradient-based meta-learning in this low-dimensional latent space.
The resulting approach, latent embedding optimization (LEO), decouples the gradient-based adaptation procedure from the underlying high-dimensional space of model parameters.
Our evaluation shows that LEO can achieve state-of-the-art performance on the competitive miniImageNet and tieredImageNet few-shot classification tasks.
Further analysis indicates LEO is able to capture uncertainty in the data, and can perform adaptation more effectively by optimizing in latent space.

assistant: text: A new meta-learning framework that learns data-dependent latent space, performs fast adaptation in the latent space, is effective for few-shot learning, has task-dependent initialization for adaptation, and works well for multimodal task distribution.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Graph Neural Networks with Noisy Labels

Paper content:
We study the robustness to symmetric label noise of GNNs training procedures.
By combining the nonlinear neural message-passing models (e.g. Graph Isomorphism Networks, GraphSAGE, etc.) with loss correction methods, we present a noise-tolerant approach for the graph classification task.
Our experiments show that test accuracy can be improved under the artificial symmetric noisy setting.

assistant: text: We apply loss correction to graph neural networks to train a more robust to noise model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Pseudo-Bayesian Learning via Direct Loss Minimization with Applications to Sparse Gaussian Process Models

Paper content:
We propose that approximate Bayesian algorithms should optimize a new criterion, directly derived from the loss, to calculate their approximate posterior which we refer to as pseudo-posterior.
Unlike standard variational inference which optimizes a lower bound on the log marginal likelihood, the new algorithms can be analyzed to provide loss guarantees on the predictions with the pseudo-posterior.
Our criterion can be used to derive new sparse Gaussian process algorithms that have error guarantees applicable to various likelihoods.

assistant: text: This paper utilizes the analysis of Lipschitz loss on a bounded hypothesis space to derive new ERM-type algorithms with strong performance guarantees that can be applied to the non-conjugate sparse GP model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Enhancing Batch Normalized Convolutional Networks using Displaced Rectifier Linear Units: A Systematic Comparative Study

Paper content:
In this paper, we turn our attention to the interworking between the activation functions and the batch normalization, which is a virtually mandatory technique to train deep networks currently.
We propose the activation function Displaced Rectifier Linear Unit (DReLU) by conjecturing that extending the identity function of ReLU to the third quadrant enhances compatibility with batch normalization.
Moreover, we used statistical tests to compare the impact of using distinct activation functions (ReLU, LReLU, PReLU, ELU, and DReLU) on the learning speed and test accuracy performance of standardized VGG and Residual Networks state-of-the-art models.
These convolutional neural networks were trained on CIFAR-100 and CIFAR-10, the most commonly used deep learning computer vision datasets.
The results showed DReLU speeded up learning in all models and datasets.
Besides, statistical significant performance assessments (p<0.05) showed DReLU enhanced the test accuracy presented by ReLU in all scenarios.
Furthermore, DReLU showed better test accuracy than any other tested activation function in all experiments with one exception, in which case it presented the second best performance.
Therefore, this work demonstrates that it is possible to increase performance replacing ReLU by an enhanced activation function.

assistant: text: This paper proposes an activation function, called displaced ReLU, to improve the performance of CNNs that use batch normalization.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Challenges in Disentangling Independent Factors of Variation

Paper content:
We study the problem of building models that disentangle independent factors of variation.
Such models encode features that can efficiently be used for classification and to transfer attributes between different images in image synthesis.
As data we use a weakly labeled training set, where labels indicate what single factor has changed between two data samples, although the relative value of the change is unknown.
This labeling is of particular interest as it may be readily available without annotation costs.
We introduce an autoencoder model and train it through constraints on image pairs and triplets.
We show the role of feature dimensionality and adversarial training theoretically and experimentally.
We formally prove the existence of the reference ambiguity, which is inherently present in the disentangling task when weakly labeled data is used.
The numerical value of a factor has different meaning in different reference frames.
When the reference depends on other factors, transferring that factor becomes ambiguous.
We demonstrate experimentally that the proposed model can successfully transfer attributes on several datasets, but show also cases when the reference ambiguity occurs.

assistant: text: It is a mostly theoretical paper that describes the challenges in disentangling factors of variation, using autoencoders and GAN.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Relational State-Space Model for Stochastic Multi-Object Systems

Paper content:
Real-world dynamical systems often consist of multiple stochastic subsystems that interact with each other.
Modeling and forecasting the behavior of such dynamics are generally not easy, due to the inherent hardness in understanding the complicated interactions and evolutions of their constituents.
This paper introduces the relational state-space model (R-SSM), a sequential hierarchical latent variable model that makes use of graph neural networks (GNNs) to simulate the joint state transitions of multiple correlated objects.
By letting GNNs cooperate with SSM, R-SSM provides a flexible way to incorporate relational information into the modeling of multi-object dynamics.
We further suggest augmenting the model with normalizing flows instantiated for vertex-indexed random variables and propose two auxiliary contrastive objectives to facilitate the learning.
The utility of R-SSM is empirically evaluated on synthetic and real time series datasets.

assistant: text: A deep hierarchical state-space model in which the state transitions of correlated objects are coordinated by graph neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hierarchical Graph-to-Graph Translation for Molecules

Paper content:
The problem of accelerating drug discovery relies heavily on automatic tools to optimize precursor molecules to afford them with better biochemical properties.
Our work in this paper substantially extends prior state-of-the-art on graph-to-graph translation methods for molecular optimization.
In particular, we realize coherent multi-resolution representations by interweaving the encoding of substructure components with the atom-level encoding of the original molecular graph.
Moreover, our graph decoder is fully autoregressive, and interleaves each step of adding a new substructure with the process of resolving its attachment to the emerging molecule.
We evaluate our model on multiple molecular optimization tasks and show that our model significantly outperforms previous state-of-the-art baselines.

assistant: text: We propose a multi-resolution, hierarchically coupled encoder-decoder for graph-to-graph translation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Disentangling Improves VAEs' Robustness to Adversarial Attacks

Paper content:
This paper is concerned with the robustness of VAEs to adversarial attacks.
We highlight that conventional VAEs are brittle under attack but that methods recently introduced for disentanglement such as β-TCVAE (Chen et al., 2018) improve robustness, as demonstrated through a variety of previously proposed adversarial attacks (Tabacof et al. (2016); Gondim-Ribeiro et al. (2018); Kos et al.(2018)).
This motivated us to develop Seatbelt-VAE, a new hierarchical disentangled VAE that is designed to be significantly more robust to adversarial attacks than existing approaches, while retaining high quality reconstructions.

assistant: text: We show that disentangled VAEs are more robust than vanilla VAEs to adversarial attacks that aim to trick them into decoding the adversarial input to a chosen target. We then develop an even more robust hierarchical disentangled VAE, Seatbelt-VAE.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Amortized Bayesian Meta-Learning

Paper content:
Meta-learning, or learning-to-learn, has proven to be a successful strategy in attacking problems in supervised learning and reinforcement learning that involve small amounts of data.
State-of-the-art solutions involve learning an initialization and/or learning algorithm using a set of training episodes so that the meta learner can generalize to an evaluation episode quickly.
These methods perform well but often lack good quantification of uncertainty, which can be vital to real-world applications when data is lacking.
We propose a meta-learning method which efficiently amortizes hierarchical variational inference across tasks, learning a prior distribution over neural network weights so that a few steps of Bayes by Backprop will produce a good task-specific approximate posterior.
We show that our method produces good uncertainty estimates on contextual bandit and few-shot learning benchmarks.

assistant: text: An adaptation to MAML-type models that accounts for posterior uncertainty in task specific latent variables by employing variational inference for task-specific parameters in a hierarchical Bayesian view of MAML.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Recurrent neural networks learn robust representations by dynamically balancing compression and expansion

Paper content:
Recordings of neural circuits in the brain reveal extraordinary dynamical richness and high variability.
At the same time, dimensionality reduction techniques generally uncover low-dimensional structures underlying these dynamics.
What determines the dimensionality of activity in neural circuits?
What is the functional role of dimensionality in behavior and task learning?
In this work we address these questions using recurrent neural network (RNN) models.
We find that, depending on the dynamics of the initial network, RNNs learn to increase and reduce dimensionality in a way that matches task demands.
These findings shed light on fundamental dynamical mechanisms by which neural networks solve tasks with robust representations that generalize to new cases.

assistant: text: Recurrent Neural Networks learn to  increase and reduce the dimensionality of their internal representation in a way that matches the task, depending on the dynamics of the initial network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 3D-SIC: 3D Semantic Instance Completion for RGB-D Scans

Paper content:
This paper introduces the task of semantic instance completion: from an incomplete RGB-D scan of a scene, we aim to detect the individual object instances comprising the scene and infer their complete object geometry.
This enables a semantically meaningful decomposition of a scanned scene into individual, complete 3D objects, including hidden and unobserved object parts.
This will open up new possibilities for interactions with object in a scene, for instance for virtual or robotic agents.
To address this task, we propose 3D-SIC, a new data-driven approach that jointly detects object instances and predicts their completed geometry.
The core idea of 3D-SIC is a novel end-to-end 3D neural network architecture that leverages joint color and geometry feature learning.
The fully-convolutional nature of our 3D network enables efficient inference of semantic instance completion for 3D scans at scale of large indoor environments in a single forward pass.
In a series evaluation, we evaluate on both real and synthetic scan benchmark data, where we outperform state-of-the-art approaches by over 15 in mAP@0.5 on ScanNet, and over 18 in mAP@0.5 on SUNCG.

assistant: text: From an incomplete RGB-D scan of a scene, we aim to detect the individual object instances comprising the scene and infer their complete object geometry.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Approximability of Discriminators Implies Diversity in GANs

Paper content:
While Generative Adversarial Networks (GANs) have empirically produced impressive results on learning complex real-world distributions, recent works have shown that they suffer from lack of diversity or mode collapse.
The theoretical work of Arora et al. (2017a) suggests a dilemma about GANs’ statistical properties: powerful discriminators cause overfitting, whereas weak discriminators cannot detect mode collapse.
By contrast, we show in this paper that GANs can in principle learn distributions in Wasserstein distance (or KL-divergence in many cases) with polynomial sample complexity, if the discriminator class has strong distinguishing power against the particular generator class (instead of against all possible generators).
For various generator classes such as mixture of Gaussians, exponential families, and invertible and injective neural networks generators, we design corresponding discriminators (which are often neural nets of specific architectures) such that the Integral Probability Metric (IPM) induced by the discriminators can provably approximate the Wasserstein distance and/or KL-divergence.
This implies that if the training is successful, then the learned distribution is close to the true distribution in Wasserstein distance or KL divergence, and thus cannot drop modes.
Our preliminary experiments show that on synthetic datasets the test IPM is well correlated with KL divergence or the Wasserstein distance, indicating that the lack of diversity in GANs may be caused by the sub-optimality in optimization instead of statistical inefficiency.

assistant: text: GANs can in principle learn distributions sample-efficiently, if the discriminator class is compact and has strong distinguishing power against the particular generator class.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Residual Non-local Attention Networks for Image Restoration

Paper content:
In this paper, we propose a residual non-local attention network for high-quality image restoration.
Without considering the uneven distribution of information in the corrupted images, previous methods are restricted by local convolutional operation and equal treatment of spatial- and channel-wise features.
To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts.
Specifically, we design trunk branch and (non-)local mask branch in each (non-)local attention block.
The trunk branch is used to extract hierarchical features.
Local and non-local mask branches aim to adaptively rescale these hierarchical features with mixed attentions.
The local mask branch concentrates on more local structures with convolutional operations, while non-local attention considers more about long-range dependencies in the whole feature map.
Furthermore, we propose residual local and non-local attention learning to train the very deep network, which further enhance the representation ability of the network.
Our proposed method can be generalized for various image restoration applications, such as image denoising, demosaicing, compression artifacts reduction, and super-resolution.
Experiments demonstrate that our method obtains comparable or better results compared with recently leading methods quantitatively and visually.

assistant: text: New state-of-the-art framework for image restoration