ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adaptive Neural Trees

Paper content:
Deep neural networks and decision trees operate on largely separate paradigms; typically, the former performs representation learning with pre-specified architectures, while the latter is characterised by learning hierarchies over pre-specified features with data-driven architectures.
We unite the two via adaptive neural trees (ANTs), a model that incorporates representation learning into edges, routing functions and leaf nodes of a decision tree, along with a backpropagation-based training algorithm that adaptively grows the architecture from primitive modules (e.g., convolutional layers).
ANTs allow increased interpretability via hierarchical clustering, e.g., learning meaningful class associations, such as separating natural vs. man-made objects.
We demonstrate this on classification and regression tasks, achieving over 99% and 90% accuracy on the MNIST and CIFAR-10 datasets, and outperforming standard neural networks, random forests and gradient boosted trees on the SARCOS dataset.
Furthermore, ANT optimisation naturally adapts the architecture to the size and complexity of the training data.

assistant: text: We propose a framework to combine decision trees and neural networks, and show on image classification tasks that it enjoys the complementary benefits of the two approaches, while addressing the limitations of prior work.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SoftLoc: Robust Temporal Localization under Label Misalignment

Paper content:
This work addresses the long-standing problem of robust event localization in the presence of temporally of misaligned labels in the training data.
We propose a novel versatile loss function that generalizes a number of training regimes from standard fully-supervised cross-entropy to count-based weakly-supervised learning.
Unlike classical models which are constrained to strictly fit the annotations during training, our soft localization learning approach relaxes the reliance on the exact position of labels instead.
Training with this new loss function exhibits strong robustness to temporal misalignment of labels, thus alleviating the burden of precise annotation of temporal sequences.
We demonstrate state-of-the-art performance against standard benchmarks in a number of challenging experiments and further show that robustness to label noise is not achieved at the expense of raw performance.

assistant: text: A new loss for training models that predict where events occur in a training sequence with noisy labels by comparing smoothed label and prediction sequence.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization

Paper content:
Transferring knowledge across tasks to improve data-efficiency is one of
the open key challenges in the area of global optimization algorithms.
Readily
available algorithms are typically designed to be universal optimizers and, thus,
often suboptimal for specific tasks.
We propose a novel transfer learning method to
obtain customized optimizers within the well-established framework of Bayesian
optimization, allowing our algorithm to utilize the proven generalization
capabilities of Gaussian processes.
Using reinforcement learning to meta-train an
acquisition function (AF) on a set of related tasks, the proposed method learns to
extract implicit structural information and to exploit it for improved data-efficiency.
We present experiments on a sim-to-real transfer task as well as on several simulated
functions and two hyperparameter search problems.
The results show that our
algorithm (1) automatically identifies structural properties of objective functions
from available source tasks or simulations, (2) performs favourably in settings with
both scarse and abundant source data, and (3) falls back to the performance level
of general AFs if no structure is present.

assistant: text: The authors propose a meta-learning based alternative to standard acquisition functions (AFs), whereby a pretrained neural network outputs acquisition values as a function of hand-chosen features.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Scalable Imitation Learning for Multi-Agent Systems with Graph Neural Networks

Paper content:
We propose an implementation of GNN that predicts and imitates the motion be- haviors from observed swarm trajectory data.
The network’s ability to capture interaction dynamics in swarms is demonstrated through transfer learning.
We finally discuss the inherent availability and challenges in the scalability of GNN, and proposed a method to improve it with layer-wise tuning and mixing of data enabled by padding.

assistant: text: Improve the scalability of graph neural networks on imitation learning and prediction of swarm motion
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Anomaly Detection with Generative Adversarial Networks

Paper content:
Many anomaly detection methods exist that perform well on low-dimensional problems however there is a notable lack of effective methods for high-dimensional spaces, such as images.
Inspired by recent successes in deep learning we propose a novel approach to anomaly detection using generative adversarial networks.
Given a sample under consideration, our method is based on searching for a good representation of that sample in the latent space of the generator; if such a representation is not found, the sample is deemed anomalous.
We achieve state-of-the-art performance on standard image benchmark datasets and visual inspection of the most anomalous samples reveals that our method does indeed return anomalies.

assistant: text: We propose a method for anomaly detection with GANs by searching the generator's latent space for good sample representations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to solve the credit assignment problem

Paper content:
Backpropagation is driving today's artificial neural networks (ANNs).
However, despite extensive research, it remains unclear if the brain implements this algorithm.
Among neuroscientists, reinforcement learning (RL) algorithms are often seen as a realistic alternative: neurons can randomly introduce change, and use unspecific feedback signals to observe their effect on the cost and thus approximate their gradient.
However, the convergence rate of such learning scales poorly with the number of involved neurons.
Here we propose a hybrid learning approach.
Each neuron uses an RL-type strategy to learn how to approximate the gradients that backpropagation would provide.
We provide proof that our approach converges to the true gradient for certain classes of networks.
In both feedforward and convolutional networks, we empirically show that our approach learns to approximate the gradient, and can match the performance of gradient-based learning.
Learning feedback weights provides a biologically plausible mechanism of achieving good performance, without the need for precise, pre-specified learning rules.

assistant: text: This paper proposes a method that addresses the "weight transport" problem by estimating the weights for the backward pass using a noise-based estimator
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment

Paper content:
Domain adaptation addresses the common problem when the target distribution generating our test data drifts from the source (training) distribution.
While absent assumptions, domain adaptation is impossible, strict conditions, e.g. covariate or label shift, enable principled algorithms.
Recently-proposed domain-adversarial approaches consist of aligning source and target encodings, often motivating this approach as minimizing two (of three) terms in a theoretical bound on target error.
Unfortunately, this minimization can cause arbitrary increases in the third term, e.g. they can break down under shifting label distributions.
We propose asymmetrically-relaxed distribution alignment, a new approach that overcomes some limitations of standard domain-adversarial algorithms.
Moreover, we characterize precise assumptions under which our algorithm is theoretically principled and demonstrate empirical benefits on both synthetic and real datasets.

assistant: text: Instead of strict distribution alignments in traditional deep domain adaptation objectives, which fails when target label distribution shifts, we propose to optimize a relaxed objective with new analysis, new algorithms, and experimental validation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to Defense by Learning to Attack

Paper content:
Adversarial training provides a principled approach for training robust neural networks.
From an optimization perspective, the adversarial training is essentially solving a minmax robust optimization problem.
The outer minimization is trying to learn a robust classifier, while the inner maximization is trying to generate adversarial samples.
Unfortunately, such a minmax problem is very difficult to solve due to the lack of convex-concave structure.
This work proposes a new adversarial training method based on a general learning-to-learn framework.
Specifically, instead of applying the existing hand-design algorithms for the inner problem, we learn an optimizer, which is parametrized as a convolutional neural network.
At the same time, a robust classifier is learned to defense the adversarial attack generated by the learned optimizer.
From the perspective of generative learning, our proposed method can be viewed as learning a deep generative model for generating adversarial samples, which is adaptive to the robust classification.
Our experiments demonstrate that our proposed method significantly outperforms existing adversarial training methods on CIFAR-10 and CIFAR-100 datasets.

assistant: text: This paper proposes a way to train image classification models to be resistant to L-infinity perturbation attacks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SELF: Learning to Filter Noisy Labels with Self-Ensembling

Paper content:
Deep neural networks (DNNs) have been shown to over-fit a dataset when being trained with noisy labels for a long enough time.
To overcome this problem, we present a simple and effective method self-ensemble label filtering (SELF) to progressively filter out the wrong labels during training.
Our method improves the task performance by gradually allowing supervision only from the potentially non-noisy (clean) labels and stops learning on the filtered noisy labels.
For the filtering, we form running averages of predictions over the entire training dataset using the network output at different training epochs.
We show that these ensemble estimates yield more accurate identification of inconsistent predictions throughout training than the single estimates of the network at the most recent training epoch.
While filtered samples are removed entirely from the supervised training loss, we dynamically leverage them via semi-supervised learning in the unsupervised loss.
We demonstrate the positive effect of such an approach on various image classification tasks under both symmetric and asymmetric label noise and at different noise ratios.
It substantially outperforms all previous works on noise-aware learning across different datasets and can be applied to a broad set of network architectures.

assistant: text: This paper proposes an algorithm for learning from data with noisy labels which alternates between updating the model and removing samples that look like they have noisy labels.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to Design RNA

Paper content:
Designing RNA molecules has garnered recent interest in medicine, synthetic biology, biotechnology and bioinformatics since many functional RNA molecules were shown to be involved in regulatory processes for transcription, epigenetics and translation.
Since an RNA's function depends on its structural properties, the RNA Design problem is to find an RNA sequence which satisfies given structural constraints.
Here, we propose a new algorithm for the RNA Design problem, dubbed LEARNA.
LEARNA uses deep reinforcement learning to train a policy network to sequentially design an entire RNA sequence given a specified target structure.
By meta-learning across 65000 different RNA Design tasks for one hour on 20 CPU cores, our extension Meta-LEARNA constructs an RNA Design policy that can be applied out of the box to solve novel RNA Design tasks.
Methodologically, for what we believe to be the first time, we jointly optimize over a rich space of architectures for the policy network, the hyperparameters of the training procedure and the formulation of the decision process.
Comprehensive empirical results on two widely-used RNA Design benchmarks, as well as a third one that we introduce, show that our approach achieves new state-of-the-art performance on the former while also being orders of magnitudes faster in reaching the previous state-of-the-art performance.
In an ablation study, we analyze the importance of our method's different components.

assistant: text: We learn to solve the RNA Design problem with reinforcement learning using meta learning and autoML approaches.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Composition-based Multi-Relational Graph Convolutional Networks

Paper content:
Graph Convolutional Networks (GCNs) have recently been shown to be quite successful in modeling graph-structured data.
However, the primary focus has been on handling simple undirected graphs.
Multi-relational graphs are a more general and prevalent form of graphs where each edge has a label and direction associated with it.
Most of the existing approaches to handle such graphs suffer from over-parameterization and are restricted to learning representations of nodes only.
In this paper, we propose CompGCN, a novel Graph Convolutional framework which jointly embeds both nodes and relations in a relational graph.
CompGCN leverages a variety of entity-relation composition operations from Knowledge Graph Embedding techniques and scales with the number of relations.
It also generalizes several of the existing multi-relational GCN methods.
We evaluate our proposed method on multiple tasks such as node classification, link prediction, and graph classification, and achieve demonstrably superior results.
We make the source code of CompGCN available to foster reproducible research.

assistant: text: This paper introduces a GCN framework for multi-relational graphs and generalizes several existing approaches to Knowledge Graph embedding into one framework.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Tangents: Fast and Easy Infinite Neural Networks in Python

Paper content:
Neural Tangents is a library designed to enable research into infinite-width neural networks.
It provides a high-level API for specifying complex and hierarchical neural network architectures.
These networks can then be trained and evaluated either at finite-width as usual or in their infinite-width limit.
Infinite-width networks can be trained analytically using exact Bayesian inference or using gradient descent via the Neural Tangent Kernel.
Additionally, Neural Tangents provides tools to study gradient descent training dynamics of wide but finite networks in either function space or weight space.
The entire library runs out-of-the-box on CPU, GPU, or TPU.
All computations can be automatically distributed over multiple accelerators with near-linear scaling in the number of devices.
Neural Tangents is available at
https://www.github.com/google/neural-tangents
We also provide an accompanying interactive Colab notebook at
https://colab.sandbox.google.com/github/google/neural-tangents/blob/master/notebooks/neural_tangents_cookbook.ipynb

assistant: text: Keras for infinite neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Linearizing Visual Processes with Deep Generative Models

Paper content:
This work studies the problem of modeling non-linear visual processes by leveraging deep generative architectures for learning linear, Gaussian models of observed sequences.
We propose a joint learning framework, combining a multivariate autoregressive model and deep convolutional generative networks.
After justification of theoretical assumptions of inearization, we propose an architecture that allows Variational Autoencoders and Generative Adversarial Networks to simultaneously learn the non-linear observation as well as the linear state-transition model from a sequence of observed frames.
Finally, we demonstrate our approach on conceptual toy examples and dynamic textures.

assistant: text: Proposes a new method that models non-linear visual process with a deep version of a linear process (Markov process).
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Self-Monitoring Navigation Agent via Auxiliary Progress Estimation

Paper content:
The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments.
This challenging task demands that the agent be aware of which instruction was completed, which instruction is needed next, which way to go, and its navigation progress towards the goal.
In this paper, we introduce a self-monitoring agent with two complementary components: (1) visual-textual co-grounding module to locate the instruction completed in the past, the instruction required for the next action, and the next moving direction from surrounding images and (2) progress monitor to ensure the grounded instruction correctly reflects the navigation progress.
We test our self-monitoring agent on a standard benchmark and analyze our proposed approach through a series of ablation studies that elucidate the contributions of the primary components.
Using our proposed method, we set the new state of the art by a significant margin (8% absolute increase in success rate on the unseen test set).
Code is available at https://github.com/chihyaoma/selfmonitoring-agent.

assistant: text: This paper describes a model for vision-and-language navigation with a panoramic visual attention and an auxillary progress monitoring loss, giving state-of-the-art results.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Regularization Matters in Policy Optimization

Paper content:
Deep Reinforcement Learning (Deep RL) has been receiving increasingly more attention  thanks to its encouraging performance on a variety of control tasks.
Yet, conventional regularization techniques in training neural networks (e.g., $L_2$ regularization, dropout) have been largely ignored in RL methods, possibly because agents are typically trained and evaluated in the same environment.
In this work, we present the first comprehensive study of regularization techniques with multiple policy optimization algorithms on continuous control tasks.
Interestingly, we find conventional regularization techniques on the policy networks can often bring large improvement on the task performance, and the improvement is typically more significant when the task is more difficult.
We also compare with the widely used entropy regularization and find $L_2$ regularization is generally better.
Our findings are further confirmed to be robust against the choice of training hyperparameters.
We also study the effects of regularizing different components and find that only regularizing the policy network is typically enough.
We hope our study provides guidance for future practices in regularizing policy optimization algorithms.

assistant: text: The authors study a set of existing direct policy optimization methods in the field of reinforcement learning and provide a detailed investigation on the effect of regulations on the performance and behavior of agents following these methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Ridge Regression: Structure, Cross-Validation, and Sketching

Paper content:
We study the following three fundamental problems about ridge regression: (1) what is the structure of the estimator?
(2) how to correctly use cross-validation to choose the regularization parameter?
and (3) how to accelerate computation without losing too much accuracy?
We consider the three problems in a unified large-data linear model.
We give a precise representation of ridge regression as a covariance matrix-dependent linear combination of the true parameter and the noise.
We study the bias of $K$-fold cross-validation for choosing the regularization parameter, and propose a simple bias-correction.
We analyze the accuracy of primal and dual sketching for ridge regression, showing they are surprisingly accurate.
Our results are illustrated by simulations and by analyzing empirical data.

assistant: text: A theoretical study of ridge regression by exploiting a new asymptotic characterisation of the ridge regression estimator.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Simultaneous Classification and Out-of-Distribution Detection Using Deep Neural Networks

Paper content:
Deep neural networks have achieved great success in classiﬁcation tasks during the last years.
However, one major problem to the path towards artiﬁcial intelligence is the inability of neural networks to accurately detect samples from novel class distributions and therefore, most of the existent classiﬁcation algorithms assume that all classes are known prior to the training stage.
In this work, we propose a methodology for training a neural network that allows it to efﬁciently detect out-of-distribution (OOD) examples without compromising much of its classiﬁcation accuracy on the test examples from known classes.
Based on the Outlier Exposure (OE) technique, we propose a novel loss function that achieves state-of-the-art results in out-of-distribution detection with OE both on image and text classiﬁcation tasks.
Additionally, the way this method was constructed makes it suitable for training any classiﬁcation algorithm that is based on Maximum Likelihood methods.

assistant: text: Proposal for a new loss function to train the network with Outlier Exposure which leads to better OOD detection compared to simple loss functions using KL divergence.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Lipschitz constant estimation of Neural Networks via sparse polynomial optimization

Paper content:
We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bound on the Lipschitz constant of neural networks.
The underlying optimization problems boil down to either linear (LP) or semidefinite (SDP) programming.
We show how to use the sparse connectivity of a network, to significantly reduce the complexity of computation.
This is specially useful for convolutional as well as pruned neural networks.
We conduct experiments on networks with random weights as well as networks trained on MNIST, showing that in the particular case of the $\ell_\infty$-Lipschitz constant, our approach yields superior estimates as compared to other baselines available in the literature.

assistant: text: LP-based upper bounds on the Lipschitz constant of Neural Networks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Mixed Setting Training Methods for Incremental Slot-Filling Tasks

Paper content:
Model training remains a dominant financial cost and time investment in machine learning applications.
Developing and debugging models often involve iterative training, further exacerbating this issue.
With growing interest in increasingly complex models, there is a need for techniques that help to reduce overall training effort.
While incremental training can save substantial time and cost by training an existing model on a small subset of data, little work has explored policies for determining when incremental training provides adequate model performance versus full retraining.
We provide a method-agnostic algorithm for deciding when to incrementally train versus fully train.
We call this setting of non-deterministic full- or incremental training ``Mixed Setting Training".
Upon evaluation in slot-filling tasks, we find that this algorithm provides a bounded error, avoids catastrophic forgetting, and results in a significant speedup over a policy of always fully training.

assistant: text: We provide a method-agnostic algorithm for deciding when to incrementally train versus fully train and it provides a significant speedup over fully training and avoids catastrophic forgetting
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Low-bit quantization and quantization-aware training for small-footprint keyword spotting

Paper content:
We investigate low-bit quantization to reduce computational cost of deep neural network (DNN) based keyword spotting (KWS).
We propose approaches to further reduce quantization bits via integrating quantization into keyword spotting model training, which we refer to as quantization-aware training.
Our experimental results on large dataset indicate that quantization-aware training can recover performance models quantized to lower bits representations.
By combining quantization-aware training and weight matrix factorization, we are able to significantly reduce model size and computation for small-footprint keyword spotting, while maintaining performance.

assistant: text: This submission proposes a combination of low-rank decomposition and quanitization approach to compress DNN models for keyword spotting.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Structured Neural Summarization

Paper content:
Summarization of long sequences into a concise statement is a core problem in natural language processing, requiring non-trivial understanding of the input.
Based on the promising results of graph neural networks on highly structured data, we develop a framework to extend existing sequence encoders with a graph component that can reason about long-distance relationships in weakly structured data such as text.
In an extensive evaluation, we show that the resulting hybrid sequence-graph models outperform both pure sequence models as well as pure graph models on a range of summarization tasks.

assistant: text: One simple trick to improve sequence models: Compose them with a graph model
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A PRIVACY-PRESERVING IMAGE CLASSIFICATION FRAMEWORK WITH A LEARNABLE OBFUSCATOR

Paper content:
Real world images often contain large amounts of private / sensitive information that should be carefully protected without reducing their utilities.
In this paper, we propose a privacy-preserving deep learning framework with a learnable ob- fuscator for the image classification task.
Our framework consists of three mod- els: learnable obfuscator, classifier and reconstructor.
The learnable obfuscator is used to remove the sensitive information in the images and extract the feature maps from them.
The reconstructor plays the role as an attacker, which tries to recover the image from the feature maps extracted by the obfuscator.
In order to best protect users’ privacy in images, we design an adversarial training methodol- ogy for our framework to optimize the obfuscator.
Through extensive evaluations on real world datasets, both the numerical metrics and the visualization results demonstrate that our framework is qualified to protect users’ privacy and achieve a relatively high accuracy on the image classification task.

assistant: text: This current work suggests using adversarial networks to obfuscate images and thus allow collecting them without privacy concerns to use them for training machine learning models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Spectral Inference Networks: Unifying Deep and Spectral Learning

Paper content:
We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization.
Spectral Inference Networks generalize Slow Feature Analysis to generic symmetric operators, and are closely related to Variational Monte Carlo methods from computational physics.
As such, they can be a powerful tool for unsupervised representation learning from video or graph-structured data.
We cast training Spectral Inference Networks as a bilevel optimization problem, which allows for online learning of multiple eigenfunctions.
We show results of training Spectral Inference Networks on problems in quantum mechanics and feature learning for videos on synthetic datasets.
Our results demonstrate that Spectral Inference Networks accurately recover eigenfunctions of linear operators and can discover interpretable representations from video in a fully unsupervised manner.

assistant: text: The authors propose to use a deep learning framework to solve the computation of the largest eigenvectors.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: One Generation Knowledge Distillation by Utilizing Peer Samples

Paper content:
Knowledge Distillation (KD) is a widely used technique in recent deep learning research to obtain small and simple models whose performance is on a par with their large and complex counterparts.
Standard Knowledge Distillation tends to be time-consuming because of the training time spent to obtain a teacher model that would then provide guidance for the student model.
It might be possible to cut short the time by training a teacher model on the fly, but it is not trivial to have such a high-capacity teacher that gives quality guidance to student models this way.
To improve this, we present a novel framework of Knowledge Distillation exploiting dark knowledge from the whole training set.
In this framework, we propose a simple and effective implementation named Distillation by Utilizing Peer Samples (DUPS) in one generation.
We verify our algorithm on numerous experiments.
Compared with standard training on modern architectures, DUPS achieves an average improvement of 1%-2% on various tasks with nearly zero extra cost.
Considering some typical Knowledge Distillation methods which are much more time-consuming, we also get comparable or even better performance using DUPS.

assistant: text: This paper proposes to address the extra computational cost of training with knowledge distillation, building on the recently proposed Snapshot Distillation technique.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Probabilistic View of Multi-agent Reinforcement Learning: A Unified Approach

Paper content:
Formulating the reinforcement learning (RL) problem in the framework of probabilistic inference not only offers a new perspective about RL, but also yields practical algorithms that are more robust and easier to train.
While this connection between RL and probabilistic inference has been extensively studied in the single-agent setting, it has not yet been fully understood in the multi-agent setup.
In this paper, we pose the problem of multi-agent reinforcement learning as the problem of performing inference in a particular graphical model.
We model the environment, as seen by each of the agents, using separate but related Markov decision processes.
We derive a practical off-policy maximum-entropy actor-critic algorithm that we call Multi-agent Soft Actor-Critic (MA-SAC) for performing approximate inference in the proposed model using variational inference.
MA-SAC can be employed in both cooperative and competitive settings.
Through experiments, we demonstrate that MA-SAC outperforms a strong baseline on several multi-agent scenarios.
While MA-SAC is one resultant multi-agent RL algorithm that can be derived from the proposed probabilistic framework, our work provides a unified view of maximum-entropy algorithms in the multi-agent setting.

assistant: text: A probabilistic framework for multi-agent reinforcement learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Through Limited Self-Supervision: Improving Time-Series Classification Without Additional Data via Auxiliary Tasks

Paper content:
Self-supervision, in which a target task is improved without external supervision, has primarily been explored in settings that assume the availability of additional data.
However, in many cases, particularly in healthcare, one may not have access to additional data (labeled or otherwise).
In such settings, we hypothesize that self-supervision based solely on the structure of the data at-hand can help.
We explore a novel self-supervision framework for time-series data, in which multiple auxiliary tasks (e.g., forecasting) are included to improve overall performance on a sequence-level target task without additional training data.
We call this approach limited self-supervision, as we limit ourselves to only the data at-hand.
We demonstrate the utility of limited self-supervision on three sequence-level classification tasks, two pertaining to real clinical data and one using synthetic data.
Within this framework, we introduce novel forms of self-supervision and demonstrate their utility in improving performance on the target task.
Our results indicate that limited self-supervision leads to a consistent improvement over a supervised baseline, across a range of domains.
In particular, for the task of identifying atrial fibrillation from small amounts of electrocardiogram data, we observe a nearly 13% improvement in the area under the receiver operating characteristics curve (AUC-ROC) relative to the baseline (AUC-ROC=0.55 vs. AUC-ROC=0.62).
Limited self-supervision applied to sequential data can aid in learning intermediate representations, making it particularly applicable in settings where data collection is difficult.

assistant: text: This paper proposes a self-supervised method for learning from time series data in healthcare settings via designing auxilliary tasks based on data's internal structure to create more labeled auxilliary training tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Semi-Supervised Semantic Dependency Parsing Using CRF Autoencoders

Paper content:
Semantic dependency parsing, which aims to find rich bi-lexical relationships, allows words to have multiple dependency heads, resulting in graph-structured representations.
We propose an approach to semi-supervised learning of semantic dependency parsers based on the CRF autoencoder framework.
Our encoder is a discriminative neural semantic dependency parser that predicts the latent parse graph of the input sentence.
Our decoder is a generative neural model that reconstructs the input sentence conditioned on the latent parse graph.
Our model is arc-factored and therefore parsing and learning are both tractable.
Experiments show our model achieves significant and consistent improvement over the supervised baseline.

assistant: text: This paper focuses on semi-supervised semantic dependency parsing using the CRF-autoencoder to train the model in a semi-supervised style, indicating effectiveness on low resource labeled data tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inference

Paper content:
Deep networks were recently suggested to face the odds between accuracy (on clean natural images) and robustness (on adversarially perturbed images) (Tsipras et al., 2019).
Such a dilemma is shown to be rooted in the inherently higher sample complexity (Schmidt et al., 2018) and/or model capacity (Nakkiran, 2019), for learning a high-accuracy and robust classifier.
In view of that, give a classification task, growing the model capacity appears to help draw a win-win between accuracy and robustness, yet at the expense of model size and latency, therefore posing challenges for resource-constrained applications.
Is it possible to co-design model accuracy, robustness and efficiency to achieve their triple wins?
This paper studies multi-exit networks associated with input-adaptive efficient inference, showing their strong promise in achieving a “sweet point" in co-optimizing model accuracy, robustness, and efficiency.
Our proposed solution, dubbed Robust Dynamic Inference Networks (RDI-Nets), allows for each input (either clean or adversarial) to adaptively choose one of the multiple output layers (early branches or the final one) to output its prediction.
That multi-loss adaptivity adds new variations and flexibility to adversarial attacks and defenses, on which we present a systematical investigation.
We show experimentally that by equipping existing backbones with such robust adaptive inference, the resulting RDI-Nets can achieve better accuracy and robustness, yet with over 30% computational savings, compared to the defended original models.

assistant: text: Is it possible to co-design model accuracy, robustness and efficiency to achieve their triple wins? Yes!
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Implementing Inductive bias for different navigation tasks through diverse RNN attrractors

Paper content:
Navigation is crucial for animal behavior and is assumed to require an internal representation of the external environment, termed a cognitive map.
The precise form of this representation is often considered to be a metric representation of space.
An internal representation, however, is judged by its contribution to performance on a given task, and may thus vary between different types of navigation tasks.
Here we train a recurrent neural network that controls an agent performing several navigation tasks in a simple environment.
To focus on internal representations, we split learning into a task-agnostic pre-training stage that modifies internal connectivity and a task-specific Q learning stage that controls the network's output.
We show that pre-training shapes the attractor landscape of the networks, leading to either a continuous attractor, discrete attractors or a disordered state.
These structures induce bias onto the Q-Learning phase, leading to a performance pattern across the tasks corresponding to metric and topological regularities.
Our results show that, in recurrent networks, inductive bias takes the form of attractor landscapes -- which can be shaped by pre-training and analyzed using dynamical systems methods.
Furthermore, we demonstrate that non-metric representations are useful for navigation tasks.

assistant: text: This paper studies the internal representations of recurrent neural networks trained on navigation tasks, and finds that RNNs pre-trained to use path integration contain 2D continuous attractors while RNNs pre-trained for landmark memory contain discrete attractors.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Neural Representation of Sketch Drawings

Paper content:
We present sketch-rnn, a recurrent neural network able to construct stroke-based drawings of common objects.
The model is trained on a dataset of human-drawn images representing many different classes.
We outline a framework for conditional and unconditional sketch generation, and describe new robust training methods for generating coherent sketch drawings in a vector format.

assistant: text: We investigate alternative to traditional pixel image modelling approaches, and propose a generative model for vector images.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Balanced and Deterministic Weight-sharing Helps Network Performance

Paper content:
Weight-sharing plays a significant role in the success of many deep neural networks, by increasing memory efficiency and incorporating useful inductive priors about the problem into the network.
But understanding how weight-sharing can be used effectively in general is a topic that has not been studied extensively.
Chen et al. (2015) proposed HashedNets, which augments a multi-layer perceptron with a hash table, as a method for neural network compression.
We generalize this method into a framework (ArbNets) that allows for efficient arbitrary weight-sharing, and use it to study the role of weight-sharing in neural networks.
We show that common neural networks can be expressed as ArbNets with different hash functions.
We also present two novel hash functions, the Dirichlet hash and the Neighborhood hash, and use them to demonstrate experimentally that balanced and deterministic weight-sharing helps with the performance of a neural network.

assistant: text: Proposing ArbNets to study weight sharing in a more systematic way by defining the weight sharing function as a hash function.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title:  The relativistic discriminator: a key element missing from standard GAN

Paper content:
In standard generative adversarial network (SGAN), the discriminator estimates the probability that the input data is real.
The generator is trained to increase the probability that fake data is real.
We argue that it should also simultaneously decrease the probability that real data is real because
1) this would account for a priori knowledge that half of the data in the mini-batch is fake,
2) this would be observed with divergence minimization, and
3) in optimal settings, SGAN would be equivalent to integral probability metric (IPM) GANs.
We show that this property can be induced by using a relativistic discriminator which estimate the probability that the given real data is more realistic than a randomly sampled fake data.
We also present a variant in which the discriminator estimate the probability that the given real data is more realistic than fake data, on average.
We generalize both approaches to non-standard GAN loss functions and we refer to them respectively as Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs).
We show that IPM-based GANs are a subset of RGANs which use the identity function.
Empirically, we observe that
1) RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts,
2) Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update (reducing the time taken for reaching the state-of-the-art by 400%), and
3) RaGANs are able to generate plausible high resolutions images (256x256) from a very small sample (N=2011), while GAN and LSGAN cannot; these images are of significantly better quality than the ones generated by WGAN-GP and SGAN with spectral normalization.
The code is freely available on https://github.com/AlexiaJM/RelativisticGAN.

assistant: text: In this work, the authors considers a variation of GAN by simultaneously decreasing the probability that real data is real for the generator.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Understanding Isomorphism Bias in Graph Data Sets 

Paper content:
In recent years there has been a rapid increase in classification methods on graph structured data.
Both in graph kernels and graph neural networks, one of the implicit assumptions of successful state-of-the-art models was that incorporating graph isomorphism features into the architecture leads to better empirical performance.
However, as we discover in this work, commonly used data sets for graph classification have repeating instances which cause the problem of isomorphism bias, i.e. artificially increasing the accuracy of the models by memorizing target information from the training set.
This prevents fair competition of the algorithms and raises a question of the validity of the obtained results.
We analyze 54 data sets, previously extensively used for graph-related tasks, on the existence of isomorphism bias, give a set of recommendations to machine learning practitioners to properly set up their models, and open source new data sets for the future experiments.

assistant: text: The authors discuss isomorphism bias in graph datasets, the overfitting effect in learning networks whenever graph isomorphism features are incorporated within the model, theoretically analogous to data leakage effects.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Size-free generalization bounds for convolutional neural networks

Paper content:
We prove bounds on the generalization error of convolutional networks.
The bounds are in terms of the training loss, the number of
parameters, the Lipschitz constant of the loss and the distance from
the weights to the initial weights.
They are independent of the
number of pixels in the input, and the height and width of hidden
feature maps.
We present experiments with CIFAR-10, along with varying
hyperparameters of a deep convolutional network, comparing our bounds
with practical generalization gaps.

assistant: text: Studies the generalization power of CNNs and improves the upper bounds of generalization errors, showing correlation between the generalization error of learned CNNs and the upper bound's dominant term.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Neural Forests: An Architecture for Tabular Data

Paper content:
Deep neural models, such as convolutional and recurrent networks, achieve phenomenal results over spatial data such as images and text.
However, when considering tabular data, gradient boosting of decision trees (GBDT) remains the method of choice.
Aiming to bridge this gap, we propose \emph{deep neural forests} (DNF)
--  a novel architecture that combines elements from decision trees as well as dense residual connections.
We present the results of extensive empirical study in which we examine the performance of GBDTs, DNFs and (deep) fully-connected networks.
These results indicate that DNFs achieve comparable results to GBDTs on tabular data, and open the door to end-to-end neural modeling of multi-modal data.
To this end, we present a successful application of DNFs as part of a hybrid architecture for a multi-modal driving scene understanding classification task.

assistant: text: An architecture for tabular data, which emulates branches of decision trees and uses dense residual connectivity