ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

Paper content:
Deep Reinforcement Learning has managed to achieve state-of-the-art results in learning control policies directly from raw pixels.
However, despite its remarkable success, it fails to generalize, a fundamental component required in a stable Artificial Intelligence system.
Using the Atari game Breakout, we demonstrate the difficulty of a trained agent in adjusting to simple modifications in the raw image, ones that a human could adapt to trivially.
In transfer learning, the goal is to use the knowledge gained from the source task to make the training of the target task faster and better.
We show that using various forms of fine-tuning, a common method for transfer learning, is not effective for adapting to such small visual changes.
In fact, it is often easier to re-train the agent from scratch than to fine-tune a trained agent.
We suggest that in some cases transfer learning can be improved by adding a dedicated component whose goal is to learn to visually map between the known domain and the new one.
Concretely, we use Unaligned Generative Adversarial Networks (GANs) to create a mapping function to translate images in the target task to corresponding images in the source task.
These mapping functions allow us to transform between various variations of the Breakout game, as well as between different levels of a Nintendo game, Road Fighter.
We show that learning this mapping is substantially more efficient than re-training.
A visualization of a trained agent playing Breakout and Road Fighter, with and without the GAN transfer, can be seen in \url{https://streamable.com/msgtm} and \url{https://streamable.com/5e2ka}.

assistant: text: We propose a method of transferring knowledge between related RL tasks using visual mappings, and demonstrate its effectiveness on visual variants of the Atari Breakout game and different levels of Road Fighter, a Nintendo car driving game.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Parametric Closed-Loop Policies for Markov Potential Games

Paper content:
Multiagent systems where the agents interact among themselves and with an stochastic environment can be formalized as stochastic games.
We study a subclass of these games, named Markov potential games (MPGs), that appear often in economic and engineering applications when the agents share some common resource.
We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards.
Previous analysis followed a variational approach that is only valid for very simple cases (convex rewards, invertible dynamics, and no coupled constraints); or considered deterministic dynamics and provided open-loop (OL) analysis, studying strategies that consist in predefined action sequences, which are not optimal for stochastic environments.
We present a closed-loop (CL) analysis for MPGs and consider parametric policies that depend on the current state and where agents adapt to stochastic transitions.
We provide easily verifiable, sufficient and necessary conditions for a stochastic game to be an MPG, even for complex parametric functions (e.g., deep neural networks); and show that a closed-loop Nash equilibrium (NE) can be found (or at least approximated) by solving a related optimal control problem (OCP).
This is useful since solving an OCP---which is a single-objective problem---is usually much simpler than solving the original set of coupled OCPs that form the game---which is a multiobjective control problem.
This is a considerable improvement over the previously standard approach for the CL analysis of MPGs, which gives no approximate solution if no NE belongs to the chosen parametric family, and which is practical only for simple parametric forms.
We illustrate the theoretical contributions with an example by applying our approach to a noncooperative communications engineering game.
We then solve the game with a deep reinforcement learning algorithm that learns policies that closely approximates an exact variational NE of the game.

assistant: text: We present general closed loop analysis for Markov potential games and show that deep reinforcement learning can be used for learning approximate closed-loop Nash equilibrium.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Cutting Down Training Memory by Re-fowarding

Paper content:
Deep Neutral Networks(DNNs) require huge GPU memory when training on modern image/video databases.
Unfortunately, the GPU memory as a hardware resource is always finite, which limits the image resolution, batch size, and learning rate that could be used for better DNN performance.
In this paper, we propose a novel training approach, called Re-forwarding, that substantially reduces memory usage in training.
Our approach automatically finds a subset of vertices in a DNN computation graph, and stores tensors only at these vertices during the first forward.
During backward, extra local forwards (called the Re-forwarding process) are conducted to compute the missing tensors between the subset of vertices.
The total memory cost becomes the sum of (1) the memory cost at the subset of vertices and (2) the maximum memory cost among local re-forwards.
Re-forwarding trades training time overheads for memory and does not compromise any performance in testing.
We propose theories and algorithms that achieve the optimal memory solutions for DNNs with either linear or arbitrary computation graphs.
Experiments show that Re-forwarding cuts down up-to 80% of training memory on popular DNNs such as Alexnet, VGG, ResNet, Densenet and Inception net.

assistant: text: This paper proposes fundamental theory and optimal algorithms for DNN training, which reduce up to 80% of training memory for popular DNNs.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy

Paper content:
In some misspecified settings, the posterior distribution in Bayesian statistics may lead to inconsistent estimates.
To fix this issue, it has been suggested to replace the likelihood by a pseudo-likelihood, that is the exponential of a loss function enjoying suitable robustness properties.
In this paper, we build a pseudo-likelihood based on the Maximum Mean Discrepancy, defined via an embedding of probability distributions into a reproducing kernel Hilbert space.
We show that this MMD-Bayes posterior is consistent and robust to model misspecification.
As the posterior obtained in this way might be intractable, we also prove that reasonable variational approximations of this posterior enjoy the same properties.
We provide details on a stochastic gradient algorithm to compute these variational approximations.
Numerical simulations indeed suggest that our estimator is more robust to misspecification than the ones based on the likelihood.

assistant: text: Robust Bayesian Estimation via Maximum Mean Discrepancy
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generalizing Reinforcement Learning to Unseen Actions

Paper content:
A fundamental trait of intelligence is the ability to achieve goals in the face of novel circumstances.
In this work, we address one such setting which requires solving a task with a novel set of actions.
Empowering machines with this ability requires generalization in the way an agent perceives its available actions along with the way it uses these actions to solve tasks.
Hence, we propose a framework to enable generalization over both these aspects: understanding an action’s functionality, and using actions to solve tasks through reinforcement learning.
Specifically, an agent interprets an action’s behavior using unsupervised representation learning over a collection of data samples reflecting the diverse properties of that action.
We employ a reinforcement learning architecture which works over these action representations, and propose regularization metrics essential for enabling generalization in a policy.
We illustrate the generalizability of the representation learning method and policy, to enable zero-shot generalization to previously unseen actions on challenging sequential decision-making environments.
Our results and videos can be found at sites.google.com/view/action-generalization/

assistant: text: We address the problem of generalization of reinforcement learning to unseen action spaces.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models

Paper content:
Many machine learning algorithms are vulnerable to almost imperceptible perturbations of their inputs.
So far it was unclear how much risk adversarial perturbations carry for the safety of real-world machine learning applications because most methods used to generate such perturbations rely either on detailed model information (gradient-based attacks) or on confidence scores such as class probabilities (score-based attacks), neither of which are available in most real-world scenarios.
In many such cases one currently needs to retreat to transfer-based attacks which rely on cumbersome substitute models, need access to the training data and can be defended against.
Here we emphasise the importance of attacks which solely rely on the final model decision.
Such decision-based attacks are (1) applicable to real-world black-box models such as autonomous cars, (2) need less knowledge and are easier to apply than transfer-based attacks and (3) are more robust to simple defences than gradient- or score-based attacks.
Previous attacks in this category were limited to simple models or simple datasets.
Here we introduce the Boundary Attack, a decision-based attack that starts from a large adversarial perturbation and then seeks to reduce the perturbation while staying adversarial.
The attack is conceptually simple, requires close to no hyperparameter tuning, does not rely on substitute models and is competitive with the best gradient-based attacks in standard computer vision tasks like ImageNet.
We apply the attack on two black-box algorithms from Clarifai.com.
The Boundary Attack in particular and the class of decision-based attacks in general open new avenues to study the robustness of machine learning models and raise new questions regarding the safety of deployed machine learning systems.
An implementation of the attack is available as part of Foolbox (https://github.com/bethgelab/foolbox).

assistant: text: A novel adversarial attack that can directly attack real-world black-box machine learning models without transfer.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Beyond Greedy Ranking: Slate Optimization via List-CVAE

Paper content:
The conventional approach to solving the recommendation problem greedily ranks
individual document candidates by prediction scores.
However, this method fails to
optimize the slate as a whole, and hence, often struggles to capture biases caused
by the page layout and document interdepedencies.
The slate recommendation
problem aims to directly find the optimally ordered subset of documents (i.e.
slates) that best serve users’ interests.
Solving this problem is hard due to the
combinatorial explosion of document candidates and their display positions on the
page.
Therefore we propose a paradigm shift from the traditional viewpoint of solving a ranking problem to a direct slate generation framework.
In this paper, we introduce List Conditional Variational Auto-Encoders (ListCVAE),
which learn the joint distribution of documents on the slate conditioned
on user responses, and directly generate full slates.
Experiments on simulated
and real-world data show that List-CVAE outperforms greedy ranking methods
consistently on various scales of documents corpora.

assistant: text: We used a CVAE type model structure to learn to directly generate slates/whole pages for recommendation systems.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Penetrating the Fog: the Path to Efficient CNN Models

Paper content:
With the increasing demand to deploy convolutional neural networks (CNNs) on mobile platforms, the sparse kernel approach was proposed, which could save more parameters than the standard convolution while maintaining accuracy.
However, despite the great potential, no prior research has pointed out how to craft an sparse kernel design with such potential (i.e., effective design), and all prior works just adopt simple combinations of existing sparse kernels such as group convolution.
Meanwhile due to the large design space it is also impossible to try all combinations of existing sparse kernels.
In this paper, we are the first in the field to consider how to craft an effective sparse kernel design by eliminating the large design space.
Specifically, we present a sparse kernel scheme to illustrate how to reduce the space from three aspects.
First, in terms of composition we remove designs composed of repeated layers.
Second, to remove designs with large accuracy degradation, we find an unified property named~\emph{information field} behind various sparse kernel designs, which could directly indicate the final accuracy.
Last, we remove designs in two cases where a better parameter efficiency could be achieved.
Additionally, we provide detailed efficiency analysis on the final 4 designs in our scheme.
Experimental results validate the idea of our scheme by showing that our scheme is able to find designs which are more efficient in using parameters and computation with similar or higher accuracy.

assistant: text: We are the first in the field to show how to craft an effective sparse kernel design from three aspects: composition, performance and efficiency.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Out-of-Order Decoding for Robust Neural Machine Translation

Paper content:
Autoregressive recurrent neural decoders that generate sequences of tokens one-by-one and left-to-right are the workhorse of modern machine translation.
In this work, we propose a new decoder architecture that can generate natural language sequences in an arbitrary order.
Along with generating tokens from a given vocabulary, our model additionally learns to select the optimal position for each produced token.
The proposed decoder architecture is fully compatible with the seq2seq framework and can be used as a drop-in replacement of any classical decoder.
We demonstrate the performance of our new decoder on the IWSLT machine translation task as well as inspect and interpret the learned decoding patterns by analyzing how the model selects new positions for each subsequent token.

assistant: text: new out-of-order decoder for neural machine translation
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
We learn to identify decision states, namely the parsimonious set of states where decisions meaningfully affect the future states an agent can reach in an environment.
We utilize the VIC framework, which maximizes an agent’s `empowerment’, ie the ability to reliably reach a diverse set of states -- and formulate a sandwich bound on the empowerment objective that allows identification of decision states.
Unlike previous work, our decision states are discovered without extrinsic rewards -- simply by interacting with the world.
Our results show that our decision states are: 1) often interpretable, and 2) lead to better exploration on downstream goal-driven tasks in partially observable environments.

assistant: text: Identify decision states (where agent can take actions that matter) without reward supervision, use it for transfer.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Latent Convolutional Models

Paper content:
We present a new latent model of natural images that can be learned on large-scale datasets.
The learning process provides a latent embedding for every image in the training dataset, as well as a deep convolutional network that maps the latent space to the image space.
After training, the new model provides a strong and universal image prior for a variety of image restoration tasks such as large-hole inpainting, superresolution, and colorization.
To model high-resolution natural images, our approach uses latent spaces of very high dimensionality (one to two orders of magnitude higher than previous latent image models).
To tackle this high dimensionality, we use latent spaces with a special manifold structure (convolutional manifolds) parameterized by a ConvNet of a certain architecture.
In the experiments, we compare the learned latent models with latent models learned by autoencoders, advanced variants of generative adversarial networks, and a strong baseline system using simpler parameterization of the latent space.
Our model outperforms the competing approaches over a range of restoration tasks.

assistant: text: We present a new deep latent model of natural images that can be trained from unlabeled datasets and can be utilized to solve various image restoration tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks

Paper content:
A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation.
This makes BPTT both computationally impractical and biologically implausible.
For this reason, full backpropagation through time is rarely used on long sequences, and truncated backpropagation through time is used as a heuristic.
However, this usually leads to biased estimates of the gradient in which longer term dependencies are ignored.
Addressing this issue, we propose an alternative algorithm, Sparse Attentive Backtracking, which might also be related to principles used by brains to learn long-term dependencies.
Sparse Attentive Backtracking learns an attention mechanism over the hidden states of the past and selectively backpropagates through paths with high attention weights.
This allows the model to learn long term dependencies while only backtracking for a small number of time steps, not just from the recent past but also from attended relevant past states.

assistant: text: Towards Efficient Credit Assignment in Recurrent Networks without Backpropagation Through Time
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: $\alpha^{\alpha}$-Rank: Scalable Multi-agent Evaluation through Evolution

Paper content:
Although challenging, strategy profile evaluation in large connected learner networks is crucial for enabling the next wave of machine learning applications.
Recently, $\alpha$-Rank, an evolutionary algorithm, has been proposed as a solution for ranking joint policy profiles in multi-agent systems.
$\alpha$-Rank claimed scalability through a polynomial time implementation with respect to the total number of pure strategy profiles.
In this paper, we formally prove that such a claim is not grounded.
In fact, we show that $\alpha$-Rank exhibits an exponential complexity in number of agents, hindering its application beyond a small finite number of joint profiles.
Realizing such a limitation, we contribute by proposing a scalable evaluation protocol that we title  $\alpha^{\alpha}$-Rank.
Our method combines evolutionary dynamics with stochastic optimization and double oracles for \emph{truly} scalable ranking with linear (in number of agents) time and memory complexities.
Our contributions allow us, for the first time, to conduct large-scale evaluation experiments of multi-agent systems, where we show successful results on large joint strategy profiles with sizes in the  order of $\mathcal{O}(2^{25})$ (i.e., $\approx \text{$33$ million strategies}$) -- a setting not evaluable using current techniques.

assistant: text: We provide a scalable solution to multi-agent evaluation with linear rate complexity in both time and memory in terms of number of agents
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: HANDLING CONCEPT DRIFT  IN WIFI-BASED INDOOR LOCALIZATION USING REPRESENTATION LEARNING

Paper content:
We outline the problem of concept drifts for time series data.
In this work, we analyze the temporal inconsistency of streaming wireless signals in the context of device-free passive indoor localization.
We show that data obtained from WiFi channel state information (CSI) can be used to train a robust system capable of performing room level localization.
One of the most challenging issues for such a system is the movement of input data distribution to an unexplored space over time, which leads to an unwanted shift in the learned boundaries of the output space.
In this work, we propose a phase and magnitude augmented feature space along with a standardization technique that is little affected by drifts.
We show that this robust representation of the data yields better learning accuracy and requires less number of retraining.

assistant: text: We introduce an augmented robust feature space for streaming wifi data that is capable of tackling concept drift for indoor localization
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Taking Apart Autoencoders: How do They Encode Geometric Shapes ?

Paper content:
We study the precise mechanisms which allow autoencoders to encode and decode a simple geometric shape, the disk.
In this carefully controlled setting, we are able to describe the specific form of the optimal solution to the minimisation problem of the training step.
We show that the autoencoder indeed approximates this solution during training.
Secondly, we identify a clear failure in the generalisation capacity of the autoencoder, namely its inability to interpolate data.
Finally, we explore several regularisation schemes to resolve the generalisation problem.
Given the great attention that has been recently given to the generative capacity of neural networks, we believe that studying in depth simple geometric cases sheds some light on the generation process and can provide a minimal requirement experimental setup for more complex architectures.

assistant: text: We study the functioning of autoencoders in a simple setting and advise new strategies for their regularisation in order to obtain bettre generalisation with latent interpolation in mind for image sythesis.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Optimistic Acceleration for Optimization

Paper content:
We consider new variants of optimization algorithms.
Our algorithms are based on the observation that mini-batch of stochastic gradients in consecutive iterations do not change drastically and consequently may be predictable.
Inspired by the similar setting in online learning literature called Optimistic Online learning, we propose two new optimistic algorithms for AMSGrad and Adam, respectively, by  exploiting the predictability of gradients.
The new algorithms combine the idea of momentum method, adaptive gradient method, and algorithms in Optimistic Online learning, which leads to speed up in training deep neural nets in practice.

assistant: text: We consider new variants of optimization algorithms for training deep nets.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: APPLICATION OF DEEP CONVOLUTIONAL NEURAL NETWORK TO PREVENT ATM FRAUD BY FACIAL DISGUISE IDENTIFICATION

Paper content:
The paper proposes and demonstrates a Deep Convolutional Neural Network (DCNN) architecture to identify users with disguised face attempting a fraudulent ATM transaction.
The recent introduction of Disguised Face Identification (DFI) framework proves the applicability of deep neural networks for this very problem.
All the ATMs nowadays incorporate a hidden camera in them and capture the footage of their users.
However, it is impossible for the police to track down the impersonators with disguised faces from the ATM footage.
The proposed deep convolutional neural network is trained to identify, in real time, whether the user in the captured image is trying to cloak his identity or not.
The output of the DCNN is then reported to the ATM to take appropriate steps and prevent the swindler from completing the transaction.
The network is trained using a dataset of images captured in similar situations as of an ATM.
The comparatively low background clutter in the images enables the network to demonstrate high accuracy in feature extraction and classification for all the different disguises.

assistant: text: Proposed System can prevent impersonators with facial disguises from completing a fraudulent transaction using a pre-trained DCNN.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Robustness May Be at Odds with Accuracy

Paper content:
We show that there exists an inherent tension between the goal of adversarial robustness and that of standard generalization.
Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy.
We demonstrate that this trade-off between the standard accuracy of a model and its robustness to adversarial perturbations provably exists even in a fairly simple and natural setting.
These findings also corroborate a similar phenomenon observed in practice.
Further, we argue that this phenomenon is a consequence of robust classifiers learning fundamentally different feature representations than  standard classifiers.
These differences, in particular, seem to result in unexpected benefits: the features learned by robust models tend to align better with salient data characteristics and human perception.

assistant: text: We show that adversarial robustness might come at the cost of standard classification performance, but also yields unexpected benefits.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Dynamically Balanced Value Estimates for Actor-Critic Methods

Paper content:
Reinforcement learning in an actor-critic setting relies on accurate value estimates of the critic.
However, the combination of function approximation, temporal difference (TD) learning and off-policy training can lead to an overestimating value function.
A solution is to use Clipped Double Q-learning (CDQ), which is used in the TD3 algorithm and computes the minimum of two critics in the TD-target.
We show that CDQ induces an underestimation bias and propose a new algorithm that accounts for this by using a weighted average of the target from CDQ and the target coming from a single critic.
The weighting parameter is adjusted during training such that the value estimates match the actual discounted return on the most recent episodes and by that it balances over- and underestimation.
Empirically, we obtain more accurate value estimates and demonstrate state of the art results on several OpenAI gym tasks.

assistant: text: A method for more accurate critic estimates in reinforcement learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: An Attention-Based Model for Learning Dynamic Interaction Networks

Paper content:
While machine learning models achieve human-comparable performance on sequential data, exploiting structured knowledge is still a challenging problem.
Spatio-temporal graphs have been proved to be a useful tool to abstract interaction graphs and previous works exploits carefully designed feed-forward architecture to preserve such structure.
We argue to scale such network design to real-world problem, a model needs to automatically learn a meaningful representation of the possible relations.
Learning such interaction structure is not trivial: on the one hand, a model has to discover the hidden relations between different problem factors in an unsupervised way; on the other hand, the mined relations have to be interpretable.
In this paper, we propose an attention module able to project a graph sub-structure in a fixed size embedding, preserving the influence that the neighbours exert on a given vertex.
On a comprehensive evaluation done on real-world as well as toy task, we found our model competitive against strong baselines.

assistant: text: A graph neural network able to automatically learn and leverage a dynamic interactive graph structure
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks

Paper content:
We analyze speed of convergence to global optimum for gradient descent training a deep linear neural network by minimizing the L2 loss over whitened data.
Convergence at a linear rate is guaranteed when the following hold: (i) dimensions of hidden layers are at least the minimum of the input and output dimensions; (ii) weight matrices at initialization are approximately balanced; and (iii) the initial loss is smaller than the loss of any rank-deficient solution.
The assumptions on initialization (conditions (ii) and (iii)) are necessary, in the sense that violating any one of them may lead to convergence failure.
Moreover, in the important case of output dimension 1, i.e. scalar regression, they are met, and thus convergence to global optimum holds, with constant probability under a random initialization scheme.
Our results significantly extend previous analyses, e.g., of deep linear residual networks (Bartlett et al., 2018).

assistant: text: We analyze gradient descent for deep linear neural networks, providing a guarantee of convergence to global optimum at a linear rate.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarial Training and Provable Defenses: Bridging the Gap

Paper content:
We propose a new method to train neural networks based on a novel combination of adversarial training and provable defenses.
The key idea is to model training as a procedure which includes both, the verifier and the adversary.
In every iteration, the verifier aims to certify the network using convex relaxation while the adversary tries to find inputs inside that convex relaxation which cause verification to fail.
We experimentally show that this training method is promising and achieves the best of both worlds – it produces a model with state-of-the-art accuracy (74.8%) and certified robustness (55.9%) on the challenging CIFAR-10 dataset with a 2/255 L-infinity perturbation.
This is a significant improvement over the currently known best results of 68.3% accuracy and 53.9% certified robustness, achieved using a 5 times larger network than our work.

assistant: text: We propose a novel combination of adversarial training and provable defenses which produces a model with state-of-the-art accuracy and certified robustness on CIFAR-10.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Compositional Obverter Communication Learning from Raw Visual Input

Paper content:
One of the distinguishing aspects of human language is its compositionality, which allows us to describe complex environments with limited vocabulary.
Previously, it has been shown that neural network agents can learn to communicate in a highly structured, possibly compositional language based on disentangled input (e.g. hand- engineered features).
Humans, however, do not learn to communicate based on well-summarized features.
In this work, we train neural agents to simultaneously develop visual perception from raw image pixels, and learn to communicate with a sequence of discrete symbols.
The agents play an image description game where the image contains factors such as colors and shapes.
We train the agents using the obverter technique where an agent introspects to generate messages that maximize its own understanding.
Through qualitative analysis, visualization and a zero-shot test, we show that the agents can develop, out of raw image pixels, a language with compositional properties, given a proper pressure from the environment.

assistant: text: We train neural network agents to develop a language with compositional properties from raw pixel input.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Wasserstein Auto-Encoders

Paper content:
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution.
WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Auto-Encoder (VAE).
This regularizer encourages the encoded training distribution to match the prior.
We compare our algorithm with several other techniques and show that it is a generalization of adversarial auto-encoders (AAE).
Our experiments show that WAE shares many of the properties of VAEs (stable training, encoder-decoder architecture, nice latent manifold structure) while generating samples of better quality.

assistant: text: We propose a new auto-encoder based on the Wasserstein distance, which improves on the sampling properties of VAE.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Task-Based Top-Down Modulation Network for Multi-Task-Learning Applications

Paper content:
A general problem that received considerable recent attention is how to perform multiple tasks in the same network, maximizing both efficiency and prediction accuracy.
A popular approach consists of a multi-branch architecture on top of a
shared backbone, jointly trained on a weighted sum of losses.
However, in many cases, the shared representation results in non-optimal performance, mainly due to an interference between conflicting gradients of uncorrelated tasks.
Recent approaches address this problem by a channel-wise modulation of the feature-maps along the shared backbone, with task specific vectors, manually or dynamically tuned.
Taking this approach a step further, we propose a novel architecture which
modulate the recognition network channel-wise, as well as spatial-wise, with an efficient top-down image-dependent computation scheme.
Our architecture uses no task-specific branches, nor task specific modules.
Instead, it uses a top-down modulation network that is shared between all of the tasks.
We show the effectiveness of our scheme by achieving on par or better results than alternative approaches on both correlated and uncorrelated sets of tasks.
We also demonstrate our advantages in terms of model size, the addition of novel tasks and interpretability.
Code will be released.

assistant: text: We propose a top-down modulation network for multi-task learning applications with several advantages over current schemes.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Aggregating explanation methods for neural networks stabilizes explanations

Paper content:
Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation.
Our contributions in this paper are twofold.
First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a single aggregated explanation.
The aggregation is more robust and aligns better with the neural network than any single explanation method..
Second, we propose a new approach to evaluating explanation methods that circumvents the need for manual evaluation and is not reliant on the alignment of neural networks and humans decision processes.

assistant: text: We show in theory and in practice that combining multiple explanation methods for DNN benefits the explanation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Target Acquisition for Handheld Virtual Panels in VR

Paper content:
The Handheld Virtual Panel (HVP) is the virtual panel attached to the non-dominant hand’s controller in virtual reality (VR).
The HVP is the go-to technique for enabling menus and toolboxes in VR devices.
In this paper, we investigate target acquisition performance for the HVP as a function of four factors: target width, target distance, the direction of approach with respect to gravity, and the angle of approach.
Our results show that all four factors have significant effects on user performance.
Based on the results, we propose guidelines towards the ergonomic and performant design of the HVP interfaces.

assistant: text: The paper investigates target acquisition for handheld virtual panels in VR and shows that target width, distance, direction of approach with respect to gravity, and angle of approach, all impact user performance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Measuring the Intrinsic Dimension of Objective Landscapes

Paper content:
Many recently trained neural networks employ large numbers of parameters to achieve good performance.
One may intuitively use the number of parameters required as a rough gauge of the difficulty of a problem.
But how accurate are such notions?
How many parameters are really needed?
In this paper we attempt to answer this question by training networks not in their native parameter space, but instead in a smaller, randomly oriented subspace.
We slowly increase the dimension of this subspace, note at which dimension solutions first appear, and define this to be the intrinsic dimension of the objective landscape.
The approach is simple to implement, computationally tractable, and produces several suggestive conclusions.
Many problems have smaller intrinsic dimensions than one might suspect, and the intrinsic dimension for a given dataset varies little across a family of models with vastly different sizes.
This latter result has the profound implication that once a parameter space is large enough to solve a problem, extra parameters serve directly to increase the dimensionality of the solution manifold.
Intrinsic dimension allows some quantitative comparison of problem difficulty across supervised, reinforcement, and other types of learning where we conclude, for example, that solving the inverted pendulum problem is 100 times easier than classifying digits from MNIST, and playing Atari Pong from pixels is about as hard as classifying CIFAR-10.
In addition to providing new cartography of the objective landscapes wandered by parameterized models, the method is a simple technique for constructively obtaining an upper bound on the minimum description length of a solution.
A byproduct of this construction is a simple approach for compressing networks, in some cases by more than 100 times.

assistant: text: We train in random subspaces of parameter space to measure how many dimensions are really needed to find a solution.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Generative Model For Electron Paths

Paper content:
Chemical reactions can be described as the stepwise redistribution of electrons in molecules.
As such, reactions are often depicted using "arrow-pushing" diagrams which show this movement as a sequence of arrows.
We propose an electron path prediction model (ELECTRO) to learn these sequences directly from raw reaction data.
Instead of predicting product molecules directly from reactant molecules in one shot, learning a model of electron movement has the benefits of
(a) being easy for chemists to interpret,
(b) incorporating constraints of chemistry, such as balanced atom counts before and after the reaction, and
(c) naturally encoding the sparsity of chemical reactions, which usually involve changes in only a small number of atoms in the reactants.
We design a method to extract approximate reaction paths from any dataset of atom-mapped reaction SMILES strings.
Our model achieves excellent performance on an important subset of the USPTO reaction dataset, comparing favorably to the strongest baselines.
Furthermore, we show that our model recovers a basic knowledge of chemistry without being explicitly trained to do so.

assistant: text: A generative model for reaction prediction that learns the mechanistic electron steps of a reaction directly from raw reaction data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learned imaging with constraints and uncertainty quantification

Paper content:
We outline new approaches to incorporate ideas from deep learning into wave-based least-squares imaging.
The aim, and main contribution of this work, is the combination of handcrafted constraints with deep convolutional neural networks, as a way to harness their remarkable ease of generating natural images.
The mathematical basis underlying our method is the expectation-maximization framework, where data are divided in batches and coupled to additional "latent" unknowns.
These unknowns are pairs of elements from the original unknown space (but now coupled to a specific data batch) and network inputs.
In this setting, the neural network controls the similarity between these additional parameters, acting as a "center" variable.
The resulting problem amounts to a maximum-likelihood estimation of the network parameters when the augmented data model is marginalized over the latent variables.

assistant: text: We combine hard handcrafted constraints with a deep prior weak constraint to perform seismic imaging and reap information on the "posterior" distribution leveraging multiplicity in the data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Reformer: The Efficient Transformer

Paper content:
Large Transformer models routinely achieve state-of-the-art results on
a number of tasks but training these models can be prohibitively costly,
especially on long sequences.
We introduce two techniques to improve
the efficiency of Transformers.
For one, we replace dot-product attention
by one that uses locality-sensitive hashing, changing its complexity
from O(L^2) to O(L), where L is the length of the sequence.
Furthermore, we use reversible residual layers instead of the standard
residuals, which allows storing activations only once in the training
process instead of N times, where N is the number of layers.
The resulting model, the Reformer, performs on par with Transformer models
while being much more memory-efficient and much faster on long sequences.

assistant: text: Efficient Transformer with locality-sensitive hashing and reversible layers
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Pretrain-KGEs: Learning Knowledge Representation from Pretrained Models for Knowledge Graph Embeddings

Paper content:
Learning knowledge graph embeddings (KGEs) is an efficient approach to knowledge graph completion.
Conventional KGEs often suffer from limited knowledge representation, which causes less accuracy especially when training on sparse knowledge graphs.
To remedy this, we present Pretrain-KGEs, a training framework for learning better knowledgeable entity and relation embeddings, leveraging the abundant linguistic knowledge from pretrained language models.
Specifically, we propose a unified approach in which we first learn entity and relation representations via pretrained language models and use the representations to initialize entity and relation embeddings for training KGE models.
Our proposed method is model agnostic in the sense that it can be applied to any variant of KGE models.
Experimental results show that our method can consistently improve results and achieve state-of-the-art performance using different KGE models such as TransE and QuatE, across four benchmark KG datasets in link prediction and triplet classification tasks.

assistant: text: We propose to learn knowledgeable entity and relation representations from Bert for knowledge graph embeddings.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Graph Constrained Reinforcement Learning for Natural Language Action Spaces

Paper content:
Interactive Fiction games are text-based simulations in which an agent interacts with the world purely through natural language.
They are ideal environments for studying how to extend reinforcement learning agents to meet the challenges of natural language understanding, partial observability, and action generation in combinatorially-large text-based action spaces.
We present KG-A2C, an agent that builds a dynamic knowledge graph while exploring and generates actions using a template-based action space.
We contend that the dual uses of the knowledge graph to reason about game state and to constrain natural language generation are the keys to scalable exploration of combinatorially large natural language actions.
Results across a wide variety of IF games show that KG-A2C outperforms current IF agents despite the exponential increase in action space size.

assistant: text: We present KG-A2C, a reinforcement learning agent that builds a dynamic knowledge graph while exploring and generates natural language using a template-based action space - outperforming all current agents on a wide set of text-based games.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Can Deep Reinforcement Learning solve Erdos-Selfridge-Spencer Games?

Paper content:
Deep reinforcement learning has achieved many recent successes, but our understanding of its strengths and limitations is hampered by the lack of rich environments in which we can fully characterize optimal behavior, and correspondingly diagnose individual actions against such a characterization.
Here we consider a family of combinatorial games, arising from work of Erdos, Selfridge, and Spencer, and we propose their use as environments for evaluating and comparing different approaches to reinforcement learning.
These games have a number of appealing features: they are challenging for current learning approaches, but they form
(i) a low-dimensional, simply parametrized environment where
(ii) there is a linear closed form solution for optimal behavior from any state, and
(iii) the difficulty of the game can be tuned by changing environment parameters in an interpretable way.
We use these Erdos-Selfridge-Spencer games not only to compare different algorithms, but also to compare approaches based on supervised and reinforcement learning, to analyze the power of multi-agent approaches in improving performance, and to evaluate generalization to environments outside the training set.

assistant: text: We adapt a family of combinatorial games with tunable difficulty and an optimal policy expressible as linear network, developing it as a rich environment for reinforcement learning, showing contrasts in performance with supervised learning, and analyzing multiagent learning and generalization.