ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: ANYTIME MINIBATCH: EXPLOITING STRAGGLERS IN ONLINE DISTRIBUTED OPTIMIZATION

Paper content:
Distributed optimization is vital in solving large-scale machine learning problems.
A widely-shared feature of distributed optimization techniques is the requirement that all nodes complete their assigned tasks in each computational epoch before the system can proceed to the next epoch.
In such settings, slow nodes, called stragglers, can greatly slow progress.
To mitigate the impact of stragglers, we propose an online distributed optimization method called Anytime Minibatch.
In this approach, all nodes are given a fixed time to compute the gradients of as many data samples as possible.
The result is a variable per-node minibatch size.
Workers then get a fixed communication time to average their minibatch gradients via several rounds of consensus, which are then used to update primal variables via dual averaging.
Anytime Minibatch prevents stragglers from holding up the system without wasting the work that stragglers can complete.
We present a convergence analysis and analyze the wall time performance.
Our numerical results show that our approach is up to 1.5 times faster in Amazon EC2 and it is up to five times faster when there is greater variability in compute node performance.

assistant: text: Accelerate distributed optimization by exploiting stragglers.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Advanced Neuroevolution: A gradient-free algorithm to train Deep Neural Networks

Paper content:
In this paper we present a novel optimization algorithm called Advanced Neuroevolution.
The aim for this algorithm is to train deep neural networks, and eventually act as an alternative to Stochastic Gradient Descent (SGD) and its variants as needed.We evaluated our algorithm on the MNIST dataset, as well as on several global optimization problems such as the Ackley function.
We find the algorithm performing relatively well for both cases, overtaking other global optimization algorithms such as Particle Swarm Optimization (PSO) and Evolution Strategies (ES).

assistant: text: A new algorithm to train deep neural networks. Tested on optimization functions and MNIST.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: I Know the Feeling: Learning to Converse with Empathy

Paper content:
Beyond understanding what is being discussed, human communication requires an awareness of what someone is feeling.
One challenge for dialogue agents is recognizing feelings in the conversation partner and replying accordingly, a key communicative skill that is trivial for humans.
Research in this area is made difficult by the paucity of suitable publicly available datasets both for emotion and dialogues.
This work proposes a new task for empathetic dialogue generation and EmpatheticDialogues, a dataset of 25k conversations grounded in emotional situations to facilitate training and evaluating dialogue systems.
Our experiments indicate that dialogue models that use our dataset are perceived to be more empathetic by human evaluators, while improving on other metrics as well (e.g. perceived relevance of responses, BLEU scores), compared to models merely trained on large-scale Internet conversation data.
We also present empirical comparisons of several ways to improve the performance of a given model by leveraging existing models or datasets without requiring lengthy re-training of the full model.

assistant: text: We improve existing dialogue systems for responding to people sharing personal stories, incorporating emotion prediction representations and also release a new benchmark and dataset of empathetic dialogues.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models

Paper content:
Deep generative neural networks have proven effective at both conditional and unconditional modeling of complex data distributions.
Conditional generation enables interactive control, but creating new controls often requires expensive retraining.
In this paper, we develop a method to condition generation without retraining the model.
By post-hoc learning latent constraints, value functions identify regions in latent space that generate outputs with desired attributes, we can conditionally sample from these regions with gradient-based optimization or amortized actor functions.
Combining attribute constraints with a universal “realism” constraint, which enforces similarity to the data distribution, we generate realistic conditional images from an unconditional variational autoencoder.
Further, using gradient-based optimization, we demonstrate identity-preserving transformations that make the minimal adjustment in latent space to modify the attributes of an image.
Finally, with discrete sequences of musical notes, we demonstrate zero-shot conditional generation, learning latent constraints in the absence of labeled data or a differentiable reward function.

assistant: text: A new approach to conditional generation by constraining the latent space of an unconditional generative model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Large-Scale Study of Curiosity-Driven Learning

Paper content:
Reinforcement learning algorithms rely on carefully engineered rewards from the environment that are extrinsic to the agent.
However, annotating each environment with hand-designed, dense rewards is difficult and not scalable, motivating the need for developing reward functions that are intrinsic to the agent.
Curiosity is such intrinsic reward function which uses prediction error as a reward signal.
In this paper:
(a) We perform the first large-scale study of purely curiosity-driven learning, i.e. {\em without any extrinsic rewards}, across $54$ standard benchmark environments, including the Atari game suite.
Our results show surprisingly good performance as well as a high degree of alignment between the intrinsic curiosity objective and the hand-designed extrinsic rewards of many games.
(b) We investigate the effect of using different feature spaces for computing prediction error and show that random features are sufficient for many popular RL game benchmarks, but learned features appear to generalize better (e.g. to novel game levels in Super Mario Bros.).
(c) We demonstrate limitations of the prediction-based rewards in stochastic setups.
Game-play videos and code are at https://doubleblindsupplementary.github.io/large-curiosity/.

assistant: text: An agent trained only with curiosity, and no extrinsic reward, does surprisingly well on 54 popular environments, including the suite of Atari games, Mario etc.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets

Paper content:
Training activation quantized neural networks involves minimizing a piecewise constant training loss whose gradient vanishes almost everywhere, which is undesirable for the standard back-propagation or chain rule.
An empirical way around this issue is to use a straight-through estimator (STE) (Bengio et al., 2013) in the backward pass only, so that the "gradient" through the modified chain rule becomes non-trivial.
Since this unusual "gradient" is certainly not the gradient of loss function, the following question arises: why searching in its negative direction minimizes the training loss?
In this paper, we provide the theoretical justification of the concept of STE by answering this question.
We consider the problem of learning a two-linear-layer network with binarized ReLU activation and Gaussian input data.
We shall refer to the unusual "gradient" given by the STE-modifed chain rule as coarse gradient.
The choice of STE is not unique.
We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient (not available for the training), and its negation is a descent direction for minimizing the population loss.
We further show the associated coarse gradient descent algorithm converges to a critical point of the population loss minimization problem.
Moreover, we show that a poor choice of STE leads to instability of the training algorithm near certain local minima, which is verified with CIFAR-10 experiments.

assistant: text: We make theoretical justification for the concept of straight-through estimator.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks

Paper content:
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level.
We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods.
The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses.
As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution.
Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.

assistant: text: Introducing the response charactrization method for interpreting cell dynamics in learned long short-term memory (LSTM) networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarial Training Can Hurt Generalization

Paper content:
While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary).
Previous work has studied this tradeoff between standard and robust accuracy, but only in the setting where no predictor performs well on both objectives in the infinite data limit.
In this paper, we show that even when the optimal predictor with infinite data performs well on both objectives, a tradeoff can still manifest itself with finite data.
Furthermore, since our construction is based on a convex learning problem, we rule out optimization concerns, thus laying bare a fundamental tension between robustness and generalization.
Finally, we show that robust self-training mostly eliminates this tradeoff by leveraging unlabeled data.

assistant: text: Even if there is no tradeoff in the infinite data limit, adversarial training can have worse standard accuracy even in a convex problem.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
This paper presents a system for immersive visualization of Non-Euclidean spaces using real-time ray tracing.
It exploits the capabilities of the new generation of GPU’s based on the NVIDIA’s Turing architecture in order to develop new methods for intuitive exploration of landscapes featuring non-trivial geometry and topology in virtual reality.

assistant: text: Immersive Visualization of the Classical Non-Euclidean Spaces using Real-Time Ray Tracing.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Sensitivity and Generalization in Neural Networks: an Empirical Study

Paper content:
In practice it is often found that large over-parameterized neural networks generalize better than their smaller counterparts, an observation that appears to conflict with classical notions of function complexity, which typically favor smaller models.
In this work, we investigate this tension between complexity and generalization through an extensive empirical exploration of two natural metrics of complexity related to sensitivity to input perturbations.
Our experiments survey thousands of models with different architectures, optimizers, and other hyper-parameters, as well as four different image classification datasets.
We find that trained neural networks are more robust to input perturbations in the vicinity of the training data manifold, as measured by the input-output Jacobian of the network, and that this correlates well with generalization.
We further establish that factors associated with poor generalization -- such as full-batch training or using random labels -- correspond to higher sensitivity, while factors associated with good generalization  -- such as data augmentation and ReLU non-linearities -- give rise to more robust functions.
Finally, we demonstrate how the input-output Jacobian norm can be predictive of generalization at the level of individual test points.

assistant: text: We perform massive experimental studies characterizing the relationships between Jacobian norms, linear regions, and generalization.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Measuring Compositionality in Representation Learning

Paper content:
Many machine learning algorithms represent input data with vector embeddings or discrete codes.
When inputs exhibit compositional structure (e.g. objects built from parts or procedures from subroutines), it is natural to ask whether this compositional structure is reflected in the the inputs’ learned representations.
While the assessment of compositionality in languages has received significant attention in linguistics and adjacent fields, the machine learning literature lacks general-purpose tools for producing graded measurements of compositional structure in more general (e.g. vector-valued) representation spaces.
We describe a procedure for evaluating compositionality by measuring how well the true representation-producing model can be approximated by a model that explicitly composes a collection of inferred representational primitives.
We use the procedure to provide formal and empirical characterizations of compositional structure in a variety of settings, exploring the relationship between compositionality and learning dynamics, human judgments, representational similarity, and generalization.
We have introduced a new evaluation method called TRE for generating graded judgments about compositional structure in representation learning problems where the structure of the observations is understood.
TRE infers a set of primitive meaning representations that, when composed, approximate the observed representations, then measures the quality of this approximation.
We have applied TRE-based analysis to four different problems in representation learning, relating compositionality to learning dynamics, linguistic compositionality, similarity and generalization.Many interesting questions regarding compositionality and representation learning remain open.
The most immediate is how to generalize TRE to the setting where oracle derivations are not available; in this case Equation 2 must be solved jointly with an unsupervised grammar induction problem BID25 .
Beyond this, it is our hope that this line of research opens up two different kinds of new work: better understanding of existing machine learning models, by providing a new set of tools for understanding their representational capacity; and better understanding of problems, by better understanding the kinds of data distributions and loss functions that give rise to compositionalor non-compositional representations of observations.

assistant: text: This paper proposes a simple procedure for evaluating compositional structure in learned representations, and uses the procedure to explore the role of compositionality in four learning problems.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Backpropagation-Free Deep Architectures with Kernels

Paper content:
One can substitute each neuron in any neural network with a kernel machine and obtain a counterpart powered by kernel machines.
The new network inherits the expressive power and architecture of the original but works in a more intuitive way since each node enjoys the simple interpretation as a hyperplane (in a reproducing kernel Hilbert space).
Further, using the kernel multilayer perceptron as an example, we prove that in classification, an optimal representation that minimizes the risk of the network can be characterized for each hidden layer.
This result removes the need of backpropagation in learning the model and can be generalized to any feedforward kernel network.
Moreover, unlike backpropagation, which turns models into black boxes, the optimal hidden representation enjoys an intuitive geometric interpretation, making the dynamics of learning in a deep kernel network simple to understand.
Empirical results are provided to validate our theory.

assistant: text: We combine kernel method with connectionist models and show that the resulting deep architectures can be trained layer-wise and have more transparent learning dynamics.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: RefNet: Automatic Essay Scoring by Pairwise Comparison

Paper content:
Automatic Essay Scoring (AES) has been an active research area as it can greatly reduce the workload of teachers and prevents subjectivity bias .
Most recent AES solutions apply deep neural network (DNN)-based models with regression, where the neural neural-based encoder learns an essay representation that helps differentiate among the essays and the corresponding essay score is inferred by a regressor.
Such DNN approach usually requires a lot of expert-rated essays as training data in order to learn a good essay representation for accurate scoring.
However, such data is usually expensive and thus is sparse.
Inspired by the observation that human usually scores an essay by comparing it with some references, we propose a Siamese framework called Referee Network (RefNet) which allows the model to compare the quality of two essays by capturing the relative features that can differentiate the essay pair.
The proposed framework can be applied as an extension to regression models as it can capture additional relative features on top of internal information.
Moreover, it intrinsically augment the data by pairing thus is ideal for handling data sparsity.
Experiment shows that our framework can significantly improve the existing regression models and achieve acceptable performance even when the training data is greatly reduced.

assistant: text: Automatically score essays on sparse data by comparing new essays with known samples with Referee Network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Reinforcement Learning Algorithm Selection

Paper content:
This paper formalises the problem of online algorithm selection in the context of Reinforcement Learning (RL).
The setup is as follows: given an episodic task and a finite number of off-policy RL algorithms, a meta-algorithm has to decide which RL algorithm is in control during the next episode so as to maximize the expected return.
The article presents a novel meta-algorithm, called Epochal Stochastic Bandit Algorithm Selection (ESBAS).
Its principle is to freeze the policy updates at each epoch, and to leave a rebooted stochastic bandit in charge of the algorithm selection.
Under some assumptions, a thorough theoretical analysis demonstrates its near-optimality considering the structural sampling budget limitations.
ESBAS is first empirically evaluated on a dialogue task where it is shown to outperform each individual algorithm in most configurations.
ESBAS is then adapted to a true online setting where algorithms update their policies after each transition, which we call SSBAS.
SSBAS is evaluated on a fruit collection task where it is shown to adapt the stepsize parameter more efficiently than the classical hyperbolic decay, and on an Atari game, where it improves the performance by a wide margin.

assistant: text: This paper formalises the problem of online algorithm selection in the context of Reinforcement Learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Zero-Shot Policy Transfer with Disentangled Attention

Paper content:
Domain adaptation is an open problem in deep reinforcement learning (RL).
Often, agents are asked to perform in environments where data is difficult to obtain.
In such settings, agents are trained in similar environments, such as simulators, and are then transferred to the original environment.
The gap between visual observations of the source and target environments often causes the agent to fail in the target environment.
We present a new RL agent, SADALA (Soft Attention DisentAngled representation Learning Agent).
SADALA first learns a compressed state representation.
It then jointly learns to ignore distracting features and solve the task presented.
SADALA's separation of important and unimportant visual features leads to robust domain transfer.
SADALA outperforms both prior disentangled-representation based RL and domain randomization approaches across RL environments (Visual Cartpole and DeepMind Lab).

assistant: text: We present an agent that uses a beta-vae to extract visual features and an attention mechanism to ignore irrelevant features from visual observations to enable robust transfer between visual domains.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Distributed Distributional Deterministic Policy Gradients

Paper content:
This work adopts the very successful distributional perspective on reinforcement learning and adapts it to the continuous control setting.
We combine this within a distributed framework for off-policy learning in order to develop what we call the Distributed Distributional Deep Deterministic Policy Gradient algorithm, D4PG.
We also combine this technique with a number of additional, simple improvements such as the use of N-step returns and prioritized experience replay.
Experimentally we examine the contribution of each of these individual components, and show how they interact, as well as their combined contributions.
Our results show that across a wide variety of simple control tasks, difficult manipulation tasks, and a set of hard obstacle-based locomotion tasks the D4PG algorithm achieves state of the art performance.

assistant: text: We develop an agent that we call the Distributional Deterministic Deep Policy Gradient algorithm, which achieves state of the art performance on a number of challenging continuous control problems.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches

Paper content:
Stochastic neural net weights are used in a variety of contexts, including regularization, Bayesian neural nets, exploration in reinforcement learning, and evolution strategies.
Unfortunately, due to the large number of weights, all the examples in a mini-batch typically share the same weight perturbation, thereby limiting the variance reduction effect of large mini-batches.
We introduce flipout, an efficient method for decorrelating the gradients within a mini-batch by implicitly sampling pseudo-independent weight perturbations for each example.
Empirically, flipout achieves the ideal linear variance reduction for fully connected networks, convolutional networks, and RNNs.
We find significant speedups in training neural networks with multiplicative Gaussian perturbations.
We show that flipout is effective at regularizing LSTMs, and outperforms previous methods.
Flipout also enables us to vectorize evolution strategies: in our experiments, a single GPU with flipout can handle the same throughput as at least 40 CPU cores using existing methods, equivalent to a factor-of-4 cost reduction on Amazon Web Services.

assistant: text: We introduce flipout, an efficient method for decorrelating the gradients computed by stochastic neural net weights within a mini-batch by implicitly sampling pseudo-independent weight perturbations for each example.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: LOSSLESS SINGLE IMAGE SUPER RESOLUTION FROM LOW-QUALITY JPG IMAGES

Paper content:
Super Resolution (SR) is a fundamental and important low-level computer vision (CV) task.
Different from traditional SR models, this study concentrates on a specific but realistic SR issue: How can we obtain satisfied SR results from compressed JPG (C-JPG) image, which widely exists on the Internet.
In general, C-JPG can release storage space while keeping considerable quality in visual.
However, further image processing operations, e.g., SR, will suffer from enlarging inner artificial details and result in unacceptable outputs.
To address this problem, we propose a novel SR structure with two specifically designed components, as well as a cycle loss.
In short, there are mainly three contributions to this paper.
First, our research can generate high-qualified SR images for prevalent C-JPG images.
Second, we propose a functional sub-model to recover information for C-JPG images, instead of the perspective of noise elimination in traditional SR approaches.
Third, we further integrate cycle loss into SR solver to build a hybrid loss function for better SR generation.
Experiments show that our approach achieves outstanding performance among state-of-the-art methods.

assistant: text: We solve the specific SR issue of low-quality JPG images by functional sub-models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the Convergence and Robustness of Batch Normalization

Paper content:
Despite its empirical success, the theoretical underpinnings of the stability, convergence and acceleration properties of batch normalization (BN) remain elusive.
In this paper, we attack this problem from a modelling approach, where we perform thorough theoretical analysis on BN applied to simplified model: ordinary least squares (OLS).
We discover that gradient descent on OLS with BN has interesting properties, including a scaling law, convergence for arbitrary learning rates for the weights, asymptotic acceleration effects, as well as insensitivity to choice of learning rates.
We then demonstrate numerically that these findings are not specific to the OLS problem and hold qualitatively for more complex supervised learning problems.
This points to a new direction towards uncovering the mathematical principles that underlies batch normalization.

assistant: text: We mathematically analyze the effect of batch normalization on a simple model and obtain key new insights that applies to general supervised learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Matching Distributions via  Optimal Transport for Semi-Supervised Learning

Paper content:
Semi-Supervised Learning (SSL) approaches have been an influential framework for the usage of unlabeled data when there is not a sufficient amount of labeled data available over the course of training.
SSL methods based on Convolutional Neural Networks (CNNs) have recently provided successful results on standard benchmark tasks such as image classification.
In this work, we consider the general setting of  SSL problem where the labeled and unlabeled data  come from the same underlying probability distribution.
We  propose a new approach that adopts  an Optimal Transport (OT) technique serving as a metric of similarity between discrete empirical probability measures to  provide pseudo-labels for the unlabeled data, which can then be used in conjunction with the initial labeled data to train the CNN model in an SSL manner.
We have evaluated and compared our proposed method with state-of-the-art SSL algorithms on standard datasets to demonstrate the superiority and effectiveness of our  SSL algorithm.

assistant: text: We propose a new algorithm based on the optimal transport to train a CNN in an SSL fashion.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Geom-GCN: Geometric Graph Convolutional Networks

Paper content:
Message-passing neural networks (MPNNs) have been successfully applied in a wide variety of applications in the real world.
However, two fundamental weaknesses of MPNNs' aggregators limit their ability to represent graph-structured data: losing the structural information of nodes in neighborhoods and lacking the ability to capture long-range dependencies in disassortative graphs.
Few studies have noticed the weaknesses from different perspectives.
From the observations on classical neural network and network geometry, we propose a novel geometric aggregation scheme for graph neural networks to overcome the two weaknesses.
The behind basic idea is the aggregation on a graph can benefit from a continuous space underlying the graph.
The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation.
We also present an implementation of the scheme in graph convolutional networks, termed Geom-GCN, to perform transductive learning on graphs.
Experimental results show the proposed Geom-GCN achieved state-of-the-art performance on a wide range of open datasets of graphs.

assistant: text: For graph neural networks, the aggregation on a graph can benefit from a continuous space underlying the graph.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Federated Adversarial Domain Adaptation

Paper content:
Federated learning improves data privacy and efficiency in machine learning performed over networks of distributed devices, such as mobile phones, IoT and wearable devices, etc.
Yet models trained with federated learning can still fail to generalize to new devices due to the problem of domain shift.
Domain shift occurs when the labeled data collected by source nodes statistically differs from the target node's unlabeled data.
In this work, we present a principled approach to the problem of federated domain adaptation, which aims to align the representations learned among the different nodes with the data distribution of the target node.
Our approach extends adversarial adaptation techniques to the constraints of the federated setting.
In addition, we devise a dynamic attention mechanism and leverage feature disentanglement to enhance knowledge transfer.
Empirically, we perform extensive experiments on several image and text classification tasks and show promising results under unsupervised federated domain adaptation setting.

assistant: text: we present a principled approach to the problem of federated domain adaptation, which aims to align the representations learned among the different nodes with the data distribution of the target node.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Uncovering Surprising Behaviors in Reinforcement Learning via Worst-case Analysis

Paper content:
Reinforcement learning agents are typically trained and evaluated according to their performance averaged over some distribution of environment settings.
But does the distribution over environment settings contain important biases, and do these lead to agents that fail in certain cases despite high average-case performance?
In this work, we consider worst-case analysis of agents over environment settings in order to detect whether there are directions in which agents may have failed to generalize.
Specifically, we consider a 3D first-person task where agents must navigate procedurally generated mazes, and where reinforcement learning agents have recently achieved human-level average-case performance.
By optimizing over the structure of mazes, we find that agents can suffer from catastrophic failures, failing to find the goal even on surprisingly simple mazes, despite their impressive average-case performance.
Additionally, we find that these failures transfer between different agents and even significantly different architectures.
We believe our findings highlight an important role for worst-case analysis in identifying whether there are directions in which agents have failed to generalize.
Our hope is that the ability to automatically identify failures of generalization will facilitate development of more general and robust agents.
To this end, we report initial results on enriching training with settings causing failure.

assistant: text: We find environment settings in which SOTA agents trained on navigation tasks display extreme failures suggesting failures in generalization.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Construction-Planning Models in Minecraft

Paper content:
Minecraft is a videogame that offers many interesting challenges for AI systems.
In this paper, we focus in construction scenarios where an agent must build a complex structure made of individual blocks.
As higher-level objects are formed of lower-level objects, the construction can naturally be modelled as a hierarchical task network.
We model a house-construction scenario in classical and HTN planning and compare the advantages and disadvantages of both kinds of models.

assistant: text: We model a house-construction scenario in Minecraft in classical and HTN planning and compare the advantages and disadvantages of both kinds of models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SSoC: Learning Spontaneous and Self-Organizing Communication for Multi-Agent Collaboration

Paper content:
Multi-agent collaboration is required by numerous real-world problems.
Although distributed setting is usually adopted by practical systems, local range communication and information aggregation still matter in fulfilling complex tasks.
For multi-agent reinforcement learning, many previous studies have been dedicated to design an effective communication architecture.
However, existing models usually suffer from an ossified communication structure, e.g., most of them predefine a particular communication mode by specifying a fixed time frequency and spatial scope for agents to communicate regardless of necessity.
Such design is incapable of dealing with multi-agent scenarios that are capricious and complicated, especially when only partial information is available.
Motivated by this, we argue that the solution is to build a spontaneous and self-organizing communication (SSoC) learning scheme.
By treating the communication behaviour as an explicit action, SSoC learns to organize communication in an effective and efficient way.
Particularly, it enables each agent to spontaneously decide when and who to send messages based on its observed states.
In this way, a dynamic inter-agent communication channel is established in an online and self-organizing manner.
The agents also learn how to adaptively aggregate the received messages and its own hidden states to execute actions.
Various experiments have been conducted to demonstrate that SSoC really learns intelligent message passing among agents located far apart.
With such agile communications, we observe that effective collaboration tactics emerge which have not been mastered by the compared baselines.

assistant: text: This paper proposes a spontaneous and self-organizing communication (SSoC) learning scheme for multi-agent RL tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Mincut Pooling in Graph Neural Networks

Paper content:
The advance of node pooling operations in Graph Neural Networks (GNNs) has lagged behind the feverish design of new message-passing techniques, and pooling remains an important and challenging endeavor for the design of deep architectures.
In this paper, we propose a pooling operation for GNNs that leverages a differentiable unsupervised loss based on the minCut optimization objective.
For each node, our method learns a soft cluster assignment vector that depends on the node features, the target inference task (e.g., a graph classification loss), and, thanks to the minCut objective, also on the connectivity structure of the graph.
Graph pooling is obtained by applying the matrix of assignment vectors to the adjacency matrix and the node features.
We validate the effectiveness of the proposed pooling method on a variety of supervised and unsupervised tasks.

assistant: text: A new pooling layer for GNNs that learns how to pool nodes, according to their features, the graph connectivity, and the dowstream task objective.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Time-Dependent Representation for Neural Event Sequence Prediction

Paper content:
Existing sequence prediction methods are mostly concerned with time-independent sequences, in which the actual time span between events is irrelevant and the distance between events is simply the difference between their order positions in the sequence.
While this time-independent view of sequences is applicable for data such as natural languages, e.g., dealing with words in a sentence, it is inappropriate and inefficient for many real world events that are observed and collected at unequally spaced points of time as they naturally arise, e.g., when a person goes to a grocery store or makes a phone call.
The time span between events can carry important information about the sequence dependence of human behaviors.
In this work, we propose a set of methods for using time in sequence prediction.
Because neural sequence models such as RNN are more amenable for handling token-like input, we propose two methods for time-dependent event representation, based on the intuition on how time is tokenized in everyday life and previous work on embedding contextualization.
We also introduce two methods for using next event duration as regularization for training a sequence prediction model.
We discuss these methods based on recurrent neural nets.
We evaluate these methods as well as baseline models on five datasets that resemble a variety of sequence prediction tasks.
The experiments revealed that the proposed methods offer accuracy gain over baseline models in a range of settings.

assistant: text: Proposed methods for time-dependent event representation and regularization for sequence prediction; Evaluated these methods on five datasets that involve a range of sequence prediction tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Stochastic Weight Averaging in Parallel: Large-Batch Training That Generalizes Well

Paper content:
We propose Stochastic Weight Averaging in Parallel (SWAP), an algorithm to accelerate DNN training.
Our algorithm uses large mini-batches to compute an approximate solution quickly and then refines it by averaging the weights of multiple models computed independently and in parallel.
The resulting models generalize equally well as those trained with small mini-batches but are produced in a substantially shorter time.
We demonstrate the reduction in training time and the good generalization performance of the resulting models on the computer vision datasets CIFAR10, CIFAR100, and ImageNet.

assistant: text: We propose SWAP, a distributed algorithm for large-batch training of neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Many tasks in natural language processing and related domains require high precision output that obeys dataset-specific constraints.
This level of fine-grained control can be difficult to obtain in large-scale neural network models.
In this work, we propose a structured latent-variable approach that adds discrete control states within a standard autoregressive neural paradigm.
Under this formulation, we can include a range of rich, posterior constraints to enforce task-specific knowledge that is effectively trained into the neural model.
This approach allows us to provide arbitrary grounding of internal model decisions, without sacrificing any representational power of neural models.
Experiments consider applications of this approach for text generation and part-of-speech induction.
For natural language generation, we find that this method improves over standard benchmarks, while also providing fine-grained control.

assistant: text: A structured latent-variable approach that adds discrete control states within a standard autoregressive neural paradigm to provide arbitrary grounding of internal model decisions, without sacrificing any representational power of neural models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: End-to-end named entity recognition and relation extraction using pre-trained language models

Paper content:
Named entity recognition (NER) and relation extraction (RE) are two important tasks in information extraction and retrieval (IE & IR).
Recent work has demonstrated that it is beneficial to learn these tasks jointly, which avoids the propagation of error inherent in pipeline-based systems and improves performance.
However, state-of-the-art joint models typically rely on external natural language processing (NLP) tools, such as dependency parsers, limiting their usefulness to domains (e.g. news) where those tools perform well.
The few neural, end-to-end models that have been proposed are trained almost completely from scratch.
In this paper, we propose a neural, end-to-end model for jointly extracting entities and their relations which does not rely on external NLP tools and which integrates a large, pre-trained language model.
Because the bulk of our model's parameters are pre-trained and we eschew recurrence for self-attention, our model is fast to train.
On 5 datasets across 3 domains, our model matches or exceeds state-of-the-art performance, sometimes by a large margin.

assistant: text: A novel, high-performing architecture for end-to-end named entity recognition and relation extraction that is fast to train.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Multi-Agent Interactions Modeling with Correlated Policies

Paper content:
In multi-agent systems, complex interacting behaviors arise due to the high correlations among agents.
However, previous work on modeling multi-agent interactions from demonstrations is primarily constrained by assuming the independence among policies and their reward structures.
In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents’ policies, which can recover agents' policies that can regenerate similar interactions.
Consequently, we develop a Decentralized Adversarial Imitation Learning algorithm with Correlated policies (CoDAIL), which allows for decentralized training and execution.
Various experiments demonstrate that CoDAIL can better regenerate complex interactions close to the demonstrators and outperforms state-of-the-art multi-agent imitation learning methods.
Our code is available at \url{https://github.com/apexrl/CoDAIL}.

assistant: text: Modeling complex multi-agent interactions under multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents’ policies.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Neural Phrase-based Machine Translation

Paper content:
In this paper, we present Neural Phrase-based Machine Translation (NPMT).
Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method.
To mitigate the monotonic alignment requirement of SWAN, we introduce a new layer to perform (soft) local reordering of input sequences.
Different from existing neural machine translation (NMT) approaches, NPMT does not use attention-based decoding mechanisms.
Instead, it directly outputs phrases in a sequential order and can decode in linear time.
Our experiments show that NPMT achieves superior performances on IWSLT 2014 German-English/English-German and IWSLT 2015 English-Vietnamese machine translation tasks compared with strong NMT baselines.
We also observe that our method produces meaningful phrases in output languages.

assistant: text: Neural phrase-based machine translation with linear decoding time
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Reward Constrained Policy Optimization

Paper content:
Solving tasks in Reinforcement Learning is no easy feat.
As the goal of the agent is to maximize the accumulated reward, it often learns to exploit loopholes and misspecifications in the reward signal resulting in unwanted behavior.
While constraints may solve this issue, there is no closed form solution for general constraints.
In this work we present a novel multi-timescale approach for constrained policy optimization, called `Reward Constrained Policy Optimization' (RCPO), which uses an alternative penalty signal to guide the policy towards a constraint satisfying one.
We prove the convergence of our approach and provide empirical evidence of its ability to train constraint satisfying policies.

assistant: text: For complex constraints in which it is not easy to estimate the gradient, we use the discounted penalty as a guiding signal. We prove that under certain assumptions it converges to a feasible solution.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Innovation Protection

Paper content:
Evolutionary-based optimization approaches have recently shown promising results in domains such as Atari and robot locomotion but less so in solving 3D tasks directly from pixels.
This paper presents a method called Deep Innovation Protection (DIP) that allows training  complex world models end-to-end for such 3D environments.
The main idea behind the approach is to employ multiobjective optimization to temporally reduce the selection pressure on specific components in a world model, allowing other components to adapt.
We investigate the emergent representations of these evolved networks, which learn a model of the world without the need for a specific forward-prediction loss.

assistant: text: Deep Innovation Protection allows evolving complex world models end-to-end for 3D tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask

Paper content:
The recent “Lottery Ticket Hypothesis” paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keep the large weights) results in models that are trainable from scratch, but only when starting from the same initial weights.
The performance of these networks often exceeds the performance of the non-sparse base model, but for reasons that were not well understood.
In this paper we study the three critical components of the Lottery Ticket (LT) algorithm, showing that each may be varied significantly without impacting the overall results.
Ablating these factors leads to new insights for why LT networks perform as well as they do.
We show why setting weights to zero is important, how signs are all you need to make the re-initialized network train, and why masking behaves like training.
Finally, we discover the existence of Supermasks, or masks that can be applied to an untrained, randomly initialized network to produce a model with performance far better than chance (86% on MNIST, 41% on CIFAR-10).

assistant: text: In neural network pruning, zeroing pruned weights is important, sign of initialization is key, and masking can be thought of as training.