ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning concise representations for regression by evolving networks of trees

Paper content:
We propose and study a method for learning interpretable representations for the task of regression.
Features are represented as networks of multi-type expression trees comprised of activation functions common in neural networks in addition to other elementary functions.
Differentiable features are trained via gradient descent, and the performance of features in a linear model is used to weight the rate of change among subcomponents of each representation.
The search process maintains an archive of representations with accuracy-complexity trade-offs to assist in generalization and interpretation.
We compare several stochastic optimization approaches within this framework.
We benchmark these variants on 100 open-source regression problems in comparison to state-of-the-art machine learning approaches.
Our main finding is that this approach produces the highest average test scores across problems while producing representations that are orders of magnitude smaller than the next best performing method (gradient boosting).
We also report a negative result in which attempts to directly optimize the disentanglement of the representation result in more highly correlated features.

assistant: text: Representing the network architecture as a set of syntax trees and optimizing their structure leads to accurate and concise regression models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Probabilistic Logic Reasoning with Graph Neural Networks

Paper content:
Markov Logic Networks (MLNs), which elegantly combine logic rules and probabilistic graphical models, can be used to address many knowledge graph problems.
However, inference in MLN is computationally intensive, making the industrial-scale application of MLN very difficult.
In recent years, graph neural networks (GNNs) have emerged as efficient and effective tools for large-scale graph problems.
Nevertheless, GNNs do not explicitly incorporate prior logic rules into the models, and may require many labeled examples for a target task.
In this paper, we explore the combination of MLNs and GNNs, and use graph neural networks for variational inference in MLN.
We propose a GNN variant, named ExpressGNN, which strikes a nice balance between the representation power and the simplicity of the model.
Our extensive experiments on several benchmark datasets demonstrate that ExpressGNN leads to effective and efficient probabilistic logic reasoning.

assistant: text: We employ graph neural networks in the variational EM framework for efficient inference and learning of Markov Logic Networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Evolving the Olfactory System

Paper content:
Flies and mice are species separated by 600 million years of evolution, yet have evolved olfactory systems that share many similarities in their anatomic and functional organization.
What functions do these shared anatomical and functional features serve, and are they optimal for odor sensing?
In this study, we address the optimality of evolutionary design in olfactory circuits by studying artificial neural networks trained to sense odors.
We found that artificial neural networks quantitatively recapitulate structures inherent in the olfactory system, including the formation of glomeruli onto a compression layer and sparse and random connectivity onto an expansion layer.
Finally, we offer theoretical justifications for each result.
Our work offers a framework to explain the evolutionary convergence of olfactory circuits, and gives insight and logic into the anatomic and functional structure of the olfactory system.

assistant: text: Artificial neural networks evolved the same structures present in the olfactory systems of flies and mice after being trained to classify odors
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Reject Illegal Inputs: Scaling Generative Classifiers with Supervised Deep Infomax

Paper content:
Deep Infomax~(DIM) is an unsupervised representation learning framework by maximizing the mutual information between the inputs and the outputs of an encoder, while probabilistic constraints are imposed on the outputs.
In this paper, we propose Supervised Deep InfoMax~(SDIM), which introduces supervised probabilistic constraints to the encoder outputs.
The supervised probabilistic constraints are equivalent to a generative classifier on high-level data representations, where class conditional log-likelihoods of samples can be evaluated.
Unlike other works building generative classifiers with conditional generative models, SDIMs scale on complex datasets, and can achieve comparable performance with discriminative counterparts.
With SDIM, we could perform \emph{classification with rejection}.
Instead of always reporting a class label, SDIM only makes predictions when test samples' largest logits surpass some pre-chosen thresholds, otherwise they will be deemed as out of the data distributions, and be rejected.
Our experiments show that SDIM with rejection policy can effectively reject illegal inputs including out-of-distribution samples and adversarial examples.

assistant: text: scale generative classifiers  on complex datasets, and evaluate their effectiveness to reject illegal inputs including out-of-distribution samples and adversarial examples.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Improving Deep Learning by Inverse Square Root Linear Units (ISRLUs)

Paper content:
We introduce the “inverse square root linear unit” (ISRLU) to speed up learning in deep neural networks.
ISRLU has better performance than ELU but has many of the same benefits.
ISRLU and ELU have similar curves and characteristics.
Both have negative values, allowing them to push mean unit activation closer to zero, and bring the normal gradient closer to the unit natural gradient, ensuring a noise- robust deactivation state, lessening the over fitting risk.
The significant performance advantage of ISRLU on traditional CPUs also carry over to more efficient HW implementations on HW/SW codesign for CNNs/RNNs.
In experiments with TensorFlow, ISRLU leads to faster learning and better generalization than ReLU on CNNs.
This work also suggests a computationally efficient variant called the “inverse square root unit” (ISRU) which can be used for RNNs.
Many RNNs use either long short-term memory (LSTM) and gated recurrent units (GRU) which are implemented with tanh and sigmoid activation functions.
ISRU has less computational complexity but still has a similar curve to tanh and sigmoid.

assistant: text: We introduce the ISRLU activation function which is continuously differentiable and faster than ELU. The related ISRU replaces tanh & sigmoid.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Mixed Precision Training With 8-bit Floating Point

Paper content:
Reduced precision computation is one of the key areas addressing the widening’compute gap’, driven by an exponential growth in deep learning applications.
In recent years, deep neural network training has largely migrated to 16-bit precision,with significant gains in performance and energy efficiency.
However, attempts to train DNNs at 8-bit precision have met with significant challenges, because of the higher precision and dynamic range requirements of back-propagation.
In this paper,  we  propose  a  method  to  train  deep  neural  networks  using  8-bit  floating point representation for weights, activations, errors, and gradients.
We demonstrate state-of-the-art accuracy across multiple data sets (imagenet-1K, WMT16)and a broader set of workloads (Resnet-18/34/50, GNMT, and Transformer) than previously reported.
We propose an enhanced loss scaling method to augment the reduced subnormal range of 8-bit floating point, to improve error propagation.
We also examine the impact of quantization noise on generalization, and propose a stochastic rounding technique to address gradient noise.
As a result of applying all these techniques,  we report slightly higher validation accuracy compared to full precision baseline.

assistant: text: We demonstrated state-of-the-art training results using 8-bit floating point representation, across Resnet, GNMT, Transformer.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Compositional Attention Networks for Machine Reasoning

Paper content:
We present Compositional Attention Networks, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning.
While many types of neural networks are effective at learning and generalizing from massive quantities of data, this model moves away from monolithic black-box architectures towards a design that provides a strong prior for iterative reasoning, enabling it to support explainable and structured learning, as well as generalization from a modest amount of data.
The model builds on the great success of existing recurrent cells such as LSTMs: It sequences a single recurrent Memory, Attention, and Control (MAC) cell, and by careful design imposes structural constraints on the operation of each cell and the interactions between them, incorporating explicit control and soft attention mechanisms into their interfaces.
We demonstrate the model's strength and robustness on the challenging CLEVR dataset for visual reasoning, achieving a new state-of-the-art 98.9% accuracy, halving the error rate of the previous best model.
More importantly, we show that the new model is more computationally efficient, data-efficient, and requires an order of magnitude less time and/or data to achieve good results.

assistant: text: We present a novel architecture, based on dynamic memory, attention and composition for the task of machine reasoning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Data-efficient Deep Reinforcement Learning for Dexterous Manipulation

Paper content:
Grasping an object and precisely stacking it on another is a difficult task for traditional robotic control or hand-engineered approaches.
Here we examine the problem in simulation and provide techniques aimed at solving it via deep reinforcement learning.
We introduce two straightforward extensions to the Deep Deterministic Policy Gradient algorithm (DDPG), which make it significantly more data-efficient and scalable.
Our results show that by making extensive use of off-policy data and replay, it is possible to find high-performance control policies.
Further, our results hint that it may soon be feasible to train successful stacking policies by collecting interactions on real robots.

assistant: text: Data-efficient deep reinforcement learning can be used to learning precise stacking policies.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: GraphQA: Protein Model Quality Assessment using Graph Convolutional Network

Paper content:
Proteins are ubiquitous molecules whose function in biological processes is determined by their 3D structure.
Experimental identification of a protein's structure can be time-consuming, prohibitively expensive, and not always possible.
Alternatively, protein folding can be modeled using computational methods, which however are not guaranteed to always produce optimal results.
GraphQA is a graph-based method to estimate the quality of protein models, that possesses favorable properties such as representation learning, explicit modeling of both sequential and 3D structure, geometric invariance and computational efficiency.
In this work, we demonstrate significant improvements of the state-of-the-art for both hand-engineered and representation-learning approaches, as well as carefully evaluating the individual contributions of GraphQA.

assistant: text: GraphQA is a graph-based method for protein Quality Assessment that improves the state-of-the-art for both hand-engineered and representation-learning approaches
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Rotational Unit of Memory 

Paper content:
The concepts of unitary evolution matrices and associative memory have boosted the field of Recurrent Neural Networks (RNN) to state-of-the-art performance in a variety of sequential tasks.
However, RNN still has a limited capacity to manipulate long-term memory.
To bypass this weakness the most successful applications of RNN use external techniques such as attention mechanisms.
In this paper we propose a novel RNN model that unifies the state-of-the-art approaches: Rotational Unit of Memory (RUM).
The core of RUM is its rotational operation, which is,  naturally,  a unitary matrix, providing architectures with the power to learn long-term dependencies by overcoming the vanishing and exploding gradients problem.
Moreover,  the rotational unit also serves as associative memory.
We evaluate our model on synthetic memorization, question answering and language modeling tasks.
RUM learns the Copying Memory task completely and improves the state-of-the-art result in the Recall task.
RUM’s performance in the bAbI Question Answering task is comparable to that of models with attention mechanism.
We also improve the state-of-the-art result to 1.189 bits-per-character (BPC) loss in the Character Level Penn Treebank (PTB) task, which is to signify the applications of RUM to real-world sequential data.
The universality of our construction, at the core of RNN, establishes RUM as a promising approach to language modeling, speech recognition and machine translation.

assistant: text: A novel RNN model which outperforms significantly the current frontier of models in a variety of sequential tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Disentangling Correlated Speaker and Noise for Speech Synthesis via Data Augmentation and Adversarial Factorization

Paper content:
To leverage crowd-sourced data to train multi-speaker text-to-speech (TTS) models that can synthesize clean speech for all speakers, it is essential to learn disentangled representations which can independently control the speaker identity and background noise in generated signals.
However, learning such representations can be challenging, due to the lack of labels describing the recording conditions of each training example, and the fact that  speakers and recording conditions are often correlated, e.g. since users often make many recordings using the same equipment.
This paper proposes three components to address this problem by: (1) formulating a conditional generative model with factorized latent variables, (2) using data augmentation to add noise that is not correlated with speaker identity and whose label is known during training, and (3) using adversarial factorization to improve disentanglement.
Experimental results demonstrate that the proposed method can disentangle speaker and noise attributes even if they are correlated in the training data, and can be used to consistently synthesize clean speech for all speakers.
Ablation studies verify the importance of each proposed component.

assistant: text: Data augmentation and adversarial training are very effective for disentangling correlated speaker and noise, enabling independent control of each attribute for text-to-speech synthesis.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Iteratively Training Look-Up Tables for Network Quantization

Paper content:
Operating deep neural networks on devices with limited resources requires the reduction of their memory footprints and computational requirements.
In this paper we introduce a training method, called look-up table quantization (LUT-Q), which learns a dictionary and assigns each weight to one of the dictionary's values.
We show that this method is very flexible and that many other techniques can be seen as special cases of LUT-Q.
For example, we can constrain the dictionary trained with LUT-Q to generate networks with pruned weight matrices or restrict the dictionary to powers-of-two to avoid the need for multiplications.
In order to obtain fully multiplier-less networks, we also introduce a multiplier-less version of batch normalization.
Extensive experiments on image recognition and object detection tasks show that LUT-Q consistently achieves better performance than other methods with the same quantization bitwidth.

assistant: text: In this paper we introduce a training method, called look-up table quantization (LUT-Q), which learns a dictionary and assigns each weight to one of the dictionary's values
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Lookahead: A Far-sighted Alternative of Magnitude-based Pruning

Paper content:
Magnitude-based pruning is one of the simplest methods for pruning neural networks.
Despite its simplicity, magnitude-based pruning and its variants demonstrated remarkable performances for pruning modern architectures.
Based on the observation that the magnitude-based pruning indeed minimizes the Frobenius distortion of a linear operator corresponding to a single layer, we develop a simple pruning method, coined lookahead pruning, by extending the single layer optimization to a multi-layer optimization.
Our experimental results demonstrate that the proposed method consistently outperforms the magnitude pruning on various networks including VGG and ResNet, particularly in the high-sparsity regime.

assistant: text: We study a multi-layer generalization of the magnitude-based pruning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generative Adversarial Network Training is a Continual Learning Problem

Paper content:
Generative Adversarial Networks (GANs) have proven to be a powerful framework for learning to draw samples from complex distributions.
However, GANs are also notoriously difficult to train, with mode collapse and oscillations a common problem.
We hypothesize that this is at least in part due to the evolution of the generator distribution and the catastrophic forgetting tendency of neural networks, which leads to the discriminator losing the ability to remember synthesized samples from previous instantiations of the generator.
Recognizing this, our contributions are twofold.
First, we show that GAN training makes for a more interesting and realistic benchmark for continual learning methods evaluation than some of the more canonical datasets.
Second, we propose leveraging continual learning techniques to augment the discriminator, preserving its ability to recognize previous generator samples.
We show that the resulting methods add only a light amount of computation, involve minimal changes to the model, and result in better overall performance on the examined image and text generation tasks.

assistant: text: Generative Adversarial Network Training is a Continual Learning Problem.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Variation Network: Learning High-level Attributes for Controlled Input Manipulation

Paper content:
This paper presents the Variation Network (VarNet), a  generative model providing means to manipulate the high-level attributes of a given input.
The originality of our approach is that VarNet is not only capable of handling pre-defined attributes but can also learn the relevant attributes of the dataset by itself.
These two settings can be easily combined  which makes VarNet applicable for a wide variety of tasks.
Further, VarNet has a sound probabilistic interpretation which grants us with  a novel way to navigate in the latent spaces as well as means to control how the  attributes are learned.
We demonstrate  experimentally that this model is capable of performing interesting input manipulation  and that the learned attributes are relevant and interpretable.

assistant: text: The Variation Network is a generative model able to learn high-level attributes without supervision that can then be used for controlled input manipulation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Do LSTMs Learn Compositionally?

Paper content:
LSTM-based language models exhibit compositionality in their representations, but how this behavior emerges over the course of training has not been explored.
Analyzing synthetic data experiments with contextual decomposition, we find that LSTMs learn long-range dependencies compositionally by building them from shorter constituents during training.

assistant: text: LSTMs learn long-range dependencies compositionally by building them from shorter constituents over the course of training.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Encoder-decoder Network as Loss Function for Summarization

Paper content:
We present a new approach to defining a sequence loss function to train a summarizer by using a secondary encoder-decoder as a loss function, alleviating a shortcoming of word level training for sequence outputs.
The technique is based on the intuition that if a summary is a good one, it should contain the most essential information from the original article, and therefore should itself be a good input sequence, in lieu of the original, from which a summary can be generated.
We present experimental results where we apply this additional loss function to a general abstractive summarizer on a news summarization dataset.
The result is an improvement in the ROUGE metric and an especially large improvement in human evaluations, suggesting enhanced performance that is competitive with specialized state-of-the-art models.

assistant: text: We present the use of a secondary encoder-decoder as a loss function to help train a summarizer.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Harmonic Structure-Based Neural Network Model for Musical Pitch Detection

Paper content:
In this paper we design a harmonic acoustic model for pitch detection.
This model arranges conventional convolution and sparse convolution in a way such that the global harmonic patterns captured by sparse convolution are composed of the enough number of local patterns captured by layers of conventional convolution.
When trained on the MAPS dataset, the harmonic model outperforms all existing pitch detection systems trained on the same dataset.
Most impressively, when trained on MAPS with simple data augmentation, the harmonic model with an LSTM layer on top surpasses an up-to-date, more complex pitch detection system trained on the MAESTRO dataset to which complicated data augmentation is applied and whose training split is an order-of-magnitude larger than the training split of MAPS.
The harmonic model has demonstrated potential to be used for advanced automatic music transcription (AMT) systems.

assistant: text: harmonic acoustic model
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unbiased scalable softmax optimization

Paper content:
Recent neural network and language models have begun to rely on softmax distributions with an extremely large number of categories.
In this context calculating the softmax normalizing constant is prohibitively expensive.
This has spurred a growing literature of efficiently computable but biased estimates of the softmax.
In this paper we present the first two unbiased algorithms for maximizing the softmax likelihood whose work per iteration is independent of the number of classes and datapoints (and does not require extra work at the end of each epoch).
We compare our unbiased methods' empirical performance to the state-of-the-art on seven real world datasets, where they comprehensively outperform all competitors.

assistant: text: Propose first methods for exactly optimizing the softmax distribution using stochastic gradient with runtime independent on the number of classes or datapoints.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Learned Representation for Scalable Vector Graphics

Paper content:
Dramatic advances in generative models have resulted in near photographic quality for artificially rendered faces, animals and other objects in the natural world.
In spite of such advances, a higher level understanding of vision and imagery does not arise from exhaustively modeling an object,  but instead identifying higher-level  attributes  that  best  summarize  the  aspects  of  an  object.
In  this  work  we attempt  to  model  the  drawing  process  of  fonts  by  building  sequential generative models of vector graphics.
This model has the benefit of providing a scale-invariant representation for imagery whose latent representation may be systematically manipulated and exploited to perform style propagation.
We demonstrate these results on a large dataset of fonts and highlight how such a model captures the statistical dependencies and richness of this dataset.
We envision that our model can find use as a tool for designers to facilitate font design.

assistant: text: We attempt to model the drawing process of fonts by building sequential generative models of vector graphics (SVGs), a highly structured representation of font characters.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Nonlinear Channels Aggregation Networks for Deep Action Recognition

Paper content:
We introduce the concept of channel aggregation in ConvNet architecture, a novel compact representation of CNN features useful for explicitly modeling the nonlinear channels encoding especially when the new unit is embedded inside of deep architectures for action recognition.
The channel aggregation is based on multiple-channels features of ConvNet and aims to be at the spot finding the optical convergence path at fast speed.
We name our proposed convolutional architecture “nonlinear channels aggregation networks (NCAN)” and its new layer “nonlinear channels aggregation layer (NCAL)”.
We theoretically motivate channels aggregation functions and empirically study their effect on convergence speed and classification accuracy.
Another contribution in this work is an efficient and effective implementation of the NCAL, speeding it up orders of magnitude.
We evaluate its performance on standard benchmarks UCF101 and HMDB51, and experimental results demonstrate that this formulation not only obtains a fast convergence but stronger generalization capability without sacrificing performance.

assistant: text: An architecture enables CNN trained on the video sequences converging rapidly
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Effective GANs for Data Distributions with Diverse Modes

Paper content:
Generative Adversarial Networks (GANs), when trained on large datasets with diverse modes, are known to produce conflated images which do not distinctly belong to any of the modes.
We hypothesize that this problem occurs due to the interaction between two facts: (1) For datasets with large variety, it is likely that the modes lie on separate manifolds.
(2) The generator (G) is formulated as a continuous function, and the input noise is derived from a connected set, due to which G's output is a connected set.
If G covers all modes, then there must be some portion of G's output which connects them.
This corresponds to undesirable, conflated images.
We develop theoretical arguments to support these intuitions.
We propose a novel method to break the second assumption via learnable discontinuities in the latent noise space.
Equivalently, it can be viewed as training several generators, thus creating discontinuities in the G function.
We also augment the GAN formulation with a classifier C that predicts which noise partition/generator produced the output images, encouraging diversity between each partition/generator.
We experiment on MNIST, celebA, STL-10, and a difficult dataset with clearly distinct modes, and show that the noise partitions correspond to different modes of the data distribution, and produce images of superior quality.

assistant: text: We introduce theory to explain the failure of GANs on complex datasets and propose a solution to fix it.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Don't Decay the Learning Rate, Increase the Batch Size

Paper content:
It is common practice to decay the learning rate.
Here we show one can usually obtain the same learning curve on both training and test sets by instead increasing the batch size during training.
This procedure is successful for stochastic gradient descent (SGD), SGD with momentum, Nesterov momentum, and Adam.
It reaches equivalent test accuracies after the same number of training epochs, but with fewer parameter updates, leading to greater parallelism and shorter training times.
We can further reduce the number of parameter updates by increasing the learning rate $\epsilon$ and scaling the batch size $B \propto \epsilon$.
Finally, one can increase the momentum coefficient $m$ and scale $B \propto 1/(1-m)$, although this tends to slightly reduce the test accuracy.
Crucially, our techniques allow us to repurpose existing training schedules for large batch training with no hyper-parameter tuning.
We train ResNet-50 on ImageNet to 76.1% validation accuracy in under 30 minutes.

assistant: text: Decaying the learning rate and increasing the batch size during training are equivalent.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: LEARNED STEP SIZE QUANTIZATION

Paper content:
Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases.
Here, we present a method for training such networks, Learned Step Size Quantization, that achieves the highest accuracy to date on the ImageNet dataset when using models, from a variety of architectures, with weights and activations quantized to 2-, 3- or 4-bits of precision, and that can train 3-bit models that reach full precision baseline accuracy.
Our approach builds upon existing methods for learning weights in quantized networks by improving how the quantizer itself is configured.
Specifically, we introduce a novel means to estimate and scale the task loss gradient at each weight and activation layer's quantizer step size, such that it can be learned in conjunction with other network parameters.
This approach works using different levels of precision as needed for a given system and requires only a simple modification of existing training code.

assistant: text: A method for learning quantization configuration for low precision networks that achieves state of the art performance for quantized networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: An image representation based convolutional network for DNA classification

Paper content:
The folding structure of the DNA molecule combined with helper molecules, also referred to as the chromatin, is highly relevant for the functional properties of DNA.
The chromatin structure is largely determined by the underlying primary DNA sequence, though the interaction is not yet fully understood.
In this paper we develop a convolutional neural network that takes an image-representation of primary DNA sequence as its input, and predicts key determinants of chromatin structure.
The method is developed such that it is capable of detecting interactions between distal elements in the DNA sequence, which are known to be highly relevant.
Our experiments show that the method outperforms several existing methods both in terms of prediction accuracy and training time.

assistant: text: A method to transform DNA sequences into 2D images using space-filling Hilbert Curves to enhance the strengths of CNNs
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Using Explainabilty to Detect Adversarial Attacks

Paper content:
Deep learning models are often sensitive to adversarial attacks, where carefully-designed input samples can cause the system to produce incorrect decisions.
Here we focus on the problem of detecting attacks, rather than robust classification, since detecting that an attack occurs may be even more important than avoiding misclassification.
We build on advances in explainability, where activity-map-like explanations are used to justify and validate decisions, by highlighting features that are involved with a classification decision.
The key observation is that it is hard to create explanations for incorrect decisions.
We propose EXAID, a novel attack-detection approach, which uses model explainability to identify images whose explanations are inconsistent with the predicted class
. Specifically, we use SHAP, which uses Shapley values in the space of the input image, to identify which input features contribute to a class decision
. Interestingly, this approach does not require to modify the attacked model, and it can be applied without modelling a specific attack
. It can therefore be applied successfully to detect unfamiliar attacks, that were unknown at the time the detection model was designed
. We evaluate EXAID on two benchmark datasets CIFAR-10 and SVHN, and against three leading attack techniques, FGSM, PGD and C&W.
We find that EXAID improves over the SoTA detection methods by a large margin across a wide range of noise levels, improving detection from 70% to over 90% for small perturbations.

assistant: text: A novel adversarial detection approach, which uses explainability methods to identify images whose explanations are inconsistent with the predicted class.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Exploration Based Language Learning for Text-Based Games

Paper content:
This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games.
Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text.
These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
Moreover, they provide a learning environment in which these skills can be acquired through interactions with an environment rather than using fixed corpora.
One aspect that makes these games particularly challenging for learning agents is the combinatorially large action space.
Existing methods for solving text-based games are limited to games that are either very simple or have an action space restricted to a predetermined set of admissible actions.
In this work, we propose to use the exploration approach of Go-Explore (Ecoffet et al., 2019) for solving text-based games.
More specifically, in an initial exploration phase, we first extract trajectories with high rewards, after which we train a policy to solve the game by imitating these trajectories.
Our experiments show that this approach outperforms existing solutions in solving text-based games, and it is more sample efficient in terms of the number of interactions with the environment.
Moreover, we show that the learned policy can generalize better than existing solutions to unseen games without using any restriction on the action space.

assistant: text: This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Autonomous Vehicle Fleet Coordination With Deep Reinforcement Learning

Paper content:
Autonomous vehicles are becoming more common in city transportation.
Companies will begin to find a need to teach these vehicles smart city fleet coordination.
Currently, simulation based modeling along with hand coded rules dictate the decision making of these autonomous vehicles.
We believe that complex intelligent behavior can be learned by these agents through Reinforcement Learning.
In this paper, we discuss our work for solving this system by adapting the Deep Q-Learning (DQN) model to the multi-agent setting.
Our approach applies deep reinforcement learning by combining convolutional neural networks with DQN to teach agents to fulfill customer demand in an environment that is partially observ-able to them.
We also demonstrate how to utilize transfer learning to teach agents to balance multiple objectives such as navigating to a charging station when its en-ergy level is low.
The two evaluations presented show that our solution has shown  hat we are successfully able to teach agents cooperation policies while balancing multiple objectives.

assistant: text: Utilized Deep Reinforcement Learning to teach agents ride-sharing fleet style coordination.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: MACHINE VS MACHINE: MINIMAX-OPTIMAL DEFENSE AGAINST ADVERSARIAL EXAMPLES

Paper content:
Recently, researchers have discovered that the state-of-the-art object classifiers can be fooled easily by small perturbations in the input unnoticeable to human eyes.
It is known that an attacker can generate strong adversarial examples if she knows the classifier parameters
. Conversely, a defender can robustify the classifier by retraining if she has the adversarial examples
. 
The cat-and-mouse game nature of attacks and defenses raises the question of the presence of equilibria in the dynamics
.
In this paper, we present a neural-network based attack class to approximate a larger but intractable class of attacks, and
formulate the attacker-defender interaction as a zero-sum leader-follower game.
We present sensitivity-penalized optimization algorithms to find minimax solutions, which are the best worst-case defenses against whitebox attacks.
Advantages of the learning-based attacks and defenses compared to gradient-based attacks and defenses are demonstrated with MNIST and CIFAR-10.

assistant: text: A game-theoretic solution to adversarial attacks and defenses.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Modifying memories in a Recurrent Neural Network Unit

Paper content:
Long Short-Term Memory (LSTM) units have the ability to memorise and use long-term dependencies between inputs to generate predictions on time series data.
We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrised by a new set of trainable weights.
This addition shows significant increases of performance on some of the tasks from the bAbI dataset.

assistant: text: Adding a new set of weights to the LSTM that rotate the cell memory improves performance on some bAbI tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Barcodes as summary of objective functions' topology

Paper content:
We apply canonical forms of gradient complexes (barcodes) to explore neural networks loss surfaces.
We present an algorithm for calculations of the objective function's barcodes of minima.
Our experiments confirm two principal observations: (1) the barcodes of minima are located in a small lower part of the range of values of objective function and (2) increase of the neural network's depth brings down the minima's barcodes.
This has natural implications for the neural network learning and the ability to generalize.

assistant: text: We apply canonical forms of gradient complexes (barcodes) to explore neural networks loss surfaces.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A shallow feature extraction network with a large receptive field for stereo matching tasks

Paper content:
Stereo matching is one of the important basic tasks in the computer vision field.
In recent years, stereo matching algorithms based on deep learning have achieved excellent performance and become the mainstream research direction.
Existing algorithms generally use deep convolutional neural networks (DCNNs) to extract more abstract semantic information, but we believe that the detailed information of the spatial structure is more important for stereo matching tasks.
Based on this point of view, this paper proposes a shallow feature extraction network with a large receptive field.
The network consists of three parts: a primary feature extraction module, an atrous spatial pyramid pooling (ASPP) module and a feature fusion module.
The primary feature extraction network contains only three convolution layers.
This network utilizes the basic feature extraction ability of the shallow network to extract and retain the detailed information of the spatial structure.
In this paper, the dilated convolution and atrous spatial pyramid pooling (ASPP) module is introduced to increase the size of receptive field.
In addition, a feature fusion module is designed, which integrates the feature maps with multiscale receptive fields and mutually complements the feature information of different scales.
We replaced the feature extraction part of the existing stereo matching algorithms with our shallow feature extraction network, and achieved state-of-the-art performance on the KITTI 2015 dataset.
Compared with the reference network, the number of parameters is reduced by 42%, and the matching accuracy is improved by 1.9%.

assistant: text: We introduced a shallow featrue extraction network with a large receptive field for stereo matching tasks, which uses a simple structure to get better performance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Self-Adversarial Learning with Comparative Discrimination for Text Generation

Paper content:
Conventional Generative Adversarial Networks (GANs) for text generation tend to have issues of reward sparsity and mode collapse that affect the quality and diversity of generated samples.
To address the issues, we propose a novel self-adversarial learning (SAL) paradigm for improving GANs' performance in text generation.
In contrast to standard GANs that use a binary classifier as its discriminator to predict whether a sample is real or generated, SAL employs a comparative discriminator which is a pairwise classifier for comparing the text quality between a pair of samples.
During training, SAL rewards the generator when its currently generated sentence is found to be better than its previously generated samples.
This self-improvement reward mechanism allows the model to receive credits more easily and avoid collapsing towards the limited number of real samples, which not only helps alleviate the reward sparsity issue but also reduces the risk of mode collapse.
Experiments on text generation benchmark datasets show that our proposed approach substantially improves both the quality and the diversity, and yields more stable performance compared to the previous GANs for text generation.

assistant: text: We propose a self-adversarial learning (SAL) paradigm which improves the generator in a self-play fashion for improving GANs' performance in text generation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Plan recognition aims to look for target plans to best explain the observed actions based on plan libraries and/or domain models.
Despite the success of previous approaches on plan recognition, they mostly rely on correct action observations.
Recent advances in visual activity recognition have the potential of enabling applications such as automated video surveillance.
Effective approaches for such problems would require the ability to recognize the plans of agents from video information.
Traditional plan recognition algorithms rely on access to detailed planning domain models.
One recent promising direction involves learning approximate (or shallow) domain models directly from the observed activity sequences.
Such plan recognition approaches expect observed action sequences as inputs.
However, visual inference results are often noisy and uncertain, typically represented as a distribution over possible actions.
In this work, we develop a visual plan recognition framework that recognizes plans with an approximate domain model learned from uncertain visual data.

assistant: text: Handling Uncertainty in Visual Perception for Plan Recognition
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding

Paper content:
For understanding generic documents, information like font sizes, column layout, and generally the positioning of words may carry semantic information that is crucial for solving a downstream document intelligence task.
Our novel BERTgrid, which is based on Chargrid by Katti et al. (2018), represents a document as a grid of contextualized word piece embedding vectors, thereby making its spatial structure and semantics accessible to the processing neural network.
The contextualized embedding vectors are retrieved from a BERT language model.
We use BERTgrid in combination with a fully convolutional network on a semantic instance segmentation task for extracting fields from invoices.
We demonstrate its performance on tabulated line item and document header field extraction.

assistant: text: Grid-based document representation with contextualized embedding vectors for documents with 2D layouts
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning

Paper content:
Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning.
In the single-agent setting, this challenge has been addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces.
Applying these techniques naively to the multi-agent setting results in agents exploring independently, without any coordination among themselves.
We argue that learning in cooperative multi-agent settings can be accelerated and improved if agents coordinate with respect to what they have explored.
In this paper we propose an approach for learning how to dynamically select between different types of intrinsic rewards which consider not just what an individual agent has explored, but all agents, such that the agents can coordinate their exploration and maximize extrinsic returns.
Concretely, we formulate the approach as a hierarchical policy where a high-level controller selects among sets of policies trained on different types of intrinsic rewards and the low-level controllers learn the action policies of all agents under these specific rewards.
We demonstrate the effectiveness of the proposed approach in a multi-agent gridworld domain with sparse rewards, and then show that our method scales up to more complex settings by evaluating on the VizDoom platform.

assistant: text: We propose several intrinsic reward functions for encouraging coordinated exploration in multi-agent problems, and introduce an approach to dynamically selecting the best exploration method for a given task, online.